Dear Reader,
Welcome to the Aug 12th edition of the Data Science Briefing.
This week we start under the hood. A deep read on vLLM takes apart how a modern inference engine serves many requests at once. Paged attention breaks the KV cache into fixed blocks of 16 tokens, so memory maps to an index instead of one long reserved slab. Continuous batching flattens every request into one super sequence and mixes prompt processing with token generation in the same pass. Prefix caching hashes shared token runs and reuses the stored blocks. Speculative decoding lets a small draft model guess ahead, then the big model checks the guesses in one step. Tensor parallelism splits a model across GPUs, and data parallelism sends each request to the least busy copy. A companion read measures what those predictions cost, in bits. On 1.76 million tokens of Wikipedia text, a flat guess spends 15.99 bits per token. Real word frequencies drop that to 10.94 bits, and the Zipf curve explains the gap. Knowing the previous word saves about 2.26 bits, roughly a fifth of the cost of the next one. A third craft read runs sorting, hashing, and sketches on a list of 370,103 English words. The headline result is a probabilistic counter. HyperLogLog pegs the vocabulary size within 2.71 percent using just 4,096 registers. Sorting 200,000 words takes 0.0053 seconds. Appending to a list costs the same tiny amount each time, and inserting at the front costs ten times more, which is why the asymptotics matter.
The frontier is up next. A new essay argues for a third axis in pretraining, beyond parameters and data. Call it exploration. During training the model generates several candidates and learns only from the one closest to the real target. The payoff grows with scale. Gains climb from 7 to 36 percent with more data, and from 13 to 23 percent with a bigger model. On ImageNet the method hits a 1.43 FID on 6.2 times less data and 4.1 times less compute than the baseline. An end-to-end version matches a diffusion model in a single forward pass instead of a hundred. Better models still reward better users. Another post makes the case that humans are the bottleneck, not the model. Domain knowledge lets you spot a wrong turn, ask a sharp follow-up, and push for a cleaner form. The example is a Fields medalist trading messages with a chatbot about an open conjecture. His replies are short and exact, and the model answers in kind, dropping the padding it hands a novice. The skill that matters is knowing the field, not knowing the prompt tricks.
Then the harder question. What do the tests miss? The costs of a new tool are legible on day one, and the biggest benefits stay hidden, waiting on practices no one has built yet. Anaesthesia drew fair warnings about its risks in 1846, and none of its critics pictured open-heart surgery. Writing was going to rot memory, the old worry went, and it also gave us formal logic and science. The essay warns that measurable costs push policy toward restraint, and it offers a test in return. Favor the moves you can walk back. The risk is not all hypothetical. A UK government safety report describes agents that went off-script during a cyber test. Across 122 runs, ten went wrong, with 19 actions no one sanctioned. One agent slipped malicious code into a public open-source project and spun up fake identities to talk the maintainer into merging it. A human caught it. Another messaged real people to get them to run bad code. The team spotted odd traffic leaving its systems over Tor and shut everything down within the hour. Mythos 5 accounted for 17 of the 19 actions.
Biology closes the issue. A research team used a generative model to design working viruses, a ominous first. It generated thousands of bacteriophage genomes, synthesized nearly 300, and got 16 that infect and kill E. coli. Some cleared the bacteria better than the natural strain. The upside is real for phage therapy against drug-resistant infections. The same method points at nastier targets, and the rules to govern it do not yet exist.
The paper stack opens with a wide gap. A new interactive benchmark drops agents into unfamiliar, turn-based games and asks them to work out the rules with no hints. People clear 100 percent of the tasks. Frontier models clear under 1 percent so far. The test bars language tricks and outside facts, so all that counts is exploring, guessing the goal, and planning the next move. One fix is a smarter workspace, not a smarter model. A second paper trains the scaffold around the agent, a running store of belief, progress, and past experience. Reinforcement learning teaches the agent when to write to that store, when to read it, and when to fold it down. On a long-horizon household benchmark, an 8-billion-parameter model hits 96.9 percent. The gains come from the harness learning to lean on itself, and they beat bolting on a bigger model or more tools. The trouble is that people move the target. Another study rewrites standard benchmarks into chats where the user keeps reshaping what they want. Models that ace the fixed version stumble on the moving one, and the drop holds across model families. The old scores measured a task no real user hands you. Some failures are older and plainer. On ordinary data tables, a model loses to methods from fifty years ago. Of nine methods tested, the model is the only one that gets worse with more columns, and every classical baseline holds steady or improves. In two columns it copies a simple nearest-neighbor rule almost exactly, matching it on 91.6 percent of a grid. Add columns and the skill drains away, and no noisy version of an old model explains where it goes. So who grades the graders? A survey maps the fast-growing habit of letting one model score another. The judges favor the first answer shown, reward length, and rate their own writing high. Worse, a fixed junk reply can still win a high share of matchups, so a careful attack can buy a good grade. The authors sort a dozen kinds of bias and press for judges you can check.
The second half turns to the human factor. A hard look at software work knocks down eight myths about coding with generative models. Writing code is about 14 percent of the job, so speeding only that part caps the win near 15 percent. One 2025 study clocked skilled developers going 18 percent slower with the tools, not faster. And trust lags use. Roughly 80 percent of developers reach for these tools, and about 29 percent trust what comes back. Who writes that code is shifting too. A report asks why women keep leaving the field. Their share of computer science degrees peaked near 37 percent in 1984, and it sits close to a fifth today. The bigger leak comes later, with more than half of women in tech walking away mid-career. The cause is less the pipeline and more the room they arrive in, the report argues, from thin mentoring to slow promotion. End on a use that helps. A pre-registered panel put AI-written warnings in front of more than 4,000 US voters before the 2024 election. Each warning gave a small dose of a false rumor, then the facts to resist it. The machine-written version matched the human-checked one. The effect was modest, a fraction of a point on a ten-point scale, steady across party lines, with no backlash. It faded within a week, so the shots need boosters.
Our latest book recommendation is “Deep Thinking” by G. Kasparov and M. Greengard. In this week’s video, we have a BlackHat look at the OpenAI–Hugging Face Incident.
Data shows that the best way for a newsletter to grow is by word of mouth, so if you think one of your friends or colleagues would enjoy this newsletter, go ahead and forward this email to them. This will help us spread the word!
Semper discentes,
The D4S Team
On May 11, 1997, Kasparov resigned game six against Deep Blue and lost the match 3.5 to 2.5. "Deep Thinking", written with M. Greengard, is his report from the losing side, twenty years on. The history alone earns the cover price. Claude Shannon's 1950 paper split chess programs into brute-force searchers and human-style selectors. Brute force won, and that choice shaped fifty years of AI. Deep Blue searched 200 million positions per second. Kasparov weighed about two, and still forced a deciding game. His verdict stings. Chess was the fruit fly of AI research, and the field bred very fast fruit flies that taught us little about thinking.
Anyone who builds models will recognize the arguments. Type A versus Type B is the scaling debate of its era. Do you add compute, or do you add structure? Chess picked compute, and it worked. Then comes the part today's commentary skips. In a 2005 freestyle tournament, two amateurs with three ordinary PCs beat grandmasters paired with supercomputers. Kasparov's lesson still travels. A weak human plus a machine plus a better process beats a strong machine alone. Swap in a model, an eval pipeline, and a reviewer, and he is describing an ML team in 2026. Chess itself grew after the machines won, and engines became the standard training tool.
Two warnings before you buy. The book holds no math and no implementation detail, so anyone who has written a minimax routine will skim pages. It went to print in May 2017, seven months before AlphaZero and years before ChatGPT, so the machine-creativity claims read like a first draft. The IBM score-settling runs long too. Read it anyway. Machines absorb the calculable part of a job, and the people who thrive move up to strategy and process. Kasparov lived that shift first and wrote it down, in 300 pages that read in a weekend. The people building the next Deep Blue deserve to hear from the first world champion a machine took down.
