Data Science Briefing #329

Issue #329

July 29, 2026

Announcements

Put your old Mac to work serving a local LLM.

Running a language model on your own hardware costs nothing per token, works on a plane, and never sends your code elsewhere. On Apple Silicon it's fast too — unified memory is a large part of why Mac Minis are back-ordered for weeks. This post covers the whole path: install llama.cpp, pull a GGUF, and serve it over an OpenAI-compatible endpoint on localhost.

The second half is what lasts. Open the GGUF, read its structure, and turn that into real judgment about quantization: what Q4_K_M actually encodes, and how to add up weights, KV cache, and buffers to know whether a model fits in RAM before you download it.

👉 Self hosting LLMs with Llama.cpp

Check it out and Subscribe so you don't miss another post.


Book of the Week

Most LLM books teach you what these models can do. T. T. P. de Souza and J. K. Regenstein Jr wrote 340 pages about what breaks when you build with them. “Large Language Models: The Hard Parts” covers the gap between a working demo and an application your organization can trust: evaluation, safety, context management, and structured output. Every chapter ships with reproducible Python code and open source tools.

The evaluation chapters alone repay the cover price. The authors build a testing framework around a concrete 10-K analysis app, then compare LangSmith, Promptfoo, and LightEval with working code for each. Most books stop at “evals matter.” This one picks the tools and shows the tradeoffs. An appendix on Ollama and llama.cpp, quantization experiment included, serves anyone who must run models on their own hardware.

Two gaps: serving at scale gets no real treatment, so engineers who own inference infrastructure will need a second book, and the tool walkthroughs will age as APIs shift. Both sit outside the stated scope. For a data scientist or ML engineer moving a prototype toward production, this is the rare LLM book that skips the hype and starts where the pain does. What it promises, it delivers..

Large Language Models: The Hard Parts

Large Language Models: The Hard Parts


Links of the Week
  1. 1. Probability and estimation on census data [stochastic.blog]
  2. 2. The Whole Premise Of Checking For Human Writing Is Daft [mohammedshehu.com]
  3. 3. DARPA, U.S. Air Force fly AI-controlled F-16 [darpa.mil]
  4. 4. Introducing Claude Opus 5 [anthropic.com]
  5. 5. The linear algebra and calculus behind every model [stochastic.blog]
  6. 6. Writing by Hand is Good for your Brain - Here's how to do it [nealstephenson.substack.com]
  7. 7. The first ATLAS report on AI [blog.google]
  8. 8. RAGOps Guide: Building and Scaling Retrieval Augmented Generation Systems [towardsdatascience.com]
  9. 9. Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs [github.com/vllm-project]
  10. 10. The Norwegian case for mediocracy [ft.com]

Papers of the Week
Video of the Week

Mathematics in the Age of AI

Mathematics in the Age of AI

All our videos are also available in our YouTube playlist.


Enjoy the newsletter?

Forward it to a friend, or subscribe to get it straight to your inbox.

Subscribe Free
← Back to Newsletter

Subscribe to get our latest content by email.
    We won't send you spam. Unsubscribe at any time.