Issue #329
July 29, 2026
Announcements
Put your old Mac to work serving a local LLM.
Running a language model on your own hardware costs nothing per token, works on a plane, and never sends your code elsewhere. On Apple Silicon it's fast too — unified memory is a large part of why Mac Minis are back-ordered for weeks. This post covers the whole path: install llama.cpp, pull a GGUF, and serve it over an OpenAI-compatible endpoint on localhost.
The second half is what lasts. Open the GGUF, read its structure, and turn that into real judgment about quantization: what Q4_K_M actually encodes, and how to add up weights, KV cache, and buffers to know whether a model fits in RAM before you download it.
👉 Self hosting LLMs with Llama.cpp
Check it out and Subscribe so you don't miss another post.
Most LLM books teach you what these models can do. T. T. P. de Souza and J. K. Regenstein Jr wrote 340 pages about what breaks when you build with them. “Large Language Models: The Hard Parts” covers the gap between a working demo and an application your organization can trust: evaluation, safety, context management, and structured output. Every chapter ships with reproducible Python code and open source tools.
The evaluation chapters alone repay the cover price. The authors build a testing framework around a concrete 10-K analysis app, then compare LangSmith, Promptfoo, and LightEval with working code for each. Most books stop at “evals matter.” This one picks the tools and shows the tradeoffs. An appendix on Ollama and llama.cpp, quantization experiment included, serves anyone who must run models on their own hardware.
Two gaps: serving at scale gets no real treatment, so engineers who own inference infrastructure will need a second book, and the tool walkthroughs will age as APIs shift. Both sit outside the stated scope. For a data scientist or ML engineer moving a prototype toward production, this is the rare LLM book that skips the hype and starts where the pain does. What it promises, it delivers..
- 1. Probability and estimation on census data [stochastic.blog]
- 2. The Whole Premise Of Checking For Human Writing Is Daft [mohammedshehu.com]
- 3. DARPA, U.S. Air Force fly AI-controlled F-16 [darpa.mil]
- 4. Introducing Claude Opus 5 [anthropic.com]
- 5. The linear algebra and calculus behind every model [stochastic.blog]
- 6. Writing by Hand is Good for your Brain - Here's how to do it [nealstephenson.substack.com]
- 7. The first ATLAS report on AI [blog.google]
- 8. RAGOps Guide: Building and Scaling Retrieval Augmented Generation Systems [towardsdatascience.com]
- 9. Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs [github.com/vllm-project]
- 10. The Norwegian case for mediocracy [ft.com]
- • Limits of Inference in Complex Systems: When Stochastic Models Become Indistinguishable (J. Aguilar, M. A. Muñoz, S. Azaele)
- • Non-Gaussian statistics of the order parameter across a phase transition (M. Allemand, G. Dupuy, P. Paquiez, N. Dupuis, A. Rançon, T. Roscilde, T. Chalopin, D. Clément)
- • Coordinative structures as scale-free networks: Cascade and percolation dynamics in motor learning with empirical validation (C. Park)
- • Re-emergence of vaccine-preventable diseases in the post-elimination era (C. K. Kumar, N. C. Lo)
- • Efficient Gillespie algorithms for spreading phenomena in large and heterogeneous higher-order networks (H. P. Maia, W. Cota, Y. Moreno, S. C. Ferreira)
- • OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews (M. Idahl, Z. Ahmadi)
- • Ten years of generative adversarial nets (GANs): a survey of the state-of-the-art (T. Chakraborty, U. Reddy, S. M. Naik, M. Panja, B. Manvitha)
- • Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes (D. Pearson, S. Shapiro, E. S. G. Venegas, S. Al-Khatib, A. P. Arzola)
Mathematics in the Age of AI
All our videos are also available in our YouTube playlist.
Enjoy the newsletter?
Forward it to a friend, or subscribe to get it straight to your inbox.
Subscribe Free