A 14-billion-parameter model claims 28 gigabytes of memory before the first token. The KV cache then grows with every token of every open request, and real traffic opens many at once. This 374-page manual lives inside that squeeze. Chi Wang runs model-inference engineering teams at Salesforce. Peiheng Hu builds distributed inference engines at NVIDIA. The two spent over eight years building AI systems together, and it shows. Their claim is blunt. Training gets the papers, and serving carries the product. Welcome to the inference era.
The method setsHands-On LLM Serving and Optimizationapart. You build a serving service from scratch, batching and streaming included, and only then meet vLLM, TensorRT-LLM, SGLang, and llama.cpp. The frameworks stop looking like magic. The arithmetic alone earns the cover price. Estimate the weight footprint, size the KV cache, compute the arithmetic intensity of prefill and decode, and you know whether a workload is compute-bound or memory-bound before renting a single GPU. Continuous batching, quantization, speculative decoding, and the four parallelisms follow, each with runnable code, and an eight-step tuning plan for Qwen3-14B ties the whole book together. Data scientists get the systems course their training skipped. Platform engineers get a build-or-buy playbook.
Two warnings before you buy. The book went to press in April 2026, pinned to Qwen3-14B and today’s engine flags, and those engines ship faster than any press run. Treat the companion repository as the living copy. The second warning concerns fit. Many notebooks want an NVIDIA GPU, and the book stays on the serving side of the wall. Training, fine-tuning, and evals sit outside its scope, so readers hunting for modeling content will find queues and schedulers instead. Read it anyway. The publisher clocks it at 11 hours, one weekend. The serving layer decides what a model costs and how fast it feels, and most teams inherit it as a black box. This book opens the box and labels the parts. The next time the latency graph spikes or the GPU bill doubles, you will know which knob to turn first.