Events

Wed Sep 9 · 2:00 PM EDT · Free

Prove Your Prompt Change Actually Helped

You changed a prompt. The score went up. But did it actually improve — or did you get lucky? In this free 30-minute lesson, learn how to run a proper paired test on two prompt versions, how to tell a real improvement from noise, and how to pick a sample size that settles the argument. No more gut-feel comparisons.

Sat Sep 12 · 9:30 AM–1:00 PM EDT

Live LLM Engineering Masterclass: Production Evals, RAG, Agents & LLMOps

A hands-on 3.5-hour deep dive into production LLM engineering — covering evals, RAG, agents, prompt engineering, observability, and LLMOps. Every technique demonstrated with running code. Organized by Packt Publishing.

Wed Sep 23 · 2:00 PM EDT · Free

Put Error Bars on Your LLM Metrics

You ran the same eval twice and got 84.2, then 81.9. Which number goes in the report? Without error bars, every score is a coin flip dressed as a fact. In 30 minutes, learn to bootstrap a confidence interval on any metric — accuracy, cost, or latency — in 20 lines of Python. No distribution assumptions required.

Wed Oct 7 · 2:00 PM EDT · Free

Stop Bad Merges with an LLM Eval Gate

A bad prompt change reaches production the same way a good one does: nobody measured either. In this free lesson, learn to wire an eval suite into GitHub Actions so regressions die in the pull-request queue — not in front of users. The full setup fits in one YAML file. Vigilance does not scale. Policy does.

Fri Oct 16 · 10:00 AM–2:00 PM EDT · $500

Build a Production-Grade LLM Eval Harness

In four hours, build a production eval harness from scratch — calibrated judges, error bars on every metric, paired tests that settle arguments, and a CI gate that blocks bad merges. You leave with a running Inspect-AI harness, the full repo, and a swap guide to run it on your own product data. Monday morning, it runs.