What Inference Actually Costs, and How to Stop It Surprising You
Per-token pricing looks trivial until it multiplies by traffic. The bill is controllable, and most of the control is in decisions made early.
Engineering
Building AI Products for Users on Bad Connections
Streaming responses, hopeful reconnects and generous payloads all assume a connection that a great many users do not have.
Retrieval-Augmented Generation, Explained Properly
The idea is simple and the failure modes are not. Most disappointing implementations fail at retrieval, not at generation.
Evaluating an AI Feature Before You Ship It
You cannot unit-test a language model, which is not the same as being unable to test it. A small hand-built evaluation set is worth more than any public benchmark.