Latency Budgets for AI Features

Set a hard latency budget and design UIs that earn attention every 200ms: optimistic updates, partial results, and streaming tokens. Cache aggressively, precompute where possible, and prefetch embeddings during idle time. If the model can't meet SLA, provide deterministic fallbacks and clear affordances.
Users forgive waiting when they see progress, but never when they lose control. Latency is a product decision as much as an engineering one.