Estimation · Multiple choice

A recommender retrieves 500 candidates per request and has a cheap distilled model and a deep model 20 times dearer per item. What keeps the cost and latency bounded?

medium Caching

Options

Why

A ranking funnel spends each model where it pays: the distilled model rates all 500, a mid-size network the best 150, and the deep model only the best

  1. Its load falls tenfold. Caching scores per user rarely hits, because candidates change on every visit, and parallel calls cut latency, not the machines they need.

Review this in your daily deck All cards in Estimation