Over half of the Instagram community visits Explore every month: a grid of photos and videos from accounts they do not follow yet, picked for them out of billions of posts. Meta's Explore system is a funnel. Candidate generation finds a few hundred posts that might interest the user, and a series of ranking passes, each more expensive per item than the last, narrows them to the 25 that fill the first page.
The expensive part is the ranking: a deep neural network that predicts likes, saves and "see fewer posts like this" is far too costly to run on every candidate. The funnel spends it only on the few that cheaper models already rated highly. Design the serving path for one Explore page, and the offline job that keeps its features and embeddings fresh.
Functional requirements
Load Explore: a user opens Explore and gets a page of 25 ranked posts. In order:
Retrieval: the retrieval service looks up accounts similar to the user's seed accounts (accounts they liked or saved posts from) in a nearest-neighbour index of account embeddings, embeddings, and samples 500 eligible posts from those accounts.
Features: the features of the 500 posts and of the user are read from the online feature store, features, in one batched call.
First pass: light, a distilled model with few features, scores the 500 and keeps the best 150.
Second pass: medium, a lightweight neural network with the dense features, scores the 150 and keeps the best 50.
Final pass: heavy, a deep multi-task network with dense and sparse features, scores the 50 and orders the best 25.
Refresh features: the given offline pipeline reads a batch of logged interactions from the warehouse and writes fresh features to features and fresh account embeddings to embeddings.
Name the nodes retrieval, embeddings, features, light, medium and heavy: the capacity and the tests in problem.proschi refer to them, and to the two use case names above.
Scale
1k Explore loads a second for the slice of traffic in this exercise.
Each load ranks 500 candidates, narrowed to 150, then 50, then 25.
One light machine scores 100k items a second in about 5 ms a batch, one medium machine 25k items a second in about 10 ms, and one heavy machine 5k items a second in about 40 ms; they cost $200, $300 and $600 a month. All three are set in problem.proschi's capacity.
The offline pipeline writes 50 batches a second.
The warehouse answers in about half a second, a few hundred queries a second per node.
The funnel's sizes (500, 150, 50, 25) and the three kinds of model are from Meta's post. The request rate, the model speeds and prices and the warehouse's numbers are assumptions for the exercise: Meta published only fleet-wide totals (65 billion features extracted and 90 million model predictions a second).
Constraints
p99 of loading Explore under 300 ms, all three passes included.
Explore available 99.95% of the time.
A request never queries the warehouse: features and embeddings are computed offline and read from online stores.
Users reach only the Explore API, never the stores or the models.
Losing any single machine must not break a latency limit.
At most $20,000 / month, the warehouse and the pipeline included.
What is given
problem.proschi declares the user, the warehouse of logged interactions (two nodes), the offline pipeline, the speed and price of each ranking model, the traffic, the requirements and the tests. Add the Explore API, retrieval and its embeddings index, the features store, the three ranking passes, their connections and the two use cases.
What this model leaves out
Ranking quality. The whole point of the deep model is that it ranks better, and the funnel trades a little of that for cost: a post the first pass drops is never seen by the deep model. Meta trains the distilled model on the main models' outputs so it drops few posts the deep model would have kept. The simulation measures load, latency and cost, not relevance, so quality appears only as the order of the passes in a test.
Feature freshness. Precomputed features are minutes to hours old. How stale is acceptable, and which features must be real time, is a product question the simulation cannot weigh.
Eligibility, integrity and diversity. Filtering out policy-violating content and spam before ranking, and spreading the page over many authors, are part of each step's label here, not separate work.
One batched call per pass. A label like x500 SCORE counts 500 items of load but one round trip, as if the candidates were sent in one batch, which is how model servers are usually called. A model server's GPU batching and its memory for features are not modelled.
Based on
Ivan Medvedev, Haotian Wu and Taylor Gordon, Powered by AI: Instagram's Explore recommender system, Meta AI blog, November 2019: over half of the Instagram community visits Explore every month; the system extracts 65 billion features and makes 90 million model predictions a second; candidate generation starts from seed accounts the person interacted with, finds similar accounts with a nearest-neighbour lookup on word2vec-like account embeddings (ig2vec), and takes media from those accounts; ineligible and likely policy-violating content is filtered out; each request samples 500 candidates, ranked in three passes: a distilled model with minimal features (500 to 150), a lightweight neural network with the full dense features (150 to 50), and a deep neural network with dense and sparse features (50 to 25, the first page); the distilled model is trained on the main models' recorded outputs; the final score is a weighted sum of predicted actions with a diversity penalty.
Vladislav Vorotilov and Ilnur Shugaepov, Scaling the Instagram Explore recommendations system, Engineering at Meta, August 2023: the funnel as retrieval, first-stage ranking, second-stage ranking and final reranking; item embeddings generated daily offline and served from an approximate nearest-neighbour index, with user and item embeddings cached for efficient inference; a lightweight first-stage model that distils the second stage; a heavier multi-task, multi-label model in the second stage.
How your design is checked
You write the design as text in Proschi. Tests run in your browser: a simulation of the traffic above checks latency, availability, cost and what happens when a machine fails. How the simulation works.
Instagram's Explore tab shows each person a grid of posts from accounts they do not follow yet. Over half of the Instagram community visits it every month, and the posts come from a pool of billions. Showing everyone the same popular grid would be cheap and dull; ranking every post for every person is impossible. Meta's answer, described in its 2019 post, is a funnel.
Candidate generation comes first. It starts from the person's seed accounts, the accounts whose posts they liked or saved, and looks up accounts that are similar to them. Similarity comes from account embeddings: Meta trains vectors (ig2vec, a word2vec-like model) on the sequences of accounts people interact with in a session, so accounts that are visited together end up near each other. A nearest-neighbour lookup finds the closest accounts, and their recent posts, filtered for eligibility, become the candidates. Each request samples 500 of them.
Ranking then takes three passes. A distilled model with minimal features scores all 500 and keeps 150. A lightweight neural network with the full set of dense features keeps 50 of those. A deep neural network with dense and sparse features orders the final 25, which fill the first page. The deep model predicts several actions at once (like, save, "see fewer posts like this"), and a weighted sum of those predictions, with a penalty for showing the same author twice, is the final score.
The problem has two use cases. Load Explore runs the funnel while the user waits. Refresh features is the offline pipeline that reads the warehouse and refills the feature store and the embedding index. The tests check that retrieval runs before the first ranker and that the rankers run from cheapest to heaviest, that features are read from the online store and never from the warehouse while a user waits, and that users reach only the Explore API.
The limits: 1k Explore loads a second, p99 (the latency 99% of requests beat) under 300 ms with all three passes included, 99.95% availability, no latency limit broken by losing any single machine, and at most $20,000 a month, the warehouse and the pipeline included. The statement also gives each model's speed and price: one light machine scores 100k items a second for $200 a month, one medium machine 25k for $300, one heavy machine 5k for $600.
machines to run the deep model on all 500 candidates, near 70% busy
140
machines it needs in the funnel, with the same headroom
15
a month for the reference design
$16,100
The request rate and the model speeds are assumptions for the exercise; the funnel's sizes are Meta's.
Pass
Items per request
Items a second
Per machine
Machines (no headroom)
Machines with headroom
light (distilled)
500
500k
100k
5
8
medium (lightweight network)
150
150k
25k
6
10
heavy (deep network)
50
50k
5k
10
15
heavy on every candidate
500
500k
5k
100
about 140 (at 70% busy)
The deep model on everything. 1k requests × 500 candidates = 500k scores a second. At 5k a machine that is 100 machines before any headroom, about 140 to stay near 70% busy: some $85,000 a month at $600 each, four times the whole budget.
The funnel. The deep model sees only 50 items per request: 50k a second, 10 machines, 15 for headroom (67% busy, 71% after losing one). The two cheaper passes add 8 and 10 machines (63% and 60% busy). All three pools together cost $13,600, a sixth of the deep model alone on everything.
Latency. The page waits for each pass in turn, so their latencies add: about 10 ms for the API, 10 ms for retrieval plus 15 ms for the index, 1 ms for the features, then 5, 10 and 40 ms for the three passes. That is about 90 ms idle, a little more under load, and about 260 ms at p99 in the simulation, under the 300 ms limit. A warehouse query, at half a second, would not fit even once.
A recommender faces a trade-off between how good a model is and how many items it can afford to score. The best model uses hundreds of features and a deep network, and it costs tens of times more per item than a simple one. The candidate pool is large, and most of it is obviously wrong for this user: a cheap model can tell that much.
So the work is split into passes. Each pass scores what the previous one kept and passes on only its best. Early passes are cheap and see many items; late passes are expensive and see few. The expensive model's load is set by the last cut, 50 items a request, not by the size of the pool. That is what keeps both the bill and the latency bounded: the latency of each pass is one batched call, and the slowest pass sees the smallest batch.
The cost of the funnel is recall: a post the first pass drops is never seen by the deep model, even if the deep model would have loved it. Meta reduces that loss by distillation: the first-pass model is trained on the recorded outputs of the main models, so it learns to imitate their ranking with far fewer features. A good funnel loses few of the items the final model would have chosen.
Ranking needs a short list to start from. Retrieval builds it without scoring individual posts. Each account is a vector; accounts that people visit together have vectors that point the same way. Given the user's seed accounts, a k-nearest-neighbour (KNN) search over those vectors returns similar accounts, with cosine distance or a dot product as the measure. Meta runs this lookup with FAISS, its library for similarity search.
The index is built offline: the embeddings come out of a training job over logged sessions, and the index is reloaded when they change. At request time it only answers lookups, which is why it can be fast. The posts of the similar accounts are then filtered for eligibility (no content that is likely to violate policy, no spam) and sampled down to 500.
Deep dive Approximate nearest neighbours
An exact KNN search compares the query with every vector, which is too slow over millions of accounts. Approximate methods (inverted lists of clusters, graph indexes such as HNSW, product quantization to shrink the vectors) find nearly the same neighbours by looking at a small part of the index. They trade a few percent of recall for orders of magnitude of speed, the same trade the ranking funnel makes. Meta's 2023 post describes item embeddings generated daily offline and served from such an approximate index.
A ranking model's inputs are features: how often this user liked posts from this author, how many saves the post got in the last day, the author's embedding. Most of them summarise long histories, so they change slowly and are expensive to compute. Computing them from raw logs while a user waits would take seconds.
So they are computed offline, in batches, from the warehouse where the interaction logs land, and written to an online feature store: a key-value store or cache that answers a batch of keys in about a millisecond. A request reads the features of all 500 candidates and of the user in one batched call, and each ranking pass uses the subset it needs. The warehouse stays an offline system: it answers in about half a second and serves a few hundred queries a second, fine for a pipeline and hopeless for 1k requests a second.
The cost is staleness: a feature is as old as the last batch that wrote it. Features that must be fresh (what the user tapped a minute ago) come from a streaming job instead; the simulation treats them all as one store.
Each model has its own cost per item and sees its own number of items, so each gets its own pool, sized from its own numbers:
machines = requests a second × items per request ÷ items one machine scores a second
then enough extra that the pool stays near 70% busy after losing one machine. Pools that are the same size are wrong for at least two of the three: the distilled model idles while the deep model queues. Separate pools also scale independently: a new, heavier final model changes only its own pool.
The API. Add an Explore API that the user calls; it will run the funnel. Nothing else is reachable from the user.
Retrieval. Add retrieval and its nearest-neighbour index embeddings. The API asks retrieval for candidates; retrieval looks up accounts near the seed accounts and returns 500 eligible posts.
Features. Add the online features store. The API reads the features of the 500 posts and of the user in one batched call before ranking.
The three passes. Add light, medium and heavy, and call them in that order. Use a fan-out label for the items each one scores, such as x500 SCORE, so that each pass carries the load of its batch.
The offline pipeline. Write "Refresh features": the given pipeline reads the warehouse, then writes features and embeddings. The serving path never touches the warehouse.
Size every pool. From the table in Back-of-the-envelope: 8, 10 and 15 machines for the passes, two or three of everything else, and a look at the budget, where the deep model is more than half of the bill.
Explain the trade-offs in decision blocks: why a funnel, why precomputed features, why one pool per model.
The deep model on every candidate. Retrieval hands all 500 posts straight to the best model. Its pool needs 100 machines before any headroom (the wrong design runs 120, at 83% busy, for $72,000 a month), several times the budget, and no cheaper pass comes first. It fails Retrieval first, then ranking from the cheapest model to the heaviest and the cost limit.
A funnel that never narrows. Three passes in the right order, but each one re-scores all 500. The early passes save the later ones nothing, and the deep pool is as large as above. Only the cost limit catches it: the order is right, the numbers are not. Each pass must keep only its best.
Features from the warehouse. The API queries the warehouse for the features of 500 posts on every request. At about half a second a query and a few hundred queries a second per node, the warehouse saturates and the p99 runs to seconds. It fails Features are precomputed and read from an online store and the p99 limit.
The same pool size for every model. The funnel is right, but every pass gets ten machines. The deep model's 50k scores a second fill its ten machines completely, requests queue there, and losing one machine makes it worse. It fails the p99 limit and survive any node failure. Size each pool from its own load.
Start with the numbers that force the funnel. Candidates per request, requests a second and the cost of one prediction. Once 500k deep-model scores a second is on the board, the funnel follows.
Name the stages and their sizes. Retrieval to hundreds, a cheap pass to about a hundred, a heavy pass to a few dozen, then re-ranking for diversity and integrity. Say what each stage uses: embeddings, few features, all features.
Expect "doesn't the first pass drop good items?" Yes, that is the trade-off. Distillation trains the cheap model to imitate the expensive one, and offline replay measures how many of the final model's picks survive the earlier cuts.
Expect "how fresh are the features?" Batch features are hours old; real-time signals (the last few taps) come from a streaming job into the same online store. Embeddings are rebuilt daily.
Expect "what if the ranking path is down?" Serve a precomputed page: Meta's 2023 post mentions precomputing some users' recommendations off peak. Popular posts are a last resort.
Expect "how do you know it works?" Online A/B tests on engagement, and offline ranking metrics such as NDCG. Meta trains the distilled model on exactly that: an NDCG loss against the main models' output.