Every file in Dropbox gets a thumbnail in the file list, and most documents can be opened as a preview in the browser without downloading them. Dropbox's content processing platform, Riviera, makes those derived outputs: a central coordinator takes each request, checks for a stored result, and dispatches the work to pools of workers, one pool per kind of transformation. A PowerPoint deck, for example, is converted to PDF and each page rendered as an image.
The expensive part is the transformation; the cheap part is serving its result again. So Riviera pre-warms: it generates the outputs of new files in the background, before anyone asks, and stores them. When a preview was not pre-warmed, it has to be generated while the user waits.
Design the upload, the background generation and the two read paths.
Functional requirements
Upload file: a user uploads a file and gets 201 once the original is stored durably in originals. Its outputs are generated later.
Pre-warm: a job from the queue asks the coordinator to generate a new file's outputs. The coordinator reads the original, gets a thumbnail from the thumbnail workers, and stores the outputs in the derived-output store. Two scenarios:
"Thumbnail and preview": the file is likely to be previewed, so the preview workers also convert it and render its pages.
"Thumbnail only": the file is unlikely to be previewed, so no preview is made in advance.
View thumbnail: a file list shows a thumbnail. Two scenarios: "Edge hit" (the CDN has it) and "Edge miss" (the CDN fetches it from the derived-output store).
Open preview: a user opens a file's preview. Three scenarios:
"Edge hit": the CDN has the page.
"Pre-warmed": the coordinator finds the page in the derived-output store.
"Generated on view": the page is not stored yet; the coordinator reads the original, has the preview workers render it, and stores the pages before answering, so the next viewer finds them.
Name the thumbnail workers thumbnailer, the preview workers previewer and the derived-output store derived: the capacity and the tests in problem.proschi refer to them, and to the use case and scenario names above.
Scale
1k uploads a second, each followed by one pre-warm job; about 40% of pre-warms make the thumbnail only.
50k thumbnail views a second, 95% of them answered at the edge.
2k preview opens a second: 80% at the edge, 18% from the stored pre-warmed pages, 2% generated on view.
A thumbnail worker renders 100 thumbnails a second in about 50 ms each; a preview worker renders 40 previews a second in about 500 ms each. Both are set in problem.proschi's capacity.
The rates and worker speeds are assumptions for the exercise: Dropbox has published only the platform's total, hundreds of thousands of transformations a second.
Constraints
An upload never waits for a transformation: p99 under 200 ms, available 99.9% of the time, and durable before it answers.
A thumbnail view never runs a worker: p99 under 100 ms, available 99.99% of the time.
p99 of opening a preview under 1.5 s, the rare on-view generation included.
A new file's outputs are generated with a p99 under 3 s.
Users reach only the API and the CDN: never the stores or the workers.
Losing any single machine must not break a latency limit.
At most $12,500 / month, both worker pools included.
What is given
problem.proschi declares the user and originals, the object storage that already holds the uploaded files (four replicas), the speed and price of a thumbnail worker and of a preview worker, the traffic, the requirements and the tests. Add the upload API, the queue, the coordinator, the two worker pools, the derived-output store, the CDN, the connections and the four use cases.
What this model leaves out
Whether a file is likely to be previewed is a scenario share here. Dropbox decides it per file with a model (Cannes, in the Based on posts); the simulation does not predict anything, so the 60/40 split is fixed.
Transformation chains (PowerPoint to PDF to page images, with the PDF step shared by other outputs) are one step to previewer. Reusing an intermediate output is not modelled.
The coordinator's check for a stored result is drawn as a read of derived; two requests for the same new file at the same moment would both generate it, and request coalescing is not simulated.
The size of each file and output, its transfer time and egress are left out: the problem is about where the work runs, not the bytes.
Based on
Andrew Cheung and Binoy Dash, How our universal content processing platform Riviera evolved for AI and beyond, Dropbox Tech Blog, July 2026: Riviera began as the preview service and became a shared platform; a central coordinator validates requests, caches responses and dispatches work to backend workers that each handle one kind of transformation; transformations chain (PowerPoint to PDF to page images); a 160×160 thumbnail is generated once and reused by previews and machine learning; more than 300 file formats, more than 100 capabilities and hundreds of thousands of transformations a second.
Win Suen, Cannes: How ML saves us $1.7M a year on document previews, Dropbox Tech Blog, January 2021: Riviera pre-generates and caches preview assets (pre-warming); on a cache miss it generates the preview on the fly while the user waits; pre-warming's CPU and storage costs are considerable; a model that skipped files unlikely to be previewed rejected about 40% of pre-warm requests in an offline test, replacing an estimated $1.7M a year of pre-warm cost with $9K of ML infrastructure.
How your design is checked
You write the design as text in Proschi. Tests run in your browser: a simulation of the traffic above checks latency, availability, cost and what happens when a machine fails. How the simulation works.
File Previews and Thumbnails: generate once, in the background, and serve many times #
Dropbox shows a thumbnail for every file and a browser preview for most documents. The system that makes them, Riviera, grew from a preview service into a platform that runs hundreds of thousands of transformations a second. This lesson builds its core idea: keep the expensive work off both the upload path and the view path.
Who uses it. Every Dropbox user: the file list needs a thumbnail per file, and opening a document in the browser needs its pages rendered as images. Behind them, other teams use the same outputs; Dropbox's post mentions that one 160×160 thumbnail serves both previews and machine-learning pipelines, so it is made once for both.
What makes it hard. Transformations are slow and varied. A thumbnail is quick, but a PowerPoint preview means converting the deck to PDF and rendering each page as an image. If any of that runs while a user waits, the user feels it; if it runs every time someone looks, the worker fleet grows with views instead of with uploads.
Functional requirements.
Upload file: store the original, answer 201.
Pre-warm: from a queue, generate a new file's thumbnail, and its preview if the file is likely to be opened, and store them. Scenarios "Thumbnail and preview" and "Thumbnail only".
View thumbnail: "Edge hit" or "Edge miss" at the CDN.
Open preview: "Edge hit", "Pre-warmed" (found in the store) or "Generated on view" (rendered while the user waits, then stored).
Non-functional requirements. Upload p99 under 200 ms and durable; thumbnail views p99 under 100 ms and 99.99% available; preview opens under 1.5 s; new outputs ready within 3 s at p99; any machine can fail; at most $12,500 a month.
What the tests check:
Uploads never wait for processing: the upload PUTs the file to originals before answering, sends to a queue, and never calls or waits for a worker pool. Pre-warm starts at the queue.
Each transformation has its own worker pool: pre-warm GETs the original before rendering, calls previewer only in "Thumbnail and preview", writes derived before it finishes, and both pools have at least two workers.
Views are served from stored outputs: thumbnail views go to the CDN before derived and never reach a worker or the originals; a pre-warmed preview never reaches a worker; a preview generated on view checks derived first and stores its pages before answering.
All of these come from the statement's scale, which is an assumption for the exercise: Dropbox has published the platform's total throughput, not per-product rates.
Quantity
Arithmetic
Result
Thumbnails to render
1 per upload
1,000/s
Previews pre-warmed
60% × 1,000
600/s
Previews generated on view
2% × 2,000
40/s
Thumbnail workers busy
1,000 ÷ 100 per worker
10
Preview workers busy
640 ÷ 40 per worker
16
Thumbnail views reaching derived
5% × 50,000
2,500/s
Preview opens past the edge
20% × 2,000
400/s
Two comparisons make the design obvious. If thumbnails were rendered on every edge miss instead of stored, the thumbnail pool would face 2,500 renders a second from views on top of 1,000 from uploads: three and a half times the work, growing with popularity rather than with new files. And if the upload waited for its preview, its latency would include half a second of rendering plus any queueing in the preview pool.
The upload's job is narrow: make the original durable and tell someone there is work to do. It writes the file to object storage, sends a message to a queue, and answers. A message to a queue is fast and does not depend on the workers being healthy, so a backlog of previews never slows an upload.
Pre-warming is Dropbox's word for generating outputs ahead of demand: a job from that queue asks the coordinator to make the new file's thumbnail and preview and store them, so that the first viewer finds them ready. It trades CPU and storage now for latency later. Dropbox's 2021 post on the subject says those costs were considerable at their volume, which is why they later trained a model to skip pre-warming previews for files unlikely to be opened; in an offline test it rejected about 40% of pre-warm requests. That is the "Thumbnail only" scenario here.
What it costs: an output is not there the instant the upload finishes, so the file list shows a placeholder for a moment, and some pre-warmed previews are never viewed. What it buys: uploads at queue speed, and views that almost never wait for a worker.
A thumbnail of a given version of a file never changes. That makes it the ideal cached object: compute it once, store it under a key that includes the file's version, and serve it to every viewer after that. A new version of the file gets a new key, so nothing ever has to be invalidated.
The store of derived outputs is object storage: durable, cheap per gigabyte and scaling out with requests. Losing an output is not a disaster (it can be regenerated from the original), but regenerating it costs a worker's time, so keeping it is the cheaper choice.
The one exception is a preview nobody made in advance. Then the coordinator checks the store, finds nothing, reads the original, has a preview worker render it and stores the result before answering, so the second viewer gets the stored copy. Dropbox's post describes exactly this: on a cache miss, the preview is generated on the fly while the user waits. Keeping that path rare is the point of pre-warming.
A worker pool is sized like any server pool: the rate it must handle divided by what one worker handles, then headroom. Here a preview worker turns out 40 previews a second, so 640 a second keeps 16 of them fully busy. At 16 they would run at 100%, with no room for a burst or a lost machine. Aim for about 70%: 16 ÷ 0.7 is 23, plus one to lose. 24 workers run at 67%, and with one lost, 23 run at 70%.
Thumbnails are the same arithmetic with different numbers: 1,000 a second at 100 per worker is 10 busy workers, about 15 with headroom. The two pools come out different sizes because their jobs differ by an order of magnitude in cost, which is the case for keeping them separate.
Deep dive Why one pool per transformation
Riviera gives each kind of transformation its own backend workers. Each pool is then sized, scaled and deployed for its own load, and a flood of expensive document conversions cannot starve cheap thumbnails. A shared pool has to be sized for the sum, and a request's latency depends on what happens to be ahead of it. The coordinator is what keeps this manageable: it composes a request into steps (a PowerPoint preview is a PDF conversion, then a page render) and sends each step to the pool that owns it, so a new format is a new worker type, not a change to every pool.
Thumbnails are small, immutable and requested 50,000 times a second. That is a CDN's job. With 95% of views answered at the edge, only 2,500 a second reach the derived store, which four object storage replicas take easily. Without a CDN, all 50,000 would go through your own servers to the store: dozens of API replicas and a much larger store, for bytes that never change.
Previews go through the CDN too. A preview miss at the edge goes to the coordinator rather than straight to the store, because only the coordinator can generate a missing page; Dropbox's post notes that the coordinator caches responses so workers are protected from repeated work.
1. Scope. Ask which outputs exist (thumbnails, previews), how often each is viewed compared with how often files are uploaded, and whether a user may see a placeholder for a moment after upload. That last answer is what permits doing the work asynchronously.
2. The write path. User → upload API → originals (synchronous, durable), then an asynchronous send to the queue, then 201. Nothing about thumbnails or previews here.
3. The processing path. "Pre-warm" starts at the queue: the coordinator reads the original, calls the thumbnail pool, calls the preview pool when the file is likely to be opened, writes the outputs to derived, and acknowledges the job. Two scenarios, one per decision.
4. The read paths. Both start at the CDN. A thumbnail miss reads derived. A preview miss goes to the coordinator, which reads derived and, only when the page is not there, renders and stores it before answering.
5. Sizing and checking. Size the two pools from the arithmetic above, give every component at least two replicas, and check the bill: the preview pool is the largest line. Then read the tests' verdict and the analysis: the upload's p99 should not move when you change the preview pool's size.
Render the thumbnail inside the upload (wrong/upload-renders-thumbnail). Tempting, because the file list then never shows a placeholder. But every upload now waits for a worker, the thumbnail pool takes both the upload's renders and the pre-warm's, and a slow pool slows uploads. It fails Uploads never wait for processing and the upload's p99.
Render again on every edge miss (wrong/transform-on-every-view). The thumbnail is made but never kept, so each CDN miss reads the original and calls the thumbnail pool. The pool's load grows with views, 2,500 extra renders a second, and saturates. It fails Views are served from stored outputs and the thumbnail view's p99.
Size the preview pool like the thumbnail pool (wrong/preview-pool-sized-like-thumbnails). Fifteen workers is right for thumbnails, but a preview worker does 40 a second, not 100, so 15 of them can do 600 against 640 needed. The pool saturates before any worker is even lost, and everything that waits for a preview queues behind it. It fails the p99 of Pre-warm and of Open preview, and survive any node failure.
Serve thumbnails through the API (wrong/thumbnails-through-the-api). Every thumbnail view goes through your own servers to the store. Those saturate at 50,000 requests a second, and sizing them for it costs more than the budget, for bytes a CDN would serve at the edge. It fails Views are served from stored outputs and the thumbnail view's p99.
Other classic mistakes.
Letting clients fetch from the derived bucket directly. It skips the CDN and exposes storage; it fails Users reach only the API and the CDN.
Generating on view without storing. The preview path's "Generated on view" must write derived before answering, or the next viewer pays again.
Pre-warming everything. It works, but it spends CPU and storage on previews nobody opens. Dropbox found it worth a model to decide.
Open with the asymmetry: "a transformation is expensive and its output is immutable, so do it once, off the request path, and cache the result everywhere." Then draw three paths: upload, processing, view.
Likely questions, with short answers:
What if a job is lost or a worker crashes? The queue redelivers it; a render is idempotent because the output key is the file version, so doing it twice overwrites the same object.
Two viewers open a preview that was not pre-warmed at the same moment? Both would render it. Coalesce concurrent requests for the same key at the coordinator, or take a short lock per key.
Which files should be pre-warmed? The ones likely to be viewed. Dropbox used a model based on file type and recent account activity; a simple rule (file type, size) is a fair first answer.
How do you add a new format? A new worker type plugged into the coordinator, reusing existing steps such as PDF to image.
What do you monitor? Queue age per transformation, worker utilisation per pool, the share of previews generated on view, and CDN hit rate.
Designing Data-Intensive Applications (Martin Kleppmann), Part III "Derived Data" and its chapter "Batch Processing": derived data as something recomputed from a source of truth.