Code that works at n = 10 can fall over at n = 10,000,000.
Live pictures of what happens in between.
When the data grows
Five common shapes of code, costed at a fixed time per step as the input grows from 10 items to 10 million. Each line is how long the work takes: the flat ones stay instant forever, the steepest stops finishing.
Assumes one operation = 10 ns
op 10 ns n up to 10M
- O(1)
- O(log n)
- O(n)
- O(n log n)
- O(n²)
| Class | What it looks like in code | Operations | Wall time | Status |
|---|---|---|---|---|
| O(1) | a hash lookup, an array index | 1 | 10 ns | instant |
| O(log n) | binary search, a B-tree index probe | 3 | 33.2 ns | instant |
| O(n) | one pass, a full table scan | 10 | 100 ns | instant |
| O(n log n) | merge sort, an ORDER BY with no index to read from | 33 | 332 ns | instant |
| O(n²) | a loop inside a loop, includes() in a map | 100 | 1 µs | instant |
Sorting a list, five ways
The same 48 numbers sorted by five methods, running for real side by side. Every lane gets the same number of steps per second, so the method that needs fewer steps finishes first, and the slow ones are still grinding long after the fast ones are done.
Real steps, counted as they happen: comparisons, or digit moves for radix sort, which never compares two numbers
sorting 48 numbers table 1,000,000 step 10 ns
| Method | Steps | Time |
|---|---|---|
| Bubble sort | 5 × 10¹¹ comparisons | 1.39 h |
| Insertion sort | 2.5 × 10¹¹ comparisons | 41.7 min |
| Quick sort | 2.77 × 10⁷ comparisons | 277 ms |
| Merge sort | 1.99 × 10⁷ comparisons | 199 ms |
| Radix sort | 1.4 × 10⁷ digit moves | 140 ms |
Loading a page from the database
One page with 3 lists on it, where every row needs a related record (say, each order's customer). Lazy loading fetches that record separately for every row: the N+1 query an ORM makes easy to write. The other three fetch the same data in a handful of trips, whatever the row count.
Assumes each database query costs a 0.8 ms round trip plus 0.15 ms to run, one after another
round trip 0.8 ms query 0.15 ms lists 3
- faint tick: one trip to fetch a list
- solid tick: one trip to fetch related rows
| Approach | Round trips so far | Page load time |
|---|---|---|
| Lazy loading | 97.9 ms | |
| Batched IN | 5.7 ms | |
| Dataloader | 3.8 ms | |
| One join per list | 2.85 ms |
Batched IN and the dataloader both make one extra trip for related rows; the difference is when. Batched IN makes it after each list (3 extra trips); the dataloader waits until every list has loaded and makes it once for the whole page. Lazy loading waits 34× longer than the joins for the same page.
When the server gets busy
How long a request takes as traffic rises, for three database connection pool sizes. Each line sits near 20 ms while there is headroom, then goes vertical, but how much headroom depends on the size of the pool: the pool of 2 is already at 56 ms when 80% busy, while the pool of 32 is still at 29 ms at 95%. The dashed line is a 250 ms target.
Assumes work averaging 20 ms per request (varying randomly), randomly timed arrivals and a queue that never turns anyone away
work 20 ms pools 2 8 32 SLA 250 ms
- one line per connection pool; the number on each wall is its size
| Connection pool | Max throughput | Busy | Response time | Status |
|---|---|---|---|---|
| 2 connections | 100/s | 10% | 20.2 ms | within SLA |
| 8 connections | 400/s | 3% | 20 ms | within SLA |
| 32 connections | 1,600/s | 1% | 20 ms | within SLA |
Everything holdsup.dev shows
The map of the site. Each card takes you straight to its live panel on the section's page. The tag on each card is the shape its curve makes, drawn beside it in the same colours the charts use.
Algorithms & data structures
The classroom material, shown at sizes the classroom never uses.
- every shape
When the data grows
Five shapes of code costed from 10 items to 10 million, and the one that stops finishing.
- n log n
Sorting algorithms
Bubble, insertion, quick, merge and radix on the same array, every step counted as it happens. Radix never compares two numbers at all.
- log
Searching
Linear scan vs binary search vs hash lookup. At a million rows one is 1,000,000 steps, one is 20, one is 1.
- quadratic
The wrong container
array.includes() inside a loop is quadratic and looks completely innocent. A Set makes it linear.
- quadratic
String building in a loop
In C# or Java, s += piece copies the whole string every iteration. A builder does not: about 16,000x apart in bytes copied at 100k pieces.
- exponential
Recursion without memoisation
Naive Fibonacci against a memoised one: the call tree drawn as it explodes, then collapsing to a line.
- cliff
Space, not time
Ten million rows in a list vs streaming them. A RAM bar filling toward the ceiling, then the OOM kill.
- linear
Big-O lies about constants
A linked list and an array are both O(n). Once the list is scattered across memory, walking it can be 100x slower. Cache locality drawn as memory jumps.
- quadratic
Nested loops over one set
Deduplicating a list by comparing every item with every later one. The triangle of comparisons drawn as an actual triangle, next to the one-pass Set that finishes first.
Databases
Where a lot of real production incidents live.
- linear
N+1 queries
Lazy loading turns one page into N+1 round trips. The join, the batched IN and the dataloader side by side.
- linear
The missing index
Sequential scan vs index scan, with rows examined as the visual. Add the index and 4,000,000 rows read becomes 8: three index pages, then straight to the matches.
- linear
OFFSET pagination
Page 1 is instant; page 10,000 reads 200,000 rows to throw away 199,980. Keyset pagination is flat forever.
- linear
COUNT(*) for a page counter
The "showing 1-20 of 4,265,795" badge costs more than the query it decorates.
- linear
SELECT * and payload width
Bytes over the wire against columns actually rendered, including the TOAST blob nobody remembered.
- quadratic
Cartesian explosion
One missing join condition turns 10k x 10k into 100,000,000 rows.
- cliff
Lock contention on a hot row
A thousand writers, one counter row. Throughput hits a ceiling, then every extra writer only adds wait.
- peak
Index write amplification
Each index speeds up one kind of read and taxes every write. The total has a lowest point, and it is not where the two lines cross.
- cliff
Connection pool exhaustion
500 app threads, 100 database connections. Where the queue forms and why the error is a timeout.
APIs & the network
Latency is a tax on conversations, not on bytes.
- cliff
The latency cliff
Flat at 20 ms while there is headroom, then vertical. How much headroom you get depends on the size of the box: a small one has nearly tripled to 56 ms at 80% busy, a big one is still under 30 ms at 95%.
- linear
Chatty vs batched
Twelve calls one after another vs six at a time vs one batched call, with the distance to the server as the slider (80 ms per round trip is about 5,300 km).
- cliff
p50 is a lie
A histogram where the average sits inside the SLA, the p99 is three seconds and the p99.9 is eight. Which users those are.
- cliff
Tail amplification
A page calling twelve services waits for the slowest, so each service's 1-in-100 slow reply lands on 11% of page loads. The maths that surprises people most.
- exponential
Retry storms
A blip, then every client retrying, then an outage that outlasts it. Then the same run with jitter and a breaker: calls fail fast for a moment and it recovers within seconds.
- exponential
Payload nesting
?expand=orders.items.product with no depth limit. Response size as a tree growing off the screen.
- linear
Serialisation cost
JSON stringify and parse from 1 KB to 100 MB, timed live in your browser up to 1 MB and extrapolated above, and the size at which gzip's CPU buys back more wire time than it costs.
- step
Cold starts
A step function, not a curve. p99 as a function of how traffic bunches up, not just how much there is.
Caching, queues & concurrency
Where good intuitions go wrong the most reliably.
- cliff
The 95% hit rate
95% hits means the 5% owns your p99 and the database still sees 50 rps at peak.
- cliff
Cache stampede
One popular key expires, every request that misses hits the database, and past about 730 requests a second it never recovers. With and without a lock.
- linear
Little's Law
Arrivals 5% over capacity add to the line every minute the surge lasts, and only spare capacity drains it. L = λW, checked live.
- peak
Universal scalability
Add workers, throughput rises, flattens, then falls. Coherency cost eating the gains: the most useful curve on the site.
- peak
Batch size tuning
Latency per event is a U and throughput a peak: batches too small drown in per-write overhead, too big wait to fill and write slowly. The best size is not "as big as possible".
- linear
Backpressure vs dropping
What a queue does when it cannot keep up: grow, block or shed. Three simulations, one slider.
- peak
Thread pool sizing
Too few and the cores sit idle, too many and they spend a fifth of their time switching while each request takes 30x longer. The curve has a top.
Architecture, cost & failure
The same curves with dollars and uptime on the y-axis.
- cliff
Vertical vs horizontal
Two cost curves crossing. The bigger box is cheaper right until it is the most expensive decision you ever made.
- step
Replicas, then shards
What each step buys, what it breaks, and the replication lag you inherit in exchange.
- cliff
Hot partitions
Sharding assumes even keys. One celebrity user and one shard carries a third of all traffic, however many shards you add.
- linear
Restore time
The backup is fine. Restoring 4 TB stage by stage takes nine hours and your RTO says one.
- linear
Cost per million requests
Two versions of the same endpoint, priced per request: an N+1 with SELECT * costs about 51 times as much every month, and nearly all of it is bytes sent.
- exponential
Availability multiplies
Twelve dependencies at 99.9% each is 98.8%: four days of downtime a year.