Latency Numbers, L1 Cache To Retry

From Alex Xu at ByteByteGo.

Alex Xu at ByteByteGo recently published “Which latency numbers should you know?”.

Let’s check it out.

(These are based on some online benchmarks e.g. Jeff Dean’s latency numbers + some other sources).

• L1 and L2 CPU caches: 1 ns, 10 ns E.g.: They are usually built onto the microprocessor chip. Unless you work with hardware directly, you probably don’t need to worry about them.

• RAM access: 100 ns E.g.: It takes around 100 ns to read data from memory. Redis is an in-memory data store, so it takes about 100 ns to read data from Redis.

• Send 1K bytes over 1 Gbps network: 10 us E.g.: It takes around 10 us to send 1KB of data from Memcached through the network.

• Read from SSD: 100 us E.g.: RocksDB is a disk-based K/V store, so the read latency is around 100 us on SSD.

• Database insert operation: 1 ms. E.g.: Postgresql commit might take 1ms. The database needs to store the data, create the index, and flush logs. All these actions take time.

• Send packet CA->Netherlands->CA: 100 ms E.g.: If we have a long-distance Zoom call, the latency might be around 100 ms.

• Retry/refresh internal: 1-10s E.g: In a monitoring system, the refresh interval is usually set to 5~10 seconds (default value on Grafana).

Notes

• 1 ns = 10^-9 seconds
• 1 us = 10^-6 seconds = 1,000 ns
• 1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns

An updated conversation about Latency times can be found here: https://gist.github.com/jboner/2841832

And a nice comprehension using solar system references is here:

Operation	Time in Nano Seconds	Astronomical Unit of Weight
L1 cache reference	0.5 ns	1/2 Earth or Five times Mars
Branch mispredict	5 ns	5 Earths
L2 cache reference	7 ns	7 Earths
Mutex lock/unlock	25 ns	Roughly [Uranus +Neptune]
Main memory reference	100 ns	Roughly Saturn + 5 Earths
Compress 1K bytes with Zippy	3,000 ns	10 Jupiters
Send 1K bytes over 1 Gbps network	10,000 ns	20 Times All the Planets of the Solar System
Read 4K randomly from SSD*	150,000 ns	1.6 times Red Dwarf Wolf 359
Read 1 MB sequentially from memory	250,000 ns	Quarter of the Sun
Round trip within same datacenter	500,000 ns	Half of the Mass of Sun
Read 1 MB sequentially from SSD*	1,000,000 ns	Sun
Disk seek	10,000,000 ns	10 Suns
Read 1 MB sequentially from disk	20,000,000 ns	Red Giant R136a2
Send packet CA->Netherlands->CA	150,000,000 ns	An Intermediate Sized Black Hole

---

Thanks. Unsurprisingly, this article was not written by me but Alex Xu at ByteByteGo .