Tail Latency

# Created: 2021-06-27; Modified: 2021-06-27

Observing Tail

Check out Jil Tene’s talk on measuring latency here (2016).

For serious quantification, there are some common QoS metrics for tail latency:

• p95, p99, …
• p99/avg
• standard deviation
• norm

Tails in Service Time

Tails in Queueing

Tails in System

Here “System” refers to a composite service built on top of several black box service units. Like Lego system, there are limited ways to combine several units, even though they can sum up quite differently.

Pipeline

Each task is processed by multiple units in sequence.

Parallel

Each task is divided into multiple sub-tasks which gets processed in parallel.