[cabfpub] What's the load on a CT log?

[Ben Laurie]

Several people have asked me this recently. Here's a nice way to estimate load.

Let's assume a single log that takes all the load.

Firstly, we see about 5,000 new certificates a day, so that's around
0.06 new certificates per second. Clearly a trivial load.

Next is load from audit (i.e. from browsers that wish to validate SCTs
accompanying certificates they see). Given some assumptions, we can
calculate the load from audit.

* Clients cache audit results.

* There are approximately b = 2.5B browsers in the world
(http://www.internetworldstats.com/stats.htm).

* The average user visits w = 89 websites a month
(http://www.creditloan.com/blog/how-the-world-spends-its-time-online/
quoting a Nielsen report). Assume these are all TLS sites.

* Assume a certificate lifetime of l = 12 months.

So, each user sees w / l new certificates a month. Each new
certificate needs to be audited, which means in practice, three web
operations (fetch STH, fetch STH consistency proof, fetch SCT
inclusion proof) - it might be a good idea to create a new API to do
all three in one go.

So, total average load is 3 * b * w / l ~ 20,000 web fetches per
second. If we optimise the API we can get that down to 7,000 qps. Each
query (in the optimised case) would be around 3 kB, which gives a
bandwidth of around 150 kb/s.

Monitors add extra load, but should only be at around the new
certificate rate - i.e. ~ .06 * number of monitors fetches per second.

IMO, this is achievable on a single machine (modulo reliability), with
some care. Clearly not a vast farm, however its done.

In practice, no one log would have to take this full load, this is a
worst case analysis.