Why Observability Keeps Ending Up on ClickHouse

Look under the hood of almost any observability product built in the last five years and you'll find the same database. Sentry's search and analytics layer, Snuba, is ClickHouse. PostHog stores every event in ClickHouse. So do SigNoz, Uptrace, highlight.io, Coroot, groundcover, Dash0, and Langfuse, which rebuilt its whole ingestion path on ClickHouse after Postgres gave out. ClickHouse the company eventually noticed the pattern too, acquired HyperDX, and now ships ClickStack as an official observability stack.

Then there are the in-house platforms. Netflix, OpenAI, Anthropic, Uber, eBay, Cloudflare, Trip.com and dozens more. Different companies, different decades of infrastructure, same storage engine for telemetry.

When that many teams independently land on one database, there's usually a mechanical reason rather than a fashionable one. This post walks through that mechanism: how ClickHouse works, what it does to the bill, and when it's still the wrong choice. If you're evaluating it as a backend, this is for you.

Telemetry has a very specific shape

A log record or a span is a wide event: a timestamp, a service name, a severity, a body, and then a pile of attributes; Kubernetes metadata, HTTP fields, tenant IDs, feature flags etc. Fifty to a few hundred fields is normal once semantic conventions and resource attributes are added.

The write pattern is an append-only firehose. Telemetry arrives in time order, at enormous volume, and never gets updated.

The read pattern is the opposite of the write pattern. Queries are almost always aggregations over a time range: error rate by service over six hours, p95 latency by endpoint, log volume by namespace. You get the point. Uber measured this on their own logging platform and found that over 80% of queries were aggregations. A query like that touches three or four fields out of hundreds, across millions of rows.

Row-oriented databases and search engines store each event as a unit. To compute an aggregate over three fields, they read past every other field of every event. And that structural mismatch is the fundamental problem that ClickHouse, or any other columnar store, solves.

ClickHouse stores each column in its own file. A query that touches 3 columns out of 50 reads roughly 6% of the bytes a row store would read for the same question. The other 47 columns are never touched.

The second effect is less obvious and matters more when considering cost. When a column lives in its own file, every value in that file has the same type and similar content. A million consecutive ServiceName values might contain thirty distinct strings. A million consecutive timestamps are nearly sequential integers. Data like that compresses extremely well, and ClickHouse lets you pick a compression codec per column: ZSTD as the general default, Delta for timestamps, Gorilla for gauge-like floats, LowCardinality dictionary encoding for strings with few distinct values.

So at this point you're probably thinking "ok what do the numbers say?". Well, let me tell you:

• The OTel collector's ClickHouse exporter README reports 7 to 11x compression on its default schema.
• ClickHouse's internal logging platform, LogHouse, runs at 431 PiB of raw telemetry stored as 27 PiB, about 16x, and that ratio has held for two years now across a 1000x growth in volume!
• An example on nginx logs got 52x with a query-friendly schema and 178x when tuned purely for compression.

So on average, telemetry is roughly 7 to 17x. Keep that number in your head for the cost section which we will get to in a minute.

The economics: a different billing mechanism

Ok so you're sold on the tech? But we all know that whether we like or not, most migrations don't happen because of the tech, they happen because of cost. In the end, it all comes down to money. So wait till you hear this:

Incumbent observability pricing bills the volume you send. Datadog lists $0.10 per GB ingested, then $2.50 per million events indexed at 30-day retention on annual terms, up to $3.75 on-demand. New Relic bills $0.40 to $0.60 per GB ingested with 8-day default retention. Retention multiplies the bill, so teams cut retention to 3 or 7 days and sample what they keep. The pricing model shapes the engineering.

ClickHouse-style economics bill the bytes you store, after compression. ClickHouse Cloud storage runs about $25 per compressed TB-month, with compute metered separately, and the floor under all of it is object storage at about $0.023 per GB-month. At 16x compression, a petabyte of raw logs is roughly 62 TB on disk, around $1,600 a month of storage. And just like that, keeping six months of logs doesn't seem as unreasonable anymore.

Who's actually running it

The case studies are unusually well documented, mostly as first-party engineering posts:

Two things stand out in the case studies in the table above. First, the Elasticsearch migrations all report the same shape of result: half the hardware and single-digit-multiple query speedups. Second, the AI labs are the newest cohort. Training clusters emit telemetry at volumes where per-GB ingest pricing produces numbers so large that you'll fall out of your chair, which is why Anthropic, OpenAI, and Character.AI all show up here.

When ClickHouse is the wrong answer

Because if you're going to hear the case for ClickHouse, you should hear the case against it too.

• Point lookups. There's no row index, so fetching one record means scanning a whole chunk of the table. Trace-ID lookups work, but they need schema care. And those kinds of queries don't belong here.
• Search. Full-text search went GA in 26.2, but there's no relevance ranking, no BM25, no fuzzy matching. If your team does a lot of ranked log search, then stay on Elasticsearch or whatever you are using.
• Small scale. Most experts will tell you to not bother below roughly 150 GiB of data, and even pro-ClickHouse writers concede that at 1 TB/day it's no simpler than its peers. Below multi-TB/day, a hosted product that just works is a fine answer.
• Self-hosting is real work. Managing disk headroom, keeping memory-hungry JOINs in check, and coordinating a cluster for high availability is no small task. So if you're a small team and this seems like a lot, go with the hosted option to reap the benefits of ClickHouse without the leg work.
• The database is not the product. Dashboards, alerting, and UI come from somewhere else: HyperDX, Grafana, or something you build. Pydantic's Logfire looked at the same facts and built on DataFusion instead. This is the part most people forget. Remember, ClickHouse is just a datastore. And you have to build the actual observability layer on top of it. Even ClickHouse's launch post admits raw-ClickHouse observability mostly worked for teams that built platforms on top.

The part of the stack that doesn't change

ClickHouse doesn't actually speak OpenTelemetry. Not the open-source version, not Cloud. It accepts database inserts, and nothing else. So between your apps and ClickHouse there is always an OpenTelemetry collector doing the translation, and that collector is where all the grunt work happens: batching data into the big inserts ClickHouse wants, mapping it onto the right tables, buffering and retrying when the database hiccups, and holding the credentials so your apps never see them.

The two paths of using ClickHouse package that collector differently though. If you want OSS ClickStack, one comes in the box, preconfigured for exactly this job. But, if you want to run Managed ClickStack on ClickHouse Cloud then you have to run your own collector and point it at Cloud. Either way you end up operating collectors, and everything you build into them; the pipelines, the sampling, the enrichment etc., carries over untouched through every backend decision you make after this one.

That collector layer is what we build at Telflo, so consider the source. But it's also just how the plumbing works, and it deserves more than a paragraph: the exporter settings, the batching numbers, the gateway topology. We're putting together a full post on wiring collectors into ClickHouse and ClickStack. This one was the why. That one will be the how.