How to filter aggregates in Kibana

I’m still quite enamored with Kibana. If you’re trying to do what Kibana was intended to do, and I often am, then it is like magic.

Unfortunately the trade off is that if you’re trying to do something that Kibana doesn’t do readily, it used to often be impossible to do it at all. This is a common occurrence in a polyglot tool set. When an entire enterprise application had one database and one programming language, both of those had to be flexible enough to do everything. Now that we expect even a modest application to have a variety of tools, we can use specialist tools that are really good inside a limited scope.

One of the things that Kibana does not do is allow aggregates to be filtered. My use case was to aggregate all transactions and show accounts with non-zero balances.

Luckily, Kibana is adding amazing new features regularly, and Vega is now part of the default install. Vega is a general purpose visualisation tool. It does everything, which lets it fill in all of the gaps in Kibana.

Vega is a declarative grammar for visualisations. Which means that it is similar to SQL or Elasticsearch in that the programmer defines an outcome, not a procedure. Vega is comfortably ten-times more complicated than the rest of Kibana, which makes it about ten-times simpler than D3.js .

There are two ways to filter an aggregate and display it with Vega in Kibana.

1. The Vega way is to filter after aggregating, as is done in this portion of the vega tutorial.
2. The Elasticsearch way is to use a bucket selector in your Elasticsearch query. Then use Vega to visualise the output. This will have the better run-time behaviour as less data travels between processes and cluster nodes.

I think that Vega is a great addition to Kibana. It greatly expands the set of possible visualisations, which gives you the flexibility to accomplish a bit more with adding another reporting tool.

Comments