An observability stack built to order

Metrics, uptime, logs and identity security brought into a single platform for an access and identity management product.

Talk to an engineer DevOps

Identity management Client

DevOps Practice

July 2024 Published

01 — The challenge

Where it started.

A complex cloud estate with no central place to see it, producing millions of logs an hour that were effectively unqueryable during an incident.

Access to Microsoft accounts had to stay continuously available, and debugging started by guessing which system to open first.

02 — What we built

The work itself.

Prometheus and Grafana. Collection and visualisation on open components, so dashboards can be changed by the team rather than requested from a vendor.

Uptime Kuma on the endpoints. External checks on the things customers actually touch, kept separate from internal health signals.

AWS OpenSearch at millions an hour. Retention and search sized for the real volume rather than a sampled one.

A bespoke CLI and GUI. A purpose-built way to pull logs out of OpenSearch quickly, because during an incident the query interface is the bottleneck.

03 — Results

What the client has now.

Uptime improved, with issues answered quickly rather than investigated slowly. Debugging time cut by making log retrieval fast enough to use mid-incident. Single sign-on and multi-factor authentication brought under the same platform as monitoring. Several disparate products replaced by one stack.

The rest of this study

What we built, what it produced, and the stack behind it.

We share the whole write-up — we would just like to know who is reading. One form opens every case study on the site for the next 30 days.

If this is your problem too

Start with a fixed-price pilot.

Two weeks on a real piece of your backlog, priced before we begin. You keep the output whether or not you continue.

Related work

Health insurance · CI/CD

From a three-hour deployment to ten minutes

Blockchain infrastructure · Monitoring