Charter operators run on checklists. Before a trip leaves the ramp, someone confirms fuel, paperwork, crew duty hours and the catering order, and nobody calls that bureaucracy. A company’s monitoring setup deserves the same treatment. ObservabilityOps, the practice of operating the systems that collect and report operational data, is largely a pre-flight discipline, and this audit turns it into questions you can answer with yes, no or “not sure”.
The framing here comes from a structured Ops Atlas entry covering ObservabilityOps, which the Atlas itself calls a label with no formal standard behind it. Treat the following as a working audit, not a certification.
Section A: Ownership and scope
- Is a named person or small group accountable for the measurement system itself? Not for every alert, and not for the services being measured. A yes here means somebody is on the hook when the system drifts.
- Do you know what you operate? Write down the collectors, storage back ends, dashboards and alert rules. An inventory that surprises you is a finding, not a failure.
- Are neighbouring jobs kept separate? Reliability targets, incident roles, security feeds and the cloud bill live elsewhere, according to the Atlas. If your keeper is secretly doing all of them, scope has crept.
Section B: Data hygiene
- Do metric labels use only bounded values? Every unique combination creates a new series. Individual customer or booking identifiers do not belong in labels. As an informal yardstick, the Atlas would like a label’s distinct values to stay in single digits and prompts an investigation once they pass roughly a hundred.
- Are there hard limits where data enters? A cap on series or labels at the point of collection protects everyone else from one careless addition. The Atlas adds a warning that, with some collectors, a breach can reject the entire scrape instead of trimming only the excess.
- Do logs carry structure? Trace and span identifiers and severity should sit in their own fields. Free-text sentences make filtering unreliable.
- Is sensitive content redacted before storage? Passenger names, passport details and payment references have no place in telemetry.
Section C: Sampling and retention
- Is your volume high enough to need sampling at all? Guidance quoted by the Atlas starts to suggest it from roughly a thousand traces per second.
- If you sample, do you keep failures and slow cases? Random slicing alone cannot promise that.
- Are retention periods chosen by value? Recent data stays hot, older data moves to cheaper tiers, and the oldest is archived. Do not accept a default simply because it came in the box.
Section D: Alerts and the alarm path
- Does every page describe something a customer would notice? Component-level noise belongs in a morning review.
- Is there an end-to-end alert-path test? The Atlas favours one black-box check over many per-component alerts, together with a heartbeat that fires continually so silence itself raises the alarm.
- Have unused alerts been retired? The audit measure is the unused-signal ratio over a ninety-day window.
Section E: Cost and value
- Do you know unit cost? Take the month’s observability spend, divide by the requests handled, and quote the result per million. A rising bill with a flat unit cost means growth; a rising unit cost means waste. Operators familiar with what drives private jet charter prices will recognise the habit of separating volume effects from rate effects.
- Is each team shown its own usage? Showback, where teams see what they consume, is level three on the Atlas’s own maturity ladder.
Section F: Freshness and coverage
- How stale is your data? Send a synthetic canary through the pipeline and measure the ninety-fifth percentile delay from event to dashboard.
- Does every service identify itself? A share of data labelled “unknown service” should sit close to zero, or you cannot say whose problem it is.
Scoring yourself
Count your yeses. Fewer than six suggests the Atlas’s first level, ad hoc, where tools exist without an owner. Between six and eleven, you probably have a central owner and some standards. Twelve or more points toward a governed product, with limits, showback and routine alert reviews. The top level in the Atlas’s scale adds optimisation: tuning sampling and retention by value so unit cost falls while coverage holds. These bands are our own rough reading, offered for orientation, and not a formal score.
What a failed answer usually means
A no is rarely a crisis. Most failures on this list trace back to one of two causes. Either nobody was assigned the question, or a default was accepted without being examined. The remedy for the first is a name and a date; for the second, a short conversation about what the data is meant to support. Resist the urge to solve every no in the same week. Rank the failures by how directly they could hide a real problem from you, start with the alarm path and the label limits, and leave tidy-up work such as dashboard cleanup for later.
It also helps to record who answered each question. If three people give three different answers to the same line, that disagreement is the finding, and it usually points at a missing standard.
A thirty-day sequence
For a small team, the Atlas outlines a first month: appoint an owner, compile the inventory, stand up the heartbeat plus an outside probe, and repair the three most damaging labels. The sequence works because every step gives the next one something to measure. If your first month leaves time over, put it into the second quarter’s work on logging standards and gateway limits.
Operators who want a gentler on-ramp to the broader ideas might begin with the explainer on how empty-leg flights work, which shows the same trade of unused capacity against cost, then return to this list with a pencil. Finish by recording the date, the score and the three weakest answers. Repeat the audit quarterly, and treat the trend as the real result.

