{"id":1416,"date":"2026-11-17T06:25:00","date_gmt":"2026-11-17T06:25:00","guid":{"rendered":"https:\/\/www.jetexe.com\/blog\/?p=1416"},"modified":"2026-11-20T11:25:00","modified_gmt":"2026-11-20T11:25:00","slug":"charter-desk-mlops-audit","status":"publish","type":"post","link":"https:\/\/www.jetexe.com\/blog\/charter-desk-mlops-audit\/","title":{"rendered":"A Pre-Departure MLOps Audit for Charter Desk Models"},"content":{"rendered":"<p>Charter operators live by pre-departure checklists: documents, weather, fuel, crew duty, catering. A pricing or demand-forecast model that sits behind a charter desk deserves its own pre-departure list, because it can quietly cost money in ways nobody sees until a quarter closes. What follows takes the practices catalogued under the MLOps label and turns them into a review sheet that fits in one working session. For every line, mark it met, partly met or missing, and record the proof you looked at rather than your impression.<\/p>\n<h2>Why a checklist and not a platform<\/h2>\n<p>The Ops Atlas entry on <a href=\"https:\/\/www.xopsschool.com\/mlops\/\">MLOps for models running in production<\/a> says plainly that the field is a practice rather than a product you can buy, and that a toolset without owners and gates leaves a team where it started. A checklist is therefore the right first instrument: it exposes where a team stands, costs nothing and is easy to repeat each quarter. The entry also points to a published scoring approach called the ML Test Score, with 28 tests in four groups, in which the overall result equals the weakest section&#8217;s total, not the average. That detail is worth borrowing, because it stops a strong area from masking a weak one.<\/p>\n<h2>Section A: traceability<\/h2>\n<ol>\n<li><strong>Can you rebuild the live model?<\/strong> Evidence: a record naming the code commit, the data snapshot&#8217;s fingerprint, the settings and the environment for the version currently serving.<\/li>\n<li><strong>Is the data versioned, not just the code?<\/strong> Evidence: a snapshot identifier that you can retrieve today.<\/li>\n<li><strong>Is reproducibility defined as tolerance?<\/strong> Evidence: a written band for acceptable metric differences between a rebuild and the original. The Atlas notes that bit-exact results are not the target, since training can be non-deterministic.<\/li>\n<li><strong>Is there a registry listing versions with lineage?<\/strong> Evidence: a single place where you can see which version is live and which one it replaced.<\/li>\n<\/ol>\n<h2>Section B: parity between training and serving<\/h2>\n<ol start=\"5\">\n<li><strong>Is there one definition for each input?<\/strong> Evidence: the same code or definition used to build features for training and for live requests.<\/li>\n<li><strong>Do you measure skew?<\/strong> Evidence: a replay test that sends identical records down both paths. In the entry&#8217;s reference build, the acceptable mismatch rate is exactly zero.<\/li>\n<li><strong>Are live inputs logged?<\/strong> Evidence: stored copies of the features that were actually served, so you can compare them with training later.<\/li>\n<\/ol>\n<h2>Section C: release control<\/h2>\n<ol start=\"8\">\n<li><strong>Does every challenger face a written gate?<\/strong> Evidence: a policy that names the test set, the metrics, the slices and a latency ceiling. The Atlas example sets a minimum overall uplift, a maximum slice loss and a response-time cap, and rejects a version that wins overall yet loses badly on one group.<\/li>\n<li><strong>Is the held-out data fixed and cut by date?<\/strong> Evidence: a dated file that nobody edits. Leakage between training and test data is a documented source of inflated results; the entry cites a survey that found errors affecting 294 papers across 17 fields.<\/li>\n<li><strong>Is exposure staged?<\/strong> Evidence: logs showing shadow traffic, then a canary share, then a live comparison.<\/li>\n<li><strong>Is the way back rehearsed?<\/strong> Evidence: a dated record of the last rollback drill, carried out by redirecting an alias or the traffic share, plus a non-learning fallback that still works.<\/li>\n<\/ol>\n<p>Operators who price empty repositioning legs will recognise why stages matter, because a mispriced route can be repeated hundreds of times before anyone notices. A useful background read is <a href=\"https:\/\/www.jetexe.com\/blog\/factors-that-affect-private-jet-charter-pricing-2\/\">the article about what moves charter prices up and down<\/a>, which shows how many inputs a pricing model must keep aligned.<\/p>\n<h2>Section D: monitoring and retraining<\/h2>\n<ol start=\"12\">\n<li><strong>Are there three layers of monitoring?<\/strong> Evidence: separate views for service health, input data and result quality.<\/li>\n<li><strong>Do you have stand-ins while labels are late?<\/strong> Evidence: tracked prediction distributions and action limits. A booking that never confirms is a late label in your world; define the proxy.<\/li>\n<li><strong>Are alert thresholds calibrated on your own history?<\/strong> Evidence: a short note recording how each limit was picked. The entry cautions that vendor defaults are not truth, and that one popular index&#8217;s familiar bands lack academic support.<\/li>\n<li><strong>Is there a retraining policy?<\/strong> Evidence: a document that lists triggers, such as a schedule, new data, degraded performance or a significant input shift, plus a staleness measurement for choosing cadence.<\/li>\n<li><strong>Do you diagnose before retraining?<\/strong> Evidence: a runbook that separates pipeline mismatch, upstream defect and real drift.<\/li>\n<\/ol>\n<h2>Section E: ownership<\/h2>\n<ol start=\"17\">\n<li><strong>Is there a named owner for the model?<\/strong> A role, not a person&#8217;s memory.<\/li>\n<li><strong>Is there a consumer list?<\/strong> Evidence: a register of every system that reads the outputs, with access control on them.<\/li>\n<li><strong>Is promotion approved by a human?<\/strong> The Atlas warns that nightly retrain-and-promote loops can produce outages through feedback effects.<\/li>\n<\/ol>\n<h2>How to run the audit without turning it into theatre<\/h2>\n<p>Run the session with the person who owns the model, someone from the commercial side who uses its output, and one sceptic who was not involved in building it. Ask each question, then request the artefact on screen. A claim that a rebuild is possible counts only if somebody actually performs it, ideally in front of the group. Treat every unanswered question as a no and move on; the value comes from the list of gaps, not from debate. Repeat the exercise after major changes and once each quarter so scores can be compared over time.<\/p>\n<h2>Scoring and what to do with it<\/h2>\n<p>Award one point for met, half for partly met and zero for missing, then total each section separately. Following the minimum-of-sections idea, your headline score is the lowest section, not the average. If that number sits near zero, the entry&#8217;s wording for a zero is blunt: it reads as research work, not a productionised system. Pick the weakest section and fix one line per week, starting with whichever line would hurt most if it failed tomorrow.<\/p>\n<p>For a plain sense of where idle capacity fits into pricing logic, you can also read <a href=\"https:\/\/www.jetexe.com\/blog\/how-empty-leg-jet-flights-work\/\">the explainer on how empty-leg flights come about<\/a>, then ask which of its inputs your own forecast model actually sees.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A 19-line checklist that turns MLOps practice into a pre-departure audit for pricing and demand models, with evidence to collect and a minimum-of-sections score.<\/p>\n","protected":false},"author":7,"featured_media":1417,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[916],"tags":[],"class_list":["post-1416","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aviation-industry"],"_links":{"self":[{"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/posts\/1416","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/comments?post=1416"}],"version-history":[{"count":1,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/posts\/1416\/revisions"}],"predecessor-version":[{"id":1542,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/posts\/1416\/revisions\/1542"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/media\/1417"}],"wp:attachment":[{"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/media?parent=1416"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/categories?post=1416"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jetexe.com\/blog\/wp-json\/wp\/v2\/tags?post=1416"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}