The Day SkyBridge Airlines' AI Agent Went From 14 Seconds to 900 Milliseconds

Mapping the Crime Scene: Why 4 Seconds Became 14

The architect pulled the trace for one slow request — "What are my rebooking options?" — and laid it out as a timeline.

User query
   │
   ▼
Orchestrator Agent (LLM call #1: understand intent)             ~1.8s
   │
   ▼
Orchestrator loads FULL tool catalog from 8 MCP servers          ~1.1s  (schema-heavy)
   │  (46 tools x ~90 tokens of schema each = ~4,100 tokens
   │   injected into context on every single turn)
   ▼
Orchestrator Agent (LLM call #2: decide which agents to invoke)  ~2.4s
   │
   ▼
Sequential calls, one after another:
   ├─ Weather Agent  → MCP call to NOAA-partner API                2.1s
   ├─ Rebooking Agent → MCP call to inventory system                3.6s
   ├─ Crew Legality Agent → MCP call to crew scheduling system       1.9s
   └─ Loyalty Agent → MCP call to tier/points DB                    1.4s
   ▼
Orchestrator Agent (LLM call #3: synthesize final answer)         2.3s
   ▼
Response to user                                                 ≈14.6s total

Five separate problems were hiding in that one trace, and they compound multiplicatively, not additively:

This is, almost verbatim, the pattern reported across the industry in 2026 as multi-agent systems moved from pilot to production: teams that bolted together 8–15 collaborating agents without disciplined orchestration ended up with systems that were unpredictable and far more expensive than a single well-designed agent, while token costs in multi-server MCP deployments grew faster than any other line item in the AI budget. SkyBridge wasn't unusual. It was typical.

The fix wasn't one silver bullet. It was six coordinated changes, and the story of implementing them is really the story of turning an "agentic system" into "an engineered distributed system that happens to have LLMs in it."

Finding One: The Toolbox Was Too Heavy

The orchestrator's biggest single latency and cost line item was schema bloat — dragging in the definitions of all 46 tools on every turn regardless of relevance.

SkyBridge's team replaced the "load everything, let the model pick" pattern with a two-tier tool discovery model, echoing an approach that has emerged across the industry (sometimes called "MCP-Zero" style routing): instead of injecting every tool schema up front, a lightweight semantic router first narrows the field.

Concretely, they:

• Built a small embedding index over tool names and one-line descriptions (not full schemas), so a router could semantically match "rebooking options after cancellation" to RebookingAgent, WeatherAgent, and CrewLegalityAgent — and correctly leave out RefundsAgent and HotelPartnerAgent for this particular question.
• Trimmed every remaining tool description down to the essential (a well-known technique: "returns eligible rebooking flights" beats a four-sentence explanation), and shortened parameter names where it didn't hurt clarity.
• Applied schema compression at the gateway layer for tools that were rarely misambiguated, keeping full descriptive schemas only for tools whose names looked similar enough to be confused (their Confluence-vs-Jira problem was RebookingAgent.search vs LoyaltyAgent.search — different services, dangerously similar shapes).

The airline lesson underneath the technique: an airline concierge doesn't need omniscience on every turn. A weather-delay conversation and a lost-baggage conversation are different tool universes. Treat "which universe am I in" as its own cheap, fast decision — don't make the expensive reasoning model re-discover it from scratch every time.

Finding Two: The Agents Were Waiting in Line

The orchestrator was calling Weather, Rebooking, Crew Legality, and Loyalty one after another, as if each needed the previous one's answer. None of them did.

The team redrew the orchestration graph as an explicit DAG (directed acyclic graph) instead of a linear chain, and made the orchestration layer itself — not the LLM's free-form reasoning — responsible for fan-out and fan-in:

Two important nuances they learned the hard way:

• Parallelizing only helps if you also fix the slowest branch. Their trace showed Rebooking (inventory system call) taking 3.6 seconds while the others finished in under 2. Running all four in parallel dropped total specialist time from ~9 seconds to ~3.6 seconds — but that 3.6-second inventory call was now the entire bottleneck. Optimizing the three faster tools would have done nothing for end-to-end latency; the real fix had to target the slow one specifically (an inventory-system read replica with a tighter SLA, plus a strict timeout with graceful degradation).
• Not everything should be an LLM decision. For this well-known workflow ("cancellation → what happens next"), the fan-out itself became a deterministic orchestration step, not something the orchestrator LLM reasoned about turn by turn. The industry lesson here is now well established: free-form, model-decided orchestration is valuable for genuinely novel or ambiguous requests, but for known, repeatable workflows, a deterministic graph (LangGraph-style, or an explicit state machine) is faster, cheaper, and — critically — auditable, because you can point at the graph and say exactly what will happen for a given input, instead of hoping the model reasons its way there the same way twice.

Finding Three: Half the Pipeline Was Overqualified

Three full frontier-model round trips (understand → decide → synthesize) for one passenger question was the single most expensive design decision in the system, in both dollars and milliseconds.

SkyBridge's team applied model routing — sending each step in the pipeline to the cheapest model capable of doing it well, rather than sending every step to the same frontier model:

This mirrors a broader shift the industry made in 2026: agentic pipelines involve a mix of operations — orchestration, tool selection, summarization, and final generation — and routing the lighter steps to smaller, faster models while reserving frontier models for genuinely hard reasoning is one of the most reliable ways to cut both cost and latency without touching accuracy on the steps that matter.

Finding Four: Nothing Remembered Anything

SkyBridge was sending its ~6,000-token system prompt (airline policy, tone guidelines, tool catalog, current fare rules) fresh on every single call, to every agent, for every passenger. None of it was cached.

They layered three kinds of caching, each solving a different problem:

One thing the team deliberately avoided, after reading up on it, was naive "cache everything" behavior: wrapping an entire long context in one cache boundary can paradoxically increase latency rather than reduce it if the boundary isn't chosen carefully. The fix is intentional placement of the cache breakpoint — cache the stable prefix, not the whole variable conversation.

Finding Five: The Passenger Was Waiting in Silence

Some latency simply can't be engineered away — a live inventory system, a crew-legality check against a regulator's rules, a real API call to a hotel partner. SkyBridge's team borrowed a lesson from voice-agent design, where the winning systems in 2026 treat perceived latency as seriously as actual latency:

• Stream everything downstream of the first useful token. Instead of waiting for the full synthesized answer, the orchestrator now streams partial results as each specialist agent returns: "Checking weather delays… Found 3 rebooking options… Confirming your crew's flight is legal to operate…" This didn't reduce total processing time, but it dropped perceived wait time dramatically, because the passenger sees continuous progress instead of a blank spinner.
• Prefetch the obvious next step. If Weather Agent comes back confirming a storm-related cancellation, the system now speculatively kicks off the Rebooking Agent's search before the orchestrator LLM has "decided" to — because for this workflow, that next step is correct with high enough confidence that the savings outweigh the rare wasted call.
• Set an explicit latency budget per hop, the way you'd set an SLA for any microservice: if the end-to-end target is 2 seconds, and guardrail evaluation needs 20–50ms, and synthesis needs 600ms, then each tool call gets an honest, enforced timeout that leaves margin — instead of hoping everything "usually" finishes in time. Track p95 and p99, not averages: a single slow tool at the tail can blow the whole budget even when the typical case looks fine.

Finding Six: The Trace Could Not Tell the Story

The scariest part of the original incident wasn't the 14-second latency. It was that nobody could answer, with confidence, why the system told a passenger the wrong thing about their bag. There was no way to reconstruct, after the fact, which tool was called, what it returned, and which reasoning step used that (possibly stale) data.

The fix was to treat traceability as a first-class architectural requirement, not an afterthought bolted on with print statements. SkyBridge adopted the emerging OpenTelemetry GenAI semantic conventions, which standardize how agent systems capture spans for model calls, tool invocations, token usage, and agent-to-agent handoffs — giving every request a structured, queryable trace instead of a wall of unstructured logs.

What this bought them concretely:

• Step-level traces, not just success/failure: for every passenger request, they could see the exact sequence — intent classified as X, tools shortlisted to {A, B, C}, tool B called with these parameters, tool B returned this payload at this timestamp, synthesis used fields P and Q from that payload to produce this sentence.
• Silent-failure detection. Agents can fail gracefully from a systems point of view — HTTP 200, well-formed JSON — while still returning stale or wrong data that the LLM then confidently repeats. Tracing alone doesn't catch this; SkyBridge added semantic evaluation of tool outputs (freshness checks, sanity bounds — "a baggage location can't be a null island coordinate") rather than trusting the status code.
• A real answer for the VP of Ops. When the next incident happened, the team could pull up the exact trace, in minutes, and show precisely which cached tool result was stale and why — turning a "the AI hallucinated" panic into a "the loyalty-tier cache TTL was too long" bug ticket.

This is also where governance and auditability intersect with performance: the same structured trace that lets you explain a decision after the fact is what lets you measure where your latency and token budget are actually going, which closes the loop back to the earlier findings.

What "Fast" Looked Like Afterward

Same question, same architecture skeleton, re-traced after the six fixes:

User query
   │
   ▼
Fast classifier (intent + tool shortlist)                        ~150ms
   │
   ▼
Deterministic fan-out (parallel, cached system prompt)
   ├─ Weather Agent            (cached tool result, TTL valid)     ~40ms
   ├─ Rebooking Agent          (inventory call, tightened SLA)    ~650ms  ← now the bottleneck, by design
   ├─ Crew Legality Agent      (parallel)                         ~400ms
   └─ Loyalty Agent            (parallel)                         ~180ms
   ▼
Streamed synthesis (frontier model, cached prefix, first token immediately) ~900ms perceived
   ▼
Response fully delivered                                        ≈1.1s total, first useful content <300ms

Fourteen seconds to roughly a second, with the perceived latency even lower because of streaming — and, just as importantly, a system that could now tell you exactly why it said what it said.

The Runbook SkyBridge Uses Now

For anyone building an orchestrator-plus-MCP-agent system, the pattern generalizes well beyond airlines:

1. Route before you reason. Don't load every tool schema into every turn — semantically shortlist tools/agents first, with a cheap model, and inject only what's relevant.
2. Parallelize independent work; keep dependent work sequential. Model the orchestration as an explicit DAG. For known workflows, prefer deterministic orchestration over letting the LLM re-derive the plan every time.
3. Right-size the model per step. Reserve frontier-model reasoning for genuinely hard steps; route classification, routing, and light summarization to smaller/faster models.
4. Cache in layers, deliberately. Prefix/prompt caching for stable system content, semantic caching for repeated user intents, TTL-based caching for tool results — and choose cache boundaries intentionally rather than caching everything by default.
5. Design for perceived latency, not just actual latency. Stream partial results, prefetch high-confidence next steps, and set explicit per-hop timeout budgets tracked at p95/p99, not average.
6. Instrument for explainability from day one. Adopt structured, standardized tracing (OpenTelemetry GenAI conventions or equivalent) so every decision, tool call, and output is reconstructable — because in a regulated, customer-facing industry like airlines, "the model hallucinated" is not an acceptable postmortem; "the cache TTL on the loyalty-tier lookup was misconfigured" is.

This article uses a fictional airline ("SkyBridge Airlines") as an illustrative composite scenario; the technical patterns, failure modes, and mitigation techniques described are drawn from documented, real-world practice in agentic AI system design.