Query Engine & LLM API

1. The Big Picture

When you send a message to Claude Code, it passes through at least four distinct layers before the model's reply reaches your terminal. Understanding those layers is the key to understanding why Claude Code behaves the way it does — why it retries, why it compacts, why it can run tools in the middle of a stream.

1. QueryEngine.submitMessage() Validates the prompt, builds the system prompt, resolves the model, records the transcript, then hands off to query().
2. query() → queryLoop() An async function* that loops until the model stops calling tools. Each iteration is one model call.
3. queryModel / callModel Calls the Anthropic API via the SDK's streaming interface, wrapping everything in withRetry().
4. Stop hooks & token budget After the model finishes each turn, external hooks run; the token budget decides whether to inject a nudge and loop again.

2. Sequence Diagram

Below is the full message flow for a single conversation turn that involves at least one tool call. Follow the arrows: the loop between queryLoop and queryModel is the heart of the agentic behavior.

3. QueryEngine — One Engine Per Conversation

QueryEngine is a stateful class instantiated once per conversation. It holds the mutable message history, total token usage, permission denials, and the abort controller. Each call to submitMessage() is one "turn" within that conversation.

QueryEngine.ts (simplified)export class QueryEngine {
  private mutableMessages: Message[]
  private abortController: AbortController
  private totalUsage: NonNullableUsage
  private permissionDenials: SDKPermissionDenial[]

  // Turn-scoped: cleared at start of each submitMessage() call
  private discoveredSkillNames = new Set<string>()

  async *submitMessage(
    prompt: string | ContentBlockParam[],
    options?: { uuid?: string; isMeta?: boolean },
  ): AsyncGenerator<SDKMessage> {
    // 1. Build system prompt (fetchSystemPromptParts)
    // 2. processUserInput — handles slash commands
    // 3. recordTranscript — persists BEFORE the API call
    // 4. yield* query({ messages, ... })
    // 5. yield final result SDKMessage
  }
}

Why transcript is written before the API call

Before query() is even called, submitMessage() persists the user's message to disk. This means a session is resumable even if the process is killed before the model ever responds. The comment in the source is illuminating:

// If the process is killed before that (e.g. user clicks Stop in
// cowork seconds after send), the transcript is left with only
// queue-operation entries; getLastSessionLog filters those out,
// returns null, and --resume fails with "No conversation found".
// Writing now makes the transcript resumable from the point the
// user message was accepted, even if no API response ever arrives.

4. Inside queryLoop() — The While(true) Core

queryLoop() in query.ts is a while(true) loop that carries a typed State object between iterations. Rather than nine separate let variables, a single state = { ... } reassignment at each continue site makes the transitions explicit and auditable.

query.tstype State = {
  messages: Message[]
  toolUseContext: ToolUseContext
  autoCompactTracking: AutoCompactTrackingState | undefined
  maxOutputTokensRecoveryCount: number
  hasAttemptedReactiveCompact: boolean
  maxOutputTokensOverride: number | undefined
  pendingToolUseSummary: Promise<ToolUseSummaryMessage | null> | undefined
  stopHookActive: boolean | undefined
  turnCount: number
  transition: Continue | undefined   // WHY we looped again
}

Continue transitions — the seven reasons to loop

The transition field records why the loop continued. This makes the continue sites self-documenting and lets tests assert which recovery path fired without inspecting message content:

5. Streaming & the API Layer

queryModel in claude.ts is an async function* that calls the Anthropic beta messages endpoint and re-yields each stream event as an internal AssistantMessage or StreamEvent.

query.ts (inner stream loop, simplified)for await (const message of deps.callModel({
  messages: prependUserContext(messagesForQuery, userContext),
  systemPrompt: fullSystemPrompt,
  thinkingConfig: toolUseContext.options.thinkingConfig,
  tools: toolUseContext.options.tools,
  signal: toolUseContext.abortController.signal,
  options: { model: currentModel, fallbackModel, ... },
})) {
  if (message.type === 'assistant') {
    assistantMessages.push(message)
    // tool_use blocks trigger needsFollowUp = true
    const toolBlocks = message.message.content
      .filter(b => b.type === 'tool_use')
    if (toolBlocks.length > 0) needsFollowUp = true
  }
  yield yieldMessage // surfaces to SDK caller / REPL
}

Streaming tool execution

When config.gates.streamingToolExecution is enabled, a StreamingToolExecutor fires tools while the stream is still open. Tools whose inputs arrive early start executing in parallel with the model still generating text, cutting latency on multi-tool turns.

// Backoff formula (from withRetry.ts)
export function getRetryDelay(
  attempt: number,
  retryAfterHeader?: string | null,
  maxDelayMs = 32000,
): number {
  if (retryAfterHeader) {
    const seconds = parseInt(retryAfterHeader, 10)
    if (!isNaN(seconds)) return seconds * 1000
  }
  const baseDelay = Math.min(
    BASE_DELAY_MS * Math.pow(2, attempt - 1),
    maxDelayMs,
  )
  const jitter = Math.random() * 0.25 * baseDelay
  return baseDelay + jitter
}

// From QueryEngine.ts — usage tracking
if (message.event.type === 'message_start') {
  currentMessageUsage = updateUsage(EMPTY_USAGE, message.event.message.usage)
}
if (message.event.type === 'message_delta') {
  currentMessageUsage = updateUsage(currentMessageUsage, message.event.usage)
  if (message.event.delta.stop_reason != null) {
    lastStopReason = message.event.delta.stop_reason
  }
}

6. Context Management & Autocompact

Before each API call, queryLoop runs a pipeline of context reduction strategies in a fixed priority order:

1. applyToolResultBudget() Caps the byte size of individual tool results. Large results are stored externally and replaced with a reference stub.
2. snipCompact (HISTORY_SNIP feature) Removes old messages from the middle of the history when they are provably not needed, freeing tokens without a full summarization pass.
3. microcompact / cached microcompact Merges consecutive tool-result/user message pairs into condensed summaries. The cached variant uses API-side cache edits to avoid retransmitting deleted blocks.
4. contextCollapse (CONTEXT_COLLAPSE feature) A read-time projection over the REPL's full history. Staged collapses are committed on each entry; the model sees a collapsed view while the UI retains the full history for scrollback.
5. autoCompact When the context approaches the blocking limit, triggers a full summarization via a forked agent. If it fires, the loop continues immediately with the post-compact messages.

// task_budget carryover across compaction (query.ts ~508)
if (params.taskBudget) {
  const preCompactContext =
    finalContextTokensFromLastResponse(messagesForQuery)
  taskBudgetRemaining = Math.max(
    0,
    (taskBudgetRemaining ?? params.taskBudget.total) - preCompactContext,
  )
}

7. Stop Hooks — Post-Turn Lifecycle

After the model finishes (no tool calls, no recovery needed), the engine calls handleStopHooks() in query/stopHooks.ts. Stop hooks are external shell scripts or commands configured by the user. They run after every turn and can:

• Produce blocking errors — injected as user messages, triggering another loop iteration
• Prevent continuation — the engine returns { reason: 'stop_hook_prevented' }
• Fire background tasks — prompt suggestions, memory extraction, auto-dream — all fire-and-forget

// --bare / SIMPLE: skip background bookkeeping
// Scripted -p calls don't want auto-memory or forked agents
// contending for resources during shutdown.
if (!isBareMode()) {
  void executePromptSuggestion(stopHookContext)
  if (feature('EXTRACT_MEMORIES') && isExtractModeActive()) {
    void extractMemoriesModule!.executeExtractMemories(...)
  }
  if (!toolUseContext.agentId) {
    void executeAutoDream(...)
  }
}

8. Token Budget — The Auto-Continue Feature

query/tokenBudget.ts implements an auto-continue feature for the SDK path. When a per-turn token budget is configured, the engine checks after each clean model stop whether the model has "used up" enough of its budget. If not, it injects a nudge message and loops again.

query/tokenBudget.tsconst COMPLETION_THRESHOLD = 0.9   // 90% used = done
const DIMINISHING_THRESHOLD = 500  // <500 new tokens = no progress

export function checkTokenBudget(
  tracker: BudgetTracker,
  agentId: string | undefined,
  budget: number | null,
  globalTurnTokens: number,
): TokenBudgetDecision {
  if (agentId || budget === null || budget <= 0) {
    return { action: 'stop', completionEvent: null }
  }
  const pct = Math.round((globalTurnTokens / budget) * 100)
  const isDiminishing =
    tracker.continuationCount >= 3 &&
    deltaSinceLastCheck < DIMINISHING_THRESHOLD &&
    tracker.lastDeltaTokens < DIMINISHING_THRESHOLD
  // Continue if under 90% AND not diminishing
  if (!isDiminishing && turnTokens < budget * COMPLETION_THRESHOLD) {
    return { action: 'continue', nudgeMessage: ... }
  }
  return { action: 'stop', ... }
}

9. Key Takeaways

10. Quiz

Five questions to check your understanding. Select an answer, then hit Check.