Context Pruning And History Management

Last updated: May 28, 2026

Assistant Context Pruning and History Management

The assistant module employs sophisticated history management logic to ensure agents maintain focus, minimize token costs, and avoid the "context blindness" that occurs when LLM windows become saturated.

History Pruning Logic

Implemented in Git commit a5a14bc0, the system uses a tiered approach to history inheritance:

1. Default Truncation

In assistant.go, the GetHistory() function enforces hard limits based on the platform:

  • WhatsApp: Last 15 messages (formerly 20). Mobile users require fast, concise context.
  • Standard Platforms (Discord/Web): Last 15 messages (formerly 30).
  • Background Delegated Agents: Limited to the last 5 messages to keep sub-tasks strictly focused on their specific objective.
  • get_capabilities truncation: Background tasks use a 40k token budget for get_capabilities calls to avoid context bloat (e8f4da9b).

2. Strategic Context Compaction

The CompactHistoryIfLarge(tokenLimit) function (in assistant.go) performs "Middle-Turn Truncation" for long-running sessions:

  1. Keep First Message: Preserves the original strategic directive (The "Goal").
  2. Keep Last N Messages: Preserves the most recent operational context (The "State").
  3. Tombstone: Replaces the middle turns with a system message: --- STRATEGIC CONTEXT TRUNCATED (%d turns removed) ---.

Memory Models: WIS / WIH / WID

To prevent data loss during pruning, the system distills facts into the Project Memory (memory.md) using the WIS model:

  • (W)hat (I)s: The current architecture, environment, and rules.
  • (W)hat (I)s (H)appened: A log of significant events and tool failures.
  • (W)hat (I)s (D)one: A checklist of completed milestones.

Token Management & HUD

The assistant calculates estimated token usage (4 characters per token) and injects a SESSION METRICS HUD into the prompt when:

  • Iteration count reaches 70% of max (e.g., 10/15 turns).
  • Context usage exceeds 50% of the window.

This HUD warns the agent to "summarize and finish" before it runs out of turns or memory.

Smart Completion Detection

As of commit d7b921e6, the ProcessReActLoop includes heuristic detection for completion keywords like "Audit Complete" or "Task Finished." If an agent provides a final text response without calling finish_task, the system can auto-terminate the loop to save redundant LLM calls.

Context Overflow Detection & Fallback Chain (May 27, 2026)

The llama client now detects context length errors and triggers a fallback chain:

1. llama-server returns HTTP 400 with context length error
2. ErrContextOverflow sentinel detected in llama_client.go
3. Falls back to gemini-2.5-flash-lite
4. If all models exhausted, session fails gracefully with detailed error

Implemented in commit 41536c47. Background tasks use non-blocking summarization with a 40k token compaction threshold (e8f4da9b).

40k Token Compaction Threshold

Proactive context compaction now triggers at 40k tokens (increased from previous thresholds) to provide more headroom before reaching model limits. Summarization is non-blocking to prevent main session loop delays.

Guidance for AI Agents

  • Assume Pruning: Do not rely on details provided 20 turns ago. If a fact is important, record it in the project memory.md.
  • Respect the HUD: When the HUD warns of nearing limits, use your remaining turns to document blockers on the task board before calling finish_task.

Cross-References