Telemetry Poller and Feedback Loops
The telemetry_poller.go operates as a continuous background service (the "Fortress Telemetry Poller"), maintaining a global, real-time state of the dLAN infrastructure.
Polling Mechanisms
The poller executes at regular intervals (typically 5 minutes) to gather metrics from disparate infrastructure endpoints.
Data Sources
- Proxmox: Queries the hypervisor for container counts, CPU/RAM usage, ZFS pool metrics, and host uptime.
- Pelican Panel: Queries game server management for total node counts and online server status.
- cPanel / WHM: Tracks offline nodes, load averages, and determines the overall cluster health status (e.g.,
ALL_CLEARvsDEGRADED). - Virtualizor: Monitors running VM statistics for VPS hosting environments.
- Financial (CoinStats): An hourly check of the Crypto Fear & Greed index and BTC dominance.
Infrastructure Feedback Loop
The gathered telemetry isn't just logged; it is injected into the active memory context of relevant agents (like Sasha or Sysadmin).
When an agent is queried about system health or receives a NOC alert, the latest telemetry snapshot provides immediate, real-world context, preventing the agent from needing to perform time-consuming manual checks before formulating an initial hypothesis.
Mermaid Diagram: Telemetry Loop
graph LR
subgraph "Infrastructure"
P[Proxmox]
C[cPanel]
V[Virtualizor]
end
P --> T[Telemetry Poller (5m)]
C --> T
V --> T
T --> M[(Global Context Cache)]
M --> A[Agent: Sasha/Sysadmin]
A --> D[Diagnose Issues]
Guidance for AI Agents
- Check Global State: Before using tools to query server health, check your injected context for the latest telemetry snapshot. The answer might already be there.
- Trend Analysis: Use the snapshot to identify broader trends (e.g., high load across the entire cPanel cluster vs. an isolated server).