03Local systems · Observability and agent operations
AEGIS Autonomous SIEM and Monitoring
I built the system around a practical division of labor: deterministic software collects, correlates, and constrains; local AI helps interpret what is unusual; and a human controls when a bounded automated remediation is authorized.
- My role
- Product architect · Full-stack engineer · Platform operator
- Timeline
- Continuously operated and evolved
The ecosystem
From Raw Signal to Human-Controlled Automated Remediation.
AEGIS is the operating spine: it receives events and active health observations, keeps the incident state, reduces noise, calls bounded analysis when useful, and carries a decision to a person without losing the evidence behind it.01 · Sense
Events and Pollers
Syslog, structured events, service checks, backups, containers, and infrastructure state.02 · Correlate
AEGIS SIEM
Normalization, service identity, incident lifecycle, deduplication, and deterministic triage.03 · Analyze
Watchman + Local AI
Novel or consequential evidence receives bounded interpretation and a proposed next action.04 · Deliver
PULSE → ntfy
Tracked delivery routes a rich decision to the right channel with resilient fallback.05 · Control
Human-Authorized Action
A person authorizes; software revalidates, remediates automatically, and reports the result.
Control principleThe automation performs the repair. The human controls whether it may proceed.
Project journey
How the system
became the system.
- 01
The need
The Lab Had Plenty of Telemetry and No Reliable Way to Know What Mattered.
Health checks, infrastructure logs, backup events, container state, and service failures were visible in different places. The real operational failure was not a lack of data; it was that important events could hide in noise or sit unnoticed until they became outages.
- 02
The starting point
Centralized Ingestion Made the Evidence Searchable—but Not Yet Actionable.
AEGIS first consolidated syslog, service health, and infrastructure events into one place. I added normalized severity, source identity, alert rules, state transitions, deduplication, cooldowns, acknowledgement, snooze, and escalation so the platform could manage an incident lifecycle rather than behave like another log viewer.
- 03
The engineering journey
The System Learned to Separate Recurring Noise From Meaningful Change.
A shared service catalog connected AEGIS health with the NEXUS command center. Deterministic triage groups repeated signatures, recognizes expected recurrence, and escalates novelty, bursts, and critical transitions. That reduces what reaches the expensive reasoning layer without asking a model to rediscover what counting and state machines already know.
- 04
The turning point
Local AI Became an Analyst Inside a Bounded Operational Loop.
Watchman and the local-agent evaluation pipeline can inspect unusual evidence, correlate related events, and recommend the next action. Autonomous rounds remain read-first: a model may request a repair, but deterministic software rechecks the claimed condition before any approved remediation. Evaluation and failure history are part of the loop so the agent can be measured rather than merely trusted.
- 05
The durable result
An End-to-End Path From Signal to Controlled Remediation.
AEGIS supplies the incident record, NEXUS supplies the fleet-wide operating view, local agents add bounded analysis, and PULSE tracks delivery to ntfy and other channels. Rich push actions let me acknowledge, investigate, or authorize a bounded automated remediation from a phone without turning the notification itself into an uncontrolled repair surface.
What remained
The durable
capability.
- One operational view across service health, infrastructure events, logs, and incident state
- Deterministic noise reduction before local-model analysis or human interruption
- Delivery tracking and channel routing through PULSE with a resilient direct-push fallback
- Human-controlled automated remediation from mobile notification through revalidation, execution, and result reporting
For technical readers
Engineering
judgment.
Use deterministic grouping and state transitions where they outperform probabilistic reasoning.
Let local AI recommend or request a repair; require human authorization and an independent precondition check before automated execution.
Keep PULSE as the delivery system and ntfy as a resilient downstream push channel rather than a second source of incident truth.
Maintain one service catalog across AEGIS and NEXUS so health identity cannot drift between the engine and the interface.
What this demonstrates
Full-Lifecycle Product and Operations Ownership
End-to-end product and operations ownership: observability, incident lifecycle, local-AI evaluation, notification reliability, safety boundaries, interfaces, and the lived work of running the platform.