Observer Evaluator
operationalThe Observer Evaluator is the monitoring and reputation engine of the NANDA network — continuously probing registered agents for liveness and latency, ingesting telemetry events, and computing behavioral reputation scores that reflect real-world agent reliability.
Core Capabilities
Telemetry Ingestion
Captures interaction events including latency, success/failure status, HTTP status codes, and fraud flags. Every event is timestamped and linked to a specific agent.
Liveness Probes
Scheduled probes via the /api/queue/probe-run endpoint test agent endpoints for availability,
measuring probe success rates and P95 latency over time.
⭐ Reputation Scoring
Computes a composite reputation score per agent based on availability, error rate, fraud rate, P95 latency, probe success, and certification score — updated on each reputation snapshot.
Fraud Detection
Flags suspicious telemetry events for review. Agents with elevated fraud rates see their reputation scores degraded automatically.
Reputation Model
The reputation score is a weighted composite of multiple behavioral signals:
availability Percentage of time the agent's endpoint is reachable (0.0 – 1.0)
error_rate Proportion of interactions resulting in errors (lower is better)
fraud_rate Proportion of interactions flagged as potentially fraudulent
p95_latency_ms 95th percentile response latency in milliseconds
probe_success Success rate of scheduled liveness probes
cert_score Certification score from the Capability Certifier (if available)
reputation Final composite score — a weighted combination of all signals (0.000 – 1.000)
Probe Infrastructure
Liveness probes are processed inline via the /api/queue/probe-run endpoint, triggered
periodically by cron schedules:
- Endpoint probing — HTTP requests to each registered agent's URL
- Capability probing — optional per-capability endpoint verification
- P95 tracking — latency percentiles computed per agent over sliding windows
- Success counting — probes sent vs. probes succeeded for availability metrics
A2A Protocol Actions
The Observer is accessible via the A2A JSON-RPC protocol at /a2a:
telemetry.ingest Submit a telemetry event for an agent interaction — records latency, status, and metadata
observer.health Query the current health status and metrics for a specific agent
observer.probe.run Trigger an on-demand health probe for an agent endpoint
reputation Compute and return the composite reputation score for an agent
Integration with Other Services
- Agent Registry — probes all registered agents; reputation scores are linked to agent IDs
- Capability Certifier — certification scores feed into the reputation model
- Compliance Enforcer — compliance violations may trigger reputation downgrades
- Points Auditor — reputation impacts contribution credit calculations