Why ACGP Exists¶
The Problem: Autonomous Agents Can Go Wrong¶
AI agents are increasingly making autonomous decisions—from customer service to financial transactions to infrastructure management. But what happens when they make mistakes?
Real Incidents (Anonymized)¶
Incident 1: The $10K AWS Bill¶
What happened: - Autonomous agent had AWS EC2 access for scaling infrastructure - Bug in cost estimation logic triggered runaway scaling - Spun up 200 GPU instances in 2 hours - Developer noticed after getting AWS billing alert - Cost: $10,000 in 2 hours
Root cause: No budget tripwire, no escalation for unusual spending patterns
How ACGP prevents this:
tripwires:
- id: "daily_aws_spending"
condition: "daily_total > 1000"
action: "HALT"
message: "Daily AWS spending limit exceeded"
Incident 2: The Refund Bot¶
What happened: - Customer service AI had authority to issue refunds up to $500 - Missing upper limit validation in production deployment - Approved 90% of refund requests over 2 weeks - Average refund: \(250 (baseline was ~\)50) - Cost: $150,000 before detection
Root cause: No anomaly detection, overly permissive policy, no escalation
How ACGP prevents this:
policies:
- action: issue_refund
if: "amount > 100"
then: "ESCALATE"
message: "Refunds over $100 require manager approval"
tripwires:
- id: "daily_refund_total"
condition: "daily_refunds > 2000"
action: "HALT"
Incident 3: The Trading Algorithm¶
What happened: - Algorithmic trading bot with autonomous execution - Flash crash triggered failsafe bug - Executed 10,000 panic sells in 45 seconds - Market order slippage compounded losses - Loss: $2.3M in 45 seconds
Root cause: No circuit breaker, no human-in-the-loop for massive positions
How ACGP prevents this:
governance_contract:
risk_level: "critical_risk"
performance_budget:
latency_budget_ms: 1000
fallback_behavior: "deny"
rules:
- action: "execute_trade"
if: "position_size > 100000"
then: "ESCALATE"
requires: "human_approval"
The Solution: Cognitive Governance¶
ACGP provides runtime governance that evaluates agent decisions before they execute, using:
1. Automatic Budget Tripwires¶
Problem: Agents can rack up costs before anyone notices
Solution: Real-time budget tracking with automatic halt
Result: Catch runaway costs in seconds, not hours
2. Action Limits by Risk¶
Problem: One-size-fits-all permissions are too coarse
Solution: Graduated interventions based on action risk
policies:
- action: issue_refund
if: amount <= 50
then: OK
- action: issue_refund
if: 50 < amount <= 500
then: ESCALATE
- action: issue_refund
if: amount > 500
then: BLOCK
Result: Low-risk actions proceed instantly, high-risk actions get review
3. Complete Audit Trail¶
Problem: Can't reconstruct what happened or why
Solution: Every decision logged with reasoning
trace = CognitiveTrace(
reasoning="Customer reported defective product...",
action="issue_refund",
parameters={"amount": 250, "order_id": "12345"}
)
result = steward.evaluate(trace)
# Logged to ReflectionDB:
# - Full reasoning chain
# - Decision and intervention
# - CTQ score and risk assessment
# - Human override (if escalated)
# - Timestamp and session context
Result: Full forensic capability for compliance and debugging
4. Gradual Autonomy¶
Problem: New agents are untested but need autonomy
Solution: Earn trust through good behavior
# New agent starts with low trust (ACL-2)
initial_acl = "ACL-2" # More oversight, higher latency
# After 30 days of good behavior:
# - 0 interventions
# - CTQ score consistently >0.85
# - No tripwire violations
# Auto-promoted to ACL-3
current_acl = "ACL-3" # More autonomy, lower latency
Result: Risk decreases as agents prove themselves
What ACGP Prevents¶
| Risk | Without ACGP | With ACGP |
|---|---|---|
| Runaway costs | Detected in billing cycle (days/weeks) | Halted in seconds via tripwires |
| Policy violations | Discovered in audit (after damage) | Prevented before execution |
| Anomalous behavior | Manual review of logs | Automatic flagging and escalation |
| Catastrophic errors | Hope and pray | Circuit breakers and HALT |
| Compliance gaps | Reconstruct from logs (maybe) | Complete audit trail always |
When You Need ACGP¶
Perfect for:¶
- Autonomous agents making decisions without constant supervision
- High-risk actions like financial transactions, data deletion, privilege grants
- Adaptive AI that learns and changes behavior
- Compliance requirements needing decision audit trails
- Multi-agent systems requiring coordinated governance
Consider carefully:¶
- Latency-sensitive operations (<10ms requirements)
- Read-only agents with no side effects
- Simple scripted bots with hard-coded logic
Not needed:¶
- Static automation with no decision-making
- Systems with existing robust human oversight
- Prototypes with no real-world consequences
Real-World Use Cases¶
Customer Service Automation¶
- Risk: Agents approve refunds, discounts, policy exceptions
- ACGP Solution: Escalate refunds >\(100, halt if daily total >\)5000
- Impact: Reduced fraud by 87%, caught 12 policy violations in first month
Infrastructure Management¶
- Risk: Agents scale resources, modify configurations, deploy code
- ACGP Solution: Block production changes without approval, halt on unusual spending
- Impact: Prevented 3 potential outages, saved $50K in cloud costs
Financial Trading¶
- Risk: Agents execute trades, adjust positions, manage risk
- ACGP Solution: Escalate large positions, deny on margin calls, circuit breakers
- Impact: Zero incidents in 18 months vs 4 incidents in prior year
Data Operations¶
- Risk: Agents query databases, export data, modify schemas
- ACGP Solution: Flag large exports, block production modifications, escalate privilege changes
- Impact: 100% compliance with data privacy regulations
Getting Started¶
Ready to add governance to your agents?
-
5-Minute Start
Add basic governance in 5 minutes with Minimal conformance
-
Production Deploy
Full governance with contracts, tripwires, and audit trails
-
Calculate Latency
Plan your governance contracts and latency budgets
-
Learn Concepts
Understand ARI framework, ACL tiers, and interventions