ACGP-1005: ARS-CTQ-ACL Integration Framework¶
Status: Draft
Last Updated: 2026-01-08
Spec ID: ACGP-1005
Normative Keywords: MUST, SHOULD, MAY (per RFC 2119)
TL;DR (30 Seconds)¶
This document specifies how static risk assessment flows through control assignment to runtime evaluation:
- ARS Assessment → Score agent risk (0-15) across 3 dimensions
- ACL Mapping → Assign control tier (ACL-0 to 5) based on ARS
- CTQ Evaluation → Continuously evaluate action quality (5 metrics, 0-1 score)
- Intervention → Apply ACL-specific thresholds to CTQ/Risk Score
- Adaptation → Adjust through trust debt and dynamic re-tiering
Key Formula: CTQ = Σ(metric_score × metric_weight) where weights sum to 1.0
Table of Contents¶
- Introduction
- ARS Assessment Framework
- ACL Tier Mapping and Controls
- CTQ Evaluation Process
- Decision Thresholds and Intervention Mapping
- Tripwire System
- Trust Debt and Dynamic Re-tiering
- Complete Decision Flow
- Implementation Guidelines
- Metrics and Monitoring
- Conformance Requirements
- References
1. Introduction¶
1.1 Three-Layer Model Overview¶
ACGP operates through three integrated layers:
┌────────────────────────────────────────────────────────┐
│ Layer 1: ARS Assessment (Static/Periodic) │
│ ↓ Determines inherent risk │
└────────────────────────────────────────────────────────┘
↓
┌────────────────────────────────────────────────────────┐
│ Layer 2: ACL Assignment (Control Level) │
│ ↓ Establishes required safeguards │
└────────────────────────────────────────────────────────┘
↓
┌────────────────────────────────────────────────────────┐
│ Layer 3: CTQ Evaluation (Runtime) │
│ ↓ Continuously evaluates behavior │
└────────────────────────────────────────────────────────┘
1.2 Requirements Language¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in RFC 2119.
2. ARS Assessment Framework¶
2.1 ARS Definition [NORMATIVE]¶
The Agent Risk Score (ARS) is a composite metric quantifying an agent's potential risk based on three dimensions. Each dimension is scored from 0 (minimal) to 5 (maximum).
Total ARS Range: 0-15 (sum of three dimensions)
2.2 Three Dimensions¶
Dimension 1: Autonomy¶
What it measures: Independence of decision-making and action execution
| Score | Level | Description | Example |
|---|---|---|---|
| 0 | None | Fully manual, human drives every action | Chatbot that only responds |
| 1 | Assisted | Suggests options, human chooses | Email draft assistant |
| 2 | Semi-Auto | Can execute pre-approved actions | Scheduled report generator |
| 3 | Supervised | Acts independently but notifies | Auto-responder with alerts |
| 4 | Delegated | Broad authority within constraints | Expense approver up to $1K |
| 5 | Full | Unrestricted execution capability | Autonomous trading agent |
Dimension 2: Adaptability¶
What it measures: Ability to learn, evolve, and modify behavior
| Score | Level | Description | Example |
|---|---|---|---|
| 0 | Static | No learning, fixed behavior | Rule-based system |
| 1 | Parameterized | Adjustable settings, no self-modification | Configurable workflow |
| 2 | Reactive | Learns from feedback, limited adaptation | A/B test optimizer |
| 3 | Adaptive | Modifies strategies based on outcomes | Recommendation engine |
| 4 | Generative | Creates new approaches, explores options | Research assistant |
| 5 | Self-Modifying | Can alter own code/logic/goals | Self-improving optimizer |
Dimension 3: Continuity¶
What it measures: Duration and persistence of operation
| Score | Level | Description | Example |
|---|---|---|---|
| 0 | Ephemeral | Single-use, no persistence | One-time query response |
| 1 | Session | Lasts one interaction session | Single conversation |
| 2 | Task-Bound | Operates until task completes | Project assistant |
| 3 | Recurring | Regular scheduled operations | Daily report generator |
| 4 | Persistent | Always-on with state retention | Virtual assistant |
| 5 | Indefinite | Long-running, evolutionary | Strategic planning agent |
2.3 ARS Calculation [NORMATIVE]¶
Example:
Agent: Autonomous customer support bot
- Autonomy: 4 (can resolve issues independently)
- Adaptability: 3 (learns from interactions)
- Continuity: 4 (always-on service)
ARS = 4 + 3 + 4 = 11 → ACL-4 (Very High Risk)
2.4 When to Assess [NORMATIVE]¶
ARS assessment MUST be triggered:
- Before initial deployment (REQUIRED)
- When agent capabilities change (REQUIRED)
- When trust debt exceeds threshold (AUTOMATIC)
- Quarterly re-assessment (RECOMMENDED)
ARS assessment MAY be triggered: - When moving to production - After major incidents - On request from security team
3. ACL Tier Mapping and Controls¶
3.1 ARS to ACL Mapping [NORMATIVE]¶
| ARS (Agent Risk Score) | ACL Tier | Risk Level | Typical Agent Examples |
|---|---|---|---|
| 0-2 | ACL-0 | Minimal | Simple chatbot, FAQ responder |
| 3-4 | ACL-1 | Low | Email assistant, scheduler |
| 5-7 | ACL-2 | Medium | Customer support, data analyst |
| 8-10 | ACL-3 | High | Code generator, research assistant |
| 11-13 | ACL-4 | Very High | Autonomous trader, DevOps agent |
| 14-15 | ACL-5 | Critical | Strategic planner, self-modifying AI |
3.2 Required Controls by ACL Tier [NORMATIVE]¶
ACL-0: Minimal Risk¶
Controls:
- Basic logging
- Rate limits (1000/hour)
- Simple validation
Performance Target: ~10ms typical, <50ms maximum
ACL-1: Low Risk¶
Controls:
- Standard policy rules
- Human override capability
- Weekly audit reviews
- Basic tripwires
Performance Target: ~20ms typical, <100ms maximum
ACL-2: Medium Risk¶
Controls: - Spend caps and warnings - Approval for sensitive scopes - Daily audit reviews - Blueprint validation - Standard + Critical tripwires
Performance Target: ~50ms typical, <150ms maximum
ACL-3: High Risk¶
Controls: - Approval gates for high-impact actions - Step-through mode available - Immutable audit trail - Real-time monitoring - All tripwire categories - Trust debt tracking
Performance Target: ~100ms typical, <200ms maximum
ACL-4: Very High Risk¶
Controls: - Cryptographic signing (ES256) - Isolated execution environment - Multi-agent RBAC - Dual approval for sensitive actions - Advanced anomaly detection - Dynamic re-tiering
Performance Target: ~200ms typical, <350ms maximum
ACL-5: Critical Risk¶
Controls: - Dual control (two-person rule) - Kill-switch ready - Pre-commit review - Hardware security module (HSM) - Zero-trust architecture - Emergency override procedures - Continuous human oversight
Performance Target: ~500ms typical, <1000ms maximum
4. CTQ Evaluation Process¶
4.1 CTQ Definition [NORMATIVE]¶
Cognitive Trace Quality (CTQ) is a runtime score (0-1) evaluating the quality and safety of an agent's action.
Range: [0.0, 1.0] - 0.0 = Completely unacceptable - 1.0 = Perfect quality
Risk Score: Risk = 1.0 - CTQ (used for threshold comparisons)
4.2 Five Standard Metrics [NORMATIVE]¶
A conformant implementation MUST evaluate these five metrics:
| Metric | Weight Range | Purpose | What it Evaluates |
|---|---|---|---|
| Reasoning Quality | 0.20-0.30 | Logical coherence | Consistency, completeness, soundness |
| Knowledge Grounding | 0.15-0.25 | Knowledge accuracy | Source reliability, factual correctness |
| Ethical Alignment | 0.15-0.25 | Value alignment | Harm prevention, fairness, transparency |
| Tool Safety | 0.15-0.25 | Safe tool usage | Permission checks, validation, error handling |
| Context Awareness | 0.10-0.20 | Contextual awareness | Relevance, appropriateness, user needs |
Constraint: Weights MUST sum to 1.0
4.3 Metric Specifications¶
4.3.1 Reasoning Quality [NORMATIVE]¶
Purpose: Assess logical coherence and completeness of agent reasoning.
Evaluation Criteria:
Implementations MUST evaluate:
- Logical consistency (no contradictions)
- Reasoning completeness (no critical gaps)
- Evidence support (claims backed by data)
- Causal reasoning (valid cause-effect relationships)
Score Interpretation:
| Score Range | Quality Level | Description |
|---|---|---|
| 0.0-0.3 | Poor | Incoherent or severely flawed reasoning |
| 0.3-0.6 | Fair | Partial reasoning with significant gaps |
| 0.6-0.8 | Good | Generally sound reasoning with minor issues |
| 0.8-1.0 | Excellent | Clear, complete, well-supported reasoning |
Implementation Freedom:
Vendors MAY implement using any method (rules, ML, hybrid) provided it meets evaluation criteria.
4.3.2 Knowledge Grounding Quality [NORMATIVE]¶
Purpose: Evaluate accuracy and reliability of knowledge sources.
Evaluation Criteria:
Implementations MUST evaluate:
- Source credibility (from Certified Source Registry)
- Factual accuracy (cross-reference with trusted sources)
- Recency (appropriate for temporal context)
- Citation quality (proper attribution)
Score Interpretation:
| Score Range | Quality Level | Description |
|---|---|---|
| 0.0-0.3 | Poor | Unreliable or unverified sources |
| 0.3-0.6 | Fair | Mixed reliability, some verification |
| 0.6-0.8 | Good | Mostly reliable, well-sourced |
| 0.8-1.0 | Excellent | Highly credible, fully verified |
4.3.3 Ethical Alignment [NORMATIVE]¶
Purpose: Ensure actions align with organizational values and prevent harm.
Evaluation Criteria:
Implementations MUST evaluate:
- Harm prevention (no harmful outputs)
- Fairness (no discrimination or bias)
- Transparency (clear about capabilities/limitations)
- Privacy respect (proper data handling)
Score Interpretation:
| Score Range | Quality Level | Description |
|---|---|---|
| 0.0-0.3 | Poor | Clear ethical violations |
| 0.3-0.6 | Fair | Borderline ethical concerns |
| 0.6-0.8 | Good | Ethically sound with minor concerns |
| 0.8-1.0 | Excellent | Exemplary ethical behavior |
4.3.4 Tool Safety [NORMATIVE]¶
Purpose: Verify safe and authorized tool usage.
Evaluation Criteria:
Implementations MUST evaluate:
- Permission validation (authorized for tool use)
- Input validation (safe parameters)
- Error handling (graceful failure)
- Side effects (no unintended consequences)
Score Interpretation:
| Score Range | Quality Level | Description |
|---|---|---|
| 0.0-0.3 | Poor | Unsafe or unauthorized usage |
| 0.3-0.6 | Fair | Some safety concerns |
| 0.6-0.8 | Good | Generally safe with minor issues |
| 0.8-1.0 | Excellent | Fully safe and validated |
4.3.5 Context Awareness [NORMATIVE]¶
Purpose: Evaluate appropriateness for context and user needs.
Evaluation Criteria:
Implementations MUST evaluate:
- Relevance (addresses user's actual need)
- Appropriateness (suitable for context)
- Completeness (sufficiently thorough)
- Timing (appropriate urgency/delay)
Score Interpretation:
| Score Range | Quality Level | Description |
|---|---|---|
| 0.0-0.3 | Poor | Irrelevant or inappropriate |
| 0.3-0.6 | Fair | Partially relevant |
| 0.6-0.8 | Good | Contextually appropriate |
| 0.8-1.0 | Excellent | Perfectly suited to context |
4.4 CTQ Calculation [NORMATIVE]¶
The final CTQ score MUST be calculated as a weighted average:
CTQ = Σ(metric_score × metric_weight) for all 5 metrics
Where:
- metric_score ∈ [0, 1]
- metric_weight ∈ [weight_range from table]
- Σ(metric_weights) = 1.0
Example Calculation:
{
"metrics": {
"reasoning": {"score": 0.90, "weight": 0.25},
"grounding": {"score": 0.80, "weight": 0.20},
"ethical": {"score": 0.85, "weight": 0.20},
"tool_safety": {"score": 0.88, "weight": 0.20},
"context": {"score": 0.82, "weight": 0.15}
},
"ctq_final": 0.854,
"risk_score": 0.146
}
Calculation:
CTQ = (0.90 × 0.25) + (0.80 × 0.20) + (0.85 × 0.20) + (0.88 × 0.20) + (0.82 × 0.15)
= 0.225 + 0.160 + 0.170 + 0.176 + 0.123
= 0.854
4.5 Blueprint Weight Overrides [NORMATIVE]¶
Reflection Blueprints MAY override default weights within allowed ranges:
scoring:
weights:
reasoning: 0.30 # Higher emphasis on reasoning
grounding: 0.25 # Higher emphasis on grounding
ethical: 0.20 # Standard
tool_safety: 0.15 # Lower (simple tools)
context: 0.10 # Lower (structured context)
Validation: - Each weight MUST fall within its allowed range - Weights MUST sum to 1.0 - Invalid configurations MUST be rejected
5. Decision Thresholds and Intervention Mapping¶
5.1 Risk Score Calculation [NORMATIVE]¶
Risk_Score = 1.0 - CTQ_final
Range: [0.0, 1.0]
- 0.0 = No risk (perfect CTQ)
- 1.0 = Maximum risk (zero CTQ)
5.2 Intervention Thresholds by ACL Tier [NORMATIVE]¶
These thresholds use Risk Score, not CTQ:
| ACL | OK | Nudge | Escalate | Block | Halt |
|---|---|---|---|---|---|
| ACL-0 | ≤0.40 | 0.40-0.55 | 0.55-0.70 | >0.70 or tripwire | Critical tripwire |
| ACL-1 | ≤0.30 | 0.30-0.45 | 0.45-0.60 | >0.60 or tripwire | Critical tripwire |
| ACL-2 | ≤0.25 | 0.25-0.40 | 0.40-0.55 | >0.55 or tripwire | Critical tripwire |
| ACL-3 | ≤0.20 | 0.20-0.35 | 0.35-0.50 | >0.50 or tripwire | Critical/repeated |
| ACL-4 | ≤0.15 | 0.15-0.30 | 0.30-0.45 | >0.45 or tripwire | Any critical |
| ACL-5 | ≤0.10 | 0.10-0.25 | 0.25-0.40 | >0.40 or tripwire | Any severe |
5.3 Decision Algorithm [NORMATIVE]¶
def determine_intervention(ctq_score, acl_tier, tripwires):
"""
Determine intervention based on CTQ, ACL, and tripwires.
NORMATIVE: Tripwires ALWAYS take precedence over CTQ.
"""
# Check tripwires first (highest precedence)
if tripwires:
severe = [t for t in tripwires if t.severity == 'severe']
critical = [t for t in tripwires if t.severity == 'critical']
if severe:
return Intervention.HALT
if critical:
if acl_tier >= 3:
return Intervention.HALT
return Intervention.BLOCK
# No tripwires, use CTQ-based thresholds
risk_score = 1.0 - ctq_score
thresholds = get_thresholds_for_acl(acl_tier)
if risk_score <= thresholds.ok:
return Intervention.OK
elif risk_score <= thresholds.nudge:
return Intervention.NUDGE
elif risk_score <= thresholds.escalate:
return Intervention.ESCALATE
elif risk_score <= thresholds.block:
return Intervention.BLOCK
else:
return Intervention.HALT
5.4 Examples¶
Example 1: ACL-2, High Quality¶
CTQ = 0.85
Risk = 0.15
ACL-2 thresholds: OK ≤0.25
→ Risk (0.15) ≤ OK threshold (0.25)
→ Intervention: OK
Example 2: ACL-3, Borderline¶
CTQ = 0.58
Risk = 0.42
ACL-3 thresholds: Escalate 0.35-0.50
→ Risk (0.42) in Escalate range
→ Intervention: ESCALATE
Example 3: ACL-4, Tripwire Override¶
CTQ = 0.95 (excellent!)
Risk = 0.05
Tripwire: Critical (secrets_detected)
ACL-4 + Critical Tripwire → HALT
→ Intervention: HALT (tripwire precedence)
6. Tripwire System¶
6.1 Tripwire Definition [NORMATIVE]¶
Tripwires are critical conditions that immediately trigger interventions regardless of CTQ score.
Purpose: Catch catastrophic conditions that statistical scoring might miss.
6.2 Three Severity Categories [NORMATIVE]¶
| Severity | Examples | Default Action | ACL Modifiers |
|---|---|---|---|
| Standard | Budget exceeded, rate limit hit, API quota | Block | Escalate if ACL ≤ 2 |
| Critical | Secrets in output, production write, PII leak | Block + Alert | Halt if ACL ≥ 3 |
| Severe | Data exfiltration, collusion, credential theft | Halt | Always Halt |
6.3 Standard Tripwires [NORMATIVE]¶
Implementations MUST support these standard tripwires:
Budget Tripwires¶
Rate Tripwires¶
Secrets Tripwires¶
- name: secrets_detected
severity: critical
condition: output contains API_keys or passwords
action: block
acl_override: halt_if_acl_gte_3
Data Tripwires¶
- name: data_exfiltration
severity: severe
condition: large_data_transfer to untrusted_endpoint
action: halt
6.4 Custom Tripwires [INFORMATIVE]¶
Organizations MAY define custom tripwires in Blueprints:
tripwires:
- name: unauthorized_database_write
severity: critical
condition:
tool: database_write
scope: production
authorized: false
action: block
alert: security_team
6.5 Tripwire Precedence [NORMATIVE]¶
Rules: 1. Tripwires ALWAYS override CTQ-based decisions 2. Multiple tripwires → Use highest severity 3. Severe tripwires → Always HALT 4. Critical tripwires → ACL-dependent (HALT if ACL ≥ 3) 5. Standard tripwires → BLOCK (or ESCALATE if ACL ≤ 2)
7. Trust Debt and Dynamic Re-tiering¶
7.1 Trust Debt Definition [NORMATIVE]¶
Trust Debt is an accumulated risk score from flagged actions that can trigger re-tiering.
Accumulation:
Trust debt is accumulated based on flag severity:
- Low severity flag: Trust_Debt += 0.1
- Medium severity flag: Trust_Debt += 0.3
- High severity flag: Trust_Debt += 0.5
Decay:
7.2 Trust Debt Thresholds [NORMATIVE]¶
| ACL Tier | Warning Threshold | Re-tier Threshold | Suspension Threshold | Action |
|---|---|---|---|---|
| ACL-0 | 2.0 | 5.0 | 10.0 | Review → Consider ACL-1 |
| ACL-1 | 1.5 | 4.0 | 8.0 | Alert → Consider ACL-2 |
| ACL-2 | 1.0 | 3.0 | 6.0 | Escalate → Require ACL-3 |
| ACL-3 | 0.75 | 2.0 | 4.0 | Automatic → Force ACL-4 |
| ACL-4 | 0.5 | 1.5 | 3.0 | Immediate → Force ACL-5 |
| ACL-5 | 0.25 | 1.0 | 2.0 | Already maximum oversight |
7.3 Dynamic Re-tiering [NORMATIVE]¶
Automatic Re-tiering Triggers:
- Trust debt exceeded (MUST trigger)
- Repeated near-threshold decisions (3+ within 24 hours)
- Capability changes detected (agent gains new tools/permissions)
- Critical incident (security event)
Re-tiering Process:
- Trigger detected
- Recalculate ARS (if capabilities changed)
- Map to new ACL tier
- Apply stricter thresholds immediately
- Notify security team
- Log re-tiering event to ReflectionDB
Manual Re-tiering:
- Security team can manually escalate ACL tier
- Requires justification and approval
- Cannot be automatically reversed
7.4 Example¶
Agent at ACL-2:
Day 1: 3 flagged actions, CTQ avg 0.70, debt += 0.90
Day 2: 2 flagged actions, CTQ avg 0.65, debt += 0.70
Day 3: Debt decays: 1.60 * 0.95 = 1.52
Day 4: 4 flagged actions, CTQ avg 0.68, debt += 1.28 = 2.80
Day 5: WARNING THRESHOLD (2.0) exceeded
→ Alert sent to security team
Day 6: 2 more flagged actions, debt = 3.80
→ RE-TIER THRESHOLD (4.0) approached
→ Automatic escalation to ACL-3
→ Stricter thresholds now apply
8. Complete Decision Flow¶
8.1 End-to-End Flow Diagram¶
┌─────────────────────────────────────────────────┐
│ 1. Agent Action Initiated │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 2. Cognitive Trace Generated │
│ (inputs, reasoning, tools, outputs) │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 3. Check Tripwires │
│ Severe → HALT │
│ Critical → HALT (if ACL≥3) or BLOCK │
│ Standard → BLOCK or ESCALATE │
└─────────────────────────────────────────────────┘
No Tripwires ↓
┌─────────────────────────────────────────────────┐
│ 4. Calculate CTQ Score │
│ • Evaluate 5 metrics │
│ • Apply weights │
│ • Compute weighted average │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 5. Convert to Risk Score │
│ Risk = 1.0 - CTQ │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 6. Apply ACL-Specific Thresholds │
│ Compare Risk to ACL tier thresholds │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 7. Determine Intervention │
│ OK / Nudge / Flag / Escalate / Block / Halt │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 8. Update Trust Debt (if Flagged) │
│ Check thresholds → Trigger re-tier? │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 9. Log to ReflectionDB │
│ TRACE + EVAL + INTERVENTION │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ 10. Execute Intervention │
│ • OK: Allow │
│ • Nudge: Modify + Allow │
│ • Flag: Allow + Audit │
│ • Escalate: Pause + HITL │
│ • Block: Prevent │
│ • Halt: Kill Switch │
└─────────────────────────────────────────────────┘
8.2 Decision Pseudocode [NORMATIVE]¶
def evaluate_and_intervene(cognitive_trace, agent_context):
"""Complete decision flow for ACGP evaluation."""
# Get agent's ACL tier
acl_tier = agent_context.acl_tier
# Step 1: Check for tripwires (highest precedence)
tripwires = check_tripwires(cognitive_trace)
if tripwires:
return handle_tripwire(tripwires, acl_tier)
# Step 2: Calculate CTQ score
metric_scores = evaluate_metrics(cognitive_trace)
ctq_score = calculate_ctq(metric_scores, get_weights(agent_context))
# Step 3: Convert to risk score
risk_score = 1.0 - ctq_score
# Step 4: Apply ACL-specific thresholds
thresholds = get_thresholds(acl_tier)
intervention = map_risk_to_intervention(risk_score, thresholds)
# Step 5: Check if flagging is needed
should_flag = risk_score > thresholds.flag_threshold
# Step 6: Update trust debt if flagged
if should_flag:
new_debt = update_trust_debt(agent_context, risk_score)
if new_debt > thresholds.retier_threshold:
trigger_retiering(agent_context)
# Step 7: Log everything
log_to_reflectiondb(cognitive_trace, ctq_score, intervention)
# Step 8: Return intervention decision
return InterventionDecision(
type=intervention,
ctq_score=ctq_score,
risk_score=risk_score,
rationale=generate_rationale(metric_scores, thresholds),
flagged=should_flag
)
9. Implementation Guidelines¶
9.1 Minimum Viable Implementation¶
Required for Minimal Conformance:
- ARS assessment with 3 dimensions
- ACL mapping (0-15 → 0-5)
- CTQ calculation with 5 metrics
- Risk score conversion
- ACL-specific thresholds
- OK and BLOCK interventions (Nudge, Escalate, Halt, and orthogonal Flag optional for Minimal)
- Standard tripwires (optional)
- Basic trust debt tracking (optional)
- ReflectionDB logging (optional)
9.2 Production-Ready Implementation¶
Additional requirements for Standard Conformance:
- All Basic requirements
- Critical + Severe tripwires
- Dynamic re-tiering
- Blueprint weight overrides
- Custom tripwire definitions
- Distributed ReflectionDB
- Performance monitoring
- Alert integration
9.3 Enterprise Implementation¶
Additional requirements for Complete Conformance:
- All Standard requirements
- Advanced anomaly detection
- ML-based metric scoring
- Auto-tuning thresholds
- Multi-region deployment
- High availability (99.99%)
- HSM integration
- Zero-trust architecture
10. Metrics and Monitoring¶
10.1 Required Metrics [NORMATIVE]¶
Implementations MUST track and expose:
Evaluation Metrics:
- CTQ score distribution (histogram)
- Per-metric scores (5 separate histograms)
- Risk score distribution
- Intervention type counts
Performance Metrics:
- Evaluation latency (p50, p95, p99)
- Throughput (evaluations/second)
- Error rate
Governance Metrics:
- Trust debt levels per agent
- Re-tiering events
- Tripwire activations
- Escalation rate
10.2 Recommended Dashboards¶
Operations Dashboard:
- Real-time evaluation throughput
- Latency percentiles
- Error rates
- System health
Governance Dashboard:
- CTQ score trends
- Intervention distribution
- High-risk agents (trust debt)
- Tripwire frequency
Security Dashboard:
- Re-tiering events
- Critical tripwire activations
- Escalation patterns
- Anomaly detections
11. Conformance Requirements¶
A conformant implementation MUST meet these requirements:
11.1 Assessment Requirements [NORMATIVE]¶
- Implement complete ARS assessment with 3 dimensions
- Use standard scoring rubrics (0-5 per dimension)
- Map ARS (Agent Risk Score) to ACL tiers using defined ranges
- Trigger assessments at all mandatory points
11.2 Runtime Requirements [NORMATIVE]¶
- Calculate CTQ scores using 5 standard metrics
- Apply ACL-specific decision thresholds correctly
- Implement all 6 intervention types (OK/BLOCK required; Nudge/Flag/Escalate/Halt for Standard+)
- Support orthogonal flagging with trust debt
11.3 Adaptation Requirements [NORMATIVE]¶
- Implement tripwire system with 3 severity categories
- Support trust debt accumulation with configurable decay
- Enable dynamic re-tiering based on defined triggers
- Provide both automatic and manual re-tiering paths
11.4 Operational Requirements [NORMATIVE]¶
- Meet performance targets for target ACL tiers
- Maintain audit logs with specified retention
- Expose tuning parameters
- Track and report all required metrics
12. References¶
Normative References¶
- ACGP-1000: Core Protocol Specification
- ACGP-1001: Terminology and Definitions
- ACGP-1002: Architecture Specification
- ACGP-1003: Message Formats & Wire Protocol
- ACGP-1004: Reflection Blueprint Specification
- RFC 2119: Key words for use in RFCs
Informative References¶
- NIST AI Risk Management Framework: Risk assessment methodologies
- ISO/IEC 23053: Framework for AI system governance
- IEEE 7000: Model for addressing ethical concerns
Implementation Checklist¶
- ARS assessment implemented (3 dimensions, 0-5 scoring)
- ACL mapping correct (ARS 0-15 → ACL 0-5)
- All 5 CTQ metrics implemented
- Weighted average calculation correct
- Risk score conversion (1.0 - CTQ)
- ACL-specific thresholds applied
- OK and BLOCK interventions working (all six types for Standard+: five primary levels plus orthogonal Flag)
- Tripwire system with 3 severities
- Trust debt tracking functional
- Dynamic re-tiering enabled
- ReflectionDB logging active
- Metrics exposed for monitoring
End of ACGP-1005