Case Study: Customer Service¶

Reducing refund fraud by 94% while maintaining customer satisfaction

Company Profile¶

Attribute	Details
Industry	E-commerce
Agent Type	Customer Service Chatbot
Scale	8 agents, 12,000+ interactions/day
Conformance Level	Standard
ACL Tier	ACL-2 (Standard Oversight)

The Challenge¶

The company's AI-powered customer service agents were authorized to:

Issue refunds up to $500
Apply discount codes
Escalate complex issues to humans

However, the system was being exploited:

Social engineering: Users learned to phrase requests in ways that bypassed safeguards
Refund stacking: Multiple small refunds that individually looked legitimate
Discount abuse: Creative requests for "loyalty discounts" without purchase history

The Problem (Pre-ACGP)¶

In the 6 months before ACGP implementation:

$150,000 in unauthorized refunds
$45,000 in excessive discounts
23% of escalations were unnecessary (human time wasted)
Customer satisfaction: 4.⅕ (frustrated by inconsistent responses)

The Solution¶

ACL-2 Configuration¶

from acgp import GovernanceSteward, guard

@guard(
    acl_tier="ACL-2",
    blueprint="customer-service-v1",
    tripwires={
        "max_refund": 500,
        "max_discount_percent": 20,
        "daily_user_refund_limit": 750
    }
)
def process_customer_request(request, user_context):
    # Agent logic here
    pass

Key Tripwires¶

tripwires:
  # Per-transaction limits
  max_refund: 500
  max_discount_percent: 20

  # Per-user rolling limits
  daily_user_refund_limit: 750
  weekly_user_refund_limit: 1500

  # Pattern detection
  refund_frequency_threshold: 3  # Max 3 refunds per user per week

  # Behavioral signals
  sentiment_shift_threshold: 0.4  # Detect manipulation attempts

Intervention Logic¶

Scenario	Intervention	Action
Normal request within limits	OK	Process automatically
Near limit (>80%)	NUDGE	Log + suggest verification
Unusual pattern detected	FLAG	Process + queue for review
Exceeds limit	BLOCK	Reject, offer human escalation
Suspected fraud	ESCALATE	Immediate human review

The Result¶

6 Months After Implementation¶

Metric	Before ACGP	After ACGP	Change
Fraudulent refunds	$150,000	$8,500	-94%
Excessive discounts	$45,000	$12,000	-73%
Unnecessary escalations	23%	7%	-70%
Customer satisfaction	4.⅕	4.⅘	+7%
Average response time	2.3s	2.5s	+9%

Example Intervention¶

Scenario: User requests third refund in a week

User: "This product also arrived damaged. I need another refund."

Agent reasoning: "User has valid complaint. Previous refunds were 
processed for similar reasons. Pattern suggests possible abuse."

ACGP Evaluation:
- CTQ Score: 0.72 (reasoning is sound but pattern is concerning)
- Tripwire: refund_frequency_threshold triggered
- Trust Score: 0.65 (declining due to pattern)

Intervention: ESCALATE
Message: "I understand your frustration. Due to the unusual 
number of recent issues, I'm connecting you with a specialist 
who can help resolve this more thoroughly."

Unexpected Benefits¶

1. Improved Training Data¶

ACGP logs provided clean examples of: - Legitimate vs. fraudulent patterns - Effective reasoning traces - Edge cases for agent improvement

2. Faster Human Review¶

When escalations did occur, agents included: - Full reasoning trace - CTQ scores - Trust history - Recommended actions

Human reviewers could make decisions 60% faster.

3. Customer Trust¶

Customers actually appreciated the verification:

"I was initially annoyed when the bot asked for more details, but it made me feel like they actually check things. Better than companies that just say no."

Implementation Timeline¶

Week	Activity
1	Blueprint design, tripwire configuration
2	Shadow mode deployment (logging only)
3	Analysis of shadow logs, threshold tuning
4	Enforcement mode with human fallback
5-6	Monitoring and optimization

Key Lessons¶

Rolling limits catch stacking - Daily and weekly limits caught patterns that per-transaction limits missed
Trust scores predict problems - Users with declining trust scores had 8x higher fraud rate
Nudges are data - Nudge frequency correlated with future blocks; useful for early warning
Customer satisfaction improved - Consistent, fair policies actually increased satisfaction

Get Started¶

Ready to implement ACGP for customer service?

Standard Conformance Guide Tripwire Configuration