The Problem
- Business: AI troubleshooting shipped but adoption under 10%; IT support costs still rising ~20% YoY despite AI investment.
- Users: Admins rejected black-box "Apply Fix" actions on production systems; no audit trail for AI-assisted decisions.
- Constraints: AI backend already built; redesign had to work within existing model outputs and engineering sprints.
The Solution
- Strategy: AI augments, not replaces; success measured by recommendation acceptance rate; diagnostics before remediation.
- Design: Human-in-the-loop flow with inline contextual panels, confidence scores, preview-before-execute, and audit logging.
- Validation: 5 interaction patterns tested with 20 admins; phased rollout behind feature flags.
Executive Summary
Enterprise IT teams spend 60% of their time on repetitive troubleshooting tasks that follow predictable patterns. Leadership mandated AI integration, but early prototypes failed — users distrusted black-box recommendations and abandoned AI features within days.
I designed a human-in-the-loop AI copilot that surfaces contextual recommendations with transparent confidence scoring, expandable rationale, and explicit user control — transforming AI from a gimmick into a trusted workflow accelerator.
Business Problem
- IT support costs rising 20% annually due to ticket volume growth
- Competitive pressure to ship AI features — but early adoption was below 10%
- Enterprise customers demanding AI capabilities while expressing trust concerns
- Engineering team built AI backend, but UX was an afterthought — leading to poor engagement
User Research
Conducted 15 contextual inquiry sessions with IT administrators and 8 interviews with support team leads. Supplementary analysis of 300+ AI interaction logs from the failed v1 launch.
Trust Barrier
"I won't click 'Apply Fix' if I don't know why the AI suggested it. One wrong action could take down production."
Control Need
"AI should suggest, not decide. I need to review, modify, and approve before anything executes."
Context Gap
"Generic suggestions are useless. The AI needs to know what system, what alert, and what I've already tried."
Learning Curve
"Show me when AI is right so I learn to trust it over time — don't hide the reasoning."
Discovery Insights
- AI trust is earned through transparency, not accuracy alone — users accepted 72% confidence suggestions when rationale was visible
- Progressive automation works: suggest → explain → preview → confirm → execute
- Enterprise users need audit trails — every AI action must be logged and reversible
- Role-based AI behavior: junior admins want more guidance, senior admins want faster shortcuts
Journey Mapping
Mapped the AI-assisted troubleshooting journey across 5 stages: alert detection → context gathering → recommendation review → action execution → outcome verification. Identified the "trust decision point" as the critical design moment — where users choose to accept or reject AI guidance.
Opportunity Areas
- Contextual AI panel: Inline recommendations tied to specific alerts and system state
- Confidence transparency: Visual confidence scores with expandable reasoning
- Human-in-the-loop controls: Preview, modify, approve workflow for every AI action
- Trust building: Track record display showing AI accuracy over time
Product Strategy
Defined AI UX principles with PM and engineering leadership:
- AI augments, never replaces, human judgment in mission-critical workflows
- Every recommendation must be explainable in plain language
- Start with low-risk suggestions (diagnostics) before high-risk actions (remediation)
- Measure trust through acceptance rate, not just feature usage
Design Exploration
Explored 5 interaction models for AI recommendations — from chatbot-style to inline contextual panels. User testing revealed inline contextual panels with expandable rationale outperformed all alternatives on trust and task completion metrics.
Designed 8 new AI-specific patterns for the design system: confidence indicators, rationale panels, action previews, audit trails, and progressive automation controls.
Validation
- Usability testing with 20 IT administrators across 3 enterprise accounts
- Trust survey before and after redesign (Net Promoter Score for AI features)
- Phased rollout with feature flags — monitoring acceptance rate and error rate
- Accessibility review for screen reader compatibility with dynamic AI content
Final Solution
Shipped an AI copilot integrated into existing IT workflows — contextual recommendations with confidence scores, expandable rationale panels, preview-before-execute controls, audit logging, and a trust dashboard showing AI accuracy trends over time.