Building AI agents in sensitive financial enterprises

Episode Highlights

• •Qamar's team built an in-house AI platform to assist agents with sensitive debt collection conversations at scale.
• •The AI acts as a co-pilot for human agents, using a human in the loop approach to flag vulnerable customers and automate tasks.
• •They chose to fine-tune their own models in-house to maintain strict compliance and have full control over quality.
• •Out-of-the-box foundation models were not viable for their use case, showing only 20-30% accuracy on real conversations.
• •Having a large, high-quality dataset of existing, anonymized conversations was the key to successfully training their models.
• •The team developed custom internal tools to manage datasets and evaluate model performance because open-source options were not suitable.
• •Continuous evaluation is critical, as shown when they created a new complaint intent after discovering misclassifications in production data.
• •AI maintenance is fundamentally different from traditional software because models are non-deterministic and can give different outputs for the same input.
• •Qamar argues that a detail-oriented QA mindset is the most important skill for an AI engineer to handle ambiguity and find failure points.