The Firefly project applies generative AI to monitor, evaluate, and validate customer-facing communications and documents across support, sales, and marketing channels. Its goal is to ensure that conversations conducted with or by AI comply with internal ethical guidelines and external regulatory standards. The system is designed as an AI-powered tool for Model Risk Management (MRM) Line 2 teams—responsible for defining and validating compliance standards within the traditional "Three Lines of Defense" framework in financial risk management. In its initial release (Version 1), Firefly enables managers to review real and synthetic conversation transcripts using AI agents, scoring them based on compliance risk and highlighting potential violations. Key Features / Highlights - Ingestion and analysis of real and synthetic customer-agent transcripts - Compliance scoring based on natural language assertions converted to AI prompts - Custom prompt engine using Retrieval-Augmented Generation (RAG) architecture - Support for various communication formats (email, chat, phone, autodialer) - Metadata-aware processing (e.g., jurisdiction, agent ID, sales context) - Modular architecture with AI agent plug-in support (ChatGPT, LLaMA, Gemini, etc