Talk
Lessons from Building an AI Audit Assistant
Auditors review more than 50 documents and answer up to 450 checklist questions before each audit. Preparation can take five days. We built Audit Assist, a production AI product that cut that time to hours. An aviation audit firm that runs more than 70 audits each year now uses it. While building it, we found that several common AI product patterns broke down. Standard RAG struggled with long, multilingual documents containing tables, images and flowcharts. Experienced auditors needed evidence they could inspect, rather than polished answers they were asked to trust. A chat interface also failed to match how auditors move from source documents to checklist responses and findings. I will share the approaches that failed, the changes that worked, and how working alongside auditors reshaped the original product brief. I will explain how we let auditors trace every finding back to its source and why the interface mattered as much as the underlying models.
