Async LLM gap-analysis pipeline
Multi-stage agent pipeline that reviews regulatory documents and emits structured gap reports.
The problem
An enterprise compliance team had to manually review hundreds of internal policy documents against regulatory frameworks every quarter. Each review took weeks, the work was repetitive, and gaps slipped through because reviewers tired before the end of the document set.
Why standard tooling didn't fit
Off-the-shelf RAG pipelines give answers, but the team needed structured outputs: every regulatory clause mapped to internal policy coverage, every gap categorised by severity, every finding traceable to a source span. A chatbot couldn't deliver that. A simple LLM script couldn't run reliably across thousands of documents.
What we built
We split the review across a set of specialised agents. The organisation's security documentation is read once and made searchable. After that, the agents work through the framework one requirement at a time. One agent decides which specialist should judge a given control. That specialist weighs the internal evidence against what the requirement actually asks for. A final pass consolidates the verdicts into a single report, ranked by how exposed each gap leaves the organisation. Because no agent has to hold the whole framework at once, the review scales with the size of the document set. Every finding cites the language it came from, so a compliance lead can challenge a conclusion without redoing the reading behind it.
Outcomes
- Quarterly review cycle compressed from 3+ weeks to under 4 hours per framework.
- Reviewer time spent on judgement calls, not document scanning.
- Auditable trail per finding, with citations and intermediate reasoning preserved.