DialFork

Benchmarks for procedural hallucination in conversational agents

GitHub · Dataset · Leaderboard

LLM agents are increasingly asked to carry out procedures: book the appointment, verify the identity, file the report. They can stay fluent, polite and factually correct while quietly stopping to follow those procedures. We call this procedural hallucination, and DialFork measures it on raw dialogue text, with labels that need no manual annotation.

The code to evaluate your methods, the paper's methods and the generation pipeline are on GitHub.

_{Made with 💛 at Idiap Research Institute · Happy forking! 🔱}