DECODA: A French Dataset for Spoken Dialogue Summarization

Dataset Description

We introduce an extended and standardized version of the DECODA corpus, designed to support research on automatic dialogue summarization, particularly in the context of task-oriented human-human spoken conversations in French.

The original DECODA corpus (Bechet et al., 2012) consists of real-world telephone conversations between agents of the Paris public transport call centre (RATP) and users of bus and metro services. Each conversation is associated with a manual transcription and, depending on the corpus partition, a human-written synopsis. The conversations cover various call types, such as itinerary requests and transport information inquiries, and contain domain-specific entities associated with a semantic ontology, including concepts related to transport and schedules.

The original corpus is divided into three parts that we called: DECODA-1, DECODA-2 and DECODA-3. While DECODA-1 contains short synopses written in a bullet-point style, DECODA-3 provides more detailed, literary-style synopses. DECODA-2 does not contain human-annotated summaries.

Our contribution builds upon this resource by providing five dataset configurations that address different challenges in dialogue summarization, including limited human annotations, differences in summary writing styles, data augmentation and the use of manual and automatic speech recognition (ASR) transcripts.

Dataset Contributions

We propose five complementary dataset configurations.

1. Decoda version v1-hum

This configuration contains only human-annotated data from DECODA-3, where summaries are written in a literary style. It provides a reference setting based exclusively on manually transcribed conversations and human-written synopses.

2. Decoda version v1-hum-bullet

This configuration combines data from DECODA-1 and DECODA-3. It brings together two different summary styles: bullet-point summaries from DECODA-1 and literary-style summaries from DECODA-3. This configuration makes it possible to investigate the impact of heterogeneous reference-summary styles on dialogue summarization.

3. Decoda version v1-hum-norm

This configuration also combines DECODA-1 and DECODA-3 but introduces a summary normalization step. Since DECODA-1 summaries are originally written in a bullet-point format, they are rewritten using ChatGPT in a one-shot generation setting to obtain a more literary style, closer to the summaries available in DECODA-3. The resulting dataset provides more stylistically consistent training targets while retaining the original human-written summaries from DECODA-3.

The summary field provides the reference summary to be used for training: normalized summaries for DECODA-1 and original human-written summaries for DECODA-3.

4. Decoda version v2-aug

This configuration extends the corpus by incorporating data from all three original DECODA subsets, while retaining manual transcriptions.

It combines:

• Human-written summaries from DECODA-3.
• Automatically generated summaries for DECODA-1 and DECODA-2, produced using ChatGPT 3.5 in a one-shot generation setting.
• Manual transcriptions of the conversations.

The automatically generated summaries aim to reproduce the literary style of DECODA-3, providing additional training data for dialogue summarization. This configuration makes it possible to investigate the impact of data augmentation while maintaining manually transcribed conversational input.

The summary field combines automatically generated summaries for DECODA-1 and DECODA-2 with human-written summaries from DECODA-3.

5. Decoda version v2-auto

This configuration uses the same data augmentation strategy as the previous configuration but relies on automatic speech recognition (ASR) transcripts instead of manual transcriptions. The automatic transcriptions are generated using WhisperX Large, allowing summarization models to be evaluated under more realistic conditions, where manually corrected transcripts are not necessarily available.

It provides two transcription formats:

• text_spk: Automatic transcription with speaker markers but without explicit conversational roles.
• text_no_spk: Automatic transcription without speaker or role markers, with dialogue turns separated by line breaks.

As in the previous configuration, the dataset combines automatically generated summaries for DECODA-1 and DECODA-2 with human-written summaries from DECODA-3.

This configuration makes it possible to investigate the robustness of dialogue summarization models when processing noisy or imperfect ASR-generated transcripts.

Dataset Structure

The dataset provides several complementary representations of each conversation, together with summary and task-related annotations.

Field	Description
filename	Unique identifier of the conversation.
text_role	Manual transcription with explicit speaker roles.
text_spk	Manual/Automatic transcription with speaker markers.
text_no_spk	Manual/Automatic transcription without speaker or role markers. Dialogue turns are separated by line breaks.
summary	Combining normalized or automatically generated summaries with original human-written summaries, depending on the dataset configuration.
calltypes	Type or purpose of the call.
label	Label associated with the call type.
part	Original DECODA subset from which the conversation originates.

Not all fields are available in every configuration. The additional normalization and ASR-related fields are provided only where relevant.

Train, Validation and Test Splits

All five configurations share the same test set of 200 examples. This ensures consistent evaluation conditions and enables fair comparisons between models trained on different dataset configurations. The different training and validation set sizes reflect the progressive integration of additional annotated and automatically generated data.

Dataset configuration	Training	Validation	Test
v1-hum	200	98	200
v1-hum-bullet	697	227	200
v1-hum-norm	697	227	200
v2-aug	1,390	396	200
v2-auto	1,390	396	200

Citation information

@inproceedings{bechet2012decoda,
  title     = {{DECODA}: A Call-Centre Human-Human Spoken Conversation Corpus},
  author    = {Bechet, Frederic and Maza, Benjamin and Bigouroux, Nicolas and Bazillon, Thierry and El-Beze, Marc and Mori, Renato and Arbillot, Emilie},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC)},
  pages     = {1343--1347},
  year      = {2012},
  url       = {https://aclanthology.org/L12-1399/}
}

@inproceedings{akani2024unified,
  title     = {Unified Framework for Spoken Language Understanding and Summarization in Task-Based Human Dialog Processing},
  author    = {Akani, E. and Bechet, F. and Favre, B. and Gemignani, R.},
  booktitle = {Proceedings of Interspeech},
  pages     = {3535--3539},
  year      = {2024},
  doi       = {10.21437/Interspeech.2024-2276}
}

@misc{akani2024increasing,
  title   = {Increasing Faithfulness in Human-Human Dialog Summarization with Spoken Language Understanding Tasks},
  author  = {Akani, E. and Favre, B. and Bechet, F. and Gemignani, R.},
  journal = {arXiv preprint arXiv:2409.10070},
  year    = {2024},
  url     = {https://arxiv.org/abs/2409.10070}
}