Please briefly describe your work and intended use. Your responses help us review access requests and understand future dataset needs. Do not include confidential or sensitive information.

Dataset Card for otoSpeech-full-duplex-turn-104h

Contact

Website: https://oto.earth
Email: agent@oto.earth

Dataset Summary

otoSpeech-full-duplex-turn-104h is an English, full-duplex conversational speech dataset for research on turn-taking and related spoken-dialogue phenomena. It contains 420 two-speaker conversations totaling approximately 104.94 hours. Each conversation includes time-aligned, channel-separated audio, a stereo combined recording, per-speaker transcripts with turn-event annotations, and session metadata.

The conversations preserve natural phenomena such as overlaps, interruptions, backchannels, fillers, laughter, floor holds, and non-speech events. This dataset is also released as TurnBench - Training Set.

Languages

English (en)

Intended Use

The dataset is intended for non-commercial research and education involving, for example:

turn-taking, endpointing, and end-of-turn prediction;
interruption, overlap, floor-hold, and backchannel detection;
conversational speech recognition and transcript modeling;
full-duplex spoken-dialogue modeling;
acoustic modeling, diarization within a recording, source separation, and related speech research; and
benchmarking and evaluation of research systems.

Use is subject to the oto Speech License v1.0. In particular, the license prohibits voice-identity uses, including voice cloning, speaker identification or verification, biometric matching, and re-identification. Production use and other commercial uses require a separate written license. For-profit organizations must receive written approval for a defined research project before using the dataset.

How to Use

This is a manually gated dataset. Request access on the repository page and authenticate before downloading:

hf auth login

The repository is organized by conversation rather than as one table row per conversation. To inspect metadata and annotations without downloading audio:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="otoearth/otoSpeech-full-duplex-turn-104h",
    repo_type="dataset",
    allow_patterns=[
        "*/metadata.json",
        "*/speaker_1_annotation_a.srt",
        "*/speaker_2_annotation_a.srt",
    ],
)
print(local_dir)

To download the two separated speaker streams, add the following patterns:

allow_patterns = [
    "*/metadata.json",
    "*/speaker_1_annotation_a.srt",
    "*/speaker_2_annotation_a.srt",
    "*/speaker_1_audio.wav",
    "*/speaker_2_audio.wav",
]

Add */combined_audio.wav only if the stereo combined rendition is also needed. The full repository is approximately 290 GB; downloading only the two separated speaker streams avoids duplicating them in the combined rendition.

Dataset Structure

The release contains 420 conversation directories. Directory names are task identifiers and are not guaranteed to be contiguous.

<task_id>/
├── combined_audio.wav
├── metadata.json
├── speaker_1_annotation_a.srt
├── speaker_1_audio.wav
├── speaker_2_annotation_a.srt
└── speaker_2_audio.wav

Dataset Statistics

Item	Value
Conversations	420
Conversation audio	approximately 104.94 hours
Mean conversation duration	approximately 14.99 minutes
Duration range	approximately 10.14–18.52 minutes
WAV files	1,260
SRT annotation files	840
Metadata files	420
Repository size	approximately 290 GB

The duration is counted once per conversation. The three WAV renditions are time-aligned representations of the same conversation and should not be added together when calculating conversational hours.

Audio Files

All audio files are 48 kHz, 32-bit floating-point WAV files.

File	Channels	Description
`speaker_1_audio.wav`	1	Speaker 1 channel.
`speaker_2_audio.wav`	1	Speaker 2 channel, time-aligned with Speaker 1.
`combined_audio.wav`	2	Stereo rendition: channel 0 is Speaker 1 and channel 1 is Speaker 2.

Annotation Files

Each speaker has one SubRip (.srt) annotation file. An entry contains a millisecond-resolution time interval followed by a bracketed event label and transcript text:

1
00:00:12,340 --> 00:00:13,020
[Acknowledgement Backchannel] Okay.

The following 17 labels occur in this release:

Acknowledgement Backchannel
Awkward Silence
Bounded Response
Channel Bleed
Continuer Backchannel
Filler
Floor-taking Competitive Interruption
Floor-taking Cooperative Interruption
Laughter
Non-floor Taking Competitive Interruption
Non-floor Taking Cooperative Interruption
Non-Speech Noise
Normal Turn
Overlap
Reaction Backchannel
Speech, Non-Linguistic
Strong Floor Hold

Label strings are case-sensitive and are stored verbatim inside square brackets. Transcript text may be empty for events without a lexical transcript.

Metadata

Each metadata.json file contains:

Field	Type	Description
`task_id`	string	Conversation/task identifier; matches the directory name.
`speaker_1_actor_id`	string	Opaque actor identifier for Speaker 1.
`speaker_2_actor_id`	string	Opaque actor identifier for Speaker 2.
`conversation_type`	string	Conversation category.
`speaker_1_actor_gender`	string	Gender value associated with Speaker 1; may be empty.
`speaker_2_actor_gender`	string	Gender value associated with Speaker 2; may be empty.

The conversation categories and session counts are:

Conversation type	Sessions
`Argumentative/Deliberative`	77
`Casual/Spontaneous`	73
`Collaborative/Problem-Solving`	65
`Instructional`	61
`Narrative/Storytelling`	81
`Task-Oriented/Transactional`	63

Dataset Creation

Collection and Recording Setup

As part of the broader otoSpeech project, this release consists of two-party, full-duplex conversations recorded with a separate channel for each participant. Sessions use topics or scenarios to encourage natural, sustained interaction rather than scripted line reading. The recordings retain timing relationships between speakers, including silence, overlap, interruption, backchanneling, and other conversational behavior.

Recording environments and microphone conditions may vary across sessions. The separated channels and combined stereo rendition are aligned sample-for-sample.

Turn-Taking Annotation

Each separated speaker channel is accompanied by an SRT file containing timestamped transcript segments and conversational-event labels. The _a suffix is retained in the filenames; this release contains one annotation track per speaker.

Considerations for Using the Data

Turn boundaries, interruption types, and backchannel categories can be context-dependent. Annotations may contain subjective decisions or errors.
Transcripts may contain recognition or transcription errors, disfluencies, incomplete words, and non-standard spelling.
Speaker, accent, topic, recording-device, and acoustic-condition distributions may be uneven.
Gender metadata is incomplete for some sessions and should not be treated as a comprehensive representation of gender identity.
Conversational audio can contain personal or sensitive information. Apply appropriate access controls and do not attempt to identify, contact, profile, or link speakers to external records.
Do not attempt to reconstruct intentionally removed content or use the data to reproduce a recognizable participant voice.

Additional Information

Licensing Information

The dataset is licensed under the oto Speech License v1.0. This is a non-commercial research-use license for datasets containing recorded human voices. It is not an open-source or Open Definition-compliant license.

In brief, the license permits approved research, education, reproducibility work, benchmarking, evaluation, and research prototypes. It imposes restrictions on corporate research, commercial use, redistribution, derived data, model artifacts, security, and voice-identity uses. This summary is non-authoritative; the full LICENSE.md controls.

For commercial licensing or questions about an intended use, contact agent@oto.earth.

Citation Information

@misc{otospeech_full_duplex_turn_104h,
  title        = {otoSpeech-full-duplex-turn-104h: Full-Duplex Conversational Speech with Turn-Taking Annotations},
  author       = {otoearth},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/otoearth/otoSpeech-full-duplex-turn-104h}},
  note         = {License: oto Speech License v1.0}
}

Downloads last month: 3,061

Total file size:

290 GB

Collection including otoearth/otoSpeech-full-duplex-turn-104h

otoSpeech Full-duplex Turn Dataset

Collection

4 items • Updated 18 days ago • 2

Article mentioning otoearth/otoSpeech-full-duplex-turn-104h

otoSpeech is the training set for TurnBench

otoearth

•

11 days ago