Dataset Viewer
The dataset viewer is not available for this subset.
Cannot get the split names for the config 'default' of the dataset.
Exception:    SplitsNotFoundError
Message:      The split names could not be parsed from the dataset config.
Traceback:    Traceback (most recent call last):
                File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 286, in get_dataset_config_info
                  for split_generator in builder._split_generators(
                                         ~~~~~~~~~~~~~~~~~~~~~~~~~^
                      StreamingDownloadManager(base_path=builder.base_path, download_config=download_config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                  )
                  ^
                File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/webdataset/webdataset.py", line 81, in _split_generators
                  first_examples = list(islice(pipeline, self.NUM_EXAMPLES_FOR_FEATURES_INFERENCE))
                File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/webdataset/webdataset.py", line 32, in _get_pipeline_from_tar
                  fs: fsspec.AbstractFileSystem = fsspec.filesystem("memory")
                                                  ~~~~~~~~~~~~~~~~~^^^^^^^^^^
                File "/usr/local/lib/python3.14/site-packages/fsspec/registry.py", line 302, in filesystem
                  cls = get_filesystem_class(protocol)
                File "/usr/local/lib/python3.14/site-packages/fsspec/registry.py", line 239, in get_filesystem_class
                  raise ValueError(f"Protocol not known: {protocol}")
              ValueError: Protocol not known: memory
              
              The above exception was the direct cause of the following exception:
              
              Traceback (most recent call last):
                File "/src/services/worker/src/worker/job_runners/config/split_names.py", line 71, in compute_split_names_from_streaming_response
                  for split in get_dataset_split_names(
                               ~~~~~~~~~~~~~~~~~~~~~~~^
                      path=dataset,
                      ^^^^^^^^^^^^^
                      config_name=config,
                      ^^^^^^^^^^^^^^^^^^^
                      token=hf_token,
                      ^^^^^^^^^^^^^^^
                  )
                  ^
                File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 340, in get_dataset_split_names
                  info = get_dataset_config_info(
                      path,
                  ...<6 lines>...
                      **config_kwargs,
                  )
                File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 291, in get_dataset_config_info
                  raise SplitsNotFoundError("The split names could not be parsed from the dataset config.") from err
              datasets.inspect.SplitsNotFoundError: The split names could not be parsed from the dataset config.

Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.

SimLingo Dataset

Overview

SimLingo-Data is a large-scale autonomous driving CARLA 2.0 dataset containing sensor data, action labels, a wide range of simulator state information, and language labels for VQA, commentary and instruction following. The driving data is collected with the privileged rule-based expert PDM-Lite.

Dataset Statistics

  • Large-scale dataset: 3,308,315 total samples (note: these are not from unique routes as the provided CARLA route files are limited)
  • Diverse Scenarios: Covers 38 complex scenarios, including urban traffic, participants violating traffic rules, and high-speed highway driving
  • Focused Evaluation: Short routes with 1 scenario (62.1%) or 3 scenarios (37.9%) per route
  • Data Types: RGB images (.jpg), LiDAR point clouds (.laz), Sensor measurements (.json.gz), Bounding boxes (.json.gz), Language annotations (.json.gz)

Dataset Structure

The dataset is organized hierarchically with the following main components:

  • data/: Raw sensor data (RGB, LiDAR, measurements, bounding boxes)
  • commentary/: Natural language descriptions of driving decisions
  • dreamer/: Instruction following data with multiple instruction/action pairs per sample
  • drivelm/: VQA data, based on DriveLM

Data Details

  • RGB Images: 1024x512 front-view camera image
  • Augmented RGB Images: 1024x512 front-view camera image with a random shift and orientation offset of the camera
  • LiDAR: Point cloud data saved in LAZ format
  • Measurements: Vehicle state, simulator state, and sensor readings in JSON format
  • Bounding Boxes: Detailed information about each object in the scene.
  • Commentary, Dreamer, VQA: Language annotations

Usage

This dataset is chunked into groups of multiple routes for efficient download and processing.

Download the whole dataset using git with Git LFS

# Clone the repository
git clone https://huggingface.co/datasets/RenzKa/simlingo

# Navigate to the directory
cd simlingo

# Pull the LFS files
git lfs pull

Download a single file with wget

# Download individual files (replace with actual file URLs from Hugging Face)
wget https://huggingface.co/datasets/RenzKa/simlingo/resolve/main/[filename].tar.gz

Extract to a single directory - please specify the location where you want to store the dataset

# Create output directory
mkdir -p database/simlingo

# Extract all archives to the same directory
for file in *.tar.gz; do
    echo "Extracting $file to database/simlingo/..."
    tar -xzf "$file" -C database/simlingo/
done

License

Please refer to the license file for usage terms and conditions.

Citation

If you use this dataset in your research, please cite:

@inproceedings{renz2025simlingo,
  title={SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment},
  author={Renz, Katrin and Chen, Long and Arani, Elahe and Sinavski, Oleg},
  booktitle={Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2025},
}
@inproceedings{sima2024drivelm,
  title={DriveLM: Driving with Graph Visual Question Answering},
  author={Chonghao Sima and Katrin Renz and Kashyap Chitta and Li Chen and Hanxue Zhang and Chengen Xie and Jens Beißwenger and Ping Luo and Andreas Geiger and Hongyang Li},
  booktitle={European Conference on Computer Vision},
  year={2024},
}
Downloads last month
4,115

Models trained or fine-tuned on RenzKa/simlingo