Skip to content

Latest commit

 

History

History
executable file
·
124 lines (79 loc) · 10.6 KB

getting_started.md

File metadata and controls

executable file
·
124 lines (79 loc) · 10.6 KB

Getting Started

The OpenScene dataset is a large-scale dataset for end-to-end planning, visual pretraining, and occupancy prediction in the field of autonomous driving.

Based on nuPlan, we provide bounding box, occupancy, and flow annotations in 3D space.

OpenScene v1.1

🔥 Change Log

  • We reorganized the meta data files and organized them by their nuPlan log files to improve usability.
  • We added more logs that have sensor data and uploaded the LiDAR raw sensor data.

❗ Must Read for the CVPR 2024 Challenge

  • For Track End-to-End Driving at Scale, please download the meta_data and the camera or/and LiDAR sensor data, depend on modalities you intend to use. Note that there is no separate competition track for camera-only planners.

  • For Track Predictive World Model, please download the meta_data, the camera and LiDAR sensor data.

  • The private test set utilized in the challenge leaderboards is exclusively provided by Motional and should not be confused with the test set.

  • It is important to note that the private test sets for the two tracks are distinct and do not share any data.

  • The input data (metadata, sensors) for the private test set will be accessible upon the opening of test server. The ground truth data will be only available on the test server operated by Motional.

Download Data

  • We recommended to download all data from OpenXLabOpenDriveLab and use provided command line interface (CLI) for acceleration.

  • The sensor data for both the trainval and test subsets amount to approximately 2TB. We recommend initially training and validating your models on the mini set.

  • 🔔 For those who already possess the nuPlan sensor data (over 20TB) locally, you have the option to directly link it to the OpenScene folder to avoid redundant downloads. We carefully aligned the folder structure with nuPlan and just downsampled the nuPlan sensor data to improve the accessibility.

  • 🔔 If you already have the OpenScene v1.0 image data, you can use it for OpenScene v1.1 as well, since almost (>98%) of all the data is present. If you want to use the occupancy label, please also download it from OpenScene v1.0. There are only a few instances of additional data in v1.1 that are missing. You can temporarily ignore those frames during training.

  • If you can't access OpenXLabOpenDriveLab, please try the HuggingFace download links below. Alternatively, you may directly download all the sensor data from nuPlan.

mini set

File Name Download Link Size
openscene_metadata_mini.tgz Google Drive / Hugging Face 509.6 MB
openscene_sensor_mini_camera OpenXLab / Hugging Face 84 GB
openscene_sensor_mini_lidar OpenXLab / Hugging Face 60 GB

trainval set

File Name Download Link Size
openscene_metadata_trainval.tgz Google Drive / Hugging Face 6.6 GB
openscene_sensor_trainval_camera OpenXLab / Hugging Face 1.1 TB
openscene_sensor_trainval_lidar OpenXLab / Hugging Face 822 GB

test set

File Name Download Link Size
openscene_metadata_test.tgz Google Drive / Hugging Face 454 MB
openscene_sensor_test_camera OpenXLab / Hugging Face 120 GB
openscene_sensor_test_lidar OpenXLab / Hugging Face 87 GB

private test set

File Name Download Link Size
openscene_metadata_private_test_wm.tgz Hugging Face 7.3 MB
openscene_sensor_private_test_wm.tgz Hugging Face 15 GB
openscene_metadata_private_test_e2e.tgz Hugging Face 4 MB
openscene_sensor_private_test_e2e.tgz Hugging Face 23.6 GB
  • private_test_wm is the private test set for Predictive World Model track.
  • private_test_e2e is the private test set for End-to-End Driving at Scale track.
  • [2024-04-09] We fix some bugs and update the metadata of private_test_wm, please replace it!

Prepare Dataset

Please follow the steps below to get familiar with the OpenScene v1.1 dataset.

  1. Download all the data manually and unzip them.
  2. Make sure the filesystem hierarchy is the same as the dataset stats.
  3. Modify and run python DriveEngine/process_data/collect_data.py to collect the meta_data in any custom split.

OpenScene v1.0

Download Data

We recommended to download from OpenXLabOpenDriveLab and use provided command line interface (CLI) for acceleration. In addition, Google DriveGoogle Drive and Baidu CloudBaidu Yun are also available. If you already have the nuPlan dataset, you only need to download the label and meta data.

Subset Google DriveGoogle Drive Baidu CloudBaidu Yun Approx. Size
mini image / label image / label 81.2G / 6.7G
trainval image / label image / label 1.1T / 95.4G
test image image 118.5G
meta data meta file meta file 6.4G
  • Mini and trainval data contain three parts -- sensor_blobs (images), meta_data, and occupancy (label).

To ensure the integrity of the downloaded data, we recommend verifying the file using its MD5 checksum after the download is complete.

Train a Occupancy Prediction Model

Baseline

We provide a baseline model based on OccNet. The baseline is currently compatible with OpenScene dataset v1.0.

Train and Test

Train model with 4 GTX3090 GPUs

./tools/dist_train.sh ./projects/configs/bevformer/bev_tiny_occ_r50_nuplan.py 4

Eval model with 4 GTX3090 GPUs

./tools/dist_test.sh ./projects/configs/bevformer/bev_tiny_occ_r50_nuplan.py ./path/to/ckpts.pth 4

Visualization

See openscene_scenario_visualization.py