Waymo vehicles have collected over 10 million autonomous miles in 25 cities; this rich and diverse set of real world experiences has helped engineers and researchers develop Waymo’s self-driving technology and innovative models and algorithms. This data is a critical ingredient for machine learning.
The Alphabet (Google’s parent company) backed company is inviting the research community to participate in the release of the Waymo Open Dataset, a high-quality multimodal sensor dataset for autonomous driving. Available free to researchers, it is comprised of high-resolution sensor data collected by Waymo self-driving vehicles. The dataset covers a wide variety of environments, from dense urban centres to suburban landscapes, as well as data collected during day and night, at dawn and dusk, in sunshine and rain.
This release contains data from 1,000 driving segments. Each segment captures 20 seconds of continuous driving, corresponding to 200,000 frames at 10 Hz per sensor. Such continuous footage gives researchers the opportunity to develop models to track and predict the behavior of other road users.
The dataset covers dense urban and suburban environments across Phoenix, AZ, Kirkland, WA, Mountain View, CA and San Francisco, CA capturing a wide spectrum of driving conditions (day and night, dawn and dusk, sun and rain). Each segment contains sensor data from five high-resolution Waymo LiDARs and five front-and-side-facing cameras. The dataset also includes LiDAR frames and images with vehicles, pedestrians, cyclists, and signage carefully labeled, capturing a total of 12 million 3D labels and 1.2 million 2D labels.
Waymo has been working on 3D perception models that fuse data from multiple cameras and LiDAR. Waymo designs their entire self-driving system — including hardware and software — to work seamlessly together, which includes choice of sensor placement and high quality temporal synchronisation.
When it comes to research in machine learning, having access to data can turn an idea into a real innovation. This data has the potential to help researchers make advances in 2D and 3D perception, and progress on areas such as domain adaptation, scene understanding and behaviour prediction.
The company hopes that the research community will generate more exciting directions with this data that will not only help to make self-driving vehicles more capable, but also impact other related fields and applications, such as computer vision and robotics.