Aria Everyday Objects (AEO)

A small-scale, real-world Project Aria dataset

with high quality static 3D oriented bounding boxes annotation

WHAT IS IT?

A new validation dataset for egocentric benchmark tasks

Research projects, like EFM3D and SceneScript, demonstrate the growing potential of using synthetic data for training large AI models. However, it's crucial to validate these models in real-world environments to ensure they effectively generalize beyond simulations.

The AEO dataset serves as a challenging real-world validation tool for 3D object detection in egocentric data, ensuring models trained in simulations can perform reliably in real-world scenarios.

DOWNLOAD THE DATASET BELOW EXPLORE THE DATA IN ARIA DATASET EXPLORER

Dataset Contents

  • Project Aria glasses data (including 2 x SLAM cameras, 1 x RGB camera, 2 x IMU, and complete sensor calibrations)
  • Aria machine perception service annotations including semi-dense point clouds and 6DoF device trajectory.
  • Manually-annotated 3D object bounding boxes

Sequence Metrics

  • 45 minutes of egocentric recordings captured by non-computer vision experts across 25 diverse, primarily indoor, real-world environments
  • 1037 3D object bounding box instances across 17 classes: Bed, Chair, Couch, Door, Floor, Lamp, Mirror, Plant, Refrigerator, Screen, Sink, Storage, Table, Wall, WallArt, WasherDryer, and Window
HOW IS AEO USED?

Used for validating egocentric foundation models, with EFM3D

AEO is used as a validation dataset in the EFM3D benchmark, the first baseline for egocentric 3d foundation models that are optimized for egocentric devices with robust 3D priors.

LEARN MORE ABOUT EFM3D

Read the EFM3D Research Paper

For more information about EFM3D benchmark and the first egocentric foundation model, EVL, read our paper here.

READ THE EFM3D RESEARCH PAPER

BibTex Citation

@article{straub24efm,
            title = {EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models},
            author ={Julian Straub and Daniel DeTone and Tianwei Shen and Nan Yang and Chris Sweeney and Richard Newcombe},
            booktitle ={arXiv preprint arXiv:2406.10224},
            year ={2024},
            url ={https://arxiv.org/abs/2406.10224},
}

Access AEO Dataset and accompanying Tools

If you are a researcher in AI or ML research, access the AEO Dataset and accompanying tools here.

ACCESS THE DATASET

By submitting your email and accessing the AEO Dataset, you agree to abide by the dataset license agreement and to receive emails in relation to the dataset.