2024 AI CITY CHALLENGE

The AI City Challenge, hosted at CVPR 2024, focuses on harnessing AI to enhance operational efficiency in physical settings such as retail and warehouse environments, and Intelligent Traffic Systems (ITS). It aims to utilize AI for actionable insights from sensor data, like camera feeds, to improve traffic safety and transportation outcomes. This year, the challenge spotlights two key areas with significant potential: retail business and ITS.

Key areas of interest include multi-camera people tracking, traffic safety analysis, naturalistic driving action recognition, fish-eye camera road object detection, and motorcycle helmet rule violation detection. We invite original contributions in these domains, leveraging computer vision, natural language processing, and deep learning for practical, large-scale applications that enhance safety and intelligence in our environments.

 

The 8th edition of the Challenge introduces exciting new tasks and significant dataset upgrades:

 

1. Traffic Safety Description and Analysis: A novel task focusing on dense video captioning of traffic safety scenarios, particularly pedestrian accidents. Using multi-camera feeds, participants will detail events leading up to incidents and ordinary scenes, offering deep insights for applications like insurance inspection processes and accident prevention.

2. Road Object Detection in Fish-Eye Cameras: This new task explores fish-eye camera analytics, featuring the open FishEye8K dataset with 8,000 images and 157K bounding boxes, alongside a challenge test set of 1,000 images from new camera footage.

3. Major Upgrades in Existing Challenges: Enhancements in datasets for existing challenges, such as a 10x increase in cameras and a 20x increase in characters for the multi-camera people tracking challenge, as well as availability of 3D annotation and camera matrices. Updates in challenge rules, like the evaluation on 3D tracking and encouragement of online tracking algorithms, are also introduced.

Participants are invited to compete in one or more of the five challenge tracks. To join, please complete the online AI City Challenge Datasets Request Form. This year’s challenges promise to push the boundaries of research and development, contributing to smarter, safer environments.

Important Dates

Workshop Committee

Shuo Wang

NVIDIA

Zheng Tang

NVIDIA

David Anastasiuo

Santa Clara University

Ming-Ching Chang

University at Albany – SUNY

Liang Zheng

Australian National University

Anuj Sharma

 Iowa State University

Pranamesh Chakraborty

Indian Institute of Technology Kanpur

Norimasa Kobori

Woven by Toyota

Jun-Wei Hsieh

National Yung-Ming Chiao-Tung University

Rama Chellappa

Johns Hopkins University

Challenge Committee

Xunlei Wu

NVIDIA

Sujit Biswas

NVIDIA

Sameer Satish Pusegaonkar

NVIDIA

Yizhou Wang

NVIDIA

Quan Kong

Woven by Toyota

Munkhjargal Gochoo

The United Arab Emirates University

Fady Al Najjar

The United Arab Emirates University

Sanjita Prajapati

Indian Institute of Technology Kanpur

Meenakshi Sumeet Arya

Iowa State University

Mohammed Shaiqur Rahman

Iowa State University

Yue Yao

Australian National University

Ping-Yang Chen

National Yung-Ming Chiao-Tung University

Munkh-Erdene Otgonbold

United Arab Emirates University

Ganzorig Batnasan

United Arab Emirates University

CITATIONS

Please cite the following papers accordingly if you choose to work with our datasets or refer to the previous challenge results:  

 

2024 challenge summary paper – The 8th AI City Challenge

@InProceedings{Shuo24AIC24,
author = {Shuo Wang and David C. Anastasiu and Zheng Tang and Ming-Ching Chang and Yue Yao and Liang Zheng and Mohammed Shaiqur Rahman and Meenakshi S. Arya and Anuj Sharma and Pranamesh Chakraborty and Sanjita Prajapati and Quan Kong and Norimasa Kobori and Munkhjargal Gochoo and Munkh-Erdene Otgonbold and Ganzorig Batnasan and Fady Alnajjar and Ping-Yang Chen and Jun-Wei Hsieh and Xunlei Wu and Sameer Satish Pusegaonkar and Yizhou Wang and Sujit Biswas and Rama Chellappa},
title = {The 8th {AI City Challenge}},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2024},
}

2023 challenge summary paper – The 7th AI City Challenge

@InProceedings{Naphade23AIC23,

author = {Milind Naphade and Shuo Wang and David C. Anastasiu and Zheng Tang and Ming-Ching Chang and Yue Yao and Liang Zheng and Mohammed Shaiqur Rahman and Meenakshi S. Arya and Anuj Sharma and Qi Feng and Vitaly Ablavsky and Stan Sclaroff and Pranamesh Chakraborty and Sanjita Prajapati and Alice Li and Shangru Li and Krishna Kunadharaju and Shenxin Jiang and Rama Chellappa},

title = {The 7th AI City Challenge},

booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},

month = {June},

year = {2023},

}

 

2022 challenge summary paper – The 6th AI City Challenge

@InProceedings{Naphade22AIC22,
author = {M. Naphade and S. Wang and D. C. Anastasiu and Z. Tang and M. Chang and Y. Yao and L. Zheng and M. Shaiqur Rahman and A. Venkatachalapathy and A. Sharma and Q. Feng and V. Ablavsky and S. Sclaroff and P. Chakraborty and A. Li and S. Li and R. Chellappa},
title = {The 6th AI City Challenge},
booktitle = {2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
month = {June},
year = {2022},pages = {3346-3355},doi = {10.1109/CVPRW56347.2022.00378},publisher = {IEEE Computer Society}
}

2021 challenge summary paper – The 5th AI City Challenge

@InProceedings{Naphade21AIC21,
author = {Milind Naphade and Shuo Wang and David C. Anastasiu and Zheng Tang and Ming-Ching Chang and Xiaodong Yang and Yue Yao and Liang Zheng and Pranamesh Chakraborty and Christian E. Lopez and Anuj Sharma and Qi Feng and Vitaly Ablavsky and Stan Sclaroff},
title = {The 5th AI City Challenge},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2021},
}

2020 challenge summary paper – The 4th AI City Challenge


@InProceedings{Naphade20AIC20,

author = {Milind Naphade and Shuo Wang and David C. Anastasiu and Zheng Tang and Ming-Ching Chang and Xiaodong Yang and Liang Zheng and Anuj Sharma and Rama Chellappa and Pranamesh Chakraborty},

title = {The 4th AI City Challenge},

booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2020},

pages = {2665–2674}

}


2019 challenge summary paper –
 The 2019 AI City Challenge


@InProceedings{Naphade19AIC19,

author = {Milind Naphade and Zheng Tang and Ming-Ching Chang and David C. Anastasiu and Anuj Sharma and Rama Chellappa and Shuo Wang and Pranamesh Chakraborty and Tingting Huang and Jenq-Neng Hwang and Siwei Lyu},
title = {The 2019 AI City Challenge},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2019},
pages = {452–460}


2018 challenge summary paper – 
The 2018 AI City Challenge


@inproceedings{Naphade18AIC18,

author={Milind Naphade and Ming-Ching Chang and Anuj Sharma and David C. Anastasiu and Vamsi Jagarlamudi and Pranamesh Chakraborty and Tingting Huang and Shuo Wang and Ming-Yu Liu and Rama Chellappa and Jenq-Neng Hwang and Siwei Lyu},
title = {The 2018 NVIDIA AI City Challenge},
booktitle = {Proc. CVPR Workshops},
pages = {53-–60}, 
year = 2018
}


2017 challenge summary paper – 
The NVIDIA AI City Challenge


@inproceedings{Naphade17AIC17,

author={Milind Naphade and David C. Anastasiu and Anuj Sharma and Vamsi Jagrlamudi and Hyeran Jeon and Kaikai Liu and Ming-Ching Chang and Siwei Lyu and Zeyu Gao},

title={The NVIDIA AI City Challenge},

booktitle = {Prof. SmartWorld},

address = {Santa Clara, CA, USA},

year = 2017

}

 

Woven Traffic Safety (WTS) Dataset

 

@article{WTS2024,

title = {WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding},

author = {Quan Kong and Yuki Kawana and Rajat Saini and Ashutosh Kumar and Jingjing Pan and Ta Gu and Yohei Ozao and Balazs Opra and David C. Anastasiu and Yoichi Sato and Norimasa Kobori},

year = {2024},

url = {https://github.com/woven-visionai/wts-dataset}

 

}

 

Synthetic Distracted Driving (SynDD2) dataset: Synthetic Distracted Driving (SynDD2) dataset for analyzing distracted behaviors and various gaze zones of a driver

 

@InProceedings{rahman2023synthetic,
author={Mohammed Shaiqur Rahman and Jiyang Wang and Senem Velipasalar Gursoy and David Anastasiu and Shuo Wang and Anuj Sharma},
title = {Synthetic Distracted Driving (SynDD2) dataset for analyzing distracted behaviors and various gaze zones of a driver},
howpublished = {arXiv:2204.08096},
year = {2023}

 

Synthetic distracted driving (SynDD1) dataset: Synthetic distracted driving (SynDD1) dataset for analyzing distracted behaviors and various gaze zones of a driver

@InProceedings{RAHMAN2023108793,
author={Mohammed Shaiqur Rahman and Archana Venkatachalapathy and Anuj Sharma and Jiyang Wang and Senem Velipasalar Gursoy and David Anastasiu and Shuo Wang},
title = {Synthetic distracted driving (SynDD1) dataset for analyzing distracted behaviors and various gaze zones of a driver},
journal = {Data in Brief},
volume = {46},
pages = {108793},
year = {2023},
issn = {2352-3409},
doi = {https://doi.org/10.1016/j.dib.2022.108793}

 

Natural language-based vehicle retrieval dataset: CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions

 

@InProceedings{Feng21CityFlowNL,
author={Qi Feng and Vitaly Ablavsky and Stan Sclaroff},
title = {CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions},
howpublished = {arXiv:2101.04741},
year = {2021}

 

Vehicle MTMC tracking & re-identification dataset – CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification


@InProceedings{Tang19CityFlow,

author = {Zheng Tang and Milind Naphade and Ming-Yu Liu and Xiaodong Yang and Stan Birchfield and Shuo Wang and Ratnesh Kumar and David Anastasiu and Jenq-Neng Hwang},

title = {CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019},

pages = {8797–8806}
}


Synthetic 3D vehicle dataset –
 Simulating Content Consistent Vehicle Datasets with Attribute Descent


@InProceedings{Yao20VehicleX,

author={Yue Yao and Liang Zheng and Xiaodong Yang and Milind Naphade and Tom Gedeon},

title = {Simulating Content Consistent Vehicle Datasets with Attribute Descent},
booktitle = {The European Conference on Computer Vision (ECCV)},

month = {August},
year = {2020},

pages = {775–791}