Datasets and High-Fidelity Models for In-Cabin Behavior Understanding

Project summary:

The overarching goal of this project is to enable the measurement, analysis, and prediction of in-car human behaviors. To achieve this goal, researchers will create a data acquisition system for the collection of large-scale in-cabin video datasets, using a multiview camera system that can allow the capture of realistic in-situ human behavior at multiple resolutions including 1) close-up facial expression, head pose, and eye gaze, 2) hand and arm movements and gestures, and 3) upper-body movements and interaction. With this dataset, researchers will develop robust and generalizable algorithms for the 4D spatiotemporal representation of behaviors. To overcome the effects of dataset bias, the research team will design a new synthetic dataset augmentation scheme to generate diverse samples for domain adaptation to handle different vehicles, passenger occlusions, and novel scenarios.