Valentina Zadrija

Throughout her lifetime, Valentina has been working in the field of machine learning and computer vision. She obtained her PhD in AI at the University of Zagreb in 2017 focusing weakly supervised learning and traffic safety. After that, she joined Rimac Automobili to work on scene understanding models and later Gideon to work on autonomous mobile robots. At Gideon, she was in charge of the product team developing the entire autonomy stack for the autonomous mobile forklift Trey. In her free time, Valentina enjoys trying out latest research papers and playing Super Mario Kart with kids.

The Hitchhiker’s guide to spatial AI: from structure from motion to LMMs

Spatial intelligence is a field of AI striving to create models that can interact with the world: deliver groceries, mow yards or load trailers. As one can imagine, there are many open challenges to get nowadays models from browsers to public spaces:

First, as robots operate in a 3D world, they need to have accurate representations to localize and navigate in it. In this lecture, Valentina will primarily talk about reconstructing the 3D world structure from camera motion and share techniques that work.

Second, LLMs learn to predict the next token from text, while spatial LMMs learn to predict the next action from the sensor logs. How to efficiently get from raw sensor data to PyTorch tensors so we can start training models in the first place?

Finally, we are training models to predict the next robot’s action without labeled data for vehicles, traffic signs, cyclists, or people. Modalities like images, controls or laser scans require different tokenization schemes and different objective functions to bring them all together. Valentina will share the latest research and findings on this front.