Van Nguyen Nguyen

Email: vanngn dot nguyen at gmail dot com

I am a Research Scientist at Axon in Seattle, where I work on computer vision and vision-language models. Before that, I was a Research Scientist at UII America Inc. in Greater Boston, working on multi-modal foundation models for medical video understanding and 3D/4D scene understanding.

I completed my PhD at IMAGINE team, École des Ponts ParisTech, under the supervision of Prof. Vincent Lepetit, where I worked on generalizable 6D pose estimation for novel objects.


News


Industrial Experience

Axon

Research Scientist
07/2026 – Present

United Imaging Intelligence

Research Scientist
06/2025 – 07/2026

Meta

Research Intern
06/2024 – 02/2025
05/2022 – 10/2022

Siemens

Research Intern
02/2019 – 07/2019

Neogia

Research Intern
07/2018 – 09/2018


PhD thesis: Pose Estimation of Novel Rigid Objects

Supervisor: Prof. Vincent Lepetit
Reviewers: Prof. Markus Vincze, Dr. Benjamin Busam
Examiners: Prof. Josef Sivic, Prof. Dima Damen, Dr. Slobodan Ilic


Publications

All of my released code is maintained on my GitHub account. GitHub stars

Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models

Meng Zheng, Samhita Marri, Anwesa Choudhuri, Benjamin Planche, Zhongpai Gao, Van Nguyen Nguyen, Terrence Chen, Girish Chowdhary, Ziyan Wu

NeurIPS 2026

A method that uses failure trajectories as adaptive negative guidance for diffusion- and flow-based VLA policies, steering action generation away from failure-prone regions toward reliable success modes.

arXiv
Direct Conditional Parameterization for N-Dimensional Splatting

Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Arun Innanje, Terrence Chen, Ziyan Wu

NeurIPS 2026

A method that directly parameterizes conditional opacity and position, eliminating costly covariance-derived slicing in N-dimensional splatting to improve rendering speed while matching or improving quality across Gaussian and Beta kernels.

CIF: Consistent Instance Field for Dynamic Scene Understanding

Junyi Wu, Van Nguyen Nguyen, Benjamin Planche, Jiachen Tao, Changchang Sun, Zhongpai Gao, Zhenghao Zhao, Anwesa Choudhuri, Gengyu Zhang, Meng Zheng, Feiran Wang, Terrence Chen, Yan Yan, Ziyan Wu

CVPR 2026

A method that represents dynamic scenes with instance-embedded deformable 3D Gaussians, disentangling visibility from persistent object identity to enable novel-view panoptic segmentation and open-vocabulary 4D querying.

arXiv project
MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding

Yuhao Su, Anwesa Choudhuri, Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Yuhan Shen, Arun Innanje, Terrence Chen, Ehsan Elhamifar, Ziyan Wu

CVPR 2026

We introduce MedVidBench, a large-scale benchmark of 531K video-instruction pairs across 8 medical sources, and MedGRPO, an RL framework with cross-dataset reward normalization and a medical LLM judge for balanced multi-dataset training.

arXiv project
Universal Beta Splatting

Rong Liu, Zhongpai Gao, Benjamin Planche, Meida Chen, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Yue Wang, Andrew Feng, Ziyan Wu

ICLR 2026

A unified framework that generalizes 3D Gaussian Splatting to N-dimensional anisotropic Beta kernels for explicit radiance field rendering. Beta kernels naturally decompose scene properties into interpretable components (surface vs. texture, diffuse vs. specular, static vs. dynamic) without explicit supervision.

arXiv project code GitHub stars GitHub forks
BOP Challenge 2024 on Model-free, Model-based Detection, and Pose Estimation of Unseen Rigid Objects

Van Nguyen Nguyen, Stephen Tyree, Andrew Guo, Médéric Fourmy, Anas Gouda, Taeyeop Lee, Sungphill Moon, Hyeontae Son, Lukas Ranftl, Jonathan Tremblay, Eric Brachmann, Bertram Drost, Vincent Lepetit, Carsten Rother, Stan Birchfield, Jiri Matas, Yann Labbé, Martin Sundermeyer, Tomáš Hodaň

CVPRW 2025 (CV4MR Best Paper Award)

The report of BOP Challenge 2024 on model-based and model-free 2D/6D object detection on BOP-Classic and new BOP-H3 datasets (HOT3D, HOPEv2, HANDAL).

arXiv project
GoTrack: Generic 6DoF Object Pose Refinement and Tracking

Van Nguyen Nguyen, Christian Forster, Sindi Shkodrani, Bugra Tekin, Cem Keskin, Vincent Lepetit, Tomáš Hodaň

CVPRW 2025

A method that refines and tracks the 6DoF pose of unseen objects from their CAD models, aligning the model’s 2D projection with the object’s appearance in an RGB image.

arXiv code GitHub stars GitHub forks
BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects

Tomas Hodan, Martin Sundermeyer, Yann Labbé, Van Nguyen Nguyen, Gu Wang, Eric Brachmann, Bertram Drost, Vincent Lepetit, Carsten Rother, Jiri Matas

CVPRW 2024

The report of BOP Challenge 2023 on state-of-the-art methods for seen and unseen object pose estimation.

arXiv project
GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence

Van Nguyen Nguyen, Thibault Groueix, Mathieu Salzmann, Vincent Lepetit

CVPR 2024

A “hybrid” template-patch correspondence approach that is fast, robust, and more accurate to estimate 6D pose of novel objects in RGB images.

arXiv code GitHub stars GitHub forks
NOPE: Novel Object Pose Estimation from a Single Image

Van Nguyen Nguyen, Thibault Groueix, Georgy Ponimatkin, Yinlin Hu, Renaud Marlet, Mathieu Salzmann, Vincent Lepetit

CVPR 2024

A method that estimates the relative pose of unseen objects from a single reference image, predicting a pose distribution that handles symmetry ambiguities.

arXiv code GitHub stars GitHub forks
OpenStreetView-5M: The Many Roads to Global Visual Geolocation

Guillaume Astruc, Nicolas Dufour, Ioannis Siglidis, Constantin Aronssohn, Nacim Bouia, Stephanie Fu, Romain Loiseau, Van Nguyen Nguyen, Charles Raude, Elliot Vincent, Lintao Xu, Hongyu Zhou, Loic Landrieu

CVPR 2024

A new benchmark for visual geolocation (~Geoguessr).

arXiv code GitHub stars GitHub forks
CNOS: A Strong Baseline for CAD-based Novel Object Segmentation

Van Nguyen Nguyen, Thibault Groueix, Georgy Ponimatkin, Vincent Lepetit, Tomáš Hodaň

ICCVW 2023 (R6D Best Method Award for 2D detection of unseen objects)

A method that can segment novel objects for a given RGB image from only their CAD models.

arXiv code GitHub stars GitHub forks
PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6DoF Tracking

Van Nguyen Nguyen+, Yuming Du+, Yang Xiao, Michaël Ramamonjisoa, Vincent Lepetit

3DV (Oral)

A method for tracking the 6D motion of objects in RGB video sequences when neither training images nor even the 3D geometry of the objects is available.

arXiv code GitHub stars GitHub forks
Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

Van Nguyen Nguyen, Yinlin Hu, Yang Xiao, Mathieu Salzmann, Vincent Lepetit

CVPR 2022

A method that can recognize objects and estimate their 3D pose in color images even under partial occlusions. Our method requires neither a training phase on these objects nor real images depicting them, only their CAD models.

arXiv code GitHub stars GitHub forks

Service

Reviewing: CVPR, ICCV, ECCV, NeurIPS, ICRA, IROS, RA-L, TPAMI, 3DV, ACCV, BMVC

Teaching: Image Processing and Artificial Vision (1st Master level) at ENPC, France in 2021, 2022

Workshop & Challenge: