Modeling the rotation of moving objects is a fundamental task in computer vision, yet SO(3) extrapolation still presents numerous challenges: (1) unknown quantities such as the moment of inertia complicate dynamics, (2) the presence of external forces and torques can lead to non-conservative kinematics, and (3) estimating evolving state trajectories under sparse, noisy observations requires robustness.
We propose modeling trajectories of noisy pose estimates on the manifold of 3D rotations in a physically and geometrically meaningful way by leveraging Neural Controlled Differential Equations guided with SO(3) Savitzky-Golay paths.
Existing extrapolation methods often rely on energy conservation or constant velocity assumptions, limiting their applicability in real-world scenarios involving non-conservative forces. In contrast, our approach is agnostic to energy and momentum conservation while being robust to input noise, making it applicable to complex, non-inertial systems. Our approach is easily integrated as a module in existing pipelines and generalizes well to trajectories with unknown physical parameters.
By learning to approximate object dynamics from noisy states during training, our model attains robust extrapolation capabilities in simulation and various real-world settings.
We consider the problem of forecasting the rotational motion of a rigid body with unknown physical properties from noisy sensor estimates in SO(3). The core of our proposed method is a neural controlled differential equation, which learns a latent representation of the underlying dynamical system with respect to a robust control path effectively handling noisy measurements.
We construct this control path directly on the manifold SO(3) by filtering noisy input observations with a Savitzky-Golay filter. This allows the CDE to learn a more robust parameterization for extrapolating rigid-body trajectory estimates than existing methods.
SO(3) Savitzky-Golay Neural CDEs: previous approaches rely on energy conservation or constant velocity assumptions. Our method can predict rotational object trajectories in various physical scenarios, including dissipative forces and objects under external conservative force fields. We design an integration control signal on the Lie group SO(3) to simultaneously handle denoising input states and serve as a numerical integration path for a CDE.
The Savitzky-Golay filter on SO(3) constructs a polynomial with global support in the Lie algebra, solvable as a single least squares problem, providing a geometrically meaningful control path that respects the manifold structure while being robust to sensor noise.
Our method robustly forecasts object rotations across diverse scenarios with noise and non-conservative forces, outperforming traditional approaches when tracking fails.
Qualitative Comparison: Our method maintains better alignment of box rotations on the Oxford MultiMotion Dataset than baseline methods, whose predicted rotations cause the projected coordinate system to drift significantly over time. Object poses are represented in world coordinates so that errors manifest as translations in pixel space.
SO(3) Trajectory Comparisons show rotational trajectories projected onto the unit sphere. The black dots represent noisy input observations from which the underlying physics are learned. Our method (blue) closely follows ground truth (cyan) trajectories on the rotation manifold, outperforming baseline methods across different scenarios.
@inproceedings{bastian2025forecasting,
title={Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)},
author={Bastian, Lennart and Rashed, Mohammad and Navab, Nassir and Birdal, Tolga},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
year={2025}
}