Learning to be Smooth:
An End-to-End Differentiable Particle Smoother

NeurIPS 2024

Ali Younis, Erik Sudderth

University of California, Irvine

Examples of state estimation in city scale environments for our method, Mixture Density Particle Smoother (bottom) which combines the output of a forward and a backward in time Mixture Density Particle Filter (MDPF, middle rows). Our method is more accurate than either the forward of backward MDPF and is able to maintain accurate tracking even when there is ambiguity in the state space.

Abstract

For challenging state estimation problems arising in domains like vision and robotics, particle-based representations attractively enable temporal reasoning about multiple posterior modes. Particle smoothers offer the potential for more accurate offline data analysis by propagating information both forward and backward in time, but have classically required human-engineered dynamics and observation models. Extending recent advances in discriminative training of particle filters, we develop a framework for low-variance propagation of gradients across long time sequences when training particle smoothers. Our “two-filter” smoother integrates particle streams that are propagated forward and backward in time, while incorporating stratification and importance weights in the resampling step to provide low-variance gradient estimates for neural network dynamics and observation models. The resulting mixture density particle smoother is substantially more accurate than state-of-the-art particle filters, as well as search-based baselines, for city-scale global vehicle localization from real-world videos and maps.

Example trajectories from the Mapillary Geo-Location (MGL) dataset with observations shown in the top row. We show the current true state and state history (black arrow and black line), the estimated posterior density of the current state (red cloud, with darker being higher probability) and the top 3 extracted modes (blue arrows) for the our MDPS as well as its forward and backward MDPFs. Due to ambiguity at early time-steps, MDPF is unable to resolve the correct intersection, and instead places probability mass at multiple intersections. By fusing both forward and backward filters, our MDPS resolves this ambiguity with probability mass focused on the correct intersection. Furthermore, MDPS provides a tighter posterior density than either MDPF-Forward or MDPF-Backward.

BibTeX


@inproceedings{younis2024mdps,
      author    = {Younis, Ali and Sudderth, Erik},
      title     = {Learning to be Smooth: An End-to-End Differentiable Particle Smoother},
      booktitle = {NeurIPS},
      year      = {2024},
}