Simulation-based Bayesian inference for multi-fingered robotic grasping

Multi-fingered robotic grasping is an undeniable stepping stone to universal picking and dexterous
manipulation. Yet, multi-fingered grippers remain challenging to control because of their rich nonsmooth contact dynamics or because of sensor noise. In this work, we aim to plan hand configurations
by performing Bayesian posterior inference through the full stochastic forward simulation of the
robot in its environment, hence robustly accounting for many of the uncertainties in the system. While
previous methods either relied on simplified surrogates of the likelihood function or attempted to
learn to directly predict maximum likelihood estimates, we bring a novel simulation-based approach
for full Bayesian inference based on a deep neural network surrogate of the likelihood-to-evidence
ratio. Hand configurations are found by directly optimizing through the resulting amortized and
differentiable expression for the posterior. The geometry of the configuration space is accounted for
by proposing a Riemannian manifold optimization procedure through the neural posterior. Simulation
and physical benchmarks demonstrate the high success rate of the procedure.