Home
Uni-Logo
 

Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics

O. Mees, Maxim Tatarchenko, Thomas Brox, W. Burgard
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
Abstract: We present a convolutional neural network for joint 3D shape prediction and viewpoint estimation from a single input image. During training, our network gets the learning signal from a silhouette of an object in the input image - a form of self-supervision. It does not require ground truth data for 3D shapes and the viewpoints. Because it relies on such a weak form of supervision, our approach can easily be applied to real-world data. We demonstrate that our method produces reasonable qualitative and quantitative results on natural images for both shape estimation and viewpoint prediction. Unlike previous approaches, our method does not require multiple views of the same object instance in the dataset, which significantly expands the applicability in practical robotics scenarios. We showcase it by using the hallucinated shapes to improve the performance on the task of grasping real-world objects both in simulation and with a PR2 robot.


Other associated files : ss3d_paper.pdf [4.7MB]  

Images and movies

 

BibTex reference

@InProceedings{TB19a,
  author       = "O. Mees and M. Tatarchenko and T. Brox and W. Burgard",
  title        = "Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics",
  booktitle    = "IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
  month        = " ",
  year         = "2019",
  url          = "http://lmbweb.informatik.uni-freiburg.de/Publications/2019/TB19a"
}

Other publications in the database