FlowControl: Optical Flow Based Visual Servoing

Max Argus1 Lukas Hermann1 Jon Long2 Thomas Brox1

1University of Freiburg

Abstract

One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.


Downloads
descriptionPaper 

Video

References

M. Argus et al., “FlowControl: Optical Flow Based Visual Servoing“, 2020. [BibTex]