[Documentation] [TitleIndex] [WordIndex

Please see the tutorials for a demo of this package using Microsoft's Kinect sensor.


This package provides a module that detects rectangles in depth images and estimates their pose. Further, the module can track these rectangles over time and publish them as ArticulatedTrack messages. From these tracks, the articulation_models/model_learner_msg can learn the kinematic model underlying the observed motion.

A demo launch file is available that runs the detector, model learner, and visualization using RVIZ (see package tutorials). This implementation is very efficient: it runs on a Intel Core2 Duo with more than 10fps.


The detector itself works as follows:

Screen shots

The following images demonstrate how the detector, tracker, and model learning works.

Motivating example

Here, the lid of a laptop is opened and closed. By tracking the pose of the laptop lid, the kinematic model of the lid is estimated from its motion and visualized using RVIZ.




The lid of a laptop is opened and closed

Visualization of the plane segmentation and the detected rectangle

Resulting kinematic model learned from a sequence of observations

Robustness against occlusions

Our approach also allows that parts of the tracked object are occluded:



Iterative pose fitting

Our approach segments the depth image into planes, and iteratively fits a rectangle to each plane. During pose fitting, our approach minimizes a cost function that is based on the number of true positive, false positive, and true negative pixels.




Segmentation and Detection

After fitting rectangles to all detected planes, our approach filters the candidates based on the pixel precision and recall. In these images, the detected planes are colored in blue/green/red and the detected rectangular objects are indicated using thick white borders.



Examples of learned kinematic models





More Information

More information (including videos, papers, presentations) can be found on the homepage of Juergen Sturm.

Report a Bug

If you run into any problems, please feel free to contact Juergen Sturm <juergen.sturm@in.tum.de>.


J├╝rgen Sturm, Kurt Konolige, Cyrill Stachniss, Wolfram Burgard. Vision-based Detection for Learning Articulation Models of Cabinet Doors and Drawers in Household Environments. In Proc. of the IEEE International Conference on Robotics and Automation (ICRA), Anchorage, USA, 2010. pdf

2024-06-22 12:36