Natural Feature Tracking
In 2008 the team members of the Christian Doppler Laboratory for Handheld Augmented Reality presented the world's first 6DOF real-time natural feature tracking system running on a mobile phone. Since then we have made many advances towards our goal of wide area markerless tracking. Nowadays, robust, high frame-rate tracking of 3D objects is not prohibitive on mobile phones anymore.
This page summarizes the various steps we made and the current status of our work onto our goal of tracking anywhere and anytime. On our way to this goal, we always only consider methods that are suitable for the mobile phone platform in terms of processing power and memory requirements.
Combined Marker & Markesless Tracking (2008)
Summary: Marker tracking has revolutionized Augmented Reality about a decade ago. However, this revolution came at the expense of visual clutter. We developed methods that allow tracking beyond the visibility of markers to improving robustness. These techniques are highly efficient in their memory and CPU usage and run at interactive frame rates on mobile phones.
Publication: Robust and Unobtrusive Marker Tracking on Mobile Phones, Daniel Wagner, Tobias Langlotz, Dieter Schmalstieg, ISMAR 2008
<More...>
Tracking by Detection (2008)
Summary: Our first attempt towards markerless tracking on mobile phones (also known as PhonySIFT) uses a heavily modified version of the well known SIFT descriptor. We replaced the slow DoG approach of the original SIFT with a FAST corner detector. The approach runs in 3-5ms on a PC and 20-40ms on a mobile phone allowing AR applications at interative frame rates.
Publication: Pose Tracking from Natural Features on Mobile Phones, Daniel Wagner, Gerhard Reitmayr, Alessandro Mulloni, Tom Drummond, Dieter Schmalstieg, ISMAR 2008
<More...>
Dedicated Detection and Tracking (2009)
Summary: In our next step to improve robustness and speed we replaced the integrated tracker of our previous approach with a dedicated tracker that takes over once an object is detected. Dedicated tracking uses active search and can be implemented extremely fast, requiring as little as ~1ms on a PC and ~6-10ms on a mobile phone per tracked object. We use exhaustive normalize cross correlation (NCC) template matching over a search window rather than the popular KLT tracker since the latter is too slow for our purposes. Additionally, NCC is highly robust to affine brightess changes. Due to the active search approach the tracker's speed is largely independent to the camera's resolution.
<More...>
Multiple Target Detection and Tracking (2009)
Summary:The detection and tracking method described above can only detect and track one object at a time, because the processing power of mobile phones does not permit to run both detection and tracking simultanously without seriously degrading overall speed. We therefore invented new methods to run both techniques simultanously at high frame rate. The generall idea is to only detect in those areas of the camera image that are not used for tracking yet as well as splitting the slower detection task over multiple frames. The whole process is designed to not exceed a predefined time budget in order to guarantee a certain frame rate. The resulting detector/tracker combinaiton can detect and track up to 6 independent objects at a frame rate of 23Hz on a mobile phone.
Publication: Multiple Target Detection and Tracking with Guaranteed Framerates
on Mobile Phones, Daniel Wagner, Dieter Schmalstieg, Horst Bischof, ISMAR 2009
<More...>
Wide area localization (2009)
Summary: The methods described above work well for detecting and tracking small planar objects, but are not suitable for pose estimation in large environments. We developed a new scale-space keypoint detector and new descriptors for fast, high-quality feature extraction on mobile phones. These new techniques allow the instant (below 1 second) estimation of a 6DOF pose from a single image in a large environment. This work is done in colaboration with our colleagues working on 3D reconstruction.
Publication: Towards Wide Area Localization on Mobile Phones, Clemens Arth, Daniel Wagner, Manfred Klopschitz,
Arnold Irschara, Dieter Schmalstieg, ISMAR 2009
<More...>
Tracking of 3D objects (2009)
Summary: The techniques developed for our work on wide area localization are now also used for our latest high-speed natural feature tracker that can detect and track real 3D objects rather than planar objects only. We perform real-time scale space keypoint detection (20Hz) on a mobile phone. The applied tracker is an extended version of the tracker used for our work on "Multiple Target Detection and Tracking".
<More...>
website maintained by
Tobias Langlotz
last updated on 2010-02-01
