SignPost 2 - Mobile AR Navigation System
SignPost II is an improved successor of the original
SignPost Application. SignPost is a Studierstube application
that is able to guide a person through an unfamiliar building. The person
wears a mobile navigation equipment that consists of:
 |
- a head mounted Glasstron optical stereo see-through display
- a helmet with an inertial tracker
- a FireWire camera
- a backpacked notebook computer with an accelerated 3D graphics
chip
- a wrist pad that allows user input, called PIP (personal
interaction panel)
|
 |
The see-through glasses enable the user
to see the real environment of the room in which he is currently standing.
SignPost allows the user to select a destination within the building and
guides him to the desired destination by overlaying hints (i.e. direction
arrows and a wireframe model of the room and its portals) on the user's
view of the scene through the glasses. The application searches the shortest
path starting from his current position and directs the user along that
path to the selected room.
To determine the user's current position,
SignPost of course needs some sort of tracking. As we cannot rely on GPS
or similar positioning systems inside a building, tracking of the user
is done with ARToolkit markers and the FireWire camera in combination
with the inertial tracker which is particularly dedicated for fast changes
in the orientation of the user. Details of tracking are described below.
Follow the links to more information on the software
design and a user's guide to the data managment
tools.
Application
Signpost II consist of four sub-applications:
| BAU |
|
Building AUgmentation. This is
the core application, which sets up the building geometry and
the tracking. This application provides configurable scene graph
of the building geometry and additional data about the building
for other applications. |
| AUG |
|
AUGmentation. This application
augments the real scenery with a wire-frame model of the building
interior. This wire-frame model can be also rendered with hidden-line-removal
using the z-buffer. This avoids the rendering of parts of the
building, which are invisible from the current position. |
| WIM |
|
World In Miniature. This application
provides a world-in-minature view of the building. The part of
the building around the current user position is displayed in
a wire-frame box. The building geometry is clipped at the border
of this box. The user position is mapped to the center of the
box. The position of the user is indicated by a small Lego-man.
This Lego-man is oriented in the WIM in same way as the user in
the real world. Two ways are provided to position the WIM relative
to the user. First the WIM can be attached to the PIP, second
the WIM is floating in front of the user, tilted by 45° so that
the user see the WIM always from top. In this mode,there are two
methods to orientate the WIM: First the wim can be world stabilized,
in which it is rotated using the rotation around the vertical
axis. The other method is to align the WIM that way, that the
top is always heading "north". |
| NAV |
|
NAVigation. This application provides
a navigation aid. It operates on a directional graph built up
from data received from the BAU Application. Any time the user
can select a destination room, and the application immediatly
calculates the shortest path from the start room to the destination
room. The start room is determined using the current user position.
The navigation aides are displayed on a HUD (head up display)
overlaid over the output of the other applications (AUG, WIM).
This HUD provides textual information (current room, destination
room, additional information) and an arrow pointing always to
the next portal through which the user have to go to reach the
destination. |
Interaction
As mentioned before, the user wears a wrist pad (the so-called PIP),
which allows him to interact with the SignPost system. The applications
mentioned in the previous section are controlled through the PIP. Above
you can see the pip with the task-bar showing the four applications.
| BAU |
|
Currently it is not possible to determine the start room at start-up
automatically. Therefore the user must select the start room himself.
The tracking system is then initialized to provide the correct
user position from this point on. This is done by selecting the
BAU application on the taskbar and the room is selected from the
list. |
| WIM |
|
For controling the WIM there
are five buttons and a slider on the PIP. The three buttons
on the left are for setting the WIM orientation. Activating
one of the first two buttons let the WIM float in front of
the user. Pressing the third buttons fixes the WIM to the
PIP. The two buttons on the right side are for switching on
or off geometry. The first one controls the visiblity of the
complete geometry of the WIM application. With the second
button, the additional geometry (e.g. Lego-man) can be switched
on or off. The slider in the middle scales the building geometry
around the center of the WIM box.
|
| AUG |
|
For controling the Augmentation there are four buttons on the
PIP. The two buttons on the left side are for switching on or
off geometry. The first one controls the visiblity of the complete
geometry of the AUG application. With the second button, the additional
geometry can be switched on or off. Pressing the first button
on the right side renders the building as wire-frame model. All
lines are visible. By pressing the second button on the right,
only the visible parts of the wire-frame are visible and all hidden
lines are not rendered. To achieve this effect, the model is rendered
as filled polygons with pixel output disabled, but z-buffer enabled.
|
| NAV |
|
The user can enter the desired destination after selecting the
NAV application. A list of all rooms is presented, from which
he can choose the one he or she wants to be guided to. The shortest
path from the current position to the destination is calculated
immediatly and the navigation hints are displayed in the HUD.
Of course, the NAV application has also a number of buttons to
control the application's geometry output. Pressing "HUD Visible"
switches the HUD geometry on or off. "HUD Arrow Visible" controls
the arrow on the HUD. "Portal Visible" controls the visible of
the next portal through which the user has to go to reach the
desired destination. By pressing "Portal Arrow Visible" the arrow,
which runs accross the floor from the current position to the
next portal, is switcvhed on or off. |
Navigation Hints
After selecting a destination room, the shortest path from the current
position to the selected destination is caluculasted. Based on that
path and the geometry information of the building, the SignPost system
guides the user through the building by providing navigation hints that
are overlaid on the user's view through the glasses as a HUD.
 |
The WIM model
floating in front of the user or it is attached to the wrist pad.
It shows a view of the floorplan around the current position,
with the person's current position denoted by a lego-man. Additionally
the scene is augmented by a wire-frame model.
|
 |
Portal arrow: The
arrow on the top right is pointing always to the next portal
to pass on the route to the destination. This portal is
highlighted by a translucent yellow rectangle.
|
 |
Room Information:
On the top left, the room where the user currently is located
and the desired destination room are displayed. |
 |
The WIM: The user
ist shown as legoman within a wire-frame / polygon model of the
rooms. |
 |
Finally: "Destinationn
Reached" is displayed when the user has reached the desired room. |
Tools: Marker-Allocator and BAUML-Viewer
SignPost relies on an XML-based data structure (so-called BAUML, Building
AUgmentation Markup Language), that holds three-dimensional geometry
information of the rooms within the building, as well as information
of ARToolkit marker placement (see 'Tracking'),
in order to allow the application to determine the current position
of the user within the building.
| Marker-Allocator |
|
The Marker-Allocator assigns to each marker in the BAUML-structure
a pattern from a set of predefined patterns in the following way:
An orthogonal grid with a fixed cell size is laid over the whole
floor. Most of the cells of the grid contain a number of markers.
Within each non-empty cell, a unique pattern, which is unique
within the 3x3 neighborhood is assigned to each marker. A new
BAUML-file with the assigned markers is generated. This method
described above allows adding new markers without changing the
existing pattern assignment: Only to the new 'unassigned' markers
a pattern is assigned, which is unique in the 3x3 neighborhood.
The width of the squares can is set to 4 meters, which is a good
trade-off between the total number of different marker patterns
(28 different patterns used) and the minimum distance of 8 meters
between indentical patterns.
|
| BAUML-Viewer |
|
 The
BAUML-Viewer displays a model of the world similar to the WIM-model.
It has a simple text-based user interface.
|
| Tracking of the person who
uses the SignPost navigation system is done by a combination
of optical and inertial tracking (blue cube on helmet). In all
rooms and hallways, which are covered by our floorplan, ARToolkit
markers were mounted on the walls every 2 - 3 meters. An ARToolkit
marker consists of a black square frame filled with some kind
of a pattern. The patterns must have no rotational symmetry,
have to be sufficiently distinct and have to be trained with
ARToolkit in order to be recognized. ARToolkit - the Augmented
Reality Toolkit - is an external component, which provides SignPost
(through OpenTracker within the Studierstube framework) with
optical tracking data.
The FireWire camera that resides
on the user's helmet delivers a video stream to ARToolkit, which
detects the markers in the video image. According to the size
and orientation of the marker on the video image, it is able
to compute the relative position of the user in respect to the
marker. By combining the relative marker location and the positional
and geometry information given in the floorplan file, the user's
movement within the environment can be tracked with certain
accuracy.
The Intersense inertial tracker,
also mounted on the user's helmet, is able to track the user's
orientation. It provides more frequent updates than the ARToolkit
tracking and therefore is optimal for rapid movements of the
user's head. Besides, it is also suited for filling gaps in
tracking, e.g. when optical tracking failed or no marker is
visible in the user's line of sight. |
  |
In order to allow reasonable tracking, ARToolkit markers have to be
placed every 2 - 3 meters. Deploying markers on our floor, which covers
several rooms and long hallways, would require several hundred different
marker patterns to be created and trained for ARToolkit. The more markers
we use, the higher the degree of similarity will be. A large set of
markers also increases the search space that has to be compared by ARToolkit.
Consequently, a large amount of markers leads to a decrease in performance
and/or a higher number of false recognitions.
|
To overcome this problem, we developed a marker partitioning-scheme
that allows re-using sets of marker. The idea is, that if the
tracking system knows the user's current location, it can rule
out large parts of the building because they will not be visible
to the camera. Thus, for two areas, which are not visible to each
other or are far away from each other, it becomes possible to
use the same marker patterns. The plan is dived into cells by
a regular grid. Each 3x3 block of cell (the cell itself and its
8-neighborhood) contains only unique marker patterns.
|
With that approach, we can generate a minimal number of disjoint sets
of markers. Two neighboring cells described above must have disjoint sets
of markers. Otherwise the camera tracking system cannot decide which cell
a certain marker belongs to, if both instances are visible from a position.
This approach improves performance in two ways: 1) The overall amount
of markers is minimized, which reduces false recognitions and minimizes
the search space of ARToolkit. 2) Starting at a given room position,
the system has only to track markers assigned to the current cell and
the neighboring cells. Thus, a lot of markers are ruled out before the
recognition process.
However, the marker re-usage leaves one question open: How does the application
determine the initial room, where the user starts the application? For
this purpose, we currently let the user determine the current room, which
is not the best solution.
Testing & Future Work
The current model used for development and testing purposes covers
parts of the 4th floor in our institute building. The floorplan file
holds geometry and adjacency information for several rooms. A total
amount of 175 markers was used. Initial experiences showed that the
system is heavily relaying on sufficient marker density. Whenever markers
are too far away (or too small) much jitter is perceived by the wireframe
model. Note, that the wireframe model can be turned off by the user,
and the jitter does have nearly no impact on the direction arrows that
are displayed to the user, because they still point to the next portal
the user is meant to go to. The jitter It can be expected, that the
jitter is reduced with a larger marker density. Gaps, which sometimes
occur between markers, are mostly bypassed by using the inertial tracker.
Unfortunately, the inertial tracker sometimes starts to drift in one
direction leading the user to bear away from his path. Heading to a
wall and viewing a marker from a close position adjusts the path shown
on the users' view to the correct path again.
Material
Video
Some videos showing the Signpost system in action:
Credits
The SignPost II application and its related tools, the Video and this
Webpage ...
... were developed, written & produced by these
people:
- Michael Knapp
- Gerhard Reitmayr
- and a lot of students who did the measurements of the building
- and finally who made all this possible: Dieter Schmalstieg
|
|
