Semantic Structure From Motion (SSFM) Sid Yingze Bao, Mohit
Bagra, Yu-Wei Chao,
and Silvio Savarese
Computer Vision Lab, University of Michigan at Ann Arbor |
||
|
||
What is it about? |
Traditional Structure from motion (SFM) aims at jointly recovering the
structure of a scene as a collection of 3D points and estimating the
camera poses from a number of input images. In this
project, called Semantic Structure from
Motion (SSFM) , we generalize this concept:
not only do we want to recover 3D points, but also recognize and
estimate the location of high level semantic scene components such as
regions and objects in 3D. As a key ingredient for this joint inference
problem, we seek to model various types of interactions between scene
components. Such interactions help regularize our solution and obtain
more accurate results than solving these problems in isolation.
Experiments on public datasets demonstrate that: 1) our framework
estimates camera poses more robustly
than SFM algorithms that use points only; 2) our framework is capable
of accurately estimating pose and location of objects, regions, and
points in the 3D scene; 3) our framework recognizes objects and regions
more accurately than state-of-the-art single image recognition
methods.
Check out our paper for details! |
Update Jun 26, 2012
|
Who might be
interested? |
Researchers
who are interested in methods for 3D reconstruction from multiple
views, object detection and recognition, scene segmentation as well as
in applications such as autonomous navigation, robotics, object
manipulation and surveillance.
|
|
Papers and citations |
download pdf and bibtex download pdf download pdf (long version) and bibtex download pdf and bibtex. Winner of the best student paper award |
|
About us |
Sid Yingze Bao is a 4th year PhD student in the Vision Lab at the University of Michigan, at Ann Arbor, EECS department Silvio Savarese is an assistant professor of Electrical and Computer Engineering at U-M and director of the Vision Lab. |
|
Results | Below
are 3 YouTube videos that illustrate the ability of SSFM to recover the
structure of a scene from multiple images and highlight the important
semantic phenomena. For more results please refer to our papers. Video credits: Mohit Bagra |
|
Soure code |
|
|
Data sets |
|