科研项目

项目列表


Stereo Vision System


立体视觉系统


Introduction

  

An FPGA-Based RGBD Imager

 

We develop a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping.  Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320~240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.

 

Fig. 1 RGBD imager system overview

 

Fig. 2 Examples of system output RGBD data

 

Papers

 

1. CHEN Lei, JIA Yunde and LI Mingxiang, An FPGA-Based RGBD Imager, Machine Vision and Applications, Machine Vision and Applications, 23(3):513-525, 2012. [PDF]

2. Lei Chen and Yunde Jia, A Parallel Reconfigurable Architecture for Real-Time Stereo Vision,  The International Conference on embedded Software and Systems (ICESS), Zhejiang, May, 2009. [PDF]

3. Yunde Jia, Xiaoxun Zhang, Mingxiang Li, Luping An, A Miniature Stereo Vision Machine (MVSM-Ⅲ) for Dense Disparity Mapping, International Conference on Pattern Recognition(ICPR2004), pages728-731, Cambridge, England. Aug. 2004. [PDF]

 

 

An Embedded Calibration Stereovision System

 

We develop an embedded calibration stereovision system that is able to be strongly online and auto calibrated without placing a calibration device in front of the stereo camera, but with the device hidden inside the cavity of the system via a half-mirror. The stereo camera simultaneously observes a scene passing through the half-mirror and the calibration device reflected from the half-mirror, and makes the formation of an embedded calibration stereo pair containing both scene and the calibration device. We use a polyhedral-mirror to generate multiple virtual images of the calibration device to occupy large part of a scene image for accurate estimation. We also use several mirrors to extend the optical path from the calibration device to the stereo camera for depth recovery of distance objects. The system can be easily used in a wide range of applications without considering variation of camera parameters.

Fig 1. A photograph of our system

 

Fig 2. An embedded calibration stereo pair

 

Papers

 

1. QIN Xiameng, YANG Jiaolong, LIANG Wei, PEI Mingtao and JIA Yunde, Stereo Camera Calibration with an Embedded Calibration Device and Scene Features, IEEE International Conference on Robotics and Biomimetics (ROBIO 2012), Guangzhou, China, 2012. [PDF]

2. JIA Yunde and QIN Xiameng, An Embedded Calibration Stereovision System, The Intelligent Vehicles Symposium (IV2012), Spain, June 2012. [PDF]

 

 

A Real-Time System for 3D Recovery of Dynamic Scene with Multiple RGBD Imagers

 

We develop a real-time 3D recovery system using multiple FPGA-based RGBD imagers. The RGBD imager developed in our lab produces color images combined with corresponding dense disparity maps encoding 3D information. Multiple RGBD imagers are externally triggered to sense the 3D world synchronously. The acquired 3D in formation of a dynamic scene from multiple viewpoints is then streamed to a PC cluster for further processing. Our system relies on depth maps and is much more suited for large scale dynamic scenes with multiple moving objects (e.g . people). 

 

Fig1. System overview

Video

Papers

1. DUAN Yong, CHEN Lei, WANG Yucheng, YANG Min, QIN Xiameng, HE Shaoyang and JIA Yunde, A Real-Time System for 3D Recovery of Dynamic Scene with Multiple RGBD Imagers, IEEE CVPR Workshop on Camera Networks & Wide Area Scene Analysis, USA, June, 2011. [PDF]