Kiyoshi Matsuo¹ Yoshimitsu Aoki²

¹Hokuyo Automatic Co., Ltd.,  ²Keio University

This paper describes a depth image enhancement method for consumer RGB-D cameras. Most existing methods use the pixel-coordinates of the aligned color image. Because the image plane generally has no relationship to the measured surfaces, the global coordinate system is not suitable to handle their local geometries. To improve enhancement accuracy, we use local tangent planes as local coordinates for the measured surfaces. Our method is composed of two steps, a calculation of the local tangents and surface reconstruction. To accurately estimate the local tangents, we propose a color heuristic calculation and an orientation correction using their positional relationships. Additionally, we propose a surface reconstruction method by ray-tracing to local tangents. In our method, accurate depth image enhancement is achieved by using the local geometries approximated by the local tangents.

Algorithm overview :                          [ high resolution (MP4, 47.8 MB), low resolution (MP4, 5.4 MB) ]
Evaluation on real sensor data : [ high resolution (MP4, 42.0 MB), low resolution (MP4, 5.0 MB) ]

Depth Image Enhancement Using Local Tangent Plane Approximations, Kiyoshi Matsuo and Yoshimitsu Aoki.
In Computer Vision and Pattern Recognition (CVPR), 2015. [ Preprint, Extended abstract, Supplementary materials, Poster ]

Depth Image Enhancement Using Local Tangent Plane Approximations, Kiyoshi Matsuo and Yoshimitsu Aoki.
( Invited talk in the 18th Meeting on Image Recognition and Understanding (MIRU). July 29, 2015. ) [ Slides, Poster ]

Depth Image Enhancement Using Local Tangent Plane Approximations, Kiyoshi Matsuo and Yoshimitsu Aoki.
( Talk in the 17th Image Processing Tokyo. November 26, 2015. ) [ Slides ]

Original codes used in the paper can not be distributed by some regulations and
I could not share any codes of this algorithm. ( I'm sorry... )

@InProceedings{Matsuo_2015_CVPR,
   author = {Matsuo, Kiyoshi and Aoki, Yoshimitsu},
   title = {Depth Image Enhancement Using Local Tangent Plane Approximations},
   booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
   month = {June},
   year = {2015}
}

Kiyoshi Matsuo (primary contact)	[URL]	[e-mail]
Yoshimitsu Aoki	[URL]	[e-mail]