Abstract

Light detection and ranging (Lidar) data can be used to capture the depth and intensity profile of a 3D scene. This modality relies on constructing, for each pixel, a histogram of time delays between emitted light pulses and detected photon arrivals. In a general setting, more than one surface can be observed in a single pixel. The problem of estimating the number of surfaces, their reflectivity, and position becomes very challenging in the low-photon regime (which equates to short acquisition times) or relatively high background levels (i.e., strong ambient illumination). This paper presents a new approach to 3D reconstruction using single-photon, single-wavelength Lidar data, which is capable of identifying multiple surfaces in each pixel. Adopting a Bayesian approach, the 3D structure to be recovered is modelled as a marked point process, and reversible jump Markov chain Monte Carlo (RJ-MCMC) moves are proposed to sample the posterior distribution of interest. In order to promote spatial correlation between points belonging to the same surface, we propose a prior that combines an area interaction process and a Strauss process. New RJ-MCMC dilation and erosion updates are presented to achieve an efficient exploration of the configuration space. To further reduce the computational load, we adopt a multiresolution approach, processing the data from a coarse to the finest scale. The experiments performed with synthetic and real data show that the algorithm obtains better reconstructions than other recently published optimization algorithms for lower execution times.

Keywords

  1. Bayesian inference
  2. 3D reconstruction
  3. Lidar
  4. low-photon imaging
  5. Poisson noise

MSC codes

  1. 62F15
  2. 62H12
  3. 62H35
  4. 62P30
  5. 62P12
  6. 65C40

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Material

Index of Supplementary Materials

Title of paper: Bayesian 3D Reconstruction of Complex Scenes from Single-Photon Lidar Data

Authors: J. Tachella, Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, S. McLaughlin, and J.-Y. Tourneret

File: M118397_01.pdf

Type: PDF

Contents: Complete RJ-MCMC acceptance rate formulas, details of some mathematical derivations used in the article and election of the impulse response in real Lidar data.

References

1.
D. Shin, F. Xu, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, Code and data used for “Computational multi-depth single-photon imaging,” https://github.com/photon-efficient-imaging/full-waveform (last accessed January 2019).
2.
Y. Altmann, R. Aspden, M. Padgett, and S. McLaughlin, A Bayesian approach to denoising of single-photon binary images, IEEE Trans. Comput. Imaging, 3 (2017), pp. 460--471.
3.
Y. Altmann, A. Maccarone, A. McCarthy, G. Newstadt, G. Buller, S. McLaughlin, and A. Hero, Robust spectral unmixing of sparse Lidar waveforms using gamma Markov random fields, IEEE Trans. Comput. Imaging, 3 (2017), pp. 658--670.
4.
Y. Altmann, S. McLaughlin, and M. Padgett, Unsupervised restoration of subsampled images constructed from geometric and binomial data, in 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), IEEE, 2018, pp. 1--5.
5.
Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, Lidar waveform-based analysis of depth images constructed using sparse single-photon data, IEEE Trans. Image Process., 25 (2016), pp. 1935--1946.
6.
Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, Robust Bayesian target detection algorithm for depth imaging from sparse single-photon data, IEEE Trans. Comput. Imaging, 2 (2016), pp. 456--467.
7.
L. Azzari and A. Foi, Variance stabilization for noisy+estimate combination in iterative Poisson denoising, IEEE Signal Process. Lett., 23 (2016), pp. 1086--1090.
8.
A. J. Baddeley and M. N. M. van Lieshout, Area-interaction point processes, Ann. Inst. Statist. Math., 47 (1995), pp. 601--619.
9.
R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, Model-based compressive sensing, IEEE Trans. Inform. Theory, 56 (2010), pp. 1982--2001.
10.
S. Brooks, A. Gelman, G. Jones, and X.-L. Meng, Handbook of Markov Chain Monte Carlo, CRC Press, Boca Raton, FL, 2011.
11.
P. L. Combettes and J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer, New York, 2011, pp. 185--212.
12.
O. Dikmen and A. T. Cemgil, Gamma Markov random fields for audio source modeling, IEEE Trans. Audio Speech Language Process., 18 (2010), pp. 589--601.
13.
D. Ferstl, C. Reinbacher, R. Ranftl, M. Ruether, and H. Bischof, Image guided depth upsampling using anisotropic total generalized variation, in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2013.
14.
J. Gao, J. Sun, J. Wei, and Q. Wang, Research of underwater target detection using a slit streak tube imaging lidar, in Proceedings of the Academic International Symposium on Optoelectronics and Microelectronics Technology (AISOMT), Harbin, China, 2012, pp. 240--243.
15.
P. J. Green, Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika, 82 (1995), pp. 711--732.
16.
T. Hakala, J. Suomalainen, S. Kaasalainen, and Y. Chen, Full waveform hyperspectral lidar for terrestrial laser scanning, Opt. Express, 20 (2012), pp. 7119--7127.
17.
A. Halimi, Y. Altmann, A. McCarthy, X. Ren, R. Tobin, G. S. Buller, and S. McLaughlin, Restoration of intensity and depth images constructed using sparse single-photon data, in Proceedings of the Signal Processing Conference (EUSIPCO), Budapest, Hungary, 2016, pp. 86--90.
18.
A. Halimi, A. Maccarone, A. McCarthy, S. McLaughlin, and G. S. Buller, Object depth profile and reflectivity restoration from sparse single-photon data acquired in underwater environments, IEEE Trans. Comput. Imaging, 3 (2017), pp. 472--484.
19.
A. Halimi, R. Tobin, A. McCarthy, S. McLaughlin, and G. S. Buller, Restoration of multilayered single-photon \textup3d lidar images, in Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), 2017, pp. 708--712.
20.
J. M. Hammersley and P. Clifford, Markov Fields on Finite Graphs and Lattices, manuscript, 1971.
21.
Z. T. Harmany, R. F. Marcia, and R. M. Willett, This is SPIRAL-TAP: Sparse Poisson Intensity Reconstruction ALgorithms---Theory and Practice, IEEE Trans. Image Process., 21 (2012), pp. 1084--1096.
22.
R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, UK, 2003.
23.
S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, Bayesian analysis of Lidar signals with multiple returns, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), pp. 2170--2180.
24.
S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, Multilayered \textup3D Lidar image construction using spatial models in a Bayesian framework, IEEE Trans. Pattern Anal. Mach. Intell., 30 (2008), pp. 1028--1040.
25.
A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. Wong, J. H. Shapiro, and V. K. Goyal, First-photon imaging, Science, 343 (2014), pp. 58--61.
26.
P. Lancaster and K. Salkauskas, Surfaces generated by moving least squares methods, Math. Comp., 37 (1981), pp. 141--158.
27.
A. Maccarone, A. McCarthy, X. Ren, R. E. Warburton, A. M. Wallace, J. Moffat, Y. Petillot, and G. S. Buller, Underwater depth imaging using time-correlated single-photon counting, Opt. Express, 23 (2015), pp. 33911--33926.
28.
C. Mallet and F. Bretar, Full-waveform topographic lidar: State-of-the-art, ISPRS J. Photogramm. Remote Sens., 64 (2009), pp. 1--16.
29.
C. Mallet, F. Lafarge, M. Roux, U. Soergel, F. Bretar, and C. Heipke, A marked point process for modeling lidar waveforms, IEEE Trans. Image Process., 19 (2010), pp. 3204--3221.
30.
A. McCarthy, R. J. Collins, N. J. Krichel, V. Fernández, A. M. Wallace, and G. S. Buller, Long-range time-of-flight scanning sensor based on high-speed time-correlated single-photon counting, Appl. Opt., 48 (2009), pp. 6241--6251.
31.
P. McCool, Y. Altmann, A. Perperidis, and S. McLaughlin, Robust Markov random field outlier detection and removal in subsampled images, in Proceedings of the Statistical Signal Processing Workshop (SSP), Palma de Mallorca, Spain, 2016, pp. 1--5.
32.
J. Mø ller and R. P. Waagepetersen, Modern statistics for spatial point processes, Scand. J. Statist., 34 (2007), pp. 643--684.
33.
I. Murray, Z. Ghahramani, and D. MacKay, MCMC for Doubly-Intractable Distributions, preprint, https://arxiv.org/abs/1206.6848, 2012.
34.
B. K. Natarajan, Sparse approximate solutions to linear systems, SIAM J. Comput., 24 (1995), pp. 227--234, https://doi.org/10.1137/S0097539792240406.
35.
M. Nilsson, Estimation of tree heights and stand volume using an airborne lidar system, Remote Sens. Environ., 56 (1996), pp. 1--7.
36.
T. Ogawa and K. Takagi, Lane recognition using on-vehicle lidar, in Proceedings of the Intelligent Vehicles Symposium, Tokyo, Japan, 2006, pp. 540--545.
37.
M. Pereyra, N. Whiteley, C. Andrieu, and J.-Y. Tourneret, Maximum marginal likelihood estimation of the granularity coefficient of a Potts-Markov random field within an MCMC algorithm, in Proceedings of the IEEE Workshop on Statistical Signal Processing (SSP), Gold Coast, Australia, 2014, pp. 121--124.
38.
J. Rapp and V. K. Goyal, A few photons among many: Unmixing signal and noise for photon-efficient active imaging, IEEE Trans. Comput. Imaging, 3 (2017), pp. 445--459.
39.
H. Rue and L. Held, Gaussian Markov Random Fields: Theory and Applications, CRC Press, Boca Raton, FL, 2005.
40.
D. Scharstein and C. Pal, Learning conditional random fields for stereo, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07), 2007, pp. 1--8.
41.
B. Schwarz, LIDAR: Mapping the world in 3D, Nature Photonics, 4 (2010), pp. 429--430.
42.
D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors, IEEE Trans. Comput. Imaging, 1 (2015), pp. 112--125.
43.
D. Shin, F. Xu, F. N. Wong, J. H. Shapiro, and V. K. Goyal, Computational multi-depth single-photon imaging, Opt. Express, 24 (2016), pp. 1873--1888.
44.
D. L. Snyder and M. I. Miller, Random Point Processes in Time and Space, Springer Science & Business Media, 2012.
45.
J. B. Tenenbaum, V. de Silva, and J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, 290 (2000), pp. 2319--2323.
46.
R. Tobin, A. Halimi, A. McCarthy, X. Ren, K. J. McEwan, S. McLaughlin, and G. S. Buller, Long-range depth profiling of camouflaged targets using single-photon detection, Opt. Eng., 57 (2017), 031303.
47.
M. N. M. van Lieshout, Markov Point Processes and Their Applications, World Scientific, 2000.
48.
A. M. Wallace, A. McCarthy, C. J. Nichol, X. Ren, S. Morak, D. Martinez-Ramirez, I. H. Woodhouse, and G. S. Buller, Design and evaluation of LiDAR for the recovery of arboreal parameters, IEEE Trans. Geosci. Remote Sens., 52 (2014), pp. 4942--4954.
49.
M. Zhou, H. Chen, L. Ren, G. Sapiro, L. Carin, and J. W. Paisley, Non-parametric Bayesian dictionary learning for sparse image representations, in Advances in Neural Information Processing Systems 22 (NIPS 2009), Curan Associates, 2009, pp. 2295--2303.
50.
M. Zhou, L. Hannah, D. B. Dunson, and L. Carin, Beta-negative binomial process and Poisson factor analysis, in Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 22, La Palma, Canary Islands, 2012, pp. 1462--1471.

Information & Authors

Information

Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 521 - 550
ISSN (online): 1936-4954

History

Submitted: 30 April 2018
Accepted: 27 December 2018
Published online: 14 March 2019

Keywords

  1. Bayesian inference
  2. 3D reconstruction
  3. Lidar
  4. low-photon imaging
  5. Poisson noise

MSC codes

  1. 62F15
  2. 62H12
  3. 62H35
  4. 62P30
  5. 62P12
  6. 65C40

Authors

Affiliations

Funding Information

STIC-AmSud Project HyperMed
Centre National de la Recherche Scientifique https://doi.org/10.13039/501100004794
QuantIC https://doi.org/10.13039/100012927
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/N003446/1, EP/M01326X/1, EP/K015338/1
Royal Academy of Engineering https://doi.org/10.13039/501100000287 : RF201617/16/31

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

There are no citations for this item

View Options

View options

PDF

View PDF

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media