Single-photon LiDAR (SPL) based on single-photon avalanche diode sensing enables time-resolved photon measurements with extreme sensitivity, offering unique potential for active 3D perception. However, real-world single photon perception remains challenging because measurement noise and complex multi-return transient phenomena jointly complicate geometric reconstruction and semantic scene understanding. Existing studies are largely limited to simulated data or small-scale controlled captures.
We introduce SP-TransientBench (STB), a real-captured multi-task benchmark for single photon perception. STB provides full time-of-flight histograms, standardized metadata, calibrated camera poses, 13-class 3D semantic annotations, and histogram-domain multi-return annotations. Dedicated data splits and evaluation protocols enable consistent benchmarking across depth estimation, multi-view 3D reconstruction, and 3D semantic segmentation.
STB contains three complementary real-captured subsets. Every view provides a full 256 × 192 × 672 SPAD histogram, resulting in approximately 168.7 GB of transient data.
A single SPAD pixel may contain several temporally separated returns from different surfaces. Instead of assigning one label per pixel, STB represents semantic labels over temporal bins and uses sequential peak peeling to identify a dominant return, label its support interval, suppress it, and reveal weaker returns.
All benchmark experiments use SPAD-only inputs, without RGB guidance or RGB–SPAD fusion.
Depth is estimated directly from raw photon time-of-flight histograms and compared with Livox reference geometry using Chamfer Distance and Recall at 1-, 3-, and 5-bin temporal tolerances. Among the evaluated methods, SSPINET achieves the lowest CD of 1.4978 m and the highest Recall@5 of 93.68%.
This track reconstructs geometry and renders novel views from calibrated transient observations. Experiments use 3, 5, or 10 input views and evaluate intensity with SSIM/LPIPS, depth with L1, and rendered histograms with PSNR. TransientNeRF obtains a histogram PSNR of 45.4251 dB in the reported 10-view setting.
The semantic track combines histogram-domain preprocessing with PointNet++, PointMLP, Point Transformer, and PointNeXt. The official split contains 8,297 training samples and 2,000 test samples. Thresholding with PointMLP reaches 90.81% OA and 52.22% mIoU.
Simulation pretraining improves semantic segmentation when only limited real data are available, but its advantage diminishes as the amount of real training data increases. With the full training set, training from scratch is comparable or slightly better, showing that simulated transients cannot fully reproduce real sensor noise, multi-return effects, and photon statistics.
If you find SP-TransientBench useful in your research, please cite our paper.
@inproceedings{dong2026sptransientbench,
title = {SP-TransientBench: A Real-Captured Single Photon Perception Benchmark},
author = {Dong, Hongzhou and Zhang, Zili and Wen, Ziting and Qiang, Yiheng and Deng, Runrong and Dong, Wenle and Jiang, Ziwen and Li, Xinyang and Lu, Rui and Sun, Shuoyao and Wang, Wenyu and Xia, Ziyi and Zheng, Haitao and Shi, Guodong and Ren, Xiaoqiang},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2026}
}