Here we provide a more complete set of statistics from the results
presented in the ICRA'15 paper Beyond Points: Evaluating Recent
3D Scan-Matching Algorithms by Magnusson,
Vaskevicius, Stoyanov, Pathak, and Birk.
For more detail about the benchmark protocol that the results are
based on, please refer to Comparing ICP variants on real-world data sets by Pomerleau et al. and
the
associated web site.
The plots below summarise the results of the local scan-matching
methods P2D-NDT and D2D-NDT in comparison with point-to-plane ICP.
These are cumulative probability plots, showing the proportion of outcomes
below any given error.
The final translation error (in metres) is shown on the horizontal axis, and the vertical axis shows the cumulative probability.
Percentiles for 5, 25, 50, 75, and 95 are shown with dashed horizontal lines. The plots also include a suggested success threshold (10 cm).
Additional plots will be added later.
Complete benchmark
Here are the results for the complete benchmark protocol, including all scan pairs and pose perturbations.
Sensitivity to overlap ratio
The following plots relate to the sensitivity to the amount of overlap between the two pairs.
Large overlap only
Here are the results for all scan pairs with more than 75% overlap, but including all pose perturbations.
Small overlap only
Here are the results for all scan pairs with less than 50% overlap, but including all pose perturbations.
Sensititivy to pose estimate
Easy poses only
Here are the results for the easy pose perturbations only, but including all scan pairs.
Hard poses only
Here are the results for the hard pose perturbations only, but including all scan pairs.
All results, tabulated
In the table below, the accuracy
results are categorised by initial pose and overlap ratio,
respectively.
Entries where
both translation and rotation error are below the thresholds
0.10 m and 0.05 rad are shaded.
Entries where translation or rotation is below the
threshold are lightly shaded.
References
F. Pomerleau, F. Colas, R. Siegwart, and S. Magnenat, Comparing ICP variants on real-world data sets, Autonomous Robots, Volume 34, Issue 3, pp 133-148, Feb. 2013.
M. Magnusson,
N. Vaskevicius,
T. Stoyanov,
K. Pathak,
and A. Birk,
Beyond Points: Evaluating Recent 3D Scan-Matching Algorithms,
Proc. ICRA, 2015.