Description

About Sparse-MVS

The sparse-MVS leader-board is based on two datasets, DTU dataset and Tanks-and-Temples (T&T) dataset. The sparse MVS setting in our leader-board selects a small proportion of the camera views by consecutively sampling a view from every sparsity = n camera index, i.e., {1, n + 1, 2n + 1, · · · }. In reality, it is also practical to sample small-batches of images at sparse viewpoints, i.e., grouping batches of views with certain Batchsize at the previously defined sparse viewpoints with a certain Sparsity. When Sparsity = 3 and Batchsize = 1, the chosen camera indexes are 1 / 4 / 7 / 10 / · · · . When Sparsity = 3 and Batchsize = 2, the chosen camera indexes are 1,2 / 4,5 / 7,8 / 10,11 / · · · .

The figure below depicts the relationship between sparsity n and the average baseline angle θ averaging over all the ground-truth points in the 22 models of the DTU dataset and 8 models of the Tanks and Temples intermediate dataset respectively. Note that, for simplicity, only the nearest view pairs are considered to calculate the baseline angle statistics.

As the sparsity increases n = 1, ..., 11, the average baseline angle θ, defined by the intersected projection rays, gradually grows in a large range, e.g. reaching more than 70° in both DTU and T&T datasets. So this sparse-MVS setting is reasonable by not only covering various degrees of sparsity but also containing irregular sampling locations.

Evaluation

The precision and recall have two metrics: the distance metric and the percentage metric. The distance metric is the original DTU dataset matlab code (download). The percentage metric is changed by the evaluation code provided by Tanks and Temples (download). The overall score for the percentage metric is measured as the f-score, and a similar measurement for the distance metric overall is given by the average of the mean precision and mean recall.