0.23.4 - Potential regression with poisson mesh reconstruction?

[antoinemacia]

HI!

We've spotted a regression on some of the scan processing we do on the latest version using rtabmap-export - it seems the poisson mesh reconstruction points end up being filtered completely - I can confirm starting to see this issue since ~4 days ago

Create and assemble the clouds... done (35s, 10497024 points).
Computing normals of the assembled cloud... done! (3s, 10497024 points)
Mesh reconstruction... depth=11
Mesh reconstruction... done (23s, 0 polygons).

This is the command I ran

rtabmap-export \
  --mesh \
  --texture \
  --opt 2 \
  --texture_size 4096 \
  --texture_range 5 \
  --poisson_depth 11 \
  --poisson_size 0.01 \
  --max_polygons 6000000 \
  --decimation 0 \
  --min_cluster 0 \
  --max_range 0 \
  --voxel 0 \
  --color_radius 0.05 \
  --depth_confidence 100 \
  --poses_format 11 \
  <database.db>

@matlabbe anything that may have changed on this front? I can provide databases if needed be

[matlabbe]

There should not have related changes to Poisson in latest releases, unless rtabmap is built against a newer PCL version than I tested.

If you can share one database reproducing that issue with that command above, I could check on my side.

EDIT: If you try with a smaller poisson depth (--poisson_depth 9), does it work? You could also set a --max_range 30 (instead of infinity) in case there is some noise in the depth image with very large values.

[antoinemacia]

@matlabbe I tested further, I can see the issue is a combination of --depth_confidence 100 and --color_radius 0.05

Command

rtabmap-export \
  --mesh \
  --texture \
  --opt 2 \
  --texture_size 4096 \
  --texture_range 5 \
  --poisson_depth 11 \
  --poisson_size 0.01 \
  --max_polygons 6000000 \
  --decimation 0 \
  --depth_confidence 100 \
  --min_cluster 0 \
  --max_range 0 \
  --voxel 0 \
  --color_radius 0.05 \
  --images_id \
  --poses_camera \
  --poses_format 11 \
  --output_dir . \
  output.db

I think the disparity in versions was running export on an older version that doesnt support depth_confidence, which is relatively recent

Removing --depth_confidence 100 seems to fix the issue

[matlabbe]

I tried the "Not working" command down to 0.22.0-humble version, and it was still failing. After using --cloud option to debug the actual point cloud used by Poisson, if the cloud is opened in MeshLab, it will complain of a lot of NaN points. The issue is that with --voxel 0, NaN points were used by Poisson, causing it to fail.

I fixed the issue in that commit: b3b8fbd

Side comments on the data, I see that even with --depth_confidence 100, the sky is kept in the point cloud, causing an effect like this:

Looking at the data in rtabmap-databaseViewer with depth confidence colormap enabled, here the 2D view and the corresponding point cloud created from that frame at 100% depth confidence (green is high confidence):


Side view at 0% and 100% confidence, we can see only the noise disappears at high confidence, but the sky stays there:

It looks like ARKit is reporting high confidence for the sky, while based on the documentation it should not. Gemini says also the same thing:

Based on Apple's ARKit documentation and technical specifications, ARKit does not typically report high depth confidence for the sky.

To remove the sky, just add --max_range 10 to your command.

EDIT: Just saw your new reply while making this answer. Removing --depth_confidence 100 did work without my new fix, but not sure why because there are still many NaN points in the point cloud. Still better to update to latest commit.

[antoinemacia]

Great, thanks!