diff --git a/README.md b/README.md index ed2e6af..9ce9971 100644 --- a/README.md +++ b/README.md @@ -353,10 +353,12 @@ By default, prepared data, caches, and experiments live under where the command is run. To use other paths, create `./vidmap-paths.toml` or set `VIDMAP__{DATA,CACHE,EXP,TESTSETS}_DIR`. +Spires and Hilti require the ROS bag reader (`pip install -e '.[dataset-preparation]'`). + Then select and prepare a benchmark dataset: ```bash -DATASET=lamar # or: euroc, eth3d_slam, crocodl +DATASET=lamar # or: euroc, eth3d_slam, crocodl, spires, hilti python -m vidmap.datasets.prepare --dataset "$DATASET" ``` @@ -366,7 +368,9 @@ prepares only the images referenced by its benchmark testset: sessions and images absent from that testset are not retained. Downloads resume after interruption and completed sequences are reused. Validated source archives are retained by default; pass `--delete-downloads` to remove them after successful -preparation. +preparation. EuRoC and ETH3D-SLAM prepare only cam0 by default; `--mode stereo` +or `--mode all` prepares both cameras, reusing existing cam0 images in the same +folder. For Spires and Hilti, add `--workers N` to prepare N recordings concurrently.
[Expected prepared dataset layouts - click to expand] @@ -375,16 +379,24 @@ Storage for the supported evaluation inventories: | Dataset | Download | Prepared | | --- | ---: | ---: | -| LaMAR | 45.2 GB | 24.0 GB | -| EuRoC | 24.7 GB | 4.7 GB | -| ETH3D-SLAM | 18.6 GB | 18.6 GB | +| LaMAR (sample-300) | 45.2 GB | 24.0 GB | +| LaMAR (complete) | 144.5 GB | 76.7 GB | +| EuRoC (monocular) | 24.7 GB | 4.7 GB | +| EuRoC (rig) | 24.7 GB | 9.6 GB | +| ETH3D-SLAM (monocular) | 18.6 GB | 18.6 GB | +| ETH3D-SLAM (rig) | 37.3 GB | 37.3 GB | | CroCoDL | 15.0 GB | 14.0 GB | +| Oxford Spires (rectified monocular, default) | 89.7 GB | 29.3 GB | +| Oxford Spires (fisheye monocular) | 89.7 GB | 15.8 GB | +| Oxford Spires (rectified rig) | 89.7 GB | 89.8 GB | +| Hilti 360° | 132.1 GB | 120 GB | Download sizes are the checksum-pinned source archives; prepared sizes cover the canonical dataset after conversion. LaMAR archives are session-level, so the download includes unused frames inside each required session ZIP; only the -94,489 testset images are retained. Retaining downloads requires roughly the sum -of both columns, plus conversion staging. +selected testset images are retained. Retaining downloads requires roughly the sum +of both columns, plus conversion staging. Hilti's prepared size is estimated from +120 panoramas sampled across all 30 recordings. After preparation, the selected dataset is available under `./local/datasets//`, with its benchmark definitions under @@ -404,6 +416,8 @@ local/ | --- | --- | | LaMAR, ETH3D-SLAM, CroCoDL | `native` | | EuRoC | `rectified` | +| Oxford Spires | `native`, `rectified` | +| Hilti 360° | `panorama` |
[Examples for each dataset - click to expand] @@ -418,24 +432,45 @@ local/datasets/ ├── euroc/machine_hall/MH_01_easy/ │ ├── preparation.json │ └── rectified/ -│ ├── images/cam0/... +│ ├── images/cam0/... # cam1/ when stereo is prepared │ └── rec/ ├── eth3d_slam/plant_1/0/ │ ├── preparation.json │ └── native/ +│ ├── images/cam0/... # cam1/ when stereo is prepared +│ └── rec/ +├── crocodl/ios-ARCHE_D2/ios_2023-07-12_20.46.14_000/ +│ ├── preparation.json +│ └── native/ │ ├── images/cam0/... │ └── rec/ -└── crocodl/ios-ARCHE_D2/ios_2023-07-12_20.46.14_000/ +├── spires/2024-03-12-keble-college-02/0/ +│ ├── preparation.json +│ ├── native/ +│ │ ├── images/cam0/... +│ │ └── rec/ +│ └── rectified/ +│ ├── images/ +│ │ ├── cam0/... +│ │ ├── cam1/... +│ │ └── cam2/... +│ └── rec/ +└── hilti/floor_1_2025-05-05_run_1/0/ ├── preparation.json - └── native/ + └── panorama/ ├── images/cam0/... └── rec/ local/testsets/ ├── lamar/CAB/sample.yaml -├── euroc/machine_hall/all.yaml -├── eth3d_slam/plant_1/all.yaml -└── crocodl/ios-ARCHE_D2/complete.yaml # Also sample-*.yaml +├── euroc/machine_hall/mono.yaml +├── eth3d_slam/plant_1/mono.yaml +├── crocodl/ios-ARCHE_D2/complete.yaml # Also sample-*.yaml +├── spires/2024-03-12-keble-college-02/ +│ ├── undistorted-mono.yaml +│ ├── fisheye-mono.yaml +│ └── undistorted-rig.yaml +└── hilti/floor_1_2025-05-05_run_1/panorama.yaml ```
@@ -448,8 +483,7 @@ Select a benchmark with `--mode`; its testset specifies the recording, image var ### Run and evaluate ```bash -# Use uncalib/base for datasets other than LaMAR. -FRONTEND_CONF=uncalib/gtkf-base +FRONTEND_CONF=uncalib/base MAPPING_CONF=uncalib/base # Omit --name below to derive the output name from the config pair. OUTPUT_NAME=default @@ -471,9 +505,7 @@ skip only this final report; per-trajectory evaluation still runs.
[LaMAR - click to expand] -Evaluate the default `sample` benchmark, a representative subset of each -ground-truth trajectory as outlined in the paper. LaMAR selects `wate_auc` by -default. +LaMAR defaults to `complete` with `wate_auc` evaluation. ```bash python -m vidmap.run_for_benchmark \ @@ -482,12 +514,25 @@ python -m vidmap.run_for_benchmark \ --mapping-conf uncalib/base ``` +For the trajectories used in the paper, prepare `sample-300` and run `sample`: + +```bash +python -m vidmap.datasets.prepare -d lamar --profile sample-300 + +python -m vidmap.run_for_benchmark \ + --dataset lamar --mode sample \ + --frontend-conf uncalib/gtkf-base \ + --mapping-conf uncalib/base +``` + +Use separate dataset/testset roots to keep both profiles. +
[EuRoC - click to expand] -Evaluate the complete EuRoC benchmark. EuRoC selects `pose_auc` by default. +EuRoC defaults to `mono` with `pose_auc` evaluation. ```bash python -m vidmap.run_for_benchmark \ @@ -496,13 +541,21 @@ python -m vidmap.run_for_benchmark \ --mapping-conf uncalib/base ``` +For calibrated stereo: + +```bash +python -m vidmap.run_for_benchmark \ + --dataset euroc --mode stereo \ + --frontend-conf calib/stereo \ + --mapping-conf calib/base +``` +
[ETH3D-SLAM - click to expand] -Evaluate the complete ETH3D-SLAM benchmark. ETH3D-SLAM selects `pose_auc` by -default. +ETH3D-SLAM defaults to `mono` with `pose_auc` evaluation. ```bash python -m vidmap.run_for_benchmark \ @@ -511,6 +564,15 @@ python -m vidmap.run_for_benchmark \ --mapping-conf uncalib/base ``` +For calibrated stereo: + +```bash +python -m vidmap.run_for_benchmark \ + --dataset eth3d_slam --mode stereo \ + --frontend-conf calib/stereo \ + --mapping-conf calib/base +``` +
@@ -527,6 +589,47 @@ python -m vidmap.run_for_benchmark \
+
+[Oxford Spires - click to expand] + +Oxford Spires has 13 scenes and defaults to rectified monocular images +(`undistorted-mono`). Select a mode with `--mode`. + +| Mode | Frontend config | Mapping config | +| --- | --- | --- | +| `undistorted-mono` (default) | `uncalib/base` | `uncalib/base` | +| `undistorted-mono` | `calib/base` | `calib/base` | +| `fisheye-mono` | `uncalib/fisheye` | `uncalib/geocalib` | +| `undistorted-rig` | `calib/rig` | `calib/base` | + +```bash +python -m vidmap.datasets.prepare --dataset spires --mode undistorted-rig + +python -m vidmap.run_for_benchmark \ + --dataset spires --mode undistorted-rig \ + --frontend-conf calib/rig --mapping-conf calib/base +``` + +Evaluation uses scale-aligned WATE-AUC on cam0. + +
+ +
+[Hilti 360° - click to expand] + +Hilti has 30 sequences and defaults to `panorama` mode. Select perspective views +with [`rig.projection`](docs/cameras.md#panoramas). + +```bash +python -m vidmap.run_for_benchmark \ + --dataset hilti \ + --frontend-conf calib/panorama --mapping-conf calib/base +``` + +Evaluation uses scale-aligned WATE-AUC on view 0. + +
+
[Results - click to expand] @@ -539,6 +642,10 @@ better), evaluated at GT-supported estimated poses: | :--- | ---: | ---: | ---: | ---: | ---: | ---: | | LaMAR (sample-300, 50 sequences) | 0.878 | 92.43 | 93.08 | 91.79 | 91.03 | 90.41 | | CroCoDL (64 sequences) | 0.233 | 95.72 | 97.27 | 97.39 | 96.56 | 98.21 | +| LaMAR (complete, 53 evaluated sequences) | 1.452 | 92.42 | 92.83 | 92.38 | 92.59 | 89.76 | +| Oxford Spires (rectified monocular, 10 sequences) | 0.130 | 98.98 | 99.10 | 99.11 | 99.10 | 99.54 | +| Oxford Spires (three-camera rig, 13 sequences) | Pending | Pending | Pending | Pending | Pending | Pending | +| Hilti (three-view, 30 sequences) | Pending | Pending | Pending | Pending | Pending | Pending | Pose AUC, averaged equally across sequences (%, higher is better): diff --git a/pyproject.toml b/pyproject.toml index cfd39b7..1eea53e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -38,6 +38,9 @@ dependencies = [ "geocalib @ git+https://github.com/cvg/GeoCalib.git@982f625d822fbb2b50c65d123fd1362a5c01ec2c", ] +[project.optional-dependencies] +dataset-preparation = ["rosbags>=0.10,<0.12"] + [tool.scikit-build] minimum-version = "build-system.requires" cmake.version = ">=3.28" diff --git a/vidmap/configs/frontend/calib/stereo.yaml b/vidmap/configs/frontend/calib/stereo.yaml new file mode 100644 index 0000000..e8b3037 --- /dev/null +++ b/vidmap/configs/frontend/calib/stereo.yaml @@ -0,0 +1,7 @@ +defaults: + - rig + +rig: + keyframe_camera: 0 + match_same_timestamp: [[0, 1]] + flag_same_timestamp_as_sequential: [[0, 1]] diff --git a/vidmap/datasets/assets/benchmark_suites.json b/vidmap/datasets/assets/benchmark_suites.json new file mode 100644 index 0000000..55d370f --- /dev/null +++ b/vidmap/datasets/assets/benchmark_suites.json @@ -0,0 +1,61 @@ +{ + "spires": { + "scenes": [ + "2024-03-12-keble-college-02", + "2024-03-12-keble-college-03", + "2024-03-12-keble-college-04", + "2024-03-12-keble-college-05", + "2024-03-13-observatory-quarter-01", + "2024-03-13-observatory-quarter-02", + "2024-03-14-blenheim-palace-01", + "2024-03-14-blenheim-palace-02", + "2024-03-14-blenheim-palace-05", + "2024-03-18-christ-church-02", + "2024-03-18-christ-church-03", + "2024-03-20-christ-church-05", + "2024-05-20-bodleian-library-02" + ], + "modes": { + "undistorted-mono": {"variant": "rectified", "num_cameras": 1}, + "fisheye-mono": {"variant": "native", "num_cameras": 1}, + "undistorted-rig": {"variant": "rectified", "num_cameras": 3} + } + }, + "hilti": { + "scenes": [ + "floor_1_2025-05-05_run_1", + "floor_1_2025-07-07_run_1", + "floor_1_2025-12-02_run_1", + "floor_2_2025-05-05_run_1", + "floor_2_2025-10-28_run_1", + "floor_2_2025-10-28_run_2", + "floor_2_2025-12-02_run_1", + "floor_2_2025-12-03_run_1", + "floor_3_2025-05-19_run_1", + "floor_3_2025-12-02_run_1", + "floor_4_2025-05-19_run_1", + "floor_4_2025-12-02_run_1", + "floor_5_2025-12-02_run_1", + "floor_6_2025-06-18_run_1", + "floor_6_2025-07-07_run_1", + "floor_6_2025-12-02_run_1", + "floor_6_2025-12-02_run_2", + "floor_7_2025-12-02_run_1", + "floor_7_2025-12-02_run_2", + "floor_7_2025-12-03_run_1", + "floor_EG_2025-10-16_run_1", + "floor_EG_2025-12-02_run_1", + "floor_EG_2025-12-02_run_2", + "floor_UG1_2025-05-19_run_1", + "floor_UG1_2025-06-18_run_1", + "floor_UG1_2025-10-16_run_1", + "floor_UG1_2025-12-02_run_1", + "floor_UG1_2025-12-02_run_2", + "floor_UG1_2025-12-03_run_1", + "floor_UG2_2025-12-02_run_1" + ], + "modes": { + "panorama": {"variant": "panorama", "num_cameras": 1} + } + } +} diff --git a/vidmap/datasets/assets/eth3d_slam/stereo_archives.tsv b/vidmap/datasets/assets/eth3d_slam/stereo_archives.tsv new file mode 100644 index 0000000..23cb1bc --- /dev/null +++ b/vidmap/datasets/assets/eth3d_slam/stereo_archives.tsv @@ -0,0 +1,56 @@ +scene split size http_etag sha256 +cables_1 training 468306334 """5cfba8ec-1be9c99e""" 62e9dec9d43f68703cd68207d9042eaaf135028a485618e03cad1c0db165cc77 +cables_2 training 37573372 """5cfba99d-23d52fc""" 2b73a4754ff4638fd6ef7f6b707770a6957fa9a3b80a79404eb870fbda9dc41e +cables_3 training 117435264 """5cfba9b0-6ffeb80""" 48ce2abfe828bf5ed8262831fa62832652426c4ccb0aa6d6ffbae38be14ad526 +camera_shake_1 training 106974159 """5cfba1e2-6604bcf""" a99161a4edc5f7d96ab76322e3c75854bb7ff138edd0d5c029fecea99648cef9 +camera_shake_2 training 286022211 """5cfba218-110c5a43""" f72c8242245fabc059d85794fa6f30b7aeb8a8d8738a3d61d5de3c14ae2d9287 +camera_shake_3 training 91536560 """5cfba288-574bcb0""" acc0b20e33a68107d56cfcabed0bcd759a614a6739f7d302405878b5feb05df1 +ceiling_1 training 566799271 """5cfba9f7-21c8aba7""" 951cd627b51397feaf455d44609c746a04186c49152d23e6a8dd5d323865d22d +ceiling_2 training 560256988 """5cfbaae3-2164d7dc""" be1f2e07d8f6b19483a223302369ad1ca3355b36da7dae5bb7625af011b21c94 +desk_3 training 810709135 """5cfbcba2-3052708f""" 72064c72412832e74ef41a594dc15d0dfbca363ed403ded1a819fa6c5c125b88 +desk_changing_1 training 1447857687 """5cfbcd30-564c8e17""" fe952331593677a1f76912dd7c00d897e8460610c8d0a1ab7154a14476558ff2 +einstein_1 training 179509025 """5cfbb47e-ab31721""" c264d228f3b757c75b14cf5ac3c8cf2308ca87d5d265a8c7112c2bc3d3a1c2d6 +einstein_2 training 528939870 """5cfbb4e4-1f86fb5e""" 1b58e596fa636f057fca3a8f83a9bfebfa0a0a322d63f5cccadcb735a872966c +einstein_flashlight training 593871147 """5cfbb6ce-2365c12b""" 168ccd4fd6032ff8f1754a760d9449bca3eff10a736f7b922d95cf32ce625bca +einstein_global_light_changes_1 training 252793489 """5cfbb7ba-f115291""" 61ed864d2f60cec06b671a4c4895b7478a7f6d685b96c861d95efaaa5d72d9c1 +einstein_global_light_changes_2 training 175055894 """5cfbb820-a6f2416""" 82423843b9c77b39af980b1915c553937353c3775fcfc294584e3bb409b44736 +einstein_global_light_changes_3 training 788101180 """5cfbb88d-2ef9783c""" d726bfbb60637f38b3fa319c2cf95a7705f17f73b2b6e6e1f323c5aa3712e54b +kidnap_1 training 322666172 """5cfbc213-133b7ebc""" 95fe4edc61916405db40e7bf06b32211828e765d572a35b8bccecf277f8b6b2d +large_loop_1 training 497813405 """5cfba3c0-1dac079d""" a8d3061b6bcdbee5044811400bcac9398cfb1636fd9a6f38d0e6145317e904ed +mannequin_1 training 230855524 """5cfba7e2-dc29364""" d4e4e6f92b1d1ceb9982068f4abfb428b8a3e58b7259b68392b9ee92bb05a4a5 +mannequin_3 training 215029137 """5cfba5b9-cd11591""" c918d453c7bc095c74f211bb76538f0f324fabcb67491d99a10af8d36e738712 +mannequin_4 training 185035148 """5cfba614-b07698c""" 491665d75e3283bbde1f3efa7533d2b439ec4649339330977f265e53a397d8d1 +mannequin_5 training 297932879 """5cfba66b-11c2184f""" bba6d1f9bba1e6a2ee38b1848b19aa3faa8a291e08cb760239d91ec806556144 +mannequin_7 training 223401150 """5cfba842-d50d4be""" 9a2da8300fb898650ddbcc7e23793ae59c391c17e48e2a647c08b4bd0c990cbe +mannequin_face_1 training 145937658 """5cfba73e-8b2d4fa""" 00738e4900fec180d50f0597f4c97c9352925fc9fd719c33636c69dad46eded2 +mannequin_face_2 training 160830114 """5cfba89d-99612a2""" 71faa4c8551da547429d944f5378cd464fd313a2456021592941f5e4ffbc8c2d +mannequin_face_3 training 209410079 """5cfba6e7-c7b581f""" 34f695285e04d53a217609714713235f4d636db93b5c83586c190af5b3083b18 +mannequin_head training 232104066 """5cfba782-dd5a082""" cc9c8d5256be577c3fa4f734389dac8e879425b042b7361f3fff716163fbf200 +motion_1 training 837644219 """5cfbb9de-31ed6fbb""" 55989272c1536370c107b43c770ba9b921d4440fade81060e114f96c2f8348da +planar_2 training 309081903 """5cfba488-126c372f""" 38bcbb0cf452a796349983839dbf6ffe8f0aa59ab6df8823b7cb3f9e1323eb4c +planar_3 training 426904075 """5cfba50d-19720a0b""" b263db944759673817d8e1317ac79af93130f9c6cf3e96e708a932c0be78b466 +plant_1 training 23623541 """5cfbb2e1-1687775""" b66d68a9e232eede4148fde0a91b1caaac32c446a4a7a61f858c49127e69424b +plant_2 training 45636091 """5cfbb2ec-2b859fb""" 2f7fc740849f70571118ca7df41b6232bb01fa6414c4feafa44334f6738a1ec9 +plant_3 training 67715213 """5cfbb301-409408d""" 336cedb4143ac8d9371b47281c24e0b915bb34e01be1aa5cc99cf1bf1b372ad7 +plant_4 training 32432886 """5cfbb31c-1eee2f6""" 38f7dc4592b26d7322412d23c21242ffcf2c703f9afadfedbe425d6ccdea4e0f +plant_5 training 62725140 """5cfbb32b-3bd1c14""" b6cdf6ed69fb0eb9e699fbb886a339c8dfd38cdf8ad4a9a3c8ee8df306a5240d +plant_scene_1 training 251665904 """5cfbb351-f001df0""" 26e0579c8e895e825c8585acb2be02632c1740fd091fdc8cc067a59418d724e5 +plant_scene_2 training 239439708 """5cfbb3bc-e458f5c""" 8a7d4daae1360513803da98b8c5eff88179fe68fd820bf6739d282bedc52f07a +plant_scene_3 training 212738765 """5cfbb423-cae22cd""" 600469371dcdf382e732092779244917bf3a27a2a4ad55da49c2781752836af7 +reflective_1 training 1645498742 """5cfbaf6f-62145176""" a3f1600855394ed600bc2e11b3238c20095a97db28617d5c3a8b8e4c9b80db4f +repetitive training 690482274 """5cfba0d9-2927ec62""" 187ecea76afd6a549d40b1f8016ca91d6c3d36ec22e464eb851fc9838ee01e5e +sfm_bench training 258391106 """5cfbd28a-f66bc42""" ffa1ac63401c479d53f6745b146508881961a7c9189f1ddd0a17aab4513d6e36 +sfm_garden training 355485017 """5cfbd18c-15304559""" fb87f4dfd325852d9788123e344b37a8198298612ae63b493aa339ae7f8ea24e +sfm_house_loop training 254016693 """5cfbd21d-f23fcb5""" c8780810ece5d1402862f62abf4d7e7264445b790c2007fdd8f1619b6b01ffcf +sfm_lab_room_1 training 94129054 """5cfbd2f1-59c4b9e""" 188fa66dfc35b3d23619009c7be482f77d72434f97d874ce62498b1ccefbe70e +sfm_lab_room_2 training 39590086 """5cfbd318-25c18c6""" ffc483753d34d2297a32db39a40383fdba559ee68679526b97bc97e1bcfe1e10 +sofa_1 training 342654754 """5cfb9b01-146c7f22""" f2de223f81f995cd715a4d465e485772fb24b862b4f2b7d7a61228f37de2b491 +sofa_2 training 242122750 """5cfb9b93-e6e7ffe""" 2c9773f36de6baeaa911154692071897e37fbb1e956197d4e7435b19517ba67a +sofa_3 training 187236617 """5cfb9bfb-b290109""" 9bad0e06b38d4bd95cc5fbc80be17872b3ec07f0a39c511a61b8c752cce07add +sofa_4 training 276821209 """5cfb9c53-107ff4d9""" fa5a2758db5f9cec3576ec074c07f363e381860065d5d4d9abea32d98575547c +sofa_shake training 176146324 """5cfb9cca-a7fc794""" 02dac10b0a3ea83ba2c097f4d4b83e73bd517c0e05ed7cd6a95a4fc7b417d738 +table_3 training 449825872 """5cfb97a5-1acfcc50""" 4f370e6df65f98ea80c3092a423164abac0549b8e0b528fade6927eef7f39fec +table_4 training 372468096 """5cfb985f-16336980""" 235dcc288a1c338f9db8606e25f87427800aae9a44b6913fada822526d9a0dbf +table_7 training 600047762 """5cfba2c4-23c40092""" 75f58be72b79150e61ec83c47c9331b028bc1c835c69d6aa566d8c05b25a328a +vicon_light_1 training 298455960 """5cfbabc1-11ca1398""" 8ea79f2536969cd07e83af72bd8615fb67d1131f995cee985e04ca19a72b72f5 +vicon_light_2 training 183749995 """5cfbac39-af3cd6b""" 3a3437546448893e514c5666bda11d2b0cc5a1c9a9ad8e3c17d1e218a43fc723 diff --git a/vidmap/datasets/assets/hilti/sources.json b/vidmap/datasets/assets/hilti/sources.json new file mode 100644 index 0000000..566f620 --- /dev/null +++ b/vidmap/datasets/assets/hilti/sources.json @@ -0,0 +1,444 @@ +{ + "shared": [ + { + "path": "config/hilti_openvins/kalibr_imucam_chain.yaml", + "url": "https://raw.githubusercontent.com/Hilti-Research/hilti-trimble-slam-challenge-2026/e0c9277e20ab6d856703d3749d52b3a0f9b43926/config/hilti_openvins/kalibr_imucam_chain.yaml", + "size": 3131, + "sha256": "0cf7880d137fccf85eaeb1bdbaf1ccb180752da386c65d70ce722a73a815c4b1" + }, + { + "path": "config/hilti_openvins/mask_cam0.png", + "url": "https://raw.githubusercontent.com/Hilti-Research/hilti-trimble-slam-challenge-2026/e0c9277e20ab6d856703d3749d52b3a0f9b43926/config/hilti_openvins/mask_cam0.png", + "size": 10254, + "sha256": "b4edcac61543eaba609216edc812c1f6c95767d62894c998eae6b63e23746a65" + }, + { + "path": "config/hilti_openvins/mask_cam1.png", + "url": "https://raw.githubusercontent.com/Hilti-Research/hilti-trimble-slam-challenge-2026/e0c9277e20ab6d856703d3749d52b3a0f9b43926/config/hilti_openvins/mask_cam1.png", + "size": 10254, + "sha256": "b4edcac61543eaba609216edc812c1f6c95767d62894c998eae6b63e23746a65" + } + ], + "scenes": { + "floor_1_2025-05-05_run_1": [ + { + "path": "data/floor_1/2025-05-05/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_1/2025-05-05/run_1/rosbag/rosbag.db3", + "size": 3985178624, + "sha256": "4c2bd0c213ced007b7232e590878373e94dee7fa89b187fe95bb0f8a0d34d161" + }, + { + "path": "groundtruth/floor_1_2025-05-05_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_1_2025-05-05_run_1.txt", + "size": 588772, + "sha256": "c617b45096d954aa46fcf78b5339b76e47bc0eaa87efd43720fed1fbd6a4d907" + } + ], + "floor_1_2025-07-07_run_1": [ + { + "path": "data/floor_1/2025-07-07/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_1/2025-07-07/run_1/rosbag/rosbag.db3", + "size": 4009676800, + "sha256": "086d82941b6d54581c70a146bd1358851f69add5f5be7d52513cc3032d90cf43" + }, + { + "path": "groundtruth/floor_1_2025-07-07_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_1_2025-07-07_run_1.txt", + "size": 602569, + "sha256": "ae8faf4ad2a2bcaadc55da848b537ffadc3f799bd66d654910017016eeb7f015" + } + ], + "floor_1_2025-12-02_run_1": [ + { + "path": "data/floor_1/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_1/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 6999293952, + "sha256": "96c9048455b2711bb1d6118c27001ad70379e148ddb4078e2d37996cdd2283d7" + }, + { + "path": "groundtruth/floor_1_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_1_2025-12-02_run_1.txt", + "size": 1240610, + "sha256": "befb7a1d847ad987010d14694b5a741479251a27ff652f5adaa3978195f89537" + } + ], + "floor_2_2025-05-05_run_1": [ + { + "path": "data/floor_2/2025-05-05/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_2/2025-05-05/run_1/rosbag/rosbag.db3", + "size": 5531398144, + "sha256": "9f66c891ae2ab22302f2119e8891f429013d939dd65403238ac6c2d485e6361a" + }, + { + "path": "groundtruth/floor_2_2025-05-05_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_2_2025-05-05_run_1.txt", + "size": 838941, + "sha256": "59108922d358d01a0803cb231602475dbb358a874cc306e64caedefd1f04f8dc" + } + ], + "floor_2_2025-10-28_run_1": [ + { + "path": "data/floor_2/2025-10-28/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_2/2025-10-28/run_1/rosbag/rosbag.db3", + "size": 3068993536, + "sha256": "9405f32bfbc828cf510b96a6235c69a9d514c5a722f5711d1bf6992388c2ff99" + }, + { + "path": "groundtruth/floor_2_2025-10-28_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_2_2025-10-28_run_1.txt", + "size": 459436, + "sha256": "44da14e5e5a6c8c695c2bd375383b0bca88a2fc703f679da636542e25c8929e1" + } + ], + "floor_2_2025-10-28_run_2": [ + { + "path": "data/floor_2/2025-10-28/run_2/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_2/2025-10-28/run_2/rosbag/rosbag.db3", + "size": 2612477952, + "sha256": "a53fc21b209ee1823614cccbd0874016f3d9cea117e5c452673cadb60bcc5299" + }, + { + "path": "groundtruth/floor_2_2025-10-28_run_2.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_2_2025-10-28_run_2.txt", + "size": 386317, + "sha256": "33ea895a4e1bf9434febfc617a5a4baa66971a38ea4ee52b6c8bf7d91c417f0f" + } + ], + "floor_2_2025-12-02_run_1": [ + { + "path": "data/floor_2/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_2/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 3560341504, + "sha256": "e65e70bef6d563e7bd4754b5f93a5af6edbefcbea30c9f59b22203a2dcf71b7e" + }, + { + "path": "groundtruth/floor_2_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_2_2025-12-02_run_1.txt", + "size": 645645, + "sha256": "4ac2691d11791f81dabfc020251621229bd11a0257cb077877e2eca8a0f21ad5" + } + ], + "floor_2_2025-12-03_run_1": [ + { + "path": "data/floor_2/2025-12-03/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_2/2025-12-03/run_1/rosbag/rosbag.db3", + "size": 4119691264, + "sha256": "d050ed81c0cd0216ec5d66bc5300cf549d2af75a6ef49b587f1ee6bc2372386c" + }, + { + "path": "groundtruth/floor_2_2025-12-03_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_2_2025-12-03_run_1.txt", + "size": 673284, + "sha256": "575af8bf797ddd94090121cd167daed2fcfd6cb6d93de71db95f79fefcdc23b7" + } + ], + "floor_3_2025-05-19_run_1": [ + { + "path": "data/floor_3/2025-05-19/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_3/2025-05-19/run_1/rosbag/rosbag.db3", + "size": 4174323712, + "sha256": "5232881a1e571cdd58234fae3c6baa86e62731101f454da2fb7cba06f9b8789c" + }, + { + "path": "groundtruth/floor_3_2025-05-19_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_3_2025-05-19_run_1.txt", + "size": 520088, + "sha256": "9ce3402032441b3849212807ed79f00c125310d7325378dd22737c84e3a64180" + } + ], + "floor_3_2025-12-02_run_1": [ + { + "path": "data/floor_3/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_3/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 3296927744, + "sha256": "7d4b974eb2ac7f1d805de2575d940524865ad3667c058bd6f25a873fbef01aaf" + }, + { + "path": "groundtruth/floor_3_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_3_2025-12-02_run_1.txt", + "size": 594643, + "sha256": "825f969d003f302be219d6d8eea06b7e36ab8e3317143787b5c359bacf087982" + } + ], + "floor_4_2025-05-19_run_1": [ + { + "path": "data/floor_4/2025-05-19/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_4/2025-05-19/run_1/rosbag/rosbag.db3", + "size": 3263307776, + "sha256": "b1775976ceec860de919357890b53e449d24759833b7e58aa7c3d2e2a3b11f94" + }, + { + "path": "groundtruth/floor_4_2025-05-19_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_4_2025-05-19_run_1.txt", + "size": 412090, + "sha256": "0620e2c5987f7b0317342baedb28fd50732c6291cc87c1e5856f544476b74487" + } + ], + "floor_4_2025-12-02_run_1": [ + { + "path": "data/floor_4/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_4/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 5149523968, + "sha256": "fd568b19ca15001d1697a6ea74ff8922eb364489d519d8e21de064bdb9df7d2d" + }, + { + "path": "groundtruth/floor_4_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_4_2025-12-02_run_1.txt", + "size": 868127, + "sha256": "52b3652cf0c7e15f40d8acd0979b8da703f6f66446759fd2439c383fa20ee9f3" + } + ], + "floor_5_2025-12-02_run_1": [ + { + "path": "data/floor_5/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_5/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 4237619200, + "sha256": "b94ee21b9c4f2342d70faa8abd86f1dedb2d6592019004384573f79f8b4b6545" + }, + { + "path": "groundtruth/floor_5_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_5_2025-12-02_run_1.txt", + "size": 719793, + "sha256": "7a1dadb07b3ee17733a73d38e490ee167e2ebf7252b441bf11e6407ea60294e2" + } + ], + "floor_6_2025-06-18_run_1": [ + { + "path": "data/floor_6/2025-06-18/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_6/2025-06-18/run_1/rosbag/rosbag.db3", + "size": 2924335104, + "sha256": "bc53a00191934a22790ead15e9edffa0d4486017074243a52246537fe486ee20" + }, + { + "path": "groundtruth/floor_6_2025-06-18_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_6_2025-06-18_run_1.txt", + "size": 311974, + "sha256": "54dca44556536182d3047d422d290450472eca9d4b8cfdad1ec42d83172c3ddc" + } + ], + "floor_6_2025-07-07_run_1": [ + { + "path": "data/floor_6/2025-07-07/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_6/2025-07-07/run_1/rosbag/rosbag.db3", + "size": 2785267712, + "sha256": "de341ed02574e74b6f11575b6747d06fc5e7838cd2c0d240cc991167bf80285b" + }, + { + "path": "groundtruth/floor_6_2025-07-07_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_6_2025-07-07_run_1.txt", + "size": 329009, + "sha256": "baace255cb241cf6af9b2da91a7ba6a2ba9b02894766d31f98ecb687ebf6b625" + } + ], + "floor_6_2025-12-02_run_1": [ + { + "path": "data/floor_6/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_6/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 4497391616, + "sha256": "5cc66e5067e6adb34ebc55295c3eb7e9617405a322313f331104e6656583eee5" + }, + { + "path": "groundtruth/floor_6_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_6_2025-12-02_run_1.txt", + "size": 766035, + "sha256": "a02b09ac9752850e2fa3c565f0e9939d073be47f4ad43931c9ce4932a5c5e3f6" + } + ], + "floor_6_2025-12-02_run_2": [ + { + "path": "data/floor_6/2025-12-02/run_2/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_6/2025-12-02/run_2/rosbag/rosbag.db3", + "size": 3333427200, + "sha256": "d5c34633a8a42976fc7f6e0ce118b8dcf6fd2b71d5dbf116ecdcf54efcad9858" + }, + { + "path": "groundtruth/floor_6_2025-12-02_run_2.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_6_2025-12-02_run_2.txt", + "size": 559320, + "sha256": "a9424f75ce2be50a476cc2d0f5f632bac5d66b9caddcfa100804f0138ae5716f" + } + ], + "floor_7_2025-12-02_run_1": [ + { + "path": "data/floor_7/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_7/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 3354808320, + "sha256": "08b87fab290ca8838356747ed928bd6dfa600b252f76b4ed86770fe498af8322" + }, + { + "path": "groundtruth/floor_7_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_7_2025-12-02_run_1.txt", + "size": 529161, + "sha256": "efb4fadcd4cdba0aa2f196d5057a0c4b22bd52d1a7e8ac53a4054fc6b4b6eb61" + } + ], + "floor_7_2025-12-02_run_2": [ + { + "path": "data/floor_7/2025-12-02/run_2/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_7/2025-12-02/run_2/rosbag/rosbag.db3", + "size": 4229259264, + "sha256": "6fb4a573bd1fca6de5394b3ccfc826cf234155af70189c95c295cf32c13ce613" + }, + { + "path": "groundtruth/floor_7_2025-12-02_run_2.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_7_2025-12-02_run_2.txt", + "size": 696771, + "sha256": "76702aa727e9e988af3fc0282db069571d797a8089611ff6636b21b77cedbca2" + } + ], + "floor_7_2025-12-03_run_1": [ + { + "path": "data/floor_7/2025-12-03/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_7/2025-12-03/run_1/rosbag/rosbag.db3", + "size": 5609529344, + "sha256": "3453f9cfb4e2d0f0893001961f35fa2fa1c6eb6afb1641097c9ba10ab8c8ad88" + }, + { + "path": "groundtruth/floor_7_2025-12-03_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_7_2025-12-03_run_1.txt", + "size": 888854, + "sha256": "ba35fcb19fec8fc6f77bd662db8e819a712dd2ba9c641320d638c896cf55d380" + } + ], + "floor_EG_2025-10-16_run_1": [ + { + "path": "data/floor_EG/2025-10-16/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_EG/2025-10-16/run_1/rosbag/rosbag.db3", + "size": 6891782144, + "sha256": "5c72def893a338b8dea472d02d8837f22209365d82f1beebbe48307ba388fb72" + }, + { + "path": "groundtruth/floor_EG_2025-10-16_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_EG_2025-10-16_run_1.txt", + "size": 1090613, + "sha256": "d0c0e5b1bfd3b948073e0541794de51159be9b84a9045e37d5050bb4d9e02653" + } + ], + "floor_EG_2025-12-02_run_1": [ + { + "path": "data/floor_EG/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_EG/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 3368960000, + "sha256": "d2700faec4a33a68e0a080eae5f2d18ef2946a0422193637383c7c57613ac8eb" + }, + { + "path": "groundtruth/floor_EG_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_EG_2025-12-02_run_1.txt", + "size": 567512, + "sha256": "3b23d275bac02ad4e0a9c80fec4a804156e380cef0174a17fefb52f956a2cfae" + } + ], + "floor_EG_2025-12-02_run_2": [ + { + "path": "data/floor_EG/2025-12-02/run_2/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_EG/2025-12-02/run_2/rosbag/rosbag.db3", + "size": 4201840640, + "sha256": "40f4ac431816b925bd7e8ee7f59266bdb5833b6166032ca1f8db9d64fcaef8a2" + }, + { + "path": "groundtruth/floor_EG_2025-12-02_run_2.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_EG_2025-12-02_run_2.txt", + "size": 731333, + "sha256": "9760de3fdffbb9dfa234da45c7ab03806c7709122203180f28351c2932340421" + } + ], + "floor_UG1_2025-05-19_run_1": [ + { + "path": "data/floor_UG1/2025-05-19/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-05-19/run_1/rosbag/rosbag.db3", + "size": 5997969408, + "sha256": "a5af3287fe0df2fc6223276930d3a1b27279228ee03428da4ff6cabefcc62961" + }, + { + "path": "groundtruth/floor_UG1_2025-05-19_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-05-19_run_1.txt", + "size": 920075, + "sha256": "a51d286349b584629098d53dfd3edb1c9c33a40b636a35ca1ce31da753ac1412" + } + ], + "floor_UG1_2025-06-18_run_1": [ + { + "path": "data/floor_UG1/2025-06-18/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-06-18/run_1/rosbag/rosbag.db3", + "size": 4719480832, + "sha256": "3f43365369d1be2348d05dafd806a3c424f328405810b1310dc7a267bda7b8f7" + }, + { + "path": "groundtruth/floor_UG1_2025-06-18_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-06-18_run_1.txt", + "size": 742284, + "sha256": "04fc27f4d4ff2402c1a5f421cc2b202b86c4a8206742465f4923fc4d926a671e" + } + ], + "floor_UG1_2025-10-16_run_1": [ + { + "path": "data/floor_UG1/2025-10-16/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-10-16/run_1/rosbag/rosbag.db3", + "size": 5849698304, + "sha256": "790587d3b61643d44b4dbeef8cc0acf4b8d4fa85d27d85b0d238a6e4cf364e8f" + }, + { + "path": "groundtruth/floor_UG1_2025-10-16_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-10-16_run_1.txt", + "size": 913652, + "sha256": "a9f2b00f730a49ca64fbd612e6e3154ae7579d1704121deb99f014dd9c3b992a" + } + ], + "floor_UG1_2025-12-02_run_1": [ + { + "path": "data/floor_UG1/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 6114402304, + "sha256": "18d122d261e50a9577262d1b4211f01f09a0567813ac16b91bf172b3cf8b6e79" + }, + { + "path": "groundtruth/floor_UG1_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-12-02_run_1.txt", + "size": 1139372, + "sha256": "f8d1546c79d298e442baa33c36f37267f123f463bbd83a5470eebbd9d39d02a3" + } + ], + "floor_UG1_2025-12-02_run_2": [ + { + "path": "data/floor_UG1/2025-12-02/run_2/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-12-02/run_2/rosbag/rosbag.db3", + "size": 5227814912, + "sha256": "b8185f422f0fdcfceac445e5b086ddec0ec22aa025ec715d98a228012f8aa058" + }, + { + "path": "groundtruth/floor_UG1_2025-12-02_run_2.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-12-02_run_2.txt", + "size": 980878, + "sha256": "572355b38476583fcb03a381fa68a1873d0ff02dfb0288aa72315ce3152db4e5" + } + ], + "floor_UG1_2025-12-03_run_1": [ + { + "path": "data/floor_UG1/2025-12-03/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG1/2025-12-03/run_1/rosbag/rosbag.db3", + "size": 3395715072, + "sha256": "b6c326b72b4d636a7e23354241381991cde734bf1c31f6bf9b6043e44fdbdd00" + }, + { + "path": "groundtruth/floor_UG1_2025-12-03_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG1_2025-12-03_run_1.txt", + "size": 600741, + "sha256": "241058c9e24a92dfb554ed03ba5929c0ec0a92de5c884a6ff8a926575d04226d" + } + ], + "floor_UG2_2025-12-02_run_1": [ + { + "path": "data/floor_UG2/2025-12-02/run_1/rosbag/rosbag.db3", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/data/floor_UG2/2025-12-02/run_1/rosbag/rosbag.db3", + "size": 5569863680, + "sha256": "1d5072b2205eac40f3e213f148cbc1e8d6bf790ec66cda0efdd2d4f5dc7017e8" + }, + { + "path": "groundtruth/floor_UG2_2025-12-02_run_1.txt", + "url": "https://huggingface.co/datasets/Hilti-Research/hilti-trimble-slam-challenge-2026/resolve/ac5fd207e10a7a45438f04d1dbc96a529b1088be/groundtruth/floor_UG2_2025-12-02_run_1.txt", + "size": 997021, + "sha256": "769445df54a5576b5e43e6ea7a2a1dfc3d9d4f6c44c78cccedf373fbe430cdef" + } + ] + } +} diff --git a/vidmap/datasets/assets/lamar/official_pose_sources.json b/vidmap/datasets/assets/lamar/official_pose_sources.json new file mode 100644 index 0000000..b4d2860 --- /dev/null +++ b/vidmap/datasets/assets/lamar/official_pose_sources.json @@ -0,0 +1,20 @@ +{ + "CAB": { + "url": "https://cvg-data.inf.ethz.ch/lamar/colmap/CAB.zip", + "pose_count": 53071, + "pose_sha256": "028afcbf40ce6aaf0c6fb4f430578a900b9a3d19a5957dbb76774191df59f8be", + "camera_sha256": "8ac4ae31c66f225752f69f0262fe14143eaebb466898c49404b232938dfe0ece" + }, + "HGE": { + "url": "https://cvg-data.inf.ethz.ch/lamar/colmap/HGE.zip", + "pose_count": 49877, + "pose_sha256": "7e5805fdf863ff3910a8c6b4f1775f4dc3e01fe75896d4960f2121abb35d072a", + "camera_sha256": "ad4a2f22ff3b735980eada98b4722d04b07e5d6dda4f63913ed5506cde2db7e9" + }, + "LIN": { + "url": "https://cvg-data.inf.ethz.ch/lamar/colmap/LIN.zip", + "pose_count": 49120, + "pose_sha256": "7f8e14136eb22e8d0bada6644761292574c08b9ba54184ad158bcf629b199f5e", + "camera_sha256": "8c46ea750b8f23c919604c03398cd3284b0eb78d52791a12293478c820a81f6a" + } +} diff --git a/vidmap/datasets/assets/lamar/testsets/CAB/complete.yaml.gz b/vidmap/datasets/assets/lamar/testsets/CAB/complete.yaml.gz new file mode 100644 index 0000000..78532e0 Binary files /dev/null and b/vidmap/datasets/assets/lamar/testsets/CAB/complete.yaml.gz differ diff --git a/vidmap/datasets/assets/lamar/testsets/HGE/complete.yaml.gz b/vidmap/datasets/assets/lamar/testsets/HGE/complete.yaml.gz new file mode 100644 index 0000000..a1a4880 Binary files /dev/null and b/vidmap/datasets/assets/lamar/testsets/HGE/complete.yaml.gz differ diff --git a/vidmap/datasets/assets/lamar/testsets/LIN/complete.yaml.gz b/vidmap/datasets/assets/lamar/testsets/LIN/complete.yaml.gz new file mode 100644 index 0000000..b29416e Binary files /dev/null and b/vidmap/datasets/assets/lamar/testsets/LIN/complete.yaml.gz differ diff --git a/vidmap/datasets/assets/spires/sources.json b/vidmap/datasets/assets/spires/sources.json new file mode 100644 index 0000000..c3e65f7 --- /dev/null +++ b/vidmap/datasets/assets/spires/sources.json @@ -0,0 +1,230 @@ +{ + "shared": [ + { + "path": "calibration/calibration-sequences-2024-03-19/cam-lidar.yaml", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/calibration/calibration-sequences-2024-03-19/cam-lidar.yaml", + "size": 1644, + "sha256": "fa6fd40646923b4ddbacef763fad15f316fc6f8019eb5ab9b3aabfd57bed46e1" + }, + { + "path": "calibration/calibration-sequences-2024-03-19/cam0.yaml", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/calibration/calibration-sequences-2024-03-19/cam0.yaml", + "size": 715, + "sha256": "2e9c3dbb83617ed3816a28d99d976b397652f3c730bce89dc50d7cf153c6fd6e" + }, + { + "path": "calibration/calibration-sequences-2024-03-19/cam1.yaml", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/calibration/calibration-sequences-2024-03-19/cam1.yaml", + "size": 716, + "sha256": "51e2f20a46bbaf2e1d02096d6b564871edfbca96d2f1c4506a3f2b9e267dd2ad" + }, + { + "path": "calibration/calibration-sequences-2024-03-19/cam2.yaml", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/calibration/calibration-sequences-2024-03-19/cam2.yaml", + "size": 715, + "sha256": "8b2433de79113fd3ab2aacc837c8d37d8e164ee703a0bdae9348a9ec6d50921b" + } + ], + "scenes": { + "2024-03-12-keble-college-02": [ + { + "path": "sequences/2024-03-12-keble-college-02/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-02/processed/trajectory/gt-tum.txt", + "size": 1199863, + "sha256": "671ed805ba33e4edd929a1e5e533837081099b443655f494ecf290c46e8f69c6" + }, + { + "path": "sequences/2024-03-12-keble-college-02/raw/rosbag/1710255615_2024-03-12-15-00-16_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-02/raw/rosbag/1710255615_2024-03-12-15-00-16_0.bag", + "size": 5128342401, + "sha256": "cade2da753b755d8c646f92f47b8985c89e9e95c8dbe8a628d98ba39596ff0da" + } + ], + "2024-03-12-keble-college-03": [ + { + "path": "sequences/2024-03-12-keble-college-03/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-03/processed/trajectory/gt-tum.txt", + "size": 1152492, + "sha256": "b560aa51dbe14f03f31d961236769822e8e99eb6d606a562d6418191f076e3ed" + }, + { + "path": "sequences/2024-03-12-keble-college-03/raw/rosbag/1710256011_2024-03-12-15-06-52_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-03/raw/rosbag/1710256011_2024-03-12-15-06-52_0.bag", + "size": 4808019426, + "sha256": "8385c4f60b436d558eef371a606c795e951185b81032b90fac3448ccde29e21b" + } + ], + "2024-03-12-keble-college-04": [ + { + "path": "sequences/2024-03-12-keble-college-04/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-04/processed/trajectory/gt-tum.txt", + "size": 2727907, + "sha256": "75c1d42b580d25be10efb9b27dfea5ef25df0f84b07386b728e72c96088f1f70" + }, + { + "path": "sequences/2024-03-12-keble-college-04/raw/rosbag/1710256650_2024-03-12-15-17-31_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-04/raw/rosbag/1710256650_2024-03-12-15-17-31_0.bag", + "size": 9467104685, + "sha256": "c36252921ee27db625b9e5f7d5e83b034a83c0dcf193dfad19a18a0182fd1769" + }, + { + "path": "sequences/2024-03-12-keble-college-04/raw/rosbag/1710256650_2024-03-12-15-26-05_1.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-04/raw/rosbag/1710256650_2024-03-12-15-26-05_1.bag", + "size": 3887183954, + "sha256": "298ed5c7dee66eb859cc783b11cae03d6eb341041c8a1107128daa2f7bcfc6de" + } + ], + "2024-03-12-keble-college-05": [ + { + "path": "sequences/2024-03-12-keble-college-05/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-05/processed/trajectory/gt-tum.txt", + "size": 2313696, + "sha256": "23d27b6f7536fa0307647a13f26c8315854c0a2cf141a374b3c504684bd0e4b1" + }, + { + "path": "sequences/2024-03-12-keble-college-05/raw/rosbag/1710257614_2024-03-12-15-33-35_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-05/raw/rosbag/1710257614_2024-03-12-15-33-35_0.bag", + "size": 9427812351, + "sha256": "00f6b8b19012d4bb212c2d0ef4916f446e191c1ceb7191c759e406a40a86a097" + }, + { + "path": "sequences/2024-03-12-keble-college-05/raw/rosbag/1710257614_2024-03-12-15-42-11_1.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-12-keble-college-05/raw/rosbag/1710257614_2024-03-12-15-42-11_1.bag", + "size": 1470324626, + "sha256": "dd5d58f0de2c8824ffb99bed385433da6af6285a6a96866bc328683587fc143d" + } + ], + "2024-03-13-observatory-quarter-01": [ + { + "path": "sequences/2024-03-13-observatory-quarter-01/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-13-observatory-quarter-01/processed/trajectory/gt-tum.txt", + "size": 1163922, + "sha256": "afb6011ad6b21413b811568337d5a0d24447efa967105dc17d29e959490519c1" + }, + { + "path": "sequences/2024-03-13-observatory-quarter-01/raw/rosbag/1710338090_2024-03-13-13-54-51_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-13-observatory-quarter-01/raw/rosbag/1710338090_2024-03-13-13-54-51_0.bag", + "size": 5466871208, + "sha256": "5da1843f9a8949fd0f7dce01e92b3c710ce524213c321b7850cba1a80d8201aa" + } + ], + "2024-03-13-observatory-quarter-02": [ + { + "path": "sequences/2024-03-13-observatory-quarter-02/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-13-observatory-quarter-02/processed/trajectory/gt-tum.txt", + "size": 1089467, + "sha256": "38334eba0983adb1c1d1aaa6ac07ce9e9980c67e91084efe2b4b3a50cc61caae" + }, + { + "path": "sequences/2024-03-13-observatory-quarter-02/raw/rosbag/1710338490_2024-03-13-14-01-30_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-13-observatory-quarter-02/raw/rosbag/1710338490_2024-03-13-14-01-30_0.bag", + "size": 5040946363, + "sha256": "3e9ec51e7a8bae016d24930cdcc6c6dda8f2b38bcd6fecd3e123a8fcbc12fdcb" + } + ], + "2024-03-14-blenheim-palace-01": [ + { + "path": "sequences/2024-03-14-blenheim-palace-01/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-01/processed/trajectory/gt-tum.txt", + "size": 1604665, + "sha256": "94c3453abfc6acbf72fda4ffc4de1a55b2de6367f7a321277ed24c085d36d2f4" + }, + { + "path": "sequences/2024-03-14-blenheim-palace-01/raw/rosbag/1710406700_2024-03-14-08-58-21_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-01/raw/rosbag/1710406700_2024-03-14-08-58-21_0.bag", + "size": 6325760912, + "sha256": "377ab4e5535ea3f15de121f67e19128a6d2c308c4df9bbd69ec997e4160a2958" + } + ], + "2024-03-14-blenheim-palace-02": [ + { + "path": "sequences/2024-03-14-blenheim-palace-02/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-02/processed/trajectory/gt-tum.txt", + "size": 1469160, + "sha256": "3d82207bf75a4c57a3644db8e5500a6bc0eec9139c0bcd1a2993bf86c075299b" + }, + { + "path": "sequences/2024-03-14-blenheim-palace-02/raw/rosbag/1710407340_2024-03-14-09-09-01_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-02/raw/rosbag/1710407340_2024-03-14-09-09-01_0.bag", + "size": 5807146911, + "sha256": "617fd4d9870bac76c1abdc2be834b927dc868f32b072c650bd04359a658d2164" + } + ], + "2024-03-14-blenheim-palace-05": [ + { + "path": "sequences/2024-03-14-blenheim-palace-05/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-05/processed/trajectory/gt-tum.txt", + "size": 1106541, + "sha256": "18dea18c3e351d7ae1f631a8869231829360be4794ecf839e0f4c083aa092f73" + }, + { + "path": "sequences/2024-03-14-blenheim-palace-05/raw/rosbag/1710410169_2024-03-14-09-56-09_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-14-blenheim-palace-05/raw/rosbag/1710410169_2024-03-14-09-56-09_0.bag", + "size": 5319780246, + "sha256": "5e0d7f9ae35e13a355368d79b55ded667359c63844957b0e6d3578a3194cd38b" + } + ], + "2024-03-18-christ-church-02": [ + { + "path": "sequences/2024-03-18-christ-church-02/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-18-christ-church-02/processed/trajectory/gt-tum.txt", + "size": 2240218, + "sha256": "01b7c1bf7e2b80f9da058dfe39b043bc7509e33c887221aa1505eb11dbde0cae" + }, + { + "path": "sequences/2024-03-18-christ-church-02/raw/rosbag/1710754066_2024-03-18-09-27-47_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-18-christ-church-02/raw/rosbag/1710754066_2024-03-18-09-27-47_0.bag", + "size": 8973742217, + "sha256": "4560361da07ea22219af83722beb621d5e1710d88bc3eab017be36dafb839bed" + }, + { + "path": "sequences/2024-03-18-christ-church-02/raw/rosbag/1710754066_2024-03-18-09-36-49_1.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-18-christ-church-02/raw/rosbag/1710754066_2024-03-18-09-36-49_1.bag", + "size": 1195460287, + "sha256": "79fdea8348dd8a4fe7b5949ce796b7884719787f1a2af6a26e64eaf9b7ad0a66" + } + ], + "2024-03-18-christ-church-03": [ + { + "path": "sequences/2024-03-18-christ-church-03/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-18-christ-church-03/processed/trajectory/gt-tum.txt", + "size": 1236394, + "sha256": "1c3e1883702f158f103f5c4d8c713f2860c0567c6a40ff9d6757b6921c0bb8fe" + }, + { + "path": "sequences/2024-03-18-christ-church-03/raw/rosbag/1710755015_2024-03-18-09-43-36_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-18-christ-church-03/raw/rosbag/1710755015_2024-03-18-09-43-36_0.bag", + "size": 5541766461, + "sha256": "98d089f404470b0bea7cdb1a8138227a2390bacf405bbcea5fce16ad816f9693" + } + ], + "2024-03-20-christ-church-05": [ + { + "path": "sequences/2024-03-20-christ-church-05/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-20-christ-church-05/processed/trajectory/gt-tum.txt", + "size": 1785838, + "sha256": "993f09497fbe78679278ba498988ee74c5265997546eabc9dc9be867f13f3119" + }, + { + "path": "sequences/2024-03-20-christ-church-05/raw/rosbag/1710926317_2024-03-20-09-18-38_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-03-20-christ-church-05/raw/rosbag/1710926317_2024-03-20-09-18-38_0.bag", + "size": 6291326026, + "sha256": "70857c06bcd979e3588ba77f9b803793692d316141579f39c09e04f19e000295" + } + ], + "2024-05-20-bodleian-library-02": [ + { + "path": "sequences/2024-05-20-bodleian-library-02/processed/trajectory/gt-tum.txt", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-05-20-bodleian-library-02/processed/trajectory/gt-tum.txt", + "size": 1200673, + "sha256": "7ceb0f9ac964e9b17cf725e07b77e726e62d998b7250796d55498742a7d0c6bc" + }, + { + "path": "sequences/2024-05-20-bodleian-library-02/raw/rosbag/1716183690_2024-05-20-06-41-31_0.bag", + "url": "https://huggingface.co/datasets/ori-drs/oxford_spires_dataset/resolve/03f4382308333aa70c3253f12acd3fbf0c7c4a15/sequences/2024-05-20-bodleian-library-02/raw/rosbag/1716183690_2024-05-20-06-41-31_0.bag", + "size": 5540105616, + "sha256": "04c06c8b45009b034c493113bad272ea6cb263395faee3222a2f974cca828e52" + } + ] + } +} diff --git a/vidmap/datasets/base.py b/vidmap/datasets/base.py index 687b313..8fe87f0 100644 --- a/vidmap/datasets/base.py +++ b/vidmap/datasets/base.py @@ -172,6 +172,7 @@ class DatasetCapabilities: class DatasetPreparation: module: str profiles: tuple[tuple[str, str], ...] = () + uses_mode: bool = False def profile(self, mode: str) -> str: profiles = dict(self.profiles) @@ -183,6 +184,8 @@ def __call__(self, scenes: tuple[str, ...] | None = None, *, mode: str) -> None: module = importlib.import_module(self.module) if self.profiles: module.main(scenes=scenes, profile=self.profile(mode)) + elif self.uses_mode: + module.main(scenes=scenes, mode=mode) else: module.main(scenes=scenes) @@ -342,6 +345,11 @@ def frontend_capture(self) -> PreparedCapture: """Copy input geometry without exposing ground-truth poses to the frontend.""" from vidmap.datasets.capture import PreparedCapture, copy_unposed_reconstruction + forced_keyframe_names = ( + frozenset(image.name for image in self.rec.images.values() if image.has_pose) + if self.input_metadata["force_gt_keyframes"] + else frozenset() + ) if self.input_metadata["rig"] is not None: from vidmap.datasets.prepare.helpers.rig_capture import load_capture from vidmap.frontend.options.rig import RigOptions @@ -358,6 +366,7 @@ def frontend_capture(self) -> PreparedCapture: capture, rec=selected, frame_timestamps_ns={fid: capture.frame_timestamps_ns[fid] for fid in selected.frames}, + forced_keyframe_names=forced_keyframe_names, ) return PreparedCapture( @@ -366,11 +375,7 @@ def frontend_capture(self) -> PreparedCapture: frame_timestamps_ns=None, provenance={}, preserve_calibration=False, - forced_keyframe_names=( - frozenset(image.name for image in self.rec.images.values() if image.has_pose) - if self.input_metadata["force_gt_keyframes"] - else frozenset() - ), + forced_keyframe_names=forced_keyframe_names, ) def camera(self, imid: int) -> pycolmap.Camera: diff --git a/vidmap/datasets/names.py b/vidmap/datasets/names.py index ca977fc..5b55003 100644 --- a/vidmap/datasets/names.py +++ b/vidmap/datasets/names.py @@ -2,9 +2,12 @@ from __future__ import annotations +import json from dataclasses import dataclass from typing import Literal +from vidmap.datasets.resources import dataset_asset + EvaluationMetric = Literal["pose_auc", "wate_auc"] @@ -28,6 +31,8 @@ def __post_init__(self) -> None: raise ValueError(f"Invalid manifest reference for dataset {self.name!r}: {self.manifest_reference!r}") +_CAPTURE_SUITES = json.loads(dataset_asset("benchmark_suites.json").read_text()) + DATASET_DEFINITIONS = ( DatasetDefinition( name="crocodl", @@ -39,24 +44,38 @@ def __post_init__(self) -> None: DatasetDefinition( name="eth3d_slam", manifest_reference="vidmap.datasets.eth3d_slam:ETH3D_SLAM_MANIFEST", - default_mode="all", + default_mode="mono", default_evaluation_metric="pose_auc", preparer_module="vidmap.datasets.prepare.eth3d_slam", ), DatasetDefinition( name="euroc", scenes=("machine_hall", "vicon_room1", "vicon_room2"), - default_mode="all", + default_mode="mono", default_evaluation_metric="pose_auc", preparer_module="vidmap.datasets.prepare.euroc", ), + DatasetDefinition( + name="hilti", + scenes=tuple(_CAPTURE_SUITES["hilti"]["scenes"]), + default_mode="panorama", + default_evaluation_metric="wate_auc", + preparer_module="vidmap.datasets.prepare.hilti", + ), DatasetDefinition( name="lamar", scenes=("CAB", "HGE", "LIN"), - default_mode="sample", + default_mode="complete", default_evaluation_metric="wate_auc", preparer_module="vidmap.datasets.prepare.lamar", ), + DatasetDefinition( + name="spires", + scenes=tuple(_CAPTURE_SUITES["spires"]["scenes"]), + default_mode="undistorted-mono", + default_evaluation_metric="wate_auc", + preparer_module="vidmap.datasets.prepare.spires", + ), ) SUPPORTED_DATASETS = tuple(definition.name for definition in DATASET_DEFINITIONS) diff --git a/vidmap/datasets/prepare/eth3d_slam.py b/vidmap/datasets/prepare/eth3d_slam.py index dbbe1c3..a334bf4 100644 --- a/vidmap/datasets/prepare/eth3d_slam.py +++ b/vidmap/datasets/prepare/eth3d_slam.py @@ -1,10 +1,11 @@ -"""Download and prepare the ground-truth ETH3D-SLAM monocular benchmark.""" +"""Download and prepare the ETH3D-SLAM mono and stereo benchmarks.""" from __future__ import annotations import argparse import csv import json +import os import re import shutil from collections.abc import Iterable, Mapping, Sequence @@ -20,31 +21,34 @@ from vidmap.datasets.eth3d_slam import ETH3D_SLAM_MANIFEST, ETH3D_SLAM_PREPARATION_VERSION from vidmap.datasets.layouts import get_dataset_layout from vidmap.datasets.prepare.helpers import filesystem as prep_fs +from vidmap.datasets.prepare.helpers.rig_capture import camera_options, frame_image_ids, write_frame_table +from vidmap.datasets.prepare.helpers.ros_benchmarks import timestamp_ns from vidmap.datasets.prepare.helpers.video_common import resumable_http_download, safe_extract_zip from vidmap.datasets.resources import dataset_asset -_URL_TEMPLATE = "https://www.eth3d.net/data/slam/datasets/{scene}_mono.zip" +_URL_TEMPLATE = "https://www.eth3d.net/data/slam/datasets/{scene}_{modality}.zip" _SOURCE_RECORD = ".source_archive.json" ETH3D_SLAM_LAYOUT = get_dataset_layout("eth3d_slam") @dataclass(frozen=True) class SourceArchive: - """One checksum-pinned archive from the official monocular release.""" + """One checksum-pinned archive from the official release.""" scene: str split: str size: int http_etag: str sha256: str + modality: str = "mono" @property def filename(self) -> str: - return f"{self.scene}_mono.zip" + return f"{self.scene}_{self.modality}.zip" @property def url(self) -> str: - return _URL_TEMPLATE.format(scene=self.scene) + return _URL_TEMPLATE.format(scene=self.scene, modality=self.modality) @property def gt_available(self) -> bool: @@ -61,9 +65,11 @@ def record(self) -> dict[str, object]: } -def source_inventory() -> tuple[SourceArchive, ...]: +def source_inventory(*, modality="mono") -> tuple[SourceArchive, ...]: """Read and validate the complete ground-truth benchmark inventory.""" - with dataset_asset("eth3d_slam", "mono_archives.tsv").open(encoding="utf-8", newline="") as file: + if modality not in {"mono", "stereo"}: + raise ValueError(f"Unknown ETH3D-SLAM modality: {modality}") + with dataset_asset("eth3d_slam", f"{modality}_archives.tsv").open(encoding="utf-8", newline="") as file: rows = tuple(csv.DictReader(file, delimiter="\t")) unordered = tuple( SourceArchive( @@ -72,6 +78,7 @@ def source_inventory() -> tuple[SourceArchive, ...]: size=int(row["size"]), http_etag=row["http_etag"], sha256=row["sha256"], + modality=modality, ) for row in rows if row["split"] == "training" @@ -114,18 +121,27 @@ def _read_unposed_names(scene_dir: Path) -> tuple[str, ...] | None: return tuple(names) -def _scene_is_complete(layout: DatasetLayout, scene: str) -> bool: +def _scene_is_complete(layout: DatasetLayout, scene: str, *, mode="mono") -> bool: images_dir = layout.images_dir(scene, "0", "native") rec_dir = layout.reconstruction_dir(scene, "0", "native") if not images_dir.is_dir() or not prep_fs.complete_colmap_model(rec_dir): return False try: reconstruction = pycolmap.Reconstruction(rec_dir) - testsets = layout.read_testsets(scene, "all") + testsets = layout.read_testsets(scene, "mono") preparation = json.loads((layout.recording_dir(scene, "0") / "preparation.json").read_text(encoding="utf-8")) except (OSError, RuntimeError, ValueError, json.JSONDecodeError): return False - image_ids = sorted(reconstruction.images) + if mode in {"stereo", "all"}: + rig = preparation["variants"]["native"]["rig"] + if rig is None or len(rig["cameras"]) != 2 or not (images_dir / "frames.csv").is_file(): + return False + if not layout.testset_path(scene, "stereo").is_file(): + return False + case = PreparedTestset("0", "native", frame_image_ids(layout.input_dir(scene, "0", "native"))) + if layout.read_testsets(scene, "stereo") != {"0": case}: + return False + image_ids = sorted(iid for iid, image in reconstruction.images.items() if image.name.startswith("cam0/")) names = {image.name for image in reconstruction.images.values()} unposed = _read_unposed_names(layout.input_dir(scene, "0", "native")) try: @@ -140,7 +156,8 @@ def _scene_is_complete(layout: DatasetLayout, scene: str) -> bool: and set(unposed) <= names and isinstance(preparation, Mapping) and preparation.get("preparation_version") == ETH3D_SLAM_PREPARATION_VERSION - and preparation["variants"] == {"native": {"rig": None, "force_gt_keyframes": True}} + and set(preparation["variants"]) == {"native"} + and preparation["variants"]["native"]["force_gt_keyframes"] and (archive is None or preparation.get("source") == archive.record()) and (archive is None or preparation.get("gt_available") is archive.gt_available) and all( @@ -150,17 +167,24 @@ def _scene_is_complete(layout: DatasetLayout, scene: str) -> bool: ) -def _pending_scenes(layout: DatasetLayout, scenes: tuple[str, ...] | None = None) -> tuple[str, ...]: +def _pending_scenes(layout: DatasetLayout, scenes: tuple[str, ...] | None = None, *, mode="mono") -> tuple[str, ...]: pending = [] for scene in layout.scenes if scenes is None else scenes: destination = layout.recording_dir(scene, "0") - testset = layout.testset_path(scene, "all") + testset = layout.testset_path(scene, "mono") + legacy = layout.testset_path(scene, "all") + if legacy.is_file() and not testset.exists(): + legacy.rename(testset) + previous = destination.with_name(f".{destination.name}.preparing") / "previous" + if previous.exists() and not destination.exists(): + previous.rename(destination) shutil.rmtree(destination.with_name(f".{destination.name}.preparing"), ignore_errors=True) testset.with_name(f".{testset.name}.preparing").unlink(missing_ok=True) if destination.exists() or testset.exists(): if not _scene_is_complete(layout, scene): raise RuntimeError(f"ETH3D-SLAM scene destination is partial or incompatible: {destination}") - continue + if _scene_is_complete(layout, scene, mode=mode): + continue pending.append(scene) return tuple(pending) @@ -210,8 +234,8 @@ def _read_groundtruth(path: Path) -> tuple[np.ndarray, np.ndarray, np.ndarray]: return timestamps, rows[:, 1:4], quaternions -def _source_images(source: Path) -> tuple[tuple[Path, float], ...]: - rgb_dir = source / "rgb" +def _source_images(source: Path, directory="rgb") -> tuple[tuple[Path, float], ...]: + rgb_dir = source / directory if not rgb_dir.is_dir(): raise RuntimeError(f"ETH3D-SLAM source scene is missing its RGB directory: {rgb_dir}") images = [] @@ -276,15 +300,17 @@ def ground_truth_poses_at_timestamps( return poses -def _convert_scene(source: Path, destination: Path, *, gt_available: bool = True) -> list[int]: - calibration = _read_calibration(source / "calibration.txt") - images = _source_images(source) - width, height = _image_dimensions(images) - poses = [None] * len(images) +def _convert_scene(source: Path, destination: Path, *, gt_available: bool = True, previous=None) -> list[int]: + streams = [_source_images(source)] + if (source / "rgb2").is_dir(): + streams.append(_source_images(source, "rgb2")) + if [path.name for path, _ in streams[0]] != [path.name for path, _ in streams[1]]: + raise ValueError(f"ETH3D-SLAM camera streams have different frame names: {source}") + poses = [None] * len(streams[0]) if gt_available: timestamps, translations, quaternions = _read_groundtruth(source / "groundtruth.txt") poses = ground_truth_poses_at_timestamps( - (timestamp for _, timestamp in images), + (timestamp for _, timestamp in streams[0]), timestamps, translations, quaternions, @@ -292,31 +318,62 @@ def _convert_scene(source: Path, destination: Path, *, gt_available: bool = True ) reconstruction = pycolmap.Reconstruction() - camera = pycolmap.Camera(model="PINHOLE", width=width, height=height, params=calibration, camera_id=0) - reconstruction.add_camera_with_trivial_rig(camera) - images_dir = destination / "images" / "cam0" rec_dir = destination / "rec" - images_dir.mkdir(parents=True) - rec_dir.mkdir() - - unposed_names = [] - for image_id, (source_image, _) in enumerate(images): - image = pycolmap.Image(camera_id=0, name=f"cam0/{source_image.name}", image_id=image_id) - pose = poses[image_id] - if pose is None: - unposed_names.append(image.name) - # COLMAP omits unregistered images when writing; the sidecar removes - # this placeholder pose on load while preserving the input image. - reconstruction.add_image_with_trivial_frame(image, pycolmap.Rigid3d()) - else: - reconstruction.add_image_with_trivial_frame(image, pose.inverse()) - shutil.copy2(source_image, images_dir / source_image.name) + rec_dir.mkdir(parents=True) + unposed_names, frame_streams = [], [] + for cid, images in enumerate(streams): + calibration = _read_calibration(source / ("calibration.txt" if cid == 0 else "calibration2.txt")) + width, height = _image_dimensions(images) + camera = pycolmap.Camera(model="PINHOLE", width=width, height=height, params=calibration, camera_id=cid) + reconstruction.add_camera_with_trivial_rig(camera) + images_dir = destination / "images" / f"cam{cid}" + images_dir.mkdir(parents=True) + frame_streams.append({timestamp_ns(path.stem): f"cam{cid}/{path.name}" for path, _ in images}) + for index, (source_image, _) in enumerate(images): + image_id = len(reconstruction.images) + image = pycolmap.Image(camera_id=cid, name=f"cam{cid}/{source_image.name}", image_id=image_id) + pose = poses[index] if cid == 0 else None + if pose is None: + unposed_names.append(image.name) + # COLMAP omits unregistered images when writing; the sidecar removes + # this placeholder pose on load while preserving the input image. + reconstruction.add_image_with_trivial_frame(image, pycolmap.Rigid3d()) + else: + reconstruction.add_image_with_trivial_frame(image, pose.inverse()) + existing = None if previous is None else previous / "images" / image.name + if existing is not None and existing.is_file(): + os.link(existing, images_dir / source_image.name) + else: + shutil.copy2(source_image, images_dir / source_image.name) reconstruction.write_binary(rec_dir) + if len(streams) > 1: + write_frame_table(destination / "images", frame_streams) with (destination / CANONICAL_UNPOSED_IMAGES_FILENAME).open("w", encoding="utf-8") as file: json.dump({"image_names": unposed_names}, file, indent=2) file.write("\n") - return sorted(reconstruction.images) + return list(range(len(streams[0]))) + + +def _rig_calibration(source, reconstruction): + cam1_from_cam0 = None + if len(reconstruction.cameras) > 1: + transform = np.loadtxt(source / "extrinsics_1_2.txt") + if ( + transform.shape != (3, 4) + or not np.isfinite(transform).all() + or not np.allclose(transform[:, :3].T @ transform[:, :3], np.eye(3), atol=1e-6) + or not np.isclose(np.linalg.det(transform[:, :3]), 1) + ): + raise ValueError("Invalid ETH3D camera transform") + # The official matrix maps the second camera into the reference camera. + cam1_from_cam0 = pycolmap.Rigid3d(transform).inverse() + cameras = [] + for cid, camera in sorted(reconstruction.cameras.items()): + # Source calibration uses integer pixel centres; COLMAP uses half-integer centres. + camera.params[2:4] += 0.5 + cameras.append(camera_options(camera, cam1_from_cam0 if cid else None)) + return dict(cameras=cameras, frames="frames.csv" if len(cameras) > 1 else None) def _prepare_scene( @@ -327,46 +384,60 @@ def _prepare_scene( ) -> None: gt_available = True if archive is None else archive.gt_available destination = layout.recording_dir(scene, "0") - testset = layout.testset_path(scene, "all") - temporary_testset = testset.with_name(f".{testset.name}.preparing") - testset.parent.mkdir(parents=True, exist_ok=True) - temporary_testset.unlink(missing_ok=True) - try: - with prep_fs.scene_staging(destination) as staging: - image_ids = _convert_scene(source, staging / "native", gt_available=gt_available) - with (staging / "preparation.json").open("w", encoding="utf-8") as file: - json.dump( - { - "gt_available": gt_available, - "preparation_version": ETH3D_SLAM_PREPARATION_VERSION, - "variants": {"native": {"rig": None, "force_gt_keyframes": True}}, - "source": None if archive is None else archive.record(), - }, - file, - indent=2, - sort_keys=True, - ) - file.write("\n") - with temporary_testset.open("x", encoding="utf-8") as file: - testset_data = PreparedTestset("0", "native", tuple(image_ids)).as_dict() - yaml.safe_dump({0: testset_data}, file, sort_keys=False, default_flow_style=True) - if not prep_fs.complete_colmap_model(staging / "native" / "rec") or not image_ids: - raise RuntimeError(f"ETH3D-SLAM conversion did not produce a complete scene: {staging}") - if testset.exists() or testset.is_symlink(): - raise RuntimeError(f"ETH3D-SLAM testset destination appeared during preparation: {testset}") - staging.rename(destination) - try: - temporary_testset.rename(testset) - except BaseException: - shutil.rmtree(destination, ignore_errors=True) - raise - finally: - temporary_testset.unlink(missing_ok=True) + with prep_fs.scene_staging(destination, replace=destination.exists()) as temporary: + staging = temporary / "recording" + image_ids = _convert_scene( + source, + staging / "native", + gt_available=gt_available, + previous=destination / "native" if destination.exists() else None, + ) + reconstruction = pycolmap.Reconstruction(staging / "native" / "rec") + rig = _rig_calibration(source, reconstruction) + preparation = { + "gt_available": gt_available, + "preparation_version": ETH3D_SLAM_PREPARATION_VERSION, + "variants": {"native": {"rig": rig, "force_gt_keyframes": True}}, + "source": None if archive is None else archive.record(), + } + if len(rig["cameras"]) > 1: + preparation["stereo_source"] = json.loads((source / _SOURCE_RECORD).read_text())[1] + (staging / "preparation.json").write_text(json.dumps(preparation, indent=2, sort_keys=True) + "\n") + cases = {"mono": PreparedTestset("0", "native", tuple(image_ids))} + if len(rig["cameras"]) > 1: + cases["stereo"] = PreparedTestset("0", "native", frame_image_ids(staging / "native")) + pending_testsets = {} + for mode, case in cases.items(): + path = layout.testset_path(scene, mode) + if path.exists(): + if layout.read_testsets(scene, mode) != {"0": case}: + raise ValueError(f"Existing testset differs from prepared scene: {path}") + continue + path.parent.mkdir(parents=True, exist_ok=True) + temporary_testset = temporary / f"{mode}.yaml" + temporary_testset.write_text(yaml.safe_dump({0: case.as_dict()}, sort_keys=False, default_flow_style=True)) + pending_testsets[path] = temporary_testset + if not prep_fs.complete_colmap_model(staging / "native" / "rec") or not image_ids: + raise RuntimeError(f"ETH3D-SLAM conversion did not produce a complete scene: {staging}") + if destination.exists(): + destination.rename(temporary / "previous") + staging.rename(destination) + published = False + try: + for path, temporary_testset in pending_testsets.items(): + temporary_testset.replace(path) + published = True + finally: + if not published: + destination.rename(staging) + for path in pending_testsets: + path.unlink(missing_ok=True) -def _extract_archive(archive: SourceArchive, downloaded: Path, root: Path) -> Path: +def _extract_archive(archive: SourceArchive, downloaded: Path, root: Path, *, camera_archives=()) -> Path: destination = root / archive.scene - expected_record = archive.record() + sources = ((archive, downloaded), *camera_archives) + expected_record = [item.record() for item, _ in sources] record_path = destination / _SOURCE_RECORD if destination.is_dir(): try: @@ -376,6 +447,9 @@ def _extract_archive(archive: SourceArchive, downloaded: Path, root: Path) -> Pa if record == expected_record: _read_calibration(destination / "calibration.txt") _source_images(destination) + if camera_archives: + _read_calibration(destination / "calibration2.txt") + _source_images(destination, "rgb2") return destination shutil.rmtree(destination) @@ -383,12 +457,16 @@ def _extract_archive(archive: SourceArchive, downloaded: Path, root: Path) -> Pa shutil.rmtree(staging, ignore_errors=True) staging.mkdir(parents=True) try: - safe_extract_zip(downloaded, staging) + for _, path in sources: + safe_extract_zip(path, staging) extracted = staging / archive.scene if not extracted.is_dir(): raise RuntimeError(f"ETH3D-SLAM archive {downloaded} did not contain {archive.scene}/") _read_calibration(extracted / "calibration.txt") _source_images(extracted) + if camera_archives: + _read_calibration(extracted / "calibration2.txt") + _source_images(extracted, "rgb2") if archive.gt_available: _read_groundtruth(extracted / "groundtruth.txt") record_path = extracted / _SOURCE_RECORD @@ -403,31 +481,48 @@ def _extract_archive(archive: SourceArchive, downloaded: Path, root: Path) -> Pa return destination -def main(delete_files: bool = False, scenes: Iterable[str] | None = None) -> None: +def main(delete_files: bool = False, scenes: Iterable[str] | None = None, *, mode="mono") -> None: + if mode not in {"mono", "stereo", "all"}: + raise ValueError(f"Unknown ETH3D-SLAM mode: {mode}") layout = ETH3D_SLAM_LAYOUT layout.data_dir.mkdir(parents=True, exist_ok=True) selected = layout.select_scenes(scenes) - pending = _pending_scenes(layout, selected) + pending = _pending_scenes(layout, selected, mode=mode) + extra_sources = ( + {archive.scene: archive for archive in source_inventory(modality="stereo")} + if mode in {"stereo", "all"} + else {} + ) for scene in pending: archive = _source_archive(scene) - print(f"Preparing ETH3D-SLAM scene {scene} ({archive.split})") - downloaded = resumable_http_download( - archive.url, - layout.data_dir / "downloads" / archive.filename, - expected_size=archive.size, - expected_digest=archive.sha256, + sources = [archive, extra_sources[scene]] if mode in {"stereo", "all"} else [archive] + downloaded = [ + resumable_http_download( + item.url, + layout.data_dir / "downloads" / item.filename, + expected_size=item.size, + expected_digest=item.sha256, + ) + for item in sources + ] + source = _extract_archive( + archive, + downloaded[0], + layout.data_dir / ".eth3d-slam-source", + camera_archives=tuple(zip(sources[1:], downloaded[1:], strict=True)), ) - source = _extract_archive(archive, downloaded, layout.data_dir / ".eth3d-slam-source") _prepare_scene(layout, scene, source, archive) - if not _scene_is_complete(layout, scene): + if not _scene_is_complete(layout, scene, mode=mode): raise RuntimeError(f"ETH3D-SLAM preparation did not produce a complete scene: {scene}") shutil.rmtree(source) if delete_files: - downloaded.unlink() + for path in downloaded: + path.unlink() def build_parser() -> argparse.ArgumentParser: - parser = argparse.ArgumentParser(description="Prepare the ground-truth ETH3D-SLAM monocular benchmark") + parser = argparse.ArgumentParser(description="Prepare the ETH3D-SLAM mono or stereo benchmark") + parser.add_argument("--mode", choices=("mono", "stereo", "all"), default="mono") parser.add_argument( "--scenes", nargs="+", @@ -444,4 +539,4 @@ def build_parser() -> argparse.ArgumentParser: if __name__ == "__main__": arguments = build_parser().parse_args() - main(delete_files=arguments.delete_downloads, scenes=arguments.scenes) + main(delete_files=arguments.delete_downloads, scenes=arguments.scenes, mode=arguments.mode) diff --git a/vidmap/datasets/prepare/euroc.py b/vidmap/datasets/prepare/euroc.py index 3087281..629ed7b 100644 --- a/vidmap/datasets/prepare/euroc.py +++ b/vidmap/datasets/prepare/euroc.py @@ -1,10 +1,11 @@ -"""Download and exactly reproduce the established monocular EuRoC benchmark.""" +"""Prepare EuRoC camera streams and benchmark selections.""" from __future__ import annotations import argparse import io import json +import os import shutil import struct import zipfile @@ -19,6 +20,8 @@ from vidmap.datasets.base import PreparedTestset from vidmap.datasets.layouts import get_dataset_layout +from vidmap.datasets.prepare.helpers.filesystem import scene_staging +from vidmap.datasets.prepare.helpers.rig_capture import camera_options, frame_image_ids, write_frame_table from vidmap.datasets.prepare.helpers.video_common import checksum, resumable_http_download from vidmap.datasets.resources import dataset_asset @@ -65,7 +68,8 @@ class CameraCalibration: class ImagePose: image_id: int name: str - camera_from_world: np.ndarray + camera_from_world: np.ndarray | None + camera_id: int = 1 def source_inventory() -> tuple[dict[str, SourceArchive], dict[str, SequenceSource]]: @@ -101,7 +105,7 @@ def source_inventory() -> tuple[dict[str, SourceArchive], dict[str, SequenceSour def parse_sensor(text: str) -> CameraCalibration: - """Parse one official EuRoC cam0 ``sensor.yaml``.""" + """Parse one official EuRoC ``sensor.yaml``.""" payload = yaml.safe_load(text) camera_model = payload["camera_model"] distortion_model = payload["distortion_model"] @@ -129,7 +133,7 @@ def parse_sensor(text: str) -> CameraCalibration: def parse_image_index(text: str) -> tuple[tuple[int, str], ...]: - """Parse the ordered official cam0 timestamp/filename CSV.""" + """Parse the ordered official camera timestamp/filename CSV.""" images = [] for line_number, line in enumerate(text.splitlines(), start=1): if not line or line.startswith("#"): @@ -235,7 +239,7 @@ def _rectification( def _rectify_png(encoded: bytes, map1: np.ndarray, map2: np.ndarray, size: tuple[int, int]) -> bytes: raw = cv2.imdecode(np.frombuffer(encoded, dtype=np.uint8), cv2.IMREAD_UNCHANGED) if raw is None: - raise ValueError("Could not decode an official EuRoC cam0 PNG") + raise ValueError("Could not decode an official EuRoC PNG") if (raw.shape[1], raw.shape[0]) != size: raise ValueError(f"EuRoC image/calibration size mismatch: {(raw.shape[1], raw.shape[0])} vs {size}") image = cv2.remap(raw, map1, map2, interpolation=cv2.INTER_LINEAR) @@ -245,15 +249,13 @@ def _rectify_png(encoded: bytes, map1: np.ndarray, map2: np.ndarray, size: tuple return output.tobytes() -def _camera_bytes(calibration: CameraCalibration, rectified: np.ndarray) -> bytes: - fx, fy, cx, cy = rectified[[0, 1, 0, 1], [0, 1, 2, 2]] - return b"".join( - ( - struct.pack(" bytes: + output = io.BytesIO() + output.write(struct.pack(" bytes: @@ -261,13 +263,14 @@ def _images_bytes(images: tuple[ImagePose, ...]) -> bytes: output.write(struct.pack(" None: def _ensure_testset(scene: str) -> None: + legacy = EUROC_LAYOUT.testset_path(scene, "all") + destination = EUROC_LAYOUT.testset_path(scene, "mono") + if legacy.is_file() and not destination.exists(): + legacy.rename(destination) _write_exact( - EUROC_LAYOUT.testset_path(scene, "all"), + destination, canonical_testset_bytes(scene), ) @@ -351,7 +358,7 @@ def _read_member(bundle: zipfile.ZipFile, member: str) -> bytes: return bundle.read(member) -def _recording_is_prepared(recording_dir: Path, sequence: SequenceSource) -> bool: +def _recording_is_prepared(recording_dir: Path, sequence: SequenceSource, *, mode="mono") -> bool: """Check the established image and reference-model contents for one recording.""" images_dir = recording_dir / "rectified" / "images" / "cam0" rec_dir = recording_dir / "rectified" / "rec" @@ -363,57 +370,83 @@ def _recording_is_prepared(recording_dir: Path, sequence: SequenceSource) -> boo ): return False preparation = json.loads(receipt.read_text(encoding="utf-8")) - if preparation["preparation_version"] != EUROC_PREPARATION_VERSION or preparation["variants"] != { - "rectified": {"rig": None, "force_gt_keyframes": True} - }: + if ( + preparation["preparation_version"] != EUROC_PREPARATION_VERSION + or set(preparation["variants"]) != {"rectified"} + or not preparation["variants"]["rectified"]["force_gt_keyframes"] + ): return False - image_paths = tuple(sorted(path for path in images_dir.iterdir() if path.is_file())) + reconstruction = pycolmap.Reconstruction(rec_dir) + ordered = tuple( + (iid, reconstruction.images[iid]) for iid in range(sequence.canonical_images) if iid in reconstruction.images + ) + image_paths = tuple(images_dir / Path(image.name).name for _, image in ordered) if ( len(image_paths) != sequence.canonical_images or image_paths[0].name != sequence.first_image or image_paths[-1].name != sequence.last_image ): return False - reconstruction = pycolmap.Reconstruction(rec_dir) - ordered = tuple(sorted(reconstruction.images.items())) - camera = next(iter(reconstruction.cameras.values()), None) + if mode in {"stereo", "all"}: + rig = preparation["variants"]["rectified"]["rig"] + if rig is None or len(rig["cameras"]) != 2 or not (images_dir.parent / "frames.csv").is_file(): + return False + if not frame_image_ids(recording_dir / "rectified"): + return False + camera = reconstruction.cameras[1] if 1 in reconstruction.cameras else None return ( camera is not None - and len(reconstruction.cameras) == 1 + and len(reconstruction.cameras) in (1, 2) and camera.model_name == "PINHOLE" and len(ordered) == sequence.canonical_images and tuple(image_id for image_id, _ in ordered) == tuple(range(sequence.canonical_images)) and tuple(image.name for _, image in ordered) == tuple(f"cam0/{path.name}" for path in image_paths) + and all((images_dir.parent / image.name).is_file() for image in reconstruction.images.values()) ) -def euroc_scene_is_prepared(scene_dir: Path) -> bool: +def euroc_scene_is_prepared(scene_dir: Path, *, mode="mono") -> bool: """Check that every recording and the shared scene testset are prepared.""" scene = scene_dir.name if scene not in EUROC_LAYOUT.scenes: return False - testset = EUROC_LAYOUT.testset_path(scene, "all") + testset = EUROC_LAYOUT.testset_path(scene, "mono") if not testset.is_file() or testset.read_bytes() != canonical_testset_bytes(scene): return False _, sequences = source_inventory() - return all( - _recording_is_prepared(scene_dir / sequence.recording, sequence) + if not all( + _recording_is_prepared(scene_dir / sequence.recording, sequence, mode=mode) for sequence in sequences.values() if sequence.scene == scene - ) + ): + return False + if mode in {"stereo", "all"}: + expected = { + seq.recording: PreparedTestset( + seq.recording, + "rectified", + frame_image_ids(EUROC_LAYOUT.input_dir(scene, seq.recording, "rectified")), + ) + for seq in sequences.values() + if seq.scene == scene + } + return EUROC_LAYOUT.read_testsets(scene, "stereo") == expected + return True -def _write_preparation_record(recording_dir: Path, sequence: SequenceSource, aggregate: SourceArchive) -> None: +def _write_preparation_record(recording_dir: Path, sequence: SequenceSource, aggregate: SourceArchive, rig) -> None: payload = { "dataset": "EuRoC MAV", "scene": sequence.scene, "recording": sequence.recording, "preparation_version": EUROC_PREPARATION_VERSION, - "variants": {"rectified": {"rig": None, "force_gt_keyframes": True}}, + "variants": {"rectified": {"rig": rig, "force_gt_keyframes": True}}, "gt_evaluable": True, - "camera": "cam0", + "cameras": [f"cam{i}" for i in range(len(rig["cameras"]))], "image_rectification": "opencv-radtan-alpha-0.0", - "temporal_sampling": "all-cam0-frames-within-ground-truth-range", + "temporal_sampling": ( + "all-cam0-frames-within-ground-truth-range" if len(rig["cameras"]) == 1 else "synchronized-frames" + ), "source_archive": aggregate.filename, "source_archive_sha256": aggregate.sha256, "sequence_archive_sha256": sequence.sha256, @@ -426,50 +459,97 @@ def _write_preparation_record(recording_dir: Path, sequence: SequenceSource, agg ) -def prepare_recording(sequence: SequenceSource, aggregate: SourceArchive, download_dir: Path) -> None: - """Prepare one official EuRoC sequence without replacing existing output.""" +def prepare_recording(sequence: SequenceSource, aggregate: SourceArchive, download_dir: Path, *, mode="mono") -> None: + """Prepare the selected camera streams together, reusing existing image files.""" destination = EUROC_LAYOUT.recording_dir(sequence.scene, sequence.recording) + previous = destination.with_name(f".{destination.name}.preparing") / "previous" + if previous.exists() and not destination.exists(): + previous.rename(destination) if destination.exists() or destination.is_symlink(): - if _recording_is_prepared(destination, sequence): + if not _recording_is_prepared(destination, sequence): + raise RuntimeError(f"Existing EuRoC recording is not the canonical prepared output: {destination}") + if _recording_is_prepared(destination, sequence, mode=mode): return - raise RuntimeError(f"Existing EuRoC recording is not the canonical prepared output: {destination}") sequence_path = _sequence_archive(sequence, aggregate, download_dir) - staging = EUROC_LAYOUT.data_dir / ".euroc-preparing" / sequence.scene / sequence.recording - staging.mkdir(parents=True, exist_ok=True) - with zipfile.ZipFile(sequence_path) as bundle: - calibration = parse_sensor(_read_member(bundle, "mav0/cam0/sensor.yaml").decode("utf-8")) - image_index = parse_image_index(_read_member(bundle, "mav0/cam0/data.csv").decode("utf-8")) - if len(image_index) != sequence.raw_cam0_images: - raise ValueError( - f"EuRoC {sequence.recording} raw cam0 count mismatch: {len(image_index)} != {sequence.raw_cam0_images}" - ) - ground_truth = parse_ground_truth( - _read_member(bundle, "mav0/state_groundtruth_estimate0/data.csv").decode("utf-8") - ) - poses = _canonical_poses(image_index, ground_truth, calibration.body_to_camera) + count = 2 if mode in {"stereo", "all"} else 1 + with ( + scene_staging(destination, replace=destination.exists()) as temporary, + zipfile.ZipFile(sequence_path) as bundle, + ): + staging = temporary / "recording" + calibrations = [parse_sensor(_read_member(bundle, f"mav0/cam{i}/sensor.yaml").decode()) for i in range(count)] + indices = [ + dict(parse_image_index(_read_member(bundle, f"mav0/cam{i}/data.csv").decode())) for i in range(count) + ] + if len(indices[0]) != sequence.raw_cam0_images: + raise ValueError(f"EuRoC {sequence.recording} raw cam0 count mismatch") + ground_truth = parse_ground_truth(_read_member(bundle, "mav0/state_groundtruth_estimate0/data.csv").decode()) + poses = list(_canonical_poses(tuple(indices[0].items()), ground_truth, calibrations[0].body_to_camera)) if ( len(poses) != sequence.canonical_images or poses[0].name != sequence.first_image or poses[-1].name != sequence.last_image ): raise ValueError(f"EuRoC {sequence.recording} canonical GT-range inventory mismatch") - rectified, map1, map2 = _rectification(calibration) - for image in poses: - contents = _read_member(bundle, f"mav0/cam0/data/{image.name}") - _write_exact( - staging / "rectified" / "images" / "cam0" / image.name, - _rectify_png(contents, map1, map2, (calibration.width, calibration.height)), + stamps = sorted(set.intersection(*(set(index) for index in indices))) + primary_names = {image.name for image in poses} + streams, cameras, options = [], [], [] + for cid, calibration in enumerate(calibrations): + matrix, map1, map2 = _rectification(calibration) + camera = pycolmap.Camera( + camera_id=cid + 1, + model="PINHOLE", + width=calibration.width, + height=calibration.height, + params=[matrix[0, 0], matrix[1, 1], matrix[0, 2], matrix[1, 2]], ) - - _write_exact(staging / "rectified" / "rec" / "cameras.bin", _camera_bytes(calibration, rectified)) - _write_exact(staging / "rectified" / "rec" / "images.bin", _images_bytes(poses)) - _write_exact(staging / "rectified" / "rec" / "points3D.bin", struct.pack(" 1 else primary_names + if cid == 0: + names |= primary_names + images_dir = staging / "rectified" / "images" / f"cam{cid}" + images_dir.mkdir(parents=True) + for name in sorted(names): + previous = destination / "rectified" / "images" / f"cam{cid}" / name + if previous.is_file(): + os.link(previous, images_dir / name) + else: + _write_exact( + images_dir / name, + _rectify_png( + _read_member(bundle, f"mav0/cam{cid}/data/{name}"), + map1, + map2, + (calibration.width, calibration.height), + ), + ) + if cid != 0 or name not in primary_names: + poses.append(ImagePose(len(poses), name, None, cid + 1)) + streams.append({stamp: f"cam{cid}/{indices[cid][stamp]}" for stamp in stamps}) + # EuRoC T_BS maps cameras into the body frame; OpenCV uses integer pixel centres. + cam_from_ref = ( + pycolmap.Rigid3d((np.linalg.inv(calibration.body_to_camera) @ calibrations[0].body_to_camera)[:3]) + if cid + else None + ) + params = camera.params.copy() + params[2:4] += 0.5 + runtime_camera = pycolmap.Camera(model="PINHOLE", width=camera.width, height=camera.height, params=params) + options.append(camera_options(runtime_camera, cam_from_ref)) + root = staging / "rectified" + _write_exact(root / "rec" / "cameras.bin", _camera_bytes(cameras)) + _write_exact(root / "rec" / "images.bin", _images_bytes(tuple(poses))) + _write_exact(root / "rec" / "points3D.bin", struct.pack(" 1: + write_frame_table(root / "images", streams) + rig = dict(cameras=options, frames="frames.csv" if count > 1 else None) + _write_preparation_record(staging, sequence, aggregate, rig) + if not _recording_is_prepared(staging, sequence, mode=mode): + raise RuntimeError(f"Prepared EuRoC recording failed final validation: {sequence.recording}") + if destination.exists(): + destination.rename(temporary / "previous") + staging.rename(destination) def _validate_zip_response(response) -> None: @@ -483,8 +563,11 @@ def main( *, download: bool = True, delete_files: bool = False, + mode: str = "mono", ) -> None: """Download official bundles and prepare every recording in the selected scenes.""" + if mode not in {"mono", "stereo", "all"}: + raise ValueError(f"Unknown EuRoC mode: {mode}") selected = EUROC_LAYOUT.select_scenes(scenes) archives, sequences = source_inventory() pending = [] @@ -497,10 +580,10 @@ def main( if destination.exists() or destination.is_symlink(): if not _recording_is_prepared(destination, sequence): raise RuntimeError(f"Existing EuRoC recording is not the canonical prepared output: {destination}") + if not _recording_is_prepared(destination, sequence, mode=mode): + pending.append(sequence) else: pending.append(sequence) - if not pending: - return download_dir = EUROC_LAYOUT.data_dir / "downloads" needed_archives = tuple(dict.fromkeys(sequence.scene for sequence in pending)) @@ -518,7 +601,19 @@ def main( archive = archives[archive_name] _validate_archive(download_dir / archive.filename, size=archive.size, sha256=archive.sha256) for sequence in pending: - prepare_recording(sequence, archives[sequence.scene], download_dir) + prepare_recording(sequence, archives[sequence.scene], download_dir, mode=mode) + if mode in {"stereo", "all"}: + for scene in selected: + cases = { + seq.recording: PreparedTestset( + seq.recording, + "rectified", + frame_image_ids(EUROC_LAYOUT.input_dir(scene, seq.recording, "rectified")), + ).as_dict() + for seq in sequences.values() + if seq.scene == scene + } + _write_exact(EUROC_LAYOUT.testset_path(scene, "stereo"), yaml.safe_dump(cases, sort_keys=False).encode()) if delete_files: for sequence in pending: (download_dir / "sequences" / f"{sequence.recording}.zip").unlink(missing_ok=True) @@ -529,6 +624,7 @@ def main( if __name__ == "__main__": parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--scenes", nargs="+", choices=EUROC_LAYOUT.scenes) + parser.add_argument("--mode", choices=("mono", "stereo", "all"), default="mono") parser.add_argument( "--no-download", action="store_true", @@ -544,4 +640,5 @@ def main( arguments.scenes, download=not arguments.no_download, delete_files=arguments.delete_downloads, + mode=arguments.mode, ) diff --git a/vidmap/datasets/prepare/helpers/filesystem.py b/vidmap/datasets/prepare/helpers/filesystem.py index 13bf4da..60bb6d6 100644 --- a/vidmap/datasets/prepare/helpers/filesystem.py +++ b/vidmap/datasets/prepare/helpers/filesystem.py @@ -74,15 +74,20 @@ def extract_archive( @contextmanager -def scene_staging(destination: Path) -> Iterator[Path]: +def scene_staging(destination: Path, *, replace: bool = False) -> Iterator[Path]: temporary = destination.with_name(f".{destination.name}.preparing") + previous = temporary / "previous" + if previous.exists() and not destination.exists(): + previous.rename(destination) shutil.rmtree(temporary, ignore_errors=True) - if destination.exists(): + if destination.exists() and not replace: raise RuntimeError(f"Prepared scene already exists: {destination}") temporary.mkdir(parents=True) try: yield temporary finally: + if previous.exists() and not destination.exists(): + previous.rename(destination) shutil.rmtree(temporary, ignore_errors=True) diff --git a/vidmap/datasets/prepare/helpers/hilti_stitching.py b/vidmap/datasets/prepare/helpers/hilti_stitching.py new file mode 100644 index 0000000..38ed5fc --- /dev/null +++ b/vidmap/datasets/prepare/helpers/hilti_stitching.py @@ -0,0 +1,202 @@ +"""Stitch Hilti fisheye pairs into equirectangular images using the official 10 m sphere.""" + +from __future__ import annotations + +import json +import sqlite3 +from collections import deque +from concurrent.futures import ThreadPoolExecutor +from contextlib import closing +from pathlib import Path +from tempfile import TemporaryDirectory + +import cv2 +import numpy as np +import yaml + +TOPICS = tuple(f"/cam{i}/image_raw/compressed" for i in (0, 1)) +_WRITE_WORKERS = 2 + + +def _write_jpeg(path, image): + if not cv2.imwrite(str(path), image, [cv2.IMWRITE_JPEG_QUALITY, 95]): + raise OSError(path) + + +def compressed_stream(connection, topic): + """Yield exact header nanoseconds and original compressed image bytes.""" + from rosbags.typesys import Stores, get_typestore + + rows = connection.execute("SELECT id, type, serialization_format FROM topics WHERE name = ?", (topic,)).fetchall() + if len(rows) != 1 or rows[0][1:] != ("sensor_msgs/msg/CompressedImage", "cdr"): + raise ValueError(f"Expected one CDR CompressedImage topic: {topic}") + typestore = get_typestore(Stores.ROS2_HUMBLE) + previous = -1 + cursor = connection.execute("SELECT data FROM messages WHERE topic_id = ? ORDER BY timestamp, id", (rows[0][0],)) + for (data,) in cursor: + message = typestore.deserialize_cdr(data, "sensor_msgs/msg/CompressedImage") + stamp = message.header.stamp + if not 0 <= stamp.nanosec < 1_000_000_000: + raise ValueError("Invalid image header nanoseconds") + timestamp = int(stamp.sec) * 1_000_000_000 + int(stamp.nanosec) + if timestamp <= previous: + raise ValueError(f"Duplicate or non-increasing header timestamp in {topic}") + previous = timestamp + yield timestamp, bytes(message.data) + + +def synchronized_pairs(stream0, stream1, counts): + """Merge exact timestamp intersections, counting discarded unpaired images.""" + streams = [iter(stream0), iter(stream1)] + current = [next(stream, None) for stream in streams] + while all(item is not None for item in current): + if current[0][0] == current[1][0]: + yield current[0][0], (current[0][1], current[1][1]) + current = [next(stream, None) for stream in streams] + else: + index = int(current[1][0] < current[0][0]) + counts[index] += 1 + current[index] = next(streams[index], None) + for index, item in enumerate(current): + if item is not None: + counts[index] += 1 + sum(1 for _ in streams[index]) + + +PANORAMA_FROM_CAM0 = np.diag([-1.0, -1.0, 1.0]) # Original physical images are upside down. + + +def read_eucm_calibration(path): + """Use the original EUCM parameters embedded in the official calibration file.""" + text = Path(path).read_text() + marker = "########### EUCM ############" + if marker not in text: + raise ValueError("Expected original EUCM calibration block in official Hilti YAML") + lines = text.split(marker, 1)[1].splitlines() + source = "\n".join(line[2:] for line in lines if line.startswith("# ")) + result = yaml.safe_load(source) + for name in ("cam0", "cam1"): + camera = result[name] + if camera["camera_model"] != "eucm" or camera["distortion_model"] != "none": + raise ValueError("Expected EUCM cameras without additional distortion") + a, b, fx, fy, cx, cy = camera["intrinsics"] + if not np.isfinite([a, b, fx, fy, cx, cy]).all() or not (0 < a < 1 and min(b, fx, fy) > 0): + raise ValueError("Invalid EUCM calibration") + return result + + +def lens_map(points, calibration, mask): + """EUCM projection with original OpenCV integer pixel centres and mask polarity.""" + a, b, fx, fy, cx, cy = calibration["intrinsics"] + x, y, z = np.moveaxis(points, -1, 0) + d = np.sqrt(b * (x * x + y * y) + z * z) + denominator = a * d + (1 - a) * z + # EUCM's invertible domain (Usenko et al.): z > -w*d. Merely + # requiring a positive projection denominator admits the folded branch. + domain_weight = a / (1 - a) if a <= 0.5 else (1 - a) / a + valid = (denominator > 1e-12) & (z > -domain_weight * d) + u = fx * np.divide(x, denominator, out=np.zeros_like(x), where=valid) + cx + v = fy * np.divide(y, denominator, out=np.zeros_like(y), where=valid) + cy + w, h = calibration["resolution"] + valid &= (u >= 0) & (v >= 0) & (u < w - 1) & (v < h - 1) + # Conservatively require all four bilinear taps to be valid. + i = np.clip(np.floor(u).astype(int), 0, w - 2) + j = np.clip(np.floor(v).astype(int), 0, h - 2) + valid &= (mask[j, i] == 0) & (mask[j + 1, i] == 0) & (mask[j, i + 1] == 0) & (mask[j + 1, i + 1] == 0) + score = np.where(valid, z / np.linalg.norm(points, axis=-1), -np.inf) + return np.stack([u, v], -1).astype(np.float32), score + + +def render_maps(calibration, masks, width, height, sphere_radius_m): + T0 = np.asarray(calibration["cam0"]["T_cam_imu"]) + T1 = np.asarray(calibration["cam1"]["T_cam_imu"]) + relative = T1 @ np.linalg.inv(T0) + yy, xx = np.indices((height, width)) + longitude = (xx + 0.5) * (2 * np.pi / width) - np.pi + latitude = (yy + 0.5) * (np.pi / height) - np.pi / 2 + rays = np.stack([np.cos(latitude) * np.sin(longitude), np.sin(latitude), np.cos(latitude) * np.cos(longitude)], -1) + points0 = sphere_radius_m * (rays @ PANORAMA_FROM_CAM0) + points1 = points0 @ relative[:3, :3].T + relative[:3, 3] + projected = [lens_map(p, calibration[f"cam{i}"], masks[i]) for i, p in enumerate((points0, points1))] + scores = np.stack([item[1] for item in projected]) + winner = np.argmax(scores, axis=0) + valid = np.isfinite(scores.max(axis=0)) + xy = np.where((winner == 0)[..., None], projected[0][0], projected[1][0]) + xy[~valid] = -1 + # Quantize before offsetting cam1 so joining the images preserves interpolation. + xy, weights = cv2.convertMaps(xy, None, cv2.CV_16SC2) + xy[..., 0][winner == 1] += calibration["cam0"]["resolution"][0] + return xy, weights, valid, relative + + +def prepare(bag, calibration, masks, output): + sphere_radius_m = 10.0 + bag, calibration, output = Path(bag).resolve(), Path(calibration).resolve(), Path(output).absolute() + if output.exists(): + raise FileExistsError(f"Refusing to overwrite {output}") + if len(masks) != 2: + raise ValueError("Provide two official masks") + calibration_data = read_eucm_calibration(calibration) + width = max(1024, 2 * min(camera["resolution"][0] for camera in calibration_data.values())) + height = width // 2 + original_masks = [] + for i, path in enumerate(masks): + mask = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + w, h = calibration_data[f"cam{i}"]["resolution"] + if mask is None or mask.shape != (h, w) or mask.dtype != np.uint8 or not set(np.unique(mask)) <= {0, 255}: + raise ValueError("Expected binary original-size OpenVINS masks (zero valid)") + original_masks.append(mask) + map_xy, map_weights, valid, relative = render_maps( + calibration_data, original_masks, width, height, sphere_radius_m + ) + resolutions = [calibration_data[f"cam{i}"]["resolution"] for i in range(2)] + source = np.zeros((max(h for w, h in resolutions), sum(w for w, h in resolutions), 3), dtype=np.uint8) + manifest = dict( + schema_version=1, + variants={"panorama": dict(rig=None, force_gt_keyframes=False)}, + preparation="hilti-equirectangular-panorama", + source_camera_model="EUCM", + image_size=[width, height], + sphere_radius_m=sphere_radius_m, + physical_baseline_m=float(np.linalg.norm(relative[:3, 3])), + geometry="central-panorama-approximation", + panorama_from_cam0=PANORAMA_FROM_CAM0.tolist(), + timestamp_unit="nanoseconds", + unpaired_timestamp_counts=[0, 0], + ) + output.parent.mkdir(parents=True, exist_ok=True) + with TemporaryDirectory(prefix=f".{output.name}-", dir=output.parent) as temporary: + staged = Path(temporary) / "prepared" + (staged / "panorama" / "images" / "cam0").mkdir(parents=True) + count = 0 + pending = deque() + with ( + closing(sqlite3.connect(bag.as_uri() + "?mode=ro", uri=True)) as connection, + ThreadPoolExecutor(max_workers=_WRITE_WORKERS) as writer, + ): + connection.execute("PRAGMA temp_store=MEMORY") + streams = [compressed_stream(connection, topic) for topic in TOPICS] + for timestamp, payloads in synchronized_pairs(*streams, manifest["unpaired_timestamp_counts"]): + if len(pending) == _WRITE_WORKERS: + pending.popleft().result() + count += 1 + offset = 0 + for payload, (w, h) in zip(payloads, resolutions, strict=True): + image = cv2.imdecode(np.frombuffer(payload, np.uint8), cv2.IMREAD_COLOR) + if image is None or image.shape != (h, w, 3): + raise ValueError("Unexpected physical image shape") + source[:h, offset : offset + w] = image + offset += w + image = cv2.remap(source, map_xy, map_weights, cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT) + name = f"panorama/images/cam0/{timestamp // 10**9}.{timestamp % 10**9:09d}.jpg" + pending.append(writer.submit(_write_jpeg, staged / name, image)) + for future in pending: + future.result() + if count < 2: + raise ValueError("At least two synchronized timestamps are required") + manifest["timestamp_count"] = count + manifest["valid_fraction"] = float(valid.mean()) + (staged / "preparation.json").write_text(json.dumps(manifest, indent=2) + "\n") + if output.exists(): + raise FileExistsError(output) + staged.rename(output) + return output / "panorama" / "images" diff --git a/vidmap/datasets/prepare/helpers/rig_capture.py b/vidmap/datasets/prepare/helpers/rig_capture.py index 1a4451e..f4bec06 100644 --- a/vidmap/datasets/prepare/helpers/rig_capture.py +++ b/vidmap/datasets/prepare/helpers/rig_capture.py @@ -1,5 +1,6 @@ """Load camera-folder inputs and prepare supplied calibration with COLMAP undistortion.""" +import csv from dataclasses import replace from pathlib import Path @@ -10,12 +11,43 @@ CaptureSnapshot, PreparedCapture, build_capture_reconstruction, + read_frame_table, resolve_capture_frames, ) from vidmap.datasets.local import select_images from vidmap.datasets.prepare.helpers.filesystem import prepare_image_directory +def camera_options(camera, cam_from_ref=None): + result = dict(camera_model_name=camera.model_name, camera_params=camera.params.tolist()) + if cam_from_ref is not None: + x, y, z, w = cam_from_ref.rotation.quat + result.update(cam_from_rig_rotation=[w, x, y, z], cam_from_rig_translation=cam_from_ref.translation.tolist()) + return result + + +def write_frame_table(image_dir, streams): + """Associate requested camera streams by their exact source timestamps.""" + stamps = sorted(set.intersection(*(set(stream) for stream in streams))) + if len(stamps) < 2: + raise ValueError("Insufficient synchronized timestamps") + with (image_dir / "frames.csv").open("w", newline="") as file: + writer = csv.writer(file) + writer.writerow(["frame_id", "timestamp_ns", "camera_id", "image_name"]) + for frame_id, stamp in enumerate(stamps, start=1): + for camera_id, stream in enumerate(streams): + writer.writerow([frame_id, stamp, camera_id, stream[stamp]]) + + +def frame_image_ids(root): + """Select the reference-model image IDs belonging to complete rig frames.""" + import pycolmap + + rec = pycolmap.Reconstruction(root / "rec") + ids = {image.name: iid for iid, image in rec.images.items()} + return tuple(ids[row["image_name"]] for row in read_frame_table(root / "images" / "frames.csv")) + + def load_capture(image_dir: str | Path, options=None, *, imnames=None) -> PreparedCapture: """Load camN/ streams; without supplied calibration, accept one cam0/ stream.""" import pycolmap diff --git a/vidmap/datasets/prepare/helpers/ros_benchmarks.py b/vidmap/datasets/prepare/helpers/ros_benchmarks.py new file mode 100644 index 0000000..e65e783 --- /dev/null +++ b/vidmap/datasets/prepare/helpers/ros_benchmarks.py @@ -0,0 +1,370 @@ +"""Shared download and preparation flow for Spires and Hilti.""" + +import argparse +import bisect +import json +import multiprocessing +from concurrent.futures import ProcessPoolExecutor, as_completed +from decimal import Decimal +from functools import partial +from pathlib import Path + +import numpy as np +import pycolmap +import yaml + +from vidmap.datasets.layouts import get_dataset_layout +from vidmap.datasets.names import get_dataset_definition +from vidmap.datasets.prepare.helpers.filesystem import scene_staging +from vidmap.datasets.prepare.helpers.video_common import checksum, resumable_http_download +from vidmap.datasets.resources import dataset_asset + +PREPARATION_VERSIONS = {"spires": 1, "hilti": 1} + + +def source_inventory(dataset): + return json.loads(dataset_asset(dataset, "sources.json").read_text()) + + +def modes(dataset): + """Map benchmark names to an image variant and the number of streams starting at cam0.""" + suite = json.loads(dataset_asset("benchmark_suites.json").read_text())[dataset] + return suite["modes"] + + +def timestamp_ns(value): + nanoseconds = Decimal(value) * 1_000_000_000 + if nanoseconds != nanoseconds.to_integral_value(): + raise ValueError(f"Timestamp is not an integer number of nanoseconds: {value}") + return int(nanoseconds) + + +def read_poses(path): + """Read TUM world-from-sensor poses.""" + timestamps, poses = [], [] + for line in Path(path).read_text().splitlines(): + fields = line.split() + if not fields or fields[0].startswith("#"): + continue + if len(fields) != 8: + raise ValueError(f"Expected timestamp xyz quaternion in {path}") + stamp = timestamp_ns(fields[0]) + values = np.asarray(fields[1:], dtype=float) + if not np.isfinite(values).all() or not np.isclose(np.linalg.norm(values[3:]), 1, atol=1e-3): + raise ValueError(f"Invalid GT pose in {path}") + if timestamps and stamp <= timestamps[-1]: + raise ValueError(f"GT timestamps must strictly increase: {path}") + timestamps.append(stamp) + poses.append(pycolmap.Rigid3d(pycolmap.Rotation3d(values[3:]), values[:3])) + if len(timestamps) < 3: + raise ValueError(f"Expected at least three GT poses: {path}") + return timestamps, poses + + +def nearest_pose(timestamps, poses, stamp, max_offset_ns): + index = bisect.bisect_left(timestamps, stamp) + candidates = range(max(0, index - 1), min(len(timestamps), index + 1)) + nearest = min(candidates, key=lambda i: (abs(timestamps[i] - stamp), timestamps[i])) + return poses[nearest] if abs(timestamps[nearest] - stamp) <= max_offset_ns else None + + +def write_ground_truth( + root, dataset, recording, variants, gt_path, camera_from_source, max_offset_ns, min_timestamp_ns=0 +): + """Write calibrated models per variant, preserving benchmark image IDs.""" + from vidmap.datasets.local import select_images + from vidmap.datasets.prepare.helpers.rig_capture import load_capture + from vidmap.frontend.options.rig import RigOptions + + # Panorama intrinsics remain placeholders; the runtime frontend calibrates its rendered views. + times, poses = read_poses(gt_path) + captures = { + name: load_capture(root / name / "images", None if spec["rig"] is None else RigOptions(**spec["rig"])) + for name, spec in variants.items() + } + image_ids, testsets = {}, {} + # Assign the existing benchmark IDs before adding other prepared stream images. + for mode, selection in modes(dataset).items(): + variant, num_cameras = selection["variant"], selection["num_cameras"] + if variant not in captures or len(captures[variant].rec.cameras) < num_cameras: + continue + capture = captures[variant] + names = ( + [image.name for _, image in sorted(capture.rec.images.items()) if image.camera_id < num_cameras] + if num_cameras > 1 + else select_images(capture.rgb_dir, None, camera_dir="cam0") + ) + for name in names: + if name not in image_ids: + image_ids[name] = len(image_ids) + 1 + testsets[mode] = dict(recording=recording, variant=variant, image_ids=[image_ids[name] for name in names]) + + for variant, capture in captures.items(): + reconstruction, posed_ids = pycolmap.Reconstruction(), set() + for camera_id, camera in sorted(capture.rec.cameras.items()): + reconstruction.add_camera_with_trivial_rig(camera) + for name in select_images(capture.rgb_dir, None, camera_dir=f"cam{camera_id}"): + if name not in image_ids: + image_ids[name] = len(image_ids) + 1 + image_id = image_ids[name] + stamp = timestamp_ns(Path(name).stem) + source_pose = ( + nearest_pose(times, poses, stamp, max_offset_ns) + if camera_id == 0 and stamp >= min_timestamp_ns + else None + ) + pose = ( + camera_from_source * source_pose.inverse() + if source_pose is not None + else pycolmap.Rigid3d(pycolmap.Rotation3d([0.0, 0.0, 0.0, -1.0]), np.zeros(3)) + ) + image = pycolmap.Image(image_id=image_id, name=name, camera_id=camera_id) + reconstruction.add_image_with_trivial_frame(image, pose) + if source_pose is not None: + posed_ids.add(image_id) + for mode, selection in testsets.items(): + if selection["variant"] == variant and len(posed_ids.intersection(selection["image_ids"])) < 3: + raise ValueError(f"Fewer than three reference-camera images have supported GT: {mode}") + destination = root / variant / "rec" + destination.mkdir() + reconstruction.write(destination) + return testsets + + +def _download(rows, root, download): + paths = [] + for row in rows: + path = root / row["path"] + if not download and not path.is_file(): + raise FileNotFoundError(f"Missing source file: {path}") + paths.append( + resumable_http_download(row["url"], path, expected_size=row["size"], expected_digest=row["sha256"]) + ) + return paths + + +def _install_testsets(layout, scene, destination): + testsets = json.loads((destination / "preparation.json").read_text())["testsets"] + for mode in testsets: + contents = yaml.safe_dump({mode: testsets[mode]}).encode() + path = layout.testset_path(scene, mode) + if path.exists() and path.read_bytes() != contents: + raise ValueError(f"Existing testset differs from prepared scene: {path}") + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_suffix(".yaml.part") + temporary.write_bytes(contents) + temporary.replace(path) + + +def scene_is_prepared(layout, scene, *, mode=None): + from vidmap.frontend.options.rig import RigOptions + + definitions = modes(layout.name) + if mode is not None and mode not in definitions: + raise ValueError(f"Unknown {layout.name} input mode: {mode}") + destination = layout.recording_dir(scene, "0") + receipt = destination / "preparation.json" + if not receipt.is_file(): + return False + manifest = json.loads(receipt.read_text()) + if manifest["sources"] != source_inventory(layout.name)["shared"] + source_inventory(layout.name)["scenes"][scene]: + return False + if manifest["schema_version"] != PREPARATION_VERSIONS[layout.name]: + return False + files = manifest["files"] + if mode is not None: + if mode not in manifest["testsets"]: + return False + definition = definitions[mode] + variant = definition["variant"] + if manifest["testsets"][mode]["variant"] != variant: + return False + image_root = Path(variant) / "images" + source = manifest["variants"][variant] + required = set() + directories = tuple(f"cam{cid}" for cid in range(definition["num_cameras"])) + if definition["num_cameras"] > 1: + options = RigOptions(**source["rig"]) + if options.frames is not None: + required.add((image_root / options.frames).as_posix()) + prefixes = (f"{variant}/rec/", *(f"{image_root}/{directory}/" for directory in directories)) + files = {name: size for name, size in files.items() if name in required or name.startswith(prefixes)} + for name, expected in files.items(): + path = destination / name + if not path.is_file() or path.stat().st_size != expected: + return False + for name, digest in manifest["metadata_sha256"].items(): + if name not in files: + continue + path = destination / name + if not path.is_file() or checksum(path, "sha256") != digest: + return False + return True + + +def _delete_downloads(rows, root): + for row in rows: + (root / row["path"]).unlink(missing_ok=True) + + +def _prepare_scene(scene, *, layout, inventory, shared, mode, download, delete_files): + dataset = layout.name + downloads = layout.data_dir / "downloads" + destination = layout.recording_dir(scene, "0") + sources = _download(inventory["scenes"][scene], downloads, download) + with scene_staging(destination, replace=True) as staging: + output = staging / "scene" + if dataset == "spires": + from vidmap.datasets.prepare.spires import prepare + + calibration = next(path for path in shared if path.name == "cam-lidar.yaml") + prepare( + [path for path in sources if path.suffix == ".bag"], + calibration.parent, + output, + mode=mode, + previous=destination if destination.exists() else None, + ) + transform = np.asarray(yaml.safe_load(calibration.read_text())["cam0"]["T_cam_lidar"]) + # Spires GT reports base_link poses; calibration is relative to the LiDAR. + base_from_lidar = np.diag([-1.0, -1.0, 1.0, 1.0]) + base_from_lidar[:3, 3] = [0, 0, 0.124] + camera_from_source = pycolmap.Rigid3d((transform @ np.linalg.inv(base_from_lidar))[:3]) + # Upstream Spires COLMAP evaluation uses evo --t_max_diff 0.01. + max_offset_ns = 10_000_000 + min_timestamp_ns = 0 + else: + from vidmap.datasets.prepare.helpers.hilti_stitching import PANORAMA_FROM_CAM0, prepare + + bag = next(path for path in sources if path.suffix == ".db3") + prepare( + bag, + next(path for path in shared if path.suffix == ".yaml"), + sorted(path for path in shared if path.suffix == ".png"), + output, + ) + camera_from_source = pycolmap.Rigid3d(pycolmap.Rotation3d(PANORAMA_FROM_CAM0), np.zeros(3)) + max_offset_ns = 20_000_000 + min_timestamp_ns = 10_005_000_000_000 + gt_path = next(path for path in sources if path.suffix == ".txt") + preparation = json.loads((output / "preparation.json").read_text()) + testsets = write_ground_truth( + output, + dataset, + "0", + preparation["variants"], + gt_path, + camera_from_source, + max_offset_ns, + min_timestamp_ns, + ) + files = { + path.relative_to(output).as_posix(): path.stat().st_size + for path in output.rglob("*") + if path.is_file() and path != output / "preparation.json" + } + hashes = { + name: checksum(output / name, "sha256") for name in files if Path(name).suffix not in {".jpg", ".png"} + } + preparation.update( + schema_version=PREPARATION_VERSIONS[dataset], + sources=inventory["shared"] + inventory["scenes"][scene], + testsets=testsets, + max_gt_offset_ns=max_offset_ns, + min_gt_timestamp_ns=min_timestamp_ns, + files=files, + metadata_sha256=hashes, + ) + (output / "preparation.json").write_text(json.dumps(preparation, indent=2) + "\n") + if destination.exists(): + destination.rename(staging / "previous") + output.rename(destination) + _install_testsets(layout, scene, destination) + if not scene_is_prepared(layout, scene): + raise RuntimeError(f"Preparation failed validation: {scene}") + if delete_files: + _delete_downloads(inventory["scenes"][scene], downloads) + + +def _initialize_worker(): + import cv2 + + cv2.setNumThreads(1) + + +def main(dataset, scenes=None, *, mode=None, download=True, delete_files=False, workers=1): + if workers < 1: + raise ValueError("workers must be positive") + mode = get_dataset_definition(dataset).default_mode if mode is None else mode + if mode != "all" and mode not in modes(dataset): + raise ValueError(f"Unknown {dataset} input mode: {mode}") + requested = modes(dataset) if mode == "all" else (mode,) + layout = get_dataset_layout(dataset) + inventory = source_inventory(dataset) + selected = layout.select_scenes(scenes) + downloads = layout.data_dir / "downloads" + pending = [] + for scene in selected: + destination = layout.recording_dir(scene, "0") + prepared = () + if destination.exists(): + if not scene_is_prepared(layout, scene): + raise RuntimeError(f"Partial or incompatible preparation: {destination}") + prepared = json.loads((destination / "preparation.json").read_text())["testsets"] + if all(item in prepared for item in requested): + _install_testsets(layout, scene, destination) + if delete_files: + _delete_downloads(inventory["scenes"][scene], downloads) + else: + pending.append(scene) + if pending: + shared = _download(inventory["shared"], downloads, download) + prepare_scene = partial( + _prepare_scene, + layout=layout, + inventory=inventory, + shared=shared, + mode=mode, + download=download, + delete_files=delete_files, + ) + if workers == 1 or len(pending) == 1: + for scene in pending: + prepare_scene(scene) + else: + with ProcessPoolExecutor( + max_workers=min(workers, len(pending)), + mp_context=multiprocessing.get_context("spawn"), + initializer=_initialize_worker, + ) as executor: + futures = [executor.submit(prepare_scene, scene) for scene in pending] + try: + for future in as_completed(futures): + future.result() + finally: + for future in futures: + future.cancel() + if delete_files and selected: + _delete_downloads(inventory["shared"], downloads) + + +def run_cli(dataset): + parser = argparse.ArgumentParser(description=f"Download and prepare the {dataset} benchmark.") + parser.add_argument("--scenes", nargs="+", choices=get_dataset_layout(dataset).scenes) + parser.add_argument( + "--mode", choices=(*modes(dataset), "all"), default=get_dataset_definition(dataset).default_mode + ) + parser.add_argument("--no-download", action="store_true") + parser.add_argument("--delete-downloads", action="store_true") + parser.add_argument( + "--workers", type=int, default=1, help="Number of recordings to prepare concurrently (default: 1)" + ) + args = parser.parse_args() + main( + dataset, + args.scenes, + mode=args.mode, + download=not args.no_download, + delete_files=args.delete_downloads, + workers=args.workers, + ) diff --git a/vidmap/datasets/prepare/hilti.py b/vidmap/datasets/prepare/hilti.py new file mode 100644 index 0000000..929dd59 --- /dev/null +++ b/vidmap/datasets/prepare/hilti.py @@ -0,0 +1,12 @@ +"""Download and prepare the Hilti 360-degree benchmark.""" + +from vidmap.datasets.prepare.helpers.ros_benchmarks import main as prepare_dataset +from vidmap.datasets.prepare.helpers.ros_benchmarks import run_cli + + +def main(scenes=None, *, mode="panorama", download=True, delete_files=False, workers=1): + prepare_dataset("hilti", scenes, mode=mode, download=download, delete_files=delete_files, workers=workers) + + +if __name__ == "__main__": + run_cli("hilti") diff --git a/vidmap/datasets/prepare/lamar.py b/vidmap/datasets/prepare/lamar.py index 6f36920..9896ae7 100644 --- a/vidmap/datasets/prepare/lamar.py +++ b/vidmap/datasets/prepare/lamar.py @@ -5,6 +5,7 @@ import argparse import gzip import hashlib +import io import json import logging import re @@ -40,6 +41,7 @@ _PREPARATION_VERSION = 1 _EXPECTED_GT_IMAGE_COUNTS = {"CAB": 206_161, "HGE": 130_953, "LIN": 117_970} _EXPECTED_SAMPLE_IMAGE_COUNTS = {"CAB": 36_334, "HGE": 27_289, "LIN": 30_866} +_EXPECTED_COMPLETE_IMAGE_COUNTS = {"CAB": 111862, "HGE": 109891, "LIN": 108450} _MODEL_FILES = ("cameras.bin", "images.bin", "points3D.bin") _SAMPLE_TARGET = re.compile(r"^(ios_.+)_\d{3}-\d+$") @@ -80,7 +82,7 @@ def _ground_truth_models_archive(downloads: Path, *, download: bool) -> Path: ) -def source_inventory(profile: str = "sample-300") -> dict[str, tuple[SourceArchive, ...]]: +def source_inventory(profile: str = "complete") -> dict[str, tuple[SourceArchive, ...]]: """Load checksum-pinned official session archives for the requested profile.""" payload = json.loads(dataset_asset("lamar", "source_archives.json").read_text(encoding="utf-8")) base_url = payload["base_url"] @@ -174,13 +176,15 @@ def rotate_camera(camera: RawCamera, rotations: int) -> RawCamera: return RawCamera(width, height, (fx, fy, cx, cy)) -def infer_rotation(camera: RawCamera, target: pycolmap.Camera) -> int: +def infer_rotation(camera: RawCamera, target: pycolmap.Camera, *, principal_point_tolerance: float = 1e-9) -> int: """Infer the unique image rotation that maps raw calibration to the GT camera.""" matches = [] for rotations in range(4): candidate = rotate_camera(camera, rotations) - if (candidate.width, candidate.height) == (target.width, target.height) and np.allclose( - candidate.params, target.params, rtol=0.0, atol=1e-9 + if ( + (candidate.width, candidate.height) == (target.width, target.height) + and np.allclose(candidate.params[:2], target.params[:2], rtol=0.0, atol=1e-9) + and np.allclose(candidate.params[2:], target.params[2:], rtol=0.0, atol=principal_point_tolerance) ): matches.append(rotations) if len(matches) != 1: @@ -205,8 +209,12 @@ def generate_testsets(scene: str, image_ids: set[int]) -> dict[str, dict[str, li return result -def canonical_testsets(scene: str, profile: str = "sample-300") -> dict[str, bytes]: +def canonical_testsets(scene: str, profile: str = "complete") -> dict[str, bytes]: """Read the exact canonical testset YAML bytes for one scene.""" + if profile == "complete": + return { + "complete": gzip.decompress(dataset_asset("lamar", "testsets", scene, "complete.yaml.gz").read_bytes()) + } if profile != "sample-300": raise ValueError(f"Unknown LaMAR profile: {profile}") return {"sample": dataset_asset("lamar", "testsets", scene, "sample-300.yaml").read_bytes()} @@ -218,7 +226,7 @@ def _parse_selection(contents: bytes) -> dict[str, list[int]]: def _sample_mapping(scene: str) -> dict[str, list[int]]: - mapping = _parse_selection(canonical_testsets(scene)["sample"]) + mapping = _parse_selection(canonical_testsets(scene, "sample-300")["sample"]) if not isinstance(mapping, dict): raise ValueError(f"Bundled LaMAR sample-300 testset is malformed for {scene}") return mapping @@ -234,7 +242,7 @@ def _sample_sessions(scene: str) -> set[str]: return sessions -def _selected_image_names(scene: str, ground_truth: pycolmap.Reconstruction, profile: str = "sample-300") -> set[str]: +def _selected_image_names(scene: str, ground_truth: pycolmap.Reconstruction, profile: str = "complete") -> set[str]: mapping = _parse_selection(next(iter(canonical_testsets(scene, profile).values()))) image_ids = {image_id for values in mapping.values() for image_id in values} by_id = {int(image.image_id): str(image.name) for image in ground_truth.images.values()} @@ -242,7 +250,9 @@ def _selected_image_names(scene: str, ground_truth: pycolmap.Reconstruction, pro if missing: raise ValueError(f"LaMAR {scene} selection references missing image IDs: {sorted(missing)[:3]}") names = {by_id[image_id] for image_id in image_ids} - expected = _EXPECTED_SAMPLE_IMAGE_COUNTS[scene] + expected = ( + _EXPECTED_SAMPLE_IMAGE_COUNTS[scene] if profile == "sample-300" else _EXPECTED_COMPLETE_IMAGE_COUNTS[scene] + ) if len(names) != expected: raise ValueError(f"LaMAR {scene} {profile} image count mismatch: expected {expected}, got {len(names)}") sessions = {Path(name).parent.as_posix() for name in names} @@ -251,7 +261,7 @@ def _selected_image_names(scene: str, ground_truth: pycolmap.Reconstruction, pro return names -def _ensure_testsets(scenes: Iterable[str], profile: str = "sample-300") -> None: +def _ensure_testsets(scenes: Iterable[str], profile: str = "complete") -> None: """Install benchmark cases with explicit recording and variant references.""" for scene in scenes: selections = {mode: {} for mode in canonical_testsets(scene, profile)} @@ -330,7 +340,7 @@ def _recording_testsets(scene, profile, ground_truth): return result -def _write_recording_model(destination, records, cameras): +def _write_recording_model(destination, records, cameras, official_poses): """Preserve source IDs and pose sentinels while flattening recording-local names.""" destination.mkdir(parents=True) selected_cameras = sorted({values[-1] for values in records.values()}) @@ -343,13 +353,18 @@ def _write_recording_model(destination, records, cameras): with (destination / "images.bin").open("xb") as stream: stream.write(struct.pack(" None: destination.mkdir(parents=True, exist_ok=True) with TemporaryDirectory(prefix=f".{scene}-", dir=destination) as temporary: @@ -422,6 +443,12 @@ def _convert_scene( for name, values in _image_records(stream): if name in required_names: records[Path(name).parent.as_posix()][name] = values + official_poses = None + if profile == "complete": + if official_models is None: + raise ValueError("The official pose archive is required") + images = {image.name: image for image in ground_truth.images.values() if image.name in required_names} + official_poses = _official_poses(scene, official_models, images) for archive in archives: recording = archive.stem if recording not in testsets: @@ -437,6 +464,7 @@ def _convert_scene( records[recording], testsets[recording], profile, + official_poses, ) if not lamar_recording_is_prepared(staging, profile=profile, recording=recording): raise RuntimeError(f"LaMAR preparation did not produce a complete recording: {recording}") @@ -471,6 +499,11 @@ def lamar_recording_is_prepared( or recording not in _profile_sessions(scene, manifest["profile"]) ): return False + if manifest["profile"] == "complete" and ( + manifest["official_pose_sha256"] != _official_source(scene)["pose_sha256"] + or manifest["official_camera_sha256"] != _official_source(scene)["camera_sha256"] + ): + return False if manifest["model_sha256"] != {name: _sha256(rec_dir / name) for name in _MODEL_FILES}: return False canonical = { @@ -494,7 +527,7 @@ def lamar_recording_is_prepared( return names == records.keys() and all(Path(name).parent.as_posix() == "cam0" for name in records) -def lamar_scene_is_prepared(dataset_root: Path, scene: str, profile: str = "sample-300") -> bool: +def lamar_scene_is_prepared(dataset_root: Path, scene: str, profile: str = "complete") -> bool: """Return whether all recordings selected by one benchmark group are ready.""" return all( lamar_recording_is_prepared(dataset_root / scene / recording, profile=profile) @@ -519,9 +552,9 @@ def _pending_recordings(scenes, profile): def _profile_sessions(scene: str, profile: str) -> set[str]: - if profile != "sample-300": - raise ValueError(f"Unknown LaMAR profile: {profile}") - return _sample_sessions(scene) + if profile == "sample-300": + return _sample_sessions(scene) + return set(_parse_selection(canonical_testsets(scene, profile)["complete"])) def _image_records(stream): @@ -538,12 +571,64 @@ def _image_records(stream): yield name.decode(), values +def _official_source(scene): + return json.loads(dataset_asset("lamar", "official_pose_sources.json").read_text())[scene] + + +def _pinhole_cameras(stream): + cameras = {} + for _ in range(struct.unpack(" None: """Download official iOS sessions and prepare selected or all LaMAR scenes.""" selected = LAMAR_LAYOUT.select_scenes(scenes) @@ -571,15 +656,24 @@ def main( elif not path.is_file() or path.stat().st_size != source.size or _sha256(path) != source.sha256: raise FileNotFoundError(f"Missing checksum-valid LaMAR archive: {path}") archive_paths.append(path) + official_models = None + if profile == "complete": + official_models = downloads / "official-colmap" / f"{scene}.zip" + if not download and not official_models.is_file(): + raise FileNotFoundError(f"Missing official pose archive: {official_models}") + resumable_http_download(_official_source(scene)["url"], official_models) _convert_scene( scene, tuple(archive_paths), LAMAR_LAYOUT.scene_dir(scene), ground_truth_models, profile=profile, + official_models=official_models, ) _ensure_testsets((scene,), profile) if delete_files: + if official_models is not None: + official_models.unlink() for path in archive_paths: path.unlink() scene_downloads = downloads / scene @@ -594,7 +688,7 @@ def main( logging.basicConfig(level=logging.INFO, format="%(message)s") parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--scenes", nargs="+", choices=LAMAR_LAYOUT.scenes) - parser.add_argument("--profile", choices=["sample-300"], default="sample-300") + parser.add_argument("--profile", choices=["sample-300", "complete"], default="complete") parser.add_argument( "--no-download", action="store_true", diff --git a/vidmap/datasets/prepare/spires.py b/vidmap/datasets/prepare/spires.py new file mode 100644 index 0000000..d5106b2 --- /dev/null +++ b/vidmap/datasets/prepare/spires.py @@ -0,0 +1,200 @@ +"""Prepare native and rectified Spires camera streams.""" + +import io +import json +import os +import shutil +from decimal import Decimal +from pathlib import Path +from tempfile import TemporaryDirectory + +import cv2 +import numpy as np +import yaml +from PIL import Image +from scipy.spatial.transform import Rotation + +from vidmap.datasets.prepare.helpers.rig_capture import write_frame_table + + +def prepare(bags, calibration_dir, output, *, mode="undistorted-mono", previous=None): + """Publish the requested camera streams atomically.""" + output = Path(output).absolute() + if output.exists() or output.is_symlink(): + raise FileExistsError(output) + output.parent.mkdir(parents=True, exist_ok=True) + with TemporaryDirectory(prefix=f".{output.name}-", dir=output.parent) as temporary: + staged = Path(temporary) / "prepared" + result = _prepare(bags, calibration_dir, staged, mode=mode, previous=previous) + if output.exists(): + raise FileExistsError(output) + staged.rename(output) + return result + + +def _prepare(bags, calibration_dir, output, *, mode, previous): + from rosbags.highlevel import AnyReader + + from vidmap.datasets.prepare.helpers.ros_benchmarks import modes + + bags = [Path(path).resolve(strict=True) for path in bags] + calroot = Path(calibration_dir).resolve(strict=True) + output = Path(output).absolute() + if not bags or len(set(bags)) != len(bags): + raise ValueError("Provide distinct raw bags in recording order") + definitions = modes("spires") + selections = definitions.values() if mode == "all" else (definitions[mode],) + streams = {} + for selection in selections: + variant = selection["variant"] + streams[variant] = max(streams[variant] if variant in streams else 0, selection["num_cameras"]) + if previous is not None: + previous = Path(previous) + metadata = json.loads((previous / "preparation.json").read_text()) + for variant, spec in metadata["variants"].items(): + streams[variant] = max(streams[variant] if variant in streams else 0, len(spec["rig"]["cameras"])) + streams = dict(sorted(streams.items())) + output.mkdir(parents=True, exist_ok=False) + pending = {variant: set() for variant in streams} + for variant, count in streams.items(): + for i in range(count): + relative = Path(variant) / "images" / f"cam{i}" + if previous is not None and (previous / relative).is_dir(): + # Link immutable images into staging; models and metadata are rebuilt separately. + shutil.copytree(previous / relative, output / relative, copy_function=os.link) + else: + (output / relative).mkdir(parents=True) + pending[variant].add(i) + camera_ids = range(max(streams.values())) + maps = {} + sizes = {} + intrinsics = {} + native_cameras = [] + for i in camera_ids: + c = yaml.safe_load((calroot / f"cam{i}.yaml").read_text()) + K = np.array(c["camera_matrix"]["data"]).reshape(3, 3) + D = np.array(c["distortion_coefficients"]["data"]) + size = (c["image_width"], c["image_height"]) + newK = K.copy() + newK[0, 2], newK[1, 2] = size[0] / 2.0, size[1] / 2.0 + # OpenCV uses integer pixel centres; COLMAP uses half-integer centres. + intrinsics[i] = [float(newK[0, 0]), float(newK[1, 1]), float(newK[0, 2] + 0.5), float(newK[1, 2] + 0.5)] + if "rectified" in pending and i in pending["rectified"]: + maps[i] = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), newK, size, cv2.CV_16SC2) + sizes[i] = size + native_cameras.append( + dict( + camera_model_name="OPENCV_FISHEYE", + camera_params=[ + float(K[0, 0]), + float(K[1, 1]), + float(K[0, 2] + 0.5), + float(K[1, 2] + 0.5), + *D.tolist(), + ], + ) + ) + extrinsics_path = calroot / "cam-lidar.yaml" + extrinsics = yaml.safe_load(extrinsics_path.read_text()) + transforms = [np.asarray(extrinsics[f"cam{i}"]["T_cam_lidar"], dtype=float) for i in camera_ids] + for transform in transforms: + if ( + transform.shape != (4, 4) + or not np.isfinite(transform).all() + or not np.allclose(transform[3], [0, 0, 0, 1]) + or not np.allclose(transform[:3, :3].T @ transform[:3, :3], np.eye(3), atol=1e-6) + or not np.isclose(np.linalg.det(transform[:3, :3]), 1) + ): + raise ValueError("Invalid camera-to-LiDAR rigid calibration") + cameras = [] + for i in camera_ids: + camera = dict(camera_model_name="PINHOLE", camera_params=intrinsics[i]) + if i: + relative = transforms[i] @ np.linalg.inv(transforms[0]) + x, y, z, w = Rotation.from_matrix(relative[:3, :3]).as_quat() + camera.update( + cam_from_rig_rotation=[float(w), float(x), float(y), float(z)], + cam_from_rig_translation=relative[:3, 3].tolist(), + ) + native_cameras[i].update( + cam_from_rig_rotation=camera["cam_from_rig_rotation"], + cam_from_rig_translation=camera["cam_from_rig_translation"], + ) + cameras.append(camera) + needed = set().union(*pending.values()) + topics = {f"/alphasense_driver_ros/cam{i}/debayered/image/compressed": i for i in needed} + seen = {i: set() for i in needed} + for bag in bags if needed else (): + with AnyReader([bag]) as reader: + connections = [c for c in reader.connections if c.topic in topics] + if set(c.topic for c in connections) != set(topics): + raise ValueError("Missing camera topic") + for connection, _, raw in reader.messages(connections=connections): + i = topics[connection.topic] + message = reader.deserialize(raw, connection.msgtype) + t = int(message.header.stamp.sec) * 10**9 + int(message.header.stamp.nanosec) + if t in seen[i]: + raise ValueError("Duplicate image header timestamp") + seen[i].add(t) + name = f"{t//10**9}.{t%10**9:09d}.jpg" + payload = bytes(message.data) + if not payload.startswith(b"\xff\xd8"): + raise ValueError("Expected original JPEG bytes") + if "native" in pending and i in pending["native"]: + (output / "native/images" / f"cam{i}" / name).write_bytes(payload) + if "rectified" in pending and i in pending["rectified"]: + image = cv2.imdecode(np.frombuffer(payload, np.uint8), cv2.IMREAD_COLOR) + if image is None or image.shape[:2] != sizes[i][::-1]: + raise ValueError("Camera dimensions changed") + image = cv2.remap(image, *maps[i], interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT) + if not cv2.imwrite(str(output / "rectified/images" / f"cam{i}" / name), image): + raise OSError("Image encoding failed") + else: + with Image.open(io.BytesIO(payload)) as image: + if image.size != sizes[i]: + raise ValueError("Camera dimensions changed") + seen = {} + variants = {} + for variant, count in streams.items(): + timestamps = { + i: {int(Decimal(path.stem) * 10**9) for path in (output / variant / "images" / f"cam{i}").glob("*.jpg")} + for i in range(count) + } + for i, values in timestamps.items(): + if i in seen and seen[i] != values: + raise ValueError(f"Native and rectified cam{i} timestamps differ") + seen[i] = values + stamps = sorted(set.intersection(*timestamps.values())) + if len(stamps) < 2: + raise ValueError("Insufficient synchronized timestamps") + frames = None + if count > 1: + frames = "frames.csv" + write_frame_table( + output / variant / "images", + [{t: f"cam{i}/{t//10**9}.{t%10**9:09d}.jpg" for t in values} for i, values in timestamps.items()], + ) + calibration = native_cameras if variant == "native" else cameras + variants[variant] = dict(rig=dict(cameras=calibration[:count], frames=frames), force_gt_keyframes=False) + stamps = set.intersection(*seen.values()) + result = dict( + variants=variants, + per_camera_frames={str(i): len(s) for i, s in seen.items()}, + rig_frames=len(stamps), + unpaired_frames={str(i): len(s) - len(stamps) for i, s in seen.items()}, + mono_frames=len(seen[0]), + ) + (output / "preparation.json").write_text(json.dumps(result, indent=2) + "\n") + return result + + +def main(scenes=None, *, mode="undistorted-mono", download=True, delete_files=False, workers=1): + from vidmap.datasets.prepare.helpers.ros_benchmarks import main as prepare_dataset + + prepare_dataset("spires", scenes, mode=mode, download=download, delete_files=delete_files, workers=workers) + + +if __name__ == "__main__": + from vidmap.datasets.prepare.helpers.ros_benchmarks import run_cli + + run_cli("spires") diff --git a/vidmap/datasets/registry.py b/vidmap/datasets/registry.py index a7a5b4f..2a94430 100644 --- a/vidmap/datasets/registry.py +++ b/vidmap/datasets/registry.py @@ -22,6 +22,24 @@ def get_dataset_spec(name: str) -> DatasetSpec: layout = get_dataset_layout(name) preparer = None if definition.preparer_module is None else DatasetPreparation(definition.preparer_module) + if name in {"spires", "hilti"}: + from vidmap.datasets.prepare.helpers.ros_benchmarks import scene_is_prepared + + preparer = DatasetPreparation(definition.preparer_module, uses_mode=True) + + def missing_artifacts(layout, scenes, mode): + return tuple( + layout.scene_dir(scene) for scene in scenes if not scene_is_prepared(layout, scene, mode=mode) + ) + + return DatasetSpec( + name=name, + layout=layout, + preparer=preparer, + missing_artifacts=missing_artifacts, + default_mode=definition.default_mode, + default_evaluation_metric=definition.default_evaluation_metric, + ) if name == "crocodl": def missing_artifacts(layout, scenes, mode): @@ -42,7 +60,7 @@ def missing_artifacts(layout, scenes, mode): default_evaluation_metric=definition.default_evaluation_metric, ) if name == "lamar": - preparer = DatasetPreparation(definition.preparer_module, (("sample", "sample-300"),)) + preparer = DatasetPreparation(definition.preparer_module, (("sample", "sample-300"), ("complete", "complete"))) def missing_artifacts(layout, scenes, mode): from vidmap.datasets.prepare.lamar import lamar_scene_is_prepared @@ -63,13 +81,33 @@ def missing_artifacts(layout, scenes, mode): default_evaluation_metric=definition.default_evaluation_metric, ) + if name == "eth3d_slam": + from vidmap.datasets.prepare.eth3d_slam import _scene_is_complete + + def missing_artifacts(layout, scenes, mode): + return tuple( + layout.scene_dir(scene) for scene in scenes if not _scene_is_complete(layout, scene, mode=mode) + ) + + return DatasetSpec( + name=name, + layout=layout, + preparer=DatasetPreparation(definition.preparer_module, uses_mode=True), + missing_artifacts=missing_artifacts, + default_mode=definition.default_mode, + default_evaluation_metric=definition.default_evaluation_metric, + ) + if name == "euroc": + preparer = DatasetPreparation(definition.preparer_module, uses_mode=True) def missing_artifacts(layout, scenes, mode): from vidmap.datasets.prepare.euroc import euroc_scene_is_prepared return tuple( - layout.scene_dir(scene) for scene in scenes if not euroc_scene_is_prepared(layout.scene_dir(scene)) + layout.scene_dir(scene) + for scene in scenes + if not euroc_scene_is_prepared(layout.scene_dir(scene), mode=mode) ) return DatasetSpec( diff --git a/vidmap/datasets/selection.py b/vidmap/datasets/selection.py index 5d081c4..f78abb4 100644 --- a/vidmap/datasets/selection.py +++ b/vidmap/datasets/selection.py @@ -51,11 +51,6 @@ def __init__(self, options: TargetSelectionOptions, dataset: str | DatasetSpec) def iter_scene_targets(self) -> Iterator[tuple[str, tuple[PreparedTarget, ...]]]: scenes = self.dataset_layout.scenes if self.options.scene is None else self.options.scene for scene in scenes: - if not self.scene_has_public_ground_truth(scene) and self.options.mode != "all": - raise ValueError( - f"{self.dataset_spec.name}/{scene} is reconstruction-only and supports mode=all; " - "pose-selected testsets require public ground truth" - ) testsets = self.dataset_spec.select_testsets(scene, self.options.mode, self.options.testset_id) targets = tuple( PreparedTarget( @@ -71,11 +66,6 @@ def iter_scene_targets(self) -> Iterator[tuple[str, tuple[PreparedTarget, ...]]] yield scene, targets def _configure_testset_reconstructions(self, scene_parser, reference_image_ids, source_reconstruction): - if self.options.mode == "all": - reference_image_names = [scene_parser.rec.images[image_id].name for image_id in reference_image_ids] - scene_parser.kf_names = reference_image_names - return reference_image_names - scene_parser.rec = pycolmap.Reconstruction() for image_id in reference_image_ids: source_image = source_reconstruction.images[image_id]