Submission

---

Instructions

The evaluation server is under construction.

Our benchmark supports one official evaluation task for public submission. Please read the details below before uploading.

• Trajectory Evaluation: Submit predicted camera trajectories for sequences in the benchmark. Your results will be evaluated against ground-truth poses using the following metrics:
  • ATE (Absolute Trajectory Error): Measures the global accuracy of the estimated trajectory.
  • Flow AUC (Area Under Curve): An accuracy metric derived from the Induced Optical Flow (IOF). It measures the percentage of pixels whose flow error falls below thresholds from 0 to 100 pixels, and reports the area under this curve. Values range from 0% (poor pose estimate) to 100% (perfect pose).
  • Mean Rotation Error: Measures the average rotational discrepancy across the trajectory.
  • Coverage: Represents the proportion of the sequence for which valid pose estimates are available. Higher coverage indicates more complete tracking.
  • Composite Score: The harmonic mean of Flow AUC and Coverage, providing a balanced metric that rewards both motion accuracy and trajectory completeness.
  Submissions should include one file per sequence containing camera poses in TUM format.
  For some sequences, we also provide synchronized Stereo + IMU data. If your method uses this inertial information, it will be marked as an inertial method on the leaderboard. You can filter to compare only such methods using the "Inertial Methods Only" checkbox.


• Novel View Synthesis (NVS) Evaluation: While we do not host a public leaderboard for NVS, you can download the NVS Benchmark and evaluate your method locally. The dataset includes six challenging video sequences designed to test view synthesis quality.
  Evaluation is based on the following metrics:
  • PSNR (Peak Signal-to-Noise Ratio): Assesses photometric accuracy by comparing predicted and ground-truth pixel values.
  • LPIPS (Learned Perceptual Image Patch Similarity): Evaluates perceptual similarity using a deep network-based metric.
  Evaluation scripts and detailed instructions are available here.

How to Submit

Please follow the instructions here to prepare and upload your trajectory submission.

To ensure fair use, each email account is limited to three submissions every seven days. Please ensure your files are correctly named and formatted to avoid evaluation errors.