OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

News

• [Jul, 2025] The challenge website is available. [MICCAI2025, APTOS2025]
• [Nov, 2024] We have repaired several damaged videos. You can now download the dataset again.
• [Oct, 2024] We realeased OphNet2024 challenge dataset ! More information can be found in Data Preparation.
• [Jul, 2024] OphNet2024 is in preparation——larger scale, more accurate, and more experimental results!
• [Jul, 2024] OphNet was accepted by ECCV2024.
• [Jun, 2024] The manuscript can be found on arXiv.

Introduction

---

Dataset Preparation

Directory Structure

OphNet-benchmark
├── annotation
│   ├── OphNet2024_surgery.csv
│   ├── OphNet2024_loca_all.csv
│   ├── OphNet2024_loca_challenge.csv
│   ├── OphNet2024_loca_challenge_phase.csv
│   ├── OphNet2024_ori_operation_trimmed.csv
│   ├── OphNet2024_ori_phase_trimmed.csv
├── data_processing
│   ├── download.sh
│   ├── clipper.py

-annotation

• OphNet2024_surgery.csv: Annotated 1,969 untrimmed videos for surgical types, with the first label as the primary surgery. Selected 743 videos for time-boundary annotation.
• OphNet2024_loca_all.csv: The original version of the time boundary annotations.
• OphNet2024_loca_challenge.csv: Map phase and operation labels with fewer than 15 clips to numeric IDs 51 and 106, which can be interpreted as renaming labels with fewer than 15 instances as "Others."
• OphNet2024_loca_challenge_phase.csv: A complete phase clip in OphNet2024_challenge.csv may be split due to covering multiple operations. Therefore, in OphNet2024_challenge_phase.csv, we merge consecutive clips of the same phase.

-data_processing

• download.sh: download files.
• clipper.py: extract clips based on annotated time boundaries from untrimmed videos.

HuggingFace

OphNet2024
├── OphNet2024_all (≈305G, 1,969 untrimmed videos--original resolution and FPS)
│   ├── OphNet2024_all.tar.gz.00
│   ├── OphNet2024_all.tar.gz.01
│   ├── ...
├── OphNet2024_trimmed_operation (≈139G, 17,508 trimmed videos from 743 videos with time-boundary annotation--original resolution and FPS)
│   ├── OphNet2024_loca_challenge_trimmed.csv
│   ├── OphNet2024_trimmed_operation.tar.gz.00
│   ├── OphNet2024_trimmed_operation.tar.gz.01
│   ├── ...
├── OphNet2024_trimmed_phase (≈139G, 14,674 trimmed videos from 743 videos with time-boundary annotation--original resolution and FPS)
│   ├── OphNet2024_loca_challenge_phase_trimmed.csv
│   ├── OphNet2024_trimmed_phase.tar.gz.00
│   ├── OphNet2024_trimmed_phase.tar.gz.01
│   ├── ...
├── Features (≈26G, features for phase/operation localization)
│   ├── csn.tar.gz
│   ├── slowfast101.tar.gz
│   ├── swin_tiny.tar.gz
│   ├── videomae.tar.gz

• OphNet2024_loca_challenge_trimmed.csv: The OphNet2024_loca_challenge.csv file with the version containing trimmed video names will be automatically created after running data_processing/cliper.py. (/OphNet2024_trimmed_operation)
• OphNet2024_loca_challenge_phase_trimmed.csv: The OphNet2024_loca_challenge_phase.csv file with the version containing trimmed video names will be automatically created after running data_processing/cliper.py. (/OphNet2024_trimmed_phase)

Download

• Label Description: The table with Chinese and English versions of surgery, phase, and operation names along with their ID mappings: OphNet2024_Label

• HuggingFace Mirror (optional, if you are in mainland China):

  export HF_ENDPOINT=https://hf-mirror.com

• Download All:

  huggingface-cli download --repo-type dataset --resume-download xioamiyh/OphNet2024 --revision main --local-dir ./

• Selective Download:

  cd ./data_processing
  bash ./download.sh

• Merge and Extract the Archive:

  cat OphNet2024_all.tar.gz.* | tar xzvf -

• Skip Downloading Trimmed Video (optional, trimming videos locally with the script):

  python data_processing/cliper.py

---

Baseline Experiments and Code

Task 1: Phase/Operation Recognition

Task 2: Phase/Operation Localization

---

Challenge

Coming soon...

Discussion Group

If you have any questions about OphNet, please add this WeChat ID: conv-not-conv

---

TO DO

• Release untrimmed videos
• Release trimmed videos--operation level
• Release trimmed videos--phase level
• Release annotation files
• Release baseline experimental results and checkpoints

---

Citation

@article{hu2024ophnet,
  title={OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding},
  author={Hu, Ming and Xia, Peng and Wang, Lin and Yan, Siyuan and Tang, Feilong and Xu, Zhongxing and Luo, Yimin and Song, Kaimin and Leitner, Jurgen and Cheng, Xuelian and others},
  journal={arXiv preprint arXiv:2406.07471},
  year={2024}
}