Camera-Robot Hand-Eye Calibration

This project implements a complete AX = XB hand-eye calibration workflow for aligning a camera coordinate system with a robot base coordinate system. It supports both synthetic validation data and real-world datasets.

Features

• Synthetic Data Generation: Create configurable datasets with noise simulation.
• Hand-Eye Solvers: Supports multiple OpenCV methods (Tsai-Lenz, Daniilidis, Park, etc.).
• Real Data Support: Automatic pattern detection (Checkerboard, Circles Grid) and pose extraction.
• Evaluation Tools: Consistency checks (residuals), robustness testing (RANSAC/subsets), and 3D visualization.
• C++ Extension: High-performance solver implementation exposed to Python.

Installation

1. Clone the repository
2. Create a virtual environment (recommended):

   python3 -m venv venv
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

   (Note: Ensure you have opencv-python, numpy, scipy, matplotlib, open3d (optional) installed.)

Usage Guide

1. Synthetic Data Workflow (Validation)

Generate synthetic data to test the solver's accuracy under noise.

# 1. Generate 20 pose pairs with default noise (0.5 deg, 0.5 cm)
python src/generate_dataset.py --pairs 20 --output samples/synthetic_dataset.json

# 2. Run Calibration
python src/solver.py --input samples/synthetic_dataset.json --method TSAI

# 3. Benchmark Robustness (Noise 0.0 -> 2.0)
python src/benchmark_noise.py --max_noise 2.0

2. Real Data Workflow

Calibrate using real images and robot poses (e.g., from data/calib_test).

# 1. Parse Data (Detects pattern, extracts poses, saves to JSON)
#    - Detects 4x11 Asymmetric Circles Grid (32mm)
python src/parse_real_data.py

# 2. Run Calibration (using Daniilidis method for robustness)
python src/solver.py --input samples/real_dataset.json --output calibration_result_real.json --method DANIILIDIS

# 3. Evaluate & Visualize
#    - Computes AX-XB residuals and RANSAC stability
python src/evaluate_real.py
#    - Visualizes 3D coordinate frames
python src/visualize_real_setup.py --save results_real_setup.png

3. C++ Performance Solver

For high-performance applications, a C++ implementation is available.

# 1. Convert standard JSON dataset to OpenCV YAML
python src/convert_to_yaml.py --input samples/synthetic_dataset.json --output samples/dataset.yml

# 2. Build and Run C++ Solver
mkdir -p cpp/build && cd cpp/build
cmake ..
make
./handeye_solver ../../samples/dataset.yml

Project Structure

• src/: Python source code (Solvers, Generators, Visualization).
• cpp/: C++ implementation.
• data/: Raw input data (images, csv poses).
• samples/: Processed JSON datasets.
• experiments/: Benchmark plots and results.

Calibration Results (Real Data)

Using the Univ. Padova dataset (Asymmetric Circles Grid):

• Method: Daniilidis (Dual Quaternion)
• Rotation Error: ~1.5 degrees (mean residual)
• Translation Error: ~7.4 cm (mean residual)
  • Note: High translation residuals are due to pattern detection ambiguity (center jumps) in the provided dataset, verified by trajectory analysis.
• Stability: Stable rotation estimate (< 0.6 deg variance) across subsets.