GitHub - IFRoS-ELTE/autonomous_recycling_robot_25: Fruit recycling robot using Scout.2 Agilex robot

Fruit Recycling Robot using Scout.2 Agilex Robot

Team: Sanghera Davina | Laroussi Rihab | Nwafor Solomon Chibuzo
Course: IFRosLab - ELTE University
Robot: Silvanus (AgileX Scout 2.0 + xArm6)

Project Overview

This project develops an autonomous fruit recycling system that detects, classifies, and sorts recyclable materials using the Silvanus robot platform.
The current implementation focuses on a finite state machine (FSM) in recycle_node_simple_multi.py that coordinates fruit detection and robotic manipulation.

Getting Started

Cloning the Repository

This repository contains a submodule (yolo-seg-fruit) for fruit detection. To clone with all submodules:

git clone --recursive https://github.com/IFRoS-ELTE/autonomous_recycling_robot_25.git
cd autonomous_recycling_robot_25

If you've already cloned the repository without submodules, initialize them with:

git submodule update --init --recursive

Repository Structure

The repository is organized into several main directories:

autonomous_recycling_robot_25/
├── demos/                          # Demonstration videos and GIFs
│   ├── IFRoS_Lab2.gif             # Main demonstration video (GIF)
│   └── IFRoS_Lab2.mp4             # Main demonstration video (MP4)
│
├── docker_setup/                   # Docker configuration files
│   ├── compose.yaml                # Docker Compose configuration
│   ├── melodic.Dockerfile          # ROS Melodic Docker image
│   ├── docker_guide.md             # Docker setup instructions
│   ├── build_melodic.sh            # Build script for Docker image
│   ├── run_melodic.sh              # Run script for Docker container
│   └── zshrc                        # Shell configuration
│
├── moveit_config/                  # MoveIt motion planning configuration
│   ├── dh_robotics_ag95_model/     # DH Robotics AG95 gripper model
│   │   ├── launch/                  # Gripper visualization launch files
│   │   ├── urdf/                    # Gripper URDF/Xacro files
│   │   └── meshes/                  # Gripper 3D mesh files
│   │
│   └── scout_xarm/                 # Scout + xArm6 combined configuration
│       ├── scout_xarm_base/         # Base robot configuration
│       ├── scout_xarm_description/ # Robot description (URDF/Xacro)
│       │   ├── scout_urdf/          # Scout mobile base URDF
│       │   ├── xarm6_urdf/          # xArm6 manipulator URDF
│       │   └── xarm6_meshes/        # xArm6 3D mesh files
│       │
│       └── scout_xarm_moveit_config/ # MoveIt planning configuration
│           ├── config/              # Planning parameters (OMPL, CHOMP, etc.)
│           └── launch/              # MoveIt launch files
│
├── recycle/                        # Main recycling robot package
│   ├── scripts/                    # Python scripts
│   │   ├── recycle_node_simple_multi.py  # Main FSM controller 
│   │   ├── grasp_planner_multi.py        # Grasp planning and execution
│   │   ├── aruco_goal_publisher.py       # ArUco marker detection
│   │   ├── region_filter_bridge.py       # Camera-to-odom transform bridge
│   │   ├── restamp_pointcloud.py         # LiDAR point cloud timestamp fix
│   │   └── odom_tf_republisher.py        # TF timestamp republisher
│   │
│   ├── launch/                     # ROS launch files
│   │   └── bringup_minimal_1.launch      # Main system bringup 
│   │
│   ├── config/                     # Configuration files
│   │   ├── controllers.yaml         # Robot controller configuration
│   │   ├── costmap_*.yaml           # Navigation costmap parameters
│   │   └── *_planner_params.yaml    # Local planner parameters
│   │
│   ├── rviz/                       # RViz visualization configurations
│   │   └── scout_xarm_bringup.rviz # Main RViz config 
│   │
│   ├── urdf/                       # Robot description files
│   │   ├── scout_v2.xacro          # Scout mobile base
│   │   └── scout_xarm6.xacro       # Combined Scout + xArm6 
│   │
│   ├── src/                        # C++ source files
│   │   ├── scout_base_node.cpp     # Scout base control node
│   │   └── scout_messenger.cpp     # Scout communication interface
│   │
│   ├── include/                    # C++ header files
│   │   └── recycle/
│   │       └── scout_messenger.hpp
│   │
│   ├── GRIPPER_SYSTEM.md           # Gripper system documentation 
│   ├── TESTING_GUIDE.md            # Testing instructions
│   ├── CMakeLists.txt              # CMake build configuration
│   └── package.xml                 # ROS package manifest
│
├── scout_ros/                      # Scout mobile base ROS packages
│   ├── scout_base/                 # Base control package
│   │   ├── launch/                 # Base launch files
│   │   └── src/                    # Base control source code
│   │
│   ├── scout_bringup/              # System bringup package
│   │   ├── launch/                 # Bringup launch files
│   │   └── scripts/                # Setup scripts
│   │
│   ├── scout_description/         # Robot description package
│   │   ├── urdf/                   # URDF/Xacro files
│   │   ├── launch/                 # Display launch files
│   │   └── rviz/                   # RViz configurations
│   │
│   └── scout_msgs/                 # Custom message definitions
│       └── msg/                    # Message files
│
├── yolo-seg-fruit/                 # Fruit detection submodule 
│   ├── catkin_ws/                  # ROS workspace
│   │   └── src/
│   │       └── integration/
│   │           └── scripts/
│   │               └── detect_fruit_depth_improved.py  # Main detection script
│   │
│   ├── docker/                     # Docker setup for detection
│   │   ├── compose.yaml
│   │   └── melodic.Dockerfile
│   │
│   ├── scripts/                    # Utility scripts
│   │   ├── build.sh                # Build Docker image
│   │   ├── start.sh                # Start container
│   │   ├── detect.sh               # Run detection
│   │   └── stop.sh                 # Stop container
│   │
│   ├── README.md                   # Detection system documentation
│   ├── SETUP_INSTRUCTIONS.md       # Setup guide
│   └── REQUIREMENTS.txt            # Python dependencies
│
└── README.md                       # This file

Key Files:

recycle/scripts/recycle_node_simple_multi.py - Main FSM controller
recycle/launch/bringup_minimal_1.launch - System bringup
recycle/GRIPPER_SYSTEM.md - Gripper documentation
yolo-seg-fruit/ - Fruit detection submodule
recycle/urdf/scout_xarm6.xacro - Combined robot model
recycle/rviz/scout_xarm_bringup.rviz - RViz visualization

Project Presentation

Take a look at our project slideshow outlining the complete waste recycling system:

https://docs.google.com/presentation/d/1JIryyOf5k8tIjwnNDefBi-1vyTzC3IGpH2LB7fFdm4U/edit?usp=sharing

Below are the video demonstrations of the system:

Current Capabilities

Fruit / waste detection using RGB-D data (detect_fruit_depth_improved.py)
Classification of detected items (e.g fruit classes)
Robotic arm manipulation to pick and place items
ArUco marker-based bin identification
FSM-based task execution inside recycle_node_simple_multi.py

Hardware Platform

Mobile Base: AgileX Scout 2.0 (4WD differential drive)
Manipulator: xArm6 robotic arm with DH Robotics AG95 gripper
- See Gripper System Documentation for detailed gripper control, force parameters, and grasping workflow
Sensors:
- Intel RealSense D435 RGB-D camera (mounted on end-effector)
Compute: Onboard computer running ROS Melodic

Software Stack

ROS Framework: ROS Melodic
Development Environment: Docker containers for distributed development
Perception: Depth-aware fruit/waste detection (detect_fruit_depth_improved.py)
Manipulation: MoveIt-based arm motion planning (called from recycle_node_simple_multi.py)
Control Logic: Custom FSM in recycle_node_simple_multi.py coordinating perception and manipulation

Task Flow

Fruit detection (detect_fruit_depth_improved.py)
- Subscribes to RGB (/camera/color/image_raw), depth (/camera/aligned_depth_to_color/image_raw), and camera info.
- Runs YOLO models to detect fruits (carrot, orange, apple) in the RGB image.
- For each detection:
  - Refines the object shape using HSV color segmentation.
  - Computes an oriented bounding box and its yaw angle (object rotation).
  - Uses the depth image and camera intrinsics to recover a 3D point in the camera frame.
  - Transforms this point into the odom frame and keeps only detections inside a fixed workspace region in front of the robot.
- Publishes a PoseStamped (position + orientation from the yaw angle) for each valid detection on:
  - /carrot_pose_camera, /orange_pose_camera, /apple_pose_camera (plus corresponding visualization markers).
Target selection and 3D pose handling (FSM)
- RecycleNode subscribes to /carrot_pose_camera, /orange_pose_camera, /apple_pose_camera, /ball_pose_camera.
- While scanning for ArUco markers, object detections are ignored to avoid interference.
- After the scan finishes and markers are stored, object detection is enabled and stabilized for 2 seconds.
- The FSM then selects the next object to process with priority:
  - orange → carrot → apple → ball.
- The selected PoseStamped (camera frame) and its type are stored as the current target.
Grasp planning and execution with xArm6
- In STATE_PICK_OBJECT, the current object pose in camera frame is adjusted by fixed calibration offsets (x−0.04 m, y+0.03 m) and transformed into the MoveIt base frame (dummy_base_link).
- For carrots, the end-effector (joint 6) is rotated to align the gripper with the object's orientation; round objects (orange, apple, ball) skip this step.
- GraspPlanner.pick_object():
  - Builds a pre‑grasp pose above the object (same x,y, higher z) and moves there via a smooth Cartesian path.
  - Moves straight down by a small offset to the final grasp pose.
  - Closes the gripper to grasp the object and then returns the arm to the stored start joint configuration.
- A gripper state check detects if the gripper closed empty; in that case, the FSM fetches a fresh pose and retries the pick up to a configured maximum.
- Detailed Documentation: See Gripper System Documentation for complete workflow, force parameters, state machine, and retry logic.
Bin selection using ArUco markers
- During STATE_RUN_SCAN, the arm performs a 4‑step sweep (left–center–right–center), while an ArUco node (aruco_goal_publisher_v2.py) detects markers and publishes their poses (/aruco_marker_pose) and TF frames (aruco_marker_<id>).
- RecycleNode matches each detected pose to the corresponding TF frame to determine its ID and stores markers:
  - ID 0 → marker_0_pose (ball bin)
  - ID 1 → marker_1_pose (carrot bin)
  - ID 2 → marker_2_pose (orange bin)
  - ID 3 → marker_3_pose (apple bin)
- STATE_WAIT_SCAN_RESULTS parses the scan results and enables object detection once the scan is complete.
Move to drop‑off pose and release
- In STATE_DROP_OBJECT, the FSM chooses the bin based on the object type:
  - ball → marker ID 0, carrot → ID 1, orange → ID 2, apple → ID 3.
- The selected marker pose is transformed into the base frame and passed to GraspPlanner.drop_object():
  - The planner computes a drop pose for the end‑effector about 25 cm above the marker (compensating for the gripper tip offset).
  - The arm follows a Cartesian path to this pose.
  - The gripper opens to release the object into the bin.
Return to home pose and wait for next item
- After dropping, STATE_RETURN_START commands the arm back to the initial joint configuration stored at startup.
- All stored object poses and the current object type are cleared.
- The FSM returns to STATE_WAIT_OBJECT and waits for the next valid fruit detection from detect_fruit_depth_improved.py, repeating the cycle.

Finite State Machine (FSM) Behavior Tree

The following behavior tree diagram illustrates the complete FSM implementation in recycle_node_simple_multi.py:

graph TD
    Start([START]) --> Root[Root: Recycle Sequence]
    
    Root --> ScanSeq[Sequence: ArUco Scan]
    ScanSeq --> InitPos[Action: Check Initial Position]
    InitPos --> SetSlowSpeed[Action: Set Slow Speed 5%]
    SetSlowSpeed --> ScanSteps[Sequence: 4-Step Scan]
    
    ScanSteps --> Step1[Step 1: Move LEFT +180°]
    Step1 --> Wait1[Wait 2s for ArUco]
    Wait1 --> Step2[Step 2: Return CENTER 0°]
    Step2 --> Step3[Step 3: Move RIGHT -180°]
    Step3 --> Wait2[Wait 2s for ArUco]
    Wait2 --> Step4[Step 4: Return CENTER 0°]
    Step4 --> RestoreSpeed[Action: Restore Speed]
    RestoreSpeed --> PublishResults[Action: Publish Scan Results]
    
    Root --> WaitScan[Wait: Scan Results]
    WaitScan --> ProcessMarkers[Action: Store Markers 0,1,2,3]
    ProcessMarkers --> EnableDetection[Action: Enable Object Detection]
    EnableDetection --> WaitStable[Wait: 2s Stabilization]
    
    WaitStable --> ObjectSelector[Selector: Check Object Priority]
    ObjectSelector --> CheckOrange{Orange<br/>detected?}
    CheckOrange -->|Yes| SelectOrange[Set: orange]
    CheckOrange -->|No| CheckCarrot{Carrot<br/>detected?}
    CheckCarrot -->|Yes| SelectCarrot[Set: carrot]
    CheckCarrot -->|No| CheckApple{Apple<br/>detected?}
    CheckApple -->|Yes| SelectApple[Set: apple]
    CheckApple -->|No| CheckBall{Ball<br/>detected?}
    CheckBall -->|Yes| SelectBall[Set: ball]
    CheckBall -->|No| WaitMore[Wait: Continue waiting]
    WaitMore --> ObjectSelector
    
    SelectOrange --> PickSeq[Sequence: Pick Object]
    SelectCarrot --> PickSeq
    SelectApple --> PickSeq
    SelectBall --> PickSeq
    
    PickSeq --> ApplyCalib[Action: Apply Calibration<br/>x-0.04m, y+0.03m]
    ApplyCalib --> TransformBase[Action: Transform to Base Frame]
    TransformBase --> RotateGripper{Object type?}
    RotateGripper -->|Carrot| AlignGripper[Action: Rotate Gripper<br/>to Match Orientation]
    RotateGripper -->|Round| SkipRotate[Skip Rotation]
    AlignGripper --> ExecutePick[Action: Execute Pick<br/>MoveIt Planning]
    SkipRotate --> ExecutePick
    
    ExecutePick --> CheckGripper{Gripper<br/>Empty?}
    CheckGripper -->|Yes| CheckRetries{Retries<br/>< 5?}
    CheckRetries -->|Yes| GetFreshPose[Action: Get Fresh Pose]
    GetFreshPose --> IncrementRetry[Action: Increment Counter]
    IncrementRetry --> PickSeq
    CheckRetries -->|No| PickFailed[Failure: Max Retries]
    CheckGripper -->|No| PickSuccess[Success: Object Grasped]
    
    PickSuccess --> DropSeq[Sequence: Drop Object]
    PickFailed --> FailedState[State: FAILED]
    
    DropSeq --> SelectMarker{Object Type?}
    SelectMarker -->|Ball| Marker0[Select: Marker ID 0]
    SelectMarker -->|Carrot| Marker1[Select: Marker ID 1]
    SelectMarker -->|Orange| Marker2[Select: Marker ID 2]
    SelectMarker -->|Apple| Marker3[Select: Marker ID 3]
    
    Marker0 --> CheckMarker0{Marker 0<br/>Available?}
    Marker1 --> CheckMarker1{Marker 1<br/>Available?}
    Marker2 --> CheckMarker2{Marker 2<br/>Available?}
    Marker3 --> CheckMarker3{Marker 3<br/>Available?}
    
    CheckMarker0 -->|No| DropFailed[Failure: Marker Missing]
    CheckMarker1 -->|No| DropFailed
    CheckMarker2 -->|No| DropFailed
    CheckMarker3 -->|No| DropFailed
    
    CheckMarker0 -->|Yes| TransformDrop[Action: Transform Marker<br/>to Base Frame]
    CheckMarker1 -->|Yes| TransformDrop
    CheckMarker2 -->|Yes| TransformDrop
    CheckMarker3 -->|Yes| TransformDrop
    
    TransformDrop --> ExecuteDrop[Action: Execute Drop<br/>25cm Above Marker]
    ExecuteDrop --> OpenGripper[Action: Open Gripper]
    OpenGripper --> DropSuccess[Success: Object Dropped]
    
    DropSuccess --> ReturnSeq[Sequence: Return to Start]
    DropFailed --> FailedState
    
    ReturnSeq --> ReturnHome[Action: Return to Initial<br/>Joint Configuration]
    ReturnHome --> ClearPoses[Action: Clear All Object Poses]
    ClearPoses --> LoopBack[Loop: Back to Wait Object]
    LoopBack --> ObjectSelector
    
    FailedState --> End([END])
    
    style Start fill:#90EE90,stroke:#2d5016,stroke-width:2px,color:#000
    style End fill:#FFB6C1,stroke:#8b3a5c,stroke-width:2px,color:#000
    style FailedState fill:#FF6B6B,stroke:#8b0000,stroke-width:2px,color:#fff
    style PickSuccess fill:#90EE90,stroke:#2d5016,stroke-width:2px,color:#000
    style DropSuccess fill:#90EE90,stroke:#2d5016,stroke-width:2px,color:#000
    style Root fill:#E6F3FF,stroke:#0066CC,stroke-width:2px,color:#000
    style ScanSeq fill:#F0E6FF,stroke:#6B46C1,stroke-width:2px,color:#000
    style PickSeq fill:#FFF9E6,stroke:#CC9900,stroke-width:2px,color:#000
    style DropSeq fill:#FFE6E6,stroke:#CC3333,stroke-width:2px,color:#000
    style ReturnSeq fill:#E6FFF0,stroke:#009966,stroke-width:2px,color:#000
    style ObjectSelector fill:#E6F7FF,stroke:#006699,stroke-width:2px,color:#000

Key Features:

Sequences: Actions executed in order (Scan, Pick, Drop, Return)
Selectors: Priority-based object selection (orange → carrot → apple → ball)
Conditions: Decision points for gripper state, retries, and marker availability
Retry Logic: Automatic retry up to 5 times if gripper closes empty
Continuous Loop: Returns to object detection after each successful cycle

System Demonstration

The following demonstrations showcase different aspects of the recycling pipeline:

Output 1

Output 2

Output 3

The following video demonstrates the complete recycling pipeline in action, showing the robot successfully detecting, picking, and sorting carrot, apple, and orange into their respective bins:

The demonstration showcases the integrated system: YOLO-based fruit detection with color segmentation for precise orientation, MoveIt-based grasping with orientation-aware manipulation for elongated objects (carrots), and ArUco marker-based bin localization for accurate sorting.

Future work may re-enable mobile navigation around the environment.

Team Responsibilities

Team Member	Primary Focus	Current Tasks
Davina Sanghera	Perception & Detection	Fruit detection, depth integration
Nwafor Solomon Chibuzo	Bringup + Manipulation & Sorting	Bringup launch and setup,MoveIt configuration, grasping & execution with xArm6 pipeline, FSM integration
Rihab Laroussi	Bin Localization & Drop Pipeline	ArUco-based bin scan and ID mapping, object-to-bin assignment, and drop motion.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Fruit Recycling Robot using Scout.2 Agilex Robot

Project Overview

Getting Started

Cloning the Repository

Repository Structure

Project Presentation

Current Capabilities

Hardware Platform

Software Stack

Task Flow

Finite State Machine (FSM) Behavior Tree

System Demonstration

Team Responsibilities

About

Uh oh!

Releases

Packages

Uh oh!

Contributors 3

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 137 Commits
demos		demos
docker_setup		docker_setup
moveit_config		moveit_config
recycle		recycle
scout_ros		scout_ros
yolo-seg-fruit @ 6754d7a		yolo-seg-fruit @ 6754d7a
.gitattributes		.gitattributes
.gitmodules		.gitmodules
README.md		README.md

IFRoS-ELTE/autonomous_recycling_robot_25

Folders and files

Latest commit

History

Repository files navigation

Fruit Recycling Robot using Scout.2 Agilex Robot

Project Overview

Getting Started

Cloning the Repository

Repository Structure

Project Presentation

Current Capabilities

Hardware Platform

Software Stack

Task Flow

Finite State Machine (FSM) Behavior Tree

System Demonstration

Team Responsibilities

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors 3

Uh oh!

Languages

Packages