Add Robust Backup System (Manual, Automatic, Integrity, and Restore)

[PStarH]

Overview

This PR introduces a comprehensive backup system for the project, including:

• Manual and automatic backup creation
• Compression and encryption support
• Integrity verification with SHA-256 checksums
• Backup listing, restore, and deletion APIs
• Automatic backup scheduling and cleanup
• Seamless integration with model training and data directories

Key Features

1. Backup Core (BackupService, BackupIntegrity)

• Manual backup : Create backups of key directories (resources, database, vector DB) via API or service call.
• Compression : Optionally compress backup files using zlib, configurable via settings.
• Encryption : Optionally encrypt backup files using Fernet symmetric encryption.
• Integrity manifest : Each backup includes a manifest with checksums and file sizes for later verification.
• Restore : Restore data from any backup, with decompression/decryption and integrity checks.
• Delete : Remove backups by ID, with safe cleanup.

2. Automatic Backup (AutoBackupManager)

• Pre-training backup : Automatically create a backup before model training starts.
• Periodic backup : Schedule regular backups during long-running training jobs.
• Failure handling : Retries with exponential backoff and logging.
• Cleanup : Automatically remove old backups based on count and size limits.

3. API Integration

• RESTful endpoints for creating, listing, restoring, and deleting backups.
• Status and error reporting for all operations.

4. Configuration

• All backup parameters (paths, compression, encryption, intervals, limits) are configurable via the main config system.

Integration

• The backup system is fully integrated with the training workflow and data storage.
• All model checkpoints, training data, and metadata are included in the backup if they reside in the configured resource/data directories.
• Automatic backup is triggered at key points in the training lifecycle.

Usage

• Manual backup : Trigger via API or service call.
• Automatic backup : Enabled by default, configurable interval and retention.
• Restore : Use API to restore any backup by ID.
• Delete : Use API to delete old or unwanted backups.

Files Added/Modified

• lpm_kernel/backup/backup_service.py
• lpm_kernel/backup/backup_integrity.py
• lpm_kernel/backup/auto_backup.py
• lpm_kernel/backup/init.py
• lpm_kernel/api/domains/backup/routes.py
• (Plus config and logging integration)

[kevin-mindverse]

Hey there. Thank you for your contribution!

according the announcement published yesterday. I've been already redirect this pr to develop. and there's a little conflict to solve :)

[PStarH]

The conflict is resolved now I think

[PStarH]

The conflict is resolved now I think