Skip to content

refactor: extract database scripts from creator classes to separate files#29

Merged
byerlikaya merged 24 commits intomainfrom
refactor/database-creators-extract-scripts-v2
Jan 19, 2026
Merged

refactor: extract database scripts from creator classes to separate files#29
byerlikaya merged 24 commits intomainfrom
refactor/database-creators-extract-scripts-v2

Conversation

@byerlikaya
Copy link
Owner

🚀 SmartRAG Pull Request

📝 Description

This PR refactors the database setup system by extracting SQL scripts from database creator classes into separate script files. This improves maintainability, follows DRY principles, and makes it easier to manage database initialization scripts.

Key Changes:

  • Extracted SQL scripts from ITestDatabaseCreator implementations into separate script files
  • Created DatabaseSetup/scripts/ directory for database creation scripts
  • Reorganized DatabaseBackups/ directory structure
  • Moved large backup files (>100MB) to DatabaseBackups/LargeFiles/ to comply with GitHub file size limits
  • Updated .gitignore to exclude large backup files while allowing smaller ones (e.g., salesmanagement.backup.bak)

🔗 Related Issue

N/A - Internal refactoring

🔄 Type of Change

Please mark the relevant options:

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🧪 Test improvements
  • 🔧 Code refactoring
  • 🚀 Release preparation

🧪 Testing

Please describe the tests that you ran to verify your changes:

  • Unit tests pass
  • Integration tests pass
  • Manual testing completed
  • SmartRAG.Demo runs without errors
  • Build succeeds with 0 errors, 0 warnings

Testing Details:

  • Verified all database creators can successfully create test databases using extracted scripts
  • Confirmed SQLite, SQL Server, PostgreSQL, and MySQL database creation works correctly
  • Tested database restoration from backup files
  • Validated that large backup files are properly excluded from Git

📋 Checklist

  • My code follows the SmartRAG style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (EN + TR)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • LoggerMessage definitions are correct (no parameter mismatches)
  • EventId assignments are unique (no conflicts)
  • Code is generic and provider-agnostic (no hardcoded domain-specific names)
  • All public APIs have XML documentation

🚨 Critical Rules Compliance

  • Generic Code: No hardcoded table/database/column names
  • Error Fixing: Only fixed the reported error, no refactoring (N/A - this is a refactoring PR)
  • Build Quality: 0 errors, 0 warnings, 0 messages
  • Language: All code elements in English only
  • SOLID/DRY: Principles followed (extracted scripts to eliminate duplication)

📸 Screenshots (if applicable)

N/A

📋 Additional Context

File Structure Changes:

  • DatabaseSetup/scripts/ - Contains database creation SQL scripts for all providers
  • DatabaseBackups/ - Contains backup files for test database restoration
  • DatabaseBackups/LargeFiles/ - Contains large backup files (>100MB) excluded from Git

Script Files Created:

  • DatabaseSetup/scripts/create_sqlite_database.sql
  • DatabaseSetup/scripts/create_sqlserver_database.sql
  • DatabaseSetup/scripts/create_postgresql_database.sql
  • DatabaseSetup/scripts/create_mysql_database.sql

Benefits:

  • Improved maintainability: SQL scripts are now in separate files, easier to edit and review
  • Better organization: Scripts and backups are in dedicated directories
  • DRY compliance: No duplicate SQL code in creator classes
  • GitHub compliance: Large files are properly excluded

🔄 Migration Guide (if breaking changes)

N/A - No breaking changes. This is a refactoring that maintains backward compatibility.

📊 Performance Impact

  • No performance impact
  • Performance improvement
  • Performance regression (explain below)

Note: This refactoring does not change runtime behavior, only improves code organization.

🔒 Security Considerations

  • No security implications
  • Security improvement
  • Security concern (explain below)

Reviewer Guidelines:

  • Check for generic, provider-agnostic code
  • Verify LoggerMessage parameter counts match format strings
  • Ensure EventId assignments are unique
  • Confirm 0 errors, 0 warnings build
  • Validate documentation updates (EN + TR)

…inciple

- Refactor database creator classes to use external SQL script files
- Reduce code duplication by ~1500 lines across all database creators
- Update DatabaseSchemaAnalyzer to support script-based approach
- Improve maintainability and separation of concerns
- Add DatabaseScripts directory with SQL scripts for SQLite, SQL Server, PostgreSQL, MySQL
- Add instnwnd.sql (Northwind sample database) for testing
- Enable script-based database initialization for better maintainability
- Document new DatabaseScripts directory structure
- Update database setup instructions to reflect script-based approach
@gitguardian
Copy link

gitguardian bot commented Jan 13, 2026

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@byerlikaya byerlikaya self-assigned this Jan 19, 2026
…ovements v3.7.0

- Added Cross-Database Mapping Detector for automatic relationship detection
- Improved SQL script extraction and database query generation
- Enhanced security with SQL/command injection prevention
- Prevented sensitive data leakage in logs and error messages
- Updated all changelog files and version numbers to 3.7.0
@byerlikaya byerlikaya merged commit 9ebdd7a into main Jan 19, 2026
11 checks passed
@byerlikaya byerlikaya deleted the refactor/database-creators-extract-scripts-v2 branch January 19, 2026 20:32
byerlikaya added a commit that referenced this pull request Feb 5, 2026
…iles (#29)

* refactor: extract SQL scripts from database creators and apply DRY principle

- Refactor database creator classes to use external SQL script files
- Reduce code duplication by ~1500 lines across all database creators
- Update DatabaseSchemaAnalyzer to support script-based approach
- Improve maintainability and separation of concerns

* feat: add database setup scripts for all supported databases

- Add DatabaseScripts directory with SQL scripts for SQLite, SQL Server, PostgreSQL, MySQL
- Add instnwnd.sql (Northwind sample database) for testing
- Enable script-based database initialization for better maintainability

* docs: update README with database script extraction changes

- Document new DatabaseScripts directory structure
- Update database setup instructions to reflect script-based approach

* refactor: reorganize database scripts into DatabaseSetup/scripts and DatabaseBackups directories

* refactor: extract database creation scripts from creator classes to separate script files

* refactor: update database services to use extracted script files and improve schema handling

* refactor: update demo and API projects to use new database setup structure

* chore: update configuration files and gitignore for new database structure

* chore: exclude large backup files from git to comply with GitHub file size limits

* chore: move large backup file to LargeFiles directory and allow salesmanagement.backup.bak

* chore: add SMARTRAG_TEST_SORULARI.md to gitignore

* feat: add cross-database mapping detector

* refactor: improve database query generation and validation logic

* docs: update changelog

* refactor: update database parser and document search services

* chore: update database setup scripts

* fix: prevent SQL injection in database creator classes

* fix: prevent command injection in database creator classes

* fix: remove shell commands to prevent command injection

* fix: prevent sensitive data leakage in logs and database handlers

* fix: remove backup file path from exception messages

* fix: remove tainted values from error log messages

* [release] feat: add cross-database mapping detector and security improvements v3.7.0

- Added Cross-Database Mapping Detector for automatic relationship detection
- Improved SQL script extraction and database query generation
- Enhanced security with SQL/command injection prevention
- Prevented sensitive data leakage in logs and error messages
- Updated all changelog files and version numbers to 3.7.0

* docs: add v3.7.0 to version history accordion in docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant