clewcrew-framework 🧠✨

Core framework for the clewcrew hallucination detection system

The clewcrew-framework provides the foundational components for building robust, scalable hallucination detection workflows. It includes workflow orchestration, state management, event handling, plugin systems, and configuration management.

🚀 Features

• Workflow Engine: Define and execute complex workflows with dependency management
• State Manager: Persistent, versioned state storage with snapshot capabilities
• Event Bus: Asynchronous pub/sub messaging system for workflow communication
• Plugin Manager: Dynamic plugin loading and lifecycle management
• Config Manager: Centralized configuration with multiple source support

📦 Installation

# Install from PyPI
pip install clewcrew-framework

# Or install from source
git clone https://github.com/clewcrew/clewcrew-framework.git
cd clewcrew-framework
pip install -e .

🎯 Quick Start

Basic Workflow Usage

from clewcrew_framework import WorkflowEngine, WorkflowStep, WorkflowContext

# Create workflow engine
engine = WorkflowEngine()

# Define workflow steps
def step1(context, results):
    print("Step 1: Data collection")
    return {"data": "sample_data"}

def step2(context, results):
    print("Step 2: Analysis")
    return {"analysis": f"Processed {results['step1']['data']}"}

# Create workflow steps
steps = [
    WorkflowStep("step1", step1),
    WorkflowStep("step2", step2, dependencies=["step1"])
]

# Register and execute workflow
engine.register_workflow("analysis_workflow", steps)

context = WorkflowContext("workflow_1")
results = await engine.execute_workflow("analysis_workflow", context)

Configuration Management

from clewcrew_framework import ConfigManager

# Initialize config manager
config_manager = ConfigManager(["./config", "./conf"])

# Load configuration from multiple sources
config = config_manager.load_config()

# Access configuration values
db_host = config_manager.get("database.host", "localhost")
api_port = config_manager.get("api.port", 8000)

# Set configuration values
config_manager.set("logging.level", "DEBUG")

Plugin System

from clewcrew_framework import PluginManager, PluginInterface

# Create plugin manager
plugin_manager = PluginManager(["./plugins"])

# Discover and load plugins
plugin_paths = plugin_manager.discover_plugins()
for path in plugin_paths:
    plugin_info = plugin_manager.load_plugin(path)

# Activate plugins
plugin_manager.activate_plugin("my_plugin")

# Get plugin information
active_plugins = plugin_manager.get_active_plugins()

🛠️ CLI Usage

The framework includes a comprehensive command-line interface:

# Workflow management
clewcrew-framework workflow my_workflow

# Plugin management
clewcrew-framework plugin list
clewcrew-framework plugin load ./plugins/my_plugin.py
clewcrew-framework plugin activate my_plugin

# Configuration management
clewcrew-framework config load ./config.yaml
clewcrew-framework config get database.host
clewcrew-framework config set logging.level DEBUG
clewcrew-framework config export config.json

🏗️ Architecture

clewcrew-framework/
├── WorkflowEngine      # Orchestrates workflow execution
├── StateManager        # Manages persistent state
├── EventBus           # Handles async messaging
├── PluginManager      # Manages plugin lifecycle
└── ConfigManager      # Centralized configuration

🔌 Plugin Development

Create custom plugins by implementing the PluginInterface:

from clewcrew_framework import PluginInterface

class MyPlugin(PluginInterface):
    def initialize(self, config):
        self.config = config
    
    def start(self):
        print("Plugin started")
    
    def stop(self):
        print("Plugin stopped")
    
    def get_info(self):
        return {"name": "MyPlugin", "version": "1.0.0"}

# Plugin metadata
__version__ = "1.0.0"
__description__ = "A sample plugin"
__author__ = "Your Name"
__tags__ = ["sample", "demo"]

📚 API Reference

WorkflowEngine

• register_workflow(name, steps): Register a new workflow
• execute_workflow(name, context): Execute a registered workflow
• get_workflow_status(name): Get status of workflow steps

StateManager

• set_state(key, value): Set a state value
• get_state(key, default): Get a state value
• save_snapshot(version, metadata): Save current state as snapshot
• load_snapshot(version): Load a specific snapshot

EventBus

• subscribe(event_name, callback): Subscribe to events
• publish(event): Publish an event asynchronously
• publish_sync(event): Publish and wait for completion

PluginManager

• discover_plugins(): Find available plugins
• load_plugin(path): Load a plugin from path
• activate_plugin(name, config): Activate a loaded plugin
• deactivate_plugin(name): Deactivate an active plugin

ConfigManager

• load_config(path): Load configuration from sources
• get(key, default): Get configuration value
• set(key, value): Set configuration value
• export_config(filepath, format): Export configuration

🧪 Testing

# Run tests
pytest tests/

# Run with coverage
pytest --cov=clewcrew_framework tests/

# Run specific test
pytest tests/test_workflow_engine.py::test_workflow_execution

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔗 Related Projects

• clewcrew-common: Common utilities and data models
• clewcrew-core: Core orchestration engine
• clewcrew-agents: Expert agent implementations
• clewcrew-recovery: Recovery engine implementations

---

Ready to build robust workflows with the clewcrew framework! 🚀✨