An open-source autonomous AI agent implementation built on xpander.ai, a complete backend for agents, and NVIDIA NIM. This reference architecture demonstrates how to build multi-step autonomous agents with persistent memory, tool orchestration, and state management using a low-abstraction approach.
This implementation provides a foundation for building agents that can:
- Access external APIs (Calendar, Meeting Services)
- Execute complex multi-turn operations (Recording setup, asset retrieval)
- Maintain state across interactions
- Process and synthesize information from various sources
The agent uses a modular design based on:
- LLM: NVIDIA NIM with llama-3.1-nemotron-ultra-253b-v1
- Foundation: xpander.ai SDK for agent orchestration
- Tools: Calendar, Recorder, Status, Email, each implemented as composable functions
- Memory: Persistent thread-based context retention
- Session Management: Dynamic session tracking for multi-step operations
meeting-recorder.mp4
post-meeting.mp4
| Component | Implementation |
|---|---|
| Agent Wrapper | Python class encapsulating LLM interaction and tool execution |
| Tool Registry | Dynamically loads available tools from xpander.ai SDK |
| Session Management | Thread-based state persistence |
| API Integration | Calendar, meeting services and email connectors |
| Token Management | Efficient token tracking and accounting |
| Error Handling | Graceful degradation with error recovery |
This agent is built on xpander.ai, a complete backend for agents that provides several key capabilities:
- Managed Agent Runtime: Deploy and run any AI agent regardless of the framework used
- Tool Orchestration: Comprehensive library of pre-built tools + ability to create custom tools
- State & Memory Management: Persistent context across agent sessions and interactions
- Tracing & Observability: Step-by-step execution analysis including model responses and API calls
- Flexible Model Support: Use any LLM provider with your own keys or xpander.ai managed options
- A2A Communication: Enable multiple agents to collaborate and pass context between each other
As a backend for agents, xpander.ai handles the complex infrastructure needed for production-grade agent deployment, allowing developers to focus on agent logic and behaviors rather than backend concerns.
# Clone the repository
git clone https://github.com/xpander-ai/nvidia-meeting-recorder-agent.git
cd nvidia-meeting-recorder-agent
# Set up environment
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
# Install xpander CLI
npm install -g xpander-cli
# Configure credentials
echo "XPANDER_API_KEY=your_key" > .env
echo "NVIDIA_NIM_API_KEY=your_key" >> .env
echo "XPANDER_AGENT_ID=your_id" >> .envThe core agent implementation is in MeetingAgent class which provides:
class MeetingAgent:
"""Wrapper around an xpander agent backed by NVIDIA NIM."""
MODEL = "nvidia/llama-3.1-nemotron-ultra-253b-v1"
def __init__(self, xpander_key: str, nim_key: str, agent_id: str):
self.client = XpanderClient(api_key=xpander_key)
self.agent: Agent= self.client.agents.get(agent_id=agent_id)
self.agent.select_llm_provider(llm_provider=LLMProvider.NVIDIA_NIM)
self.llm = OpenAI(base_url="https://integrate.api.nvidia.com/v1",
api_key=nim_key)The agent executes a multi-step processing loop that:
- Processes user input
- Manages context (including time)
- Generates LLM responses
- Extracts and executes tool calls
- Updates state and returns results
# Get upcoming meetings
result, thread_id = meeting_agent.run("List my upcoming meetings for the next 3 days")# Start recording a specific meeting
result, thread_id = meeting_agent.run(
"Create a recorder for the onboarding meeting",
thread_id=thread_id
)
# Check recording status
result, thread_id = meeting_agent.run(
"Check the recorder status and give me the asset links",
thread_id=thread_id
)# Email meeting assets with summary
result, thread_id = meeting_agent.run(
"Email the video & transcript to team@example.com with a summary",
thread_id=thread_id
)This implementation leverages key capabilities of xpander.ai as a backend for agents, making it ideal for extension:
- Visual Workbench: Rapidly test and debug agent behavior through the xpander.ai Agent Workbench
- Tool Management: Add custom tools or use pre-built tools from the library
- Framework Flexibility: Switch between different LLM providers without code changes
- State Persistence: Thread management across agent calls handled automatically
- Multi-endpoint Triggering: Expose agents via API, Agent-to-Agent (A2A), or communication platforms
- Custom UI Integration: Wrap agent with web UI, embed in existing applications, or integrate with Slack/Teams
- Observability: Monitor execution paths, performance metrics, and error patterns
- Versioning & Lifecycle: Deploy multiple versions and manage agent lifecycle
- Additional Tools: Extend the agent with new tools for different meeting platforms or data sources
- UI Development: Build custom interfaces on top of the agent API
- Multi-agent Orchestration: Connect multiple specialized agents working together
- Domain Adaptation: Customize the agent for specific industries or use cases
For detailed development instructions and to access the full implementation, refer to the Jupyter notebook.
This project uses the Xpander SDK which has its own licensing terms. Refer to the Xpander documentation for details.