Initial MCP integration

docs/1.getting-started/4.package.md

@@ -21,6 +21,7 @@ The structure of the resulting package is the following:
 | :file_folder: `hasura`        | Arbitrary Hasura metadata to apply during configuration                                                                              |
 | :file_folder: `hooks`         | User-defined callbacks to run manually or by schedule                                                                                |
 | :file_folder: `models`        | DipDup ORM models to store data in the database                                                                                      |
+| :file_folder: `mcp`           | Custom MCP rools and resources                                                                                                       |
 | :file_folder: `sql`           | SQL scripts and queries to run manually or on specific events                                                                        |
 | :file_folder: `types`         | Automatically generated Pydantic dataclasses for contract types                                                                      |
 | `dipdup.yaml`                 | Root DipDup config; can be expanded with env-specific files                                                                          |

docs/5.advanced/7.mcp.md

@@ -0,0 +1,203 @@
+---
+title: "MCP"
+description: "MCP is an open protocol that standardizes how applications provide context to LLMs. This document explains how to integrate DipDup with MCP clients and implement custom tools and resources."
+---
+
+# MCP integration
+
+::banner{type="warning"}
+**You are early!**
+
+MCP is a pretty recent technology, so many tools might not work as expected. Consult the [Known issues](#known-issues) section for more information and open the ticket in [GitHub issues](https://github.com/dipdup-io/dipdup/issues) if you encounter any problems.
+::
+
+## Introduction
+
+The **Model Context Protocol** (MCP) is an open protocol that enables seamless integration between LLM applications and external data sources and tools. Whether you're building an AI-powered IDE, enhancing a chat interface, or creating custom AI workflows, MCP provides a standardized way to connect LLMs with the context they need.
+
+There are three types of MCP primitives (citing from the docs):
+
+- **Resources** are **application-controlled** and allow to expose data and content that can be read by clients and used as context for LLM interactions.
+- **Tools** are **model-controlled** and enable servers to expose executable functionality to clients, such as interacting with external systems, performing computations, and taking actions in the real world.
+- **Prompts** are **user-controlled** and enable servers to define reusable prompt templates and workflows that clients can easily surface to users and LLMs
+
+**DipDup** provides an MCP server with several built-in primitives to help LLMs understand the context of your project and the current state of the indexer. You can also implement your own tools and resources specific to your project.
+
+## Running MCP server
+
+DipDup MCP server runs in a separate process via the SSE transport. To start it, run the following command:
+
+```shell
+dipdup mcp run
+```
+
+Make sure that you use the same database connection settings as for the indexer.
+
+By default, the server listens on `http://127.0.0.1:9999`. You can change the settings in the `mcp` section of the configuration file.
+
+```yaml [dipdup.yaml]
+mcp:
+  host: 127.0.0.1
+  port: 9999
+```
+
+## Connecting clients
+
+### Cursor
+
+To configure MCP servers go to `File -> Preferences -> Cursor Settings -> MCP`.
+
+Add the following configuration to `~/.cursor/mcp.json`:
+
+```json [~/.cursor/mcp.json]
+{
+  "mcpServers": {
+    "local": {
+      "url": "http://127.0.0.1:9999/sse"
+    }
+  }
+}
+```
+
+If your server is running, but Cursor doesn't connect, try "Reload Window" in the Command Palette.
+
+### VSCode (Copilot)
+
+There is no official support for MCP in VSCode yet.
+
+There are two extensions available in repository to add MCP capabilities to **Copilot** assistant. None of them worked out-of-the-box at the time of writing this document.
+
+- [Copilot MCP](https://marketplace.visualstudio.com/items?itemName=AutomataLabs.copilot-mcp) (issue [#20](https://github.com/VikashLoomba/copilot-mcp/issues/20))
+- [MCP-Client](https://marketplace.visualstudio.com/items?itemName=m1self.mcp-client)
+
+### Claude Desktop
+
+[Claude Desktop](https://claude.ai/download) currently only supports stdio-based MCP servers. You can use [supercorp-ai/supergateway](https://github.com/supercorp-ai/supergateway) tool to connect DipDup MCP server to Claude Desktop:
+
+```shell [Terminal]
+npx -y supergateway --sse http://127.0.0.1:9999
+```
+
+There is also [lightconetech/mcp-gateway](https://github.com/lightconetech/mcp-gateway) tool available for the same purpose.
+
+## Implementing MCP primitives
+
+To make your server more useful, you can implement custom MCP primitives specific to your project. For example, let's implement a tool that provides some information about token holders.
+
+If you used DipDup prior to 8.3 release, run `dipdup init` command to update the project structure.
+
+Create a new file `tools.py` in the `mcp` project directory. In this example, we define a new tool `TopHolders` that returns the top 10 token holders sorted by their balance.
+
+```python [mcp/tools.py]
+from dipdup import mcp
+
+from demo_evm_events import models
+
+@mcp.tool('TopHolders', 'Get top token holders')
+async def tool_holders() -> str:
+    holders = await models.Holder.filter().order_by('-balance').limit(10).all()
+    res = 'Top holders:\n\n'
+    for holder in holders:
+        res += f'- {holder.address}: {holder.balance}\n'
+    return res
+```
+
+### Using application context
+
+You can use the application context the same way as in handlers and hooks. Use the `mcp.get_ctx()` function to get the context object.
+
+```python
+from dipdup import mcp
+from dipdup.context import McpContext
+
+@mcp.tool(...)
+async def tool():
+    ctx: McpContext = mcp.get_ctx()
+
+    ctx.logger.info('Hello from MCP tool!')
+```
+
+### Calling other callbacks
+
+You can use the `dipdup.mcp` functions to read other resources or call tools from your callbacks.
+
+```python
+from dipdup import mcp
+
+@mcp.tool(...)
+async def tool():
+    result = await mcp.call_tool('my_tool', {})
+```
+
+For a low-level access you can use `dipdup.mcp.server` singleton to interact with the running server.
+
+### Interacting with running indexer
+
+DipDup provides [management API](../7.references/4.api.md) to interact with the running indexer. For example you can use it to add indexes in runtime. First, add running indexer as a HTTP datasource:
+
+```yaml [dipdup.yaml]
+datasources:
+  indexer:
+    kind: http
+    # NOTE: Default for Compose stack
+    url: http://api:46339
+```
+
+Then, call this datasource from your MCP tool:
+
+```python
+from dipdup import mcp
+
+@mcp.tool(...)
+async def tool():
+    ctx = mcp.get_ctx()
+    datasource = ctx.get_http_datasource('indexer')
+    response = await datasource.post(
+        '/add_index',
+        params={
+            'name': 'my_index',
+            'template': 'my_template',
+            'values': {'param': 'value'},
+            'first_level': 0,
+            'last_level': 1000,
+        }
+    )
+```
+
+## Running in Docker
+
+You can find a Compose file in `deploy/` project directory that runs both the indexer and the MCP server in Docker containers. To use this manifest, use the following command:
+
+```shell [Terminal]
+make up COMPOSE=deploy/compose.mcp.yaml
+```
+
+### Debugging and troubleshooting
+
+To check if your tool is working correctly before using it in the client, you can use the [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector) app:
+
+1. Run `npx @modelcontextprotocol/inspector`
+2. Open `http://127.0.0.1:5173/` in your browser
+3. Choose SSE transport and connect to `http://127.0.0.1:9999/sse`.
+
+You can also enable full logging to get some insights into the MCP server:
+
+```yaml [dipdup.yaml]
+logging:
+  '': DEBUG
+```
+
+## Known issues
+
+- **DipDup** doesn't support custom prompts at the moment.
+- **Cursor** fails to discover resources exposed by DipDup server. Tools work fine. (0.47.8 tested)
+- **VSCode** community extensions for MCP don't work out-of-the-box.
+- **Claude Desktop** doesn't support SSE-based MCP servers; gateway tools are required.
+
+## Further reading
+
+- [For Server Developers - Model Context Protocol](https://modelcontextprotocol.io/quickstart/server)
+- [Example Servers - Model Context Protocol](https://modelcontextprotocol.io/examples)
+- [qpd-v/mcp-guide](https://github.com/qpd-v/mcp-guide) -  a tutorial server that helps users understand MCP concepts, provides interactive examples, and demonstrates best practices for building MCP integrations. Contains [awesome-list](https://raw.githubusercontent.com/qpd-v/mcp-guide/refs/heads/main/src/servers/serverlist.txt) of related projects.
+- [anjor/coinmarket-mcp-server](https://github.com/anjor/coinmarket-mcp-server) - example MCP server for CoinMarketCap API.
+- [ample-education/cursor-resources](https://github.com/ample-education/cursor-resources) - a collection of resources to maximize productivity with Cursor: prompts, rules, MCP servers etc.

docs/7.references/1.cli.md

@@ -203,6 +203,24 @@ Discord:               <a class="reference external" href="https://discord.gg/aG
 </section>
 
 
+## mcp
+
+<p>Commands related to MCP integration.</p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>dipdup<span class="w"> </span>mcp<span class="w"> </span><span class="o">[</span>OPTIONS<span class="o">]</span><span class="w"> </span>COMMAND<span class="w"> </span><span class="o">[</span>ARGS<span class="o">]</span>...
+</pre></div>
+</div>
+
+
+### run
+
+<p>Run MCP server.</p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>dipdup<span class="w"> </span>mcp<span class="w"> </span>run<span class="w"> </span><span class="o">[</span>OPTIONS<span class="o">]</span>
+</pre></div>
+</div>
+</section>
+</section>
+
+
 ## migrate
 
 <p>Migrate project to the new spec version.</p>

docs/7.references/2.config.md

@@ -10,7 +10,7 @@ description: "Config file reference"
 
 ## dipdup.config.DipDupConfig
 
-<em class="property"><span class="k"><span class="pre">class</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">dipdup.config.</span></span><span class="sig-name descname"><span class="pre">DipDupConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">*args</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">spec_version</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">package</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">datasources=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">database=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">runtimes=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">contracts=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">indexes=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">templates=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">jobs=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hooks=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hasura=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sentry=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">prometheus=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">api=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">advanced=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">custom=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">logging='INFO'</span></span></em><span class="sig-paren">)</span></dt>
+<em class="property"><span class="k"><span class="pre">class</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">dipdup.config.</span></span><span class="sig-name descname"><span class="pre">DipDupConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">*args</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">spec_version</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">package</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">datasources=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">database=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">runtimes=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">contracts=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">indexes=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">templates=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">jobs=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hooks=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hasura=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sentry=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">prometheus=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">api=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">advanced=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">custom=&lt;factory&gt;</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">logging='INFO'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">mcp=None</span></span></em><span class="sig-paren">)</span></dt>
 <dd><p>DipDup project configuration file</p>
 <dl class="field-list simple">
 <dt class="field-odd" style="color: var(--txt-primary);">Parameters<span class="colon">:</span></dt>
@@ -32,6 +32,7 @@ description: "Config file reference"
 <li><p><strong>advanced</strong> (<a class="reference internal" href="#dipdupconfigadvancedconfig" title="dipdup.config.AdvancedConfig" target="_self"><em>AdvancedConfig</em></a>) – Advanced config</p></li>
 <li><p><strong>custom</strong> (<em>dict</em><em>[</em><em>str</em><em>, </em><em>Any</em><em>]</em>) – User-defined configuration to use in callbacks</p></li>
 <li><p><strong>logging</strong> (<em>dict</em><em>[</em><em>str</em><em>, </em><em>str</em><em> | </em><em>int</em><em>] </em><em>| </em><em>str</em><em> | </em><em>int</em>) – Modify logging verbosity</p></li>
+<li><p><strong>mcp</strong> (<em>McpConfig</em><em> | </em><em>None</em>) – MCP server config</p></li>
 
 </ul>
 </dd>