[SECRETS-CONFIG] [ENHANCEMENT] Template Variable Substitution

[chris-schra]

Product Requirements Document: Template Variable Substitution for MCP Funnel

Executive Summary

This PRD outlines the implementation of a template variable substitution system for MCP Funnel configurations. This feature will enable users to reference secrets and environment variables using template syntax (e.g., ${secret:GITHUB_TOKEN}) throughout their configuration files, eliminating the need to hardcode sensitive values and improving configuration portability.

Problem Statement

Current Limitations

1. No Variable Substitution: Currently, MCP Funnel's secretProviders system only makes environment variables available to spawned processes. There's no way to reference these secrets in other configuration fields.

2. Hardcoded Secrets: Users must hardcode sensitive values directly in configuration files or use workarounds like Docker's --env-file flag.

3. Limited HTTP Authentication Support: Remote MCP servers requiring authentication headers cannot securely reference tokens from secret providers.

4. Configuration Portability: Configurations cannot be shared between team members or environments without manual token replacement.

Evidence from Codebase

• No template substitution logic exists in packages/mcp/src/config.ts
• Server spawning in packages/mcp/src/index.ts:831-835 passes configuration values as-is
• Test file packages/mcp/src/registry/config-generator.test.ts:line-with-${API_TOKEN} shows placeholder syntax but no actual implementation
• Config generator documentation mentions "Replace placeholder values" as a manual step (packages/mcp/src/registry/config-generator.ts)

User Stories

Story 1: GitHub API Authentication

As a developer using GitHub's hosted MCP server
I want to reference my GitHub token from a .env file in the authorization header
So that I don't have to hardcode my token in the configuration

{
  "servers": {
    "github": {
      "url": "https://api.githubcopilot.com/mcp/",
      "headers": {
        "Authorization": "Bearer ${secret:GITHUB_PERSONAL_ACCESS_TOKEN}"
      }
    }
  },
  "secretProviders": [
    { "type": "dotenv", "config": { "path": ".env" } }
  ]
}

Story 2: Dynamic Docker Environment Variables

As a developer using Docker-based MCP servers
I want to dynamically pass environment variables to Docker containers
So that I can use MCP Funnel's secret management instead of Docker's --env-file

{
  "servers": {
    "custom-server": {
      "command": "docker",
      "args": [
        "run",
        "-e",
        "API_KEY=${secret:API_KEY}",
        "-e",
        "DB_URL=${secret:DATABASE_URL}",
        "-i",
        "--rm",
        "my-mcp-server"
      ],
      "secretProviders": [
        { "type": "dotenv", "config": { "path": ".env.production" } }
      ]
    }
  }
}

Story 3: Environment-Specific Configuration

As a DevOps engineer
I want to use environment variables to configure server endpoints
So that the same configuration works across dev, staging, and production

{
  "servers": {
    "api": {
      "url": "${env:API_ENDPOINT:-https://api.example.com}/mcp",
      "headers": {
        "X-Environment": "${env:ENVIRONMENT:-development}"
      }
    }
  }
}

Story 4: File-Based Secrets

As a security-conscious developer
I want to read secrets from files (e.g., Kubernetes mounted secrets)
So that I can integrate with existing secret management systems

{
  "servers": {
    "secure-server": {
      "command": "node",
      "args": ["server.js"],
      "env": {
        "LICENSE_KEY": "${file:/var/secrets/license.txt}",
        "CERT_CONTENT": "${file:/etc/ssl/cert.pem}"
      }
    }
  }
}

Proposed Solution

Template Syntax

${source:key}           - Basic substitution
${source:key:-default}  - With default value
${source:key:?error}   - Error if not found

Shell-style references like $VAR are parsed as sugar for ${env:VAR}, keeping the canonical ${source:key} syntax while supporting existing configs.

Supported Sources

1. secret: - Values from resolved secret providers

   • Example: ${secret:API_KEY}
   • Resolution: Looks up in secrets resolved via secretProviders

2. env: - Environment variables from current process

   • Example: ${env:HOME}
   • Resolution: Reads from process.env

3. file: - Contents from files

   • Example: ${file:/path/to/secret}
   • Resolution: Reads file contents, trims whitespace

4. config: - Cross-references within configuration

   • Example: ${config:servers.github.url}
   • Resolution: JSON path lookup within current config

Implementation Architecture

1. Template Resolution Phase

Create new module: packages/mcp/src/template-resolver.ts

export interface TemplateContext {
  secrets: Record<string, string>;  // From secretProviders
  env: Record<string, string>;      // From process.env
  config: ProxyConfig;              // Current configuration
}

export class TemplateResolver {
  constructor(private context: TemplateContext) {}

  async resolve(value: string): Promise<string> {
    // Regex: /\$\{([^:}]+):([^}]+)\}/g
    // Parse source and key
    // Apply resolution based on source type
    // Handle defaults and errors
  }

  async resolveObject(obj: any): Promise<any> {
    // Recursively resolve all string values in object
  }
}

2. Integration Points

Configuration Loading (packages/mcp/src/config-loader.ts):

// After loading config
const secrets = await resolveSecretsFromConfig(config.defaultSecretProviders);
const resolver = new TemplateResolver({ secrets, env: process.env, config });
const resolvedConfig = await resolver.resolveObject(config);

Server Connection (packages/mcp/src/index.ts:761-810):

private async resolveServerEnvironment(targetServer: TargetServer) {
  // ... existing secret resolution ...

  // NEW: Apply template resolution to server config
  const resolver = new TemplateResolver({
    secrets: finalEnv,
    env: process.env,
    config: this._config
  });

  targetServer = await resolver.resolveObject(targetServer);
  return finalEnv;
}

3. Security Considerations

• No Logging: Never log resolved template values
• Validation: Validate file paths to prevent directory traversal
• Permissions: Check file read permissions before accessing
• Local files only: file: source resolves paths on the local filesystem; remote URLs are out of scope for the MVP
• Size limits: Reject files larger than the configurable maximum (default 1 MB) to mitigate DoS vectors
• Escaping: Properly escape resolved values for shell commands

Architecture & Type Contracts

Define explicit interfaces so additional sources slot in without refactoring core logic:

export interface TemplateSourceResolver {
  readonly source: TemplateSourceType;
  resolve(token: TemplateToken, ctx: TemplateResolutionContext): Promise<string>;
  canResolve(token: TemplateToken): boolean;
}

export interface TemplateResolutionContext {
  readonly secrets: Record<string, string>;
  readonly env: NodeJS.ProcessEnv;
  readonly config: Record<string, unknown>;
  readonly resolver: TemplateResolver;
}

export interface TemplateToken {
  raw: string;
  source: TemplateSourceType;
  key: string;
  defaultValue?: string;
  errorMessage?: string;
}

export type TemplateSourceType = 'secret' | 'env' | 'file' | 'config';

TemplateResolver maintains a registry of TemplateSourceResolver instances. New sources register through a small factory so future additions (e.g., vault:) are additive.

Configuration Schema Updates

Update packages/mcp/src/config.ts:

// Add template validation to string fields
const TemplateStringSchema = z.string().refine(
  (val) => {
    // Validate template syntax if present
    const templateRegex = /\$\{([^:}]+):([^}]+)\}/g;
    // Check for valid sources: secret, env, file, config
    return true; // Actual validation logic
  },
  { message: "Invalid template syntax" }
);

// Update schemas to use TemplateStringSchema where appropriate
export const TargetServerSchema = z.object({
  name: z.string(),
  command: TemplateStringSchema,
  args: z.array(TemplateStringSchema).optional(),
  // ... etc
});

Testing Requirements

Unit Tests

Location: packages/mcp/src/template-resolver.test.ts

1. Basic substitution: ${secret:KEY} → resolved value
2. Default values: ${env:MISSING:-default} → "default"
3. Error handling: ${secret:MISSING:?Required} → throws error
4. Nested templates: ${secret:${env:KEY_NAME}} → resolved
5. Escaping: \${literal} → "${literal}"
6. File reading: Mock file system for file: source tests
7. Shell alias: $MY_VAR resolves identically to ${env:MY_VAR}

Integration Tests

Location: packages/mcp/src/secrets/template-integration.test.ts

1. End-to-end flow: Config with templates → resolved server spawn
2. Docker args: Verify -e KEY=value correctly substituted
3. HTTP headers: Verify Authorization header substitution
4. Multiple providers: Test precedence with multiple secret sources
5. Server env merge: Ensure template expansion composes with resolved passthrough env (e.g., Windows PATH)
6. Error scenarios: Missing required values, invalid syntax

Security Tests

1. Path traversal: ${file:../../etc/passwd} → blocked
2. Command injection: ${env:USER;rm -rf /} → safely escaped
3. Circular references: ${config:a} → ${config:b} → ${config:a} → detected
4. File size guard: Files larger than limit trigger validation error

Implementation Plan

1. Core resolver
   • Implement TemplateResolver with token parsing, default/error handling, and registry of TemplateSourceResolvers.
   • Add concrete resolvers for secret, env, file, and config sources; include shell alias handling in the env resolver.
2. Configuration wiring
   • Update config-loader to run the resolver across loaded configs before normalization.
   • Extend schema types (TemplateStringSchema) so template syntax validation happens during parse.
3. Runtime integration
   • Apply template resolution to server spawn paths (resolveServerEnvironment, command args, headers).
   • Ensure resolved values merge cleanly with filtered passthrough env on all platforms.
4. File access safeguards
   • Enforce local path resolution, size limits, and permission checks in the file resolver.
5. Testing
   • Unit tests for resolver/token parsing, default/error branches, shell alias, and each source.
   • Integration tests covering Docker args, HTTP headers, env merge (including Windows PATH), and failure scenarios.
   • Security tests for traversal, command injection, circular references, and oversize files.
6. Documentation
   • Update README and example configs to show template usage.
   • Add notes on security best practices and file-source constraints.

Alternative Solutions Considered

1. Environment Variable Expansion Only

• Pros: Simpler, follows shell conventions
• Cons: Limited to env vars, no secret provider integration
• Decision: Rejected - doesn't leverage secretProviders system

2. JavaScript Expression Evaluation

• Pros: More flexible, supports complex logic
• Cons: Security risks, complexity, performance overhead
• Decision: Rejected - too complex and risky

3. External Preprocessing Tool

• Pros: Separation of concerns, language agnostic
• Cons: Additional dependency, complex deployment
• Decision: Rejected - poor developer experience

Resolved Questions

• Async resolution scope: file: remains local-only; remote URLs are explicitly out of scope for the MVP to avoid network exfiltration paths.
• Caching policy: Skip resolver caching for now—configuration payloads are small, and eager caching complicates invalidation without clear benefit.
• Syntax sugar: Support bash-style $VAR as a direct alias for ${env:VAR} while keeping ${source:key} as the canonical form.
• Validation timing: Perform syntax validation during config load; runtime errors surface only when referenced values are actually missing.

Appendix

A. Current Code References

Files requiring modification:

• packages/mcp/src/config.ts - Schema updates
• packages/mcp/src/config-loader.ts - Template resolution integration
• packages/mcp/src/index.ts:761-810 - Server environment resolution
• packages/mcp/src/index.ts:831-835 - Server spawning

New files to create:

• packages/mcp/src/template-resolver.ts - Core resolution logic
• packages/mcp/src/template-resolver.test.ts - Unit tests
• packages/mcp/src/secrets/template-integration.test.ts - Integration tests

B. Example Configurations

Before (Current)

{
  "servers": {
    "github": {
      "command": "docker",
      "args": ["run", "--env-file", ".env", "-i", "--rm", "ghcr.io/github/github-mcp-server"]
    }
  }
}

After (With Templates)

{
  "servers": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_TOKEN": "${secret:GITHUB_PERSONAL_ACCESS_TOKEN}"
      }
    }
  },
  "secretProviders": [
    { "type": "dotenv", "config": { "path": ".env" } }
  ]
}

C. Compatibility Matrix

Feature	Current	With Templates	Breaking Change
Hardcoded values	✅	✅	No
Environment variables	Via spawn env	${env:VAR}	No
Secret providers	Process env only	${secret:KEY}	No
Docker --env-file	Required	Optional	No
HTTP headers	Hardcoded only	${secret:TOKEN}	No
File contents	Not supported	${file:path}	No