Skip to content

Migrate Authentication from Microsoft Entra ID to Keycloak (Terraform-based) #1

@shoootyou

Description

@shoootyou

Role: You are a DevOps automation engineer specialized in Identity and Access Management with Terraform, Keycloak, and multi-service application deployments.

Task:
Convert the current authentication and authorization setup from Microsoft Entra ID to Keycloak, maintaining all current security and agentic behavior. The new configuration must be fully automated through Terraform (under /terraform) and integrate with the existing multi-service app structure:

  • products-web → Streamlit UI (Frontend)
  • products-agent → FastAPI Agent API
  • products-mcp → Model Context Protocol (MCP) Server

🧩 Current Architecture Summary

  • Each service authenticates via OAuth2 + JWT validation using Microsoft Entra ID.
  • products-agent acts as an OBO (On-Behalf-Of) intermediary, exchanging and validating tokens between the UI and MCP.
  • products-mcp uses HCP Vault to generate dynamic database credentials and validate identity scopes.
  • All configuration and environment variables are generated via Terraform (export-env.sh), and deployments occur via Docker Compose or AWS bastion host.

🎯 Goal

Replace Microsoft Entra ID with Keycloak as the identity provider across all components, keeping the same functional and security model.


🛠️ Required Actions

  1. Terraform Integration

    • Use the Keycloak Terraform provider.
    • Create a new Terraform module under /terraform/keycloak that:
      • Deploys or references a Keycloak realm (e.g., confused-deputy-realm).
      • Creates clients for:
        • products-web (public client, PKCE flow)
        • products-agent (confidential client with client secret)
        • products-mcp (confidential client)
      • Defines realm roles for user groups (e.g., readonly, admin).
      • Adds protocol mappers for group/role claims in JWTs.
      • Configures OBO token exchange between clients using Keycloak’s token_exchange feature.
    • Expose Terraform outputs for each client ID and secret to be used in .env files.
  2. Environment & App Configuration

    • Update each app’s .env files generated by export-env.sh:
      • Replace all MS_ENTRA_* variables with KEYCLOAK_* equivalents:
        • KEYCLOAK_URL, KEYCLOAK_REALM, KEYCLOAK_CLIENT_ID, KEYCLOAK_CLIENT_SECRET
      • Update redirect URIs and audience claims for each service.
    • Adjust JWT validation logic in products-agent and products-mcp:
      • Use Keycloak’s OpenID Connect discovery endpoint for public keys.
      • Replace Entra ID issuer/audience validation with Keycloak’s issuer.
      • Preserve the same OBO flow between products-web → products-agent → products-mcp.
  3. Docker Compose Integration

    • Add a Keycloak service to docker-compose/ for local testing:
      keycloak:
        image: quay.io/keycloak/keycloak:latest
        command: start-dev --import-realm
        ports:
          - "8080:8080"
        environment:
          KEYCLOAK_ADMIN: admin
          KEYCLOAK_ADMIN_PASSWORD: admin
    • Mount the exported realm JSON file generated by Terraform (if applicable).
  4. Documentation Updates

    • Update /README.md and /terraform/README.md:
      • Replace all mentions of Microsoft Entra ID with Keycloak.
      • Include instructions for initializing and applying the new Keycloak configuration:
        cd terraform
        terraform init
        terraform apply -target=module.keycloak
        ./export-env.sh local
      • Note: JWT validation and token delegation now use Keycloak OIDC endpoints.

🧾 Expected Output

  • Complete Terraform configuration for Keycloak under /terraform/keycloak/.
  • Updated .env templates for all apps (products-web, products-agent, products-mcp).
  • Optional docker-compose snippet to run Keycloak locally.
  • Updated documentation reflecting Keycloak-based auth flow.
  • Maintained end-to-end secure OBO token exchange between services.

✅ Closure Conditions

  • All apps can authenticate via Keycloak and communicate securely.
  • Terraform deploys the realm, clients, and roles automatically.
  • Token validation, delegation, and policy enforcement remain consistent with the previous Entra ID setup.
  • No manual steps are required outside Terraform and Docker Compose.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions