Melody Engine

Prompt based instrumental music generation.

✨ Features

Melody Engine is a thin but practical wrapper on MagentaRT, providing:

1. REST API for programmatic music generation
2. Serialized job queue to avoid GPU thrashing
3. Containerized runtime for reproducible deployment
4. Web interface that hides MagentaRT's operational complexity

🧰 Requirements

You would need a machine with a powerful Nvidia GPU, and a linux host with podman or docker installed. The MagentaRT's official repository suggests a GPU with at least 40GB VRAM, but in my experience, a powerful enough GPU like NVIDIA RTX A5000 with 24GB VRAM works just fine, especially for what I have experimented with.

🚀 Getting started

Use the pre-built container image from Docker Hub to run Melody Engine.

1. Pull the latest container image from Docker Hub.

   podman image pull docker.io/zbhavyai/melody-engine:latest

2. Create output and cache directories on the host

   mkdir -p "$(pwd)/outputs" && mkdir -p "$HOME/.cache/melody-engine"

3. Run the container. This will load MagentaRT on your GPU and start a uvicorn server on port 8080.

   podman container run \
      --detach \
      --name melody-engine \
      --restart unless-stopped \
      --publish 8080:8080 \
      --security-opt=label=disable \
      --env TF_GPU_ALLOCATOR=cuda_malloc_async \
      --device=nvidia.com/gpu=all \
      --volume "$(pwd)/outputs:/opt/app/outputs:rw,Z" \
      --volume "$HOME/.cache/melody-engine:/magenta-realtime/cache:rw,Z" \
      docker.io/zbhavyai/melody-engine:latest

Alternatively, build and run container image locally.

1. Build the container image.

   make container-build

2. Run the container. This will load MagentaRT on your GPU and start a uvicorn server on port 8080.

   make container-run

Once the container is running, you can interact with the Melody Engine web interface or REST API.

1. Access the web interface at localhost:8080 or explore the API documentation at localhost:8080/docs.

2. Generated audio files are saved locally in the outputs directory, or can be downloaded from the web interface or REST API.

🎧 Example

Generate a 1-hour spacey electronica track using the REST API.

1. Submit a generation job. The response will include a job ID, which uniquely identifies your generation request.

   curl \
      --request POST \
      --location 'localhost:8080/api/v1/jobs' \
      --header 'Content-Type: application/json' \
      --data '{
         "prompt": "spacey electronica with drifting pads and gentle rhythmic motion",
         "duration_s": 3600,
         "gain_db": 0,
         "format": "mp3"
      }'

2. Poll the status of your job.

   curl \
      --request GET \
      --location 'localhost:8080/api/v1/jobs/{job_id}'

3. Once the job is complete, you may download the generated audio file.

   curl \
      --request GET \
      --location 'localhost:8080/api/v1/jobs/{job_id}/download' \
      --output 'generated_track.mp3'