Skip to content

Commit 929da5e

Browse files
devnendevnen
authored
Colab instructions
1 parent 500a946 commit 929da5e

File tree

1 file changed

+296
-0
lines changed

1 file changed

+296
-0
lines changed

‎README_Colab.md‎

Lines changed: 296 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,296 @@
1+
# README_Colab.md — Run Chatterbox TTS Server on Google Colab (T4 GPU)
2+
3+
This guide shows how to run **Chatterbox-TTS-Server** in a fresh Google Colab notebook with a T4 GPU, using an isolated micromamba environment to avoid Colab package conflicts.
4+
You will open the Web UI via Colab’s built-in port proxy: Colab displays a `https://localhost:PORT/` link that actually points to an externally reachable `*.colab.*` URL. [web:146]
5+
6+
---
7+
8+
## What you will get
9+
10+
After completing Cells **1 → 4**:
11+
- The Web UI opens in your browser from the Colab proxy link. [web:146]
12+
- The Turbo model downloads on first run and loads on GPU.
13+
- The server status endpoint reports the model is loaded (Cell 4 prints `/api/model-info`). [file:21]
14+
15+
---
16+
17+
## Important rules (read first)
18+
19+
- **Do not use “Run all.”** Cell 4 runs the server in the foreground and will keep running while the server is up, so “Run all” will either hang or behave unexpectedly.
20+
- Run cells **one-by-one**, waiting for each cell to finish before running the next cell.
21+
- Keep the notebook tab open while using the Web UI; if the runtime disconnects, the server stops.
22+
23+
---
24+
25+
## First-time setup (Cells 1 → 4)
26+
27+
### Cell 1 — Create isolated Python 3.11 environment (micromamba)
28+
29+
```
30+
%%bash
31+
set -e
32+
33+
cd /content
34+
35+
# Download micromamba into /content/bin/micromamba
36+
curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
37+
38+
# Create a clean env (isolated from Colab’s global packages)
39+
./bin/micromamba create -y -n cb311 -c conda-forge python=3.11 pip
40+
41+
echo "✅ micromamba ready at /content/bin/micromamba"
42+
echo "✅ env created: cb311"
43+
```
44+
45+
---
46+
47+
### Cell 2 — Install PyTorch (CUDA 12.1) + ONNX + Chatterbox fork
48+
49+
Notes:
50+
- This uses **absolute paths** (`/content/bin/micromamba`) so it works regardless of current directory.
51+
- It force-reinstalls the package to avoid stale cached installs.
52+
53+
```
54+
%%bash
55+
set -e
56+
57+
cd /content
58+
MICROMAMBA="/content/bin/micromamba"
59+
60+
# Sanity check
61+
ls -lah "$MICROMAMBA"
62+
63+
# Upgrade pip tooling inside the env
64+
"$MICROMAMBA" run -n cb311 python -m pip install -U pip setuptools wheel
65+
66+
echo "📦 Installing PyTorch 2.5.1 (CUDA 12.1)..."
67+
"$MICROMAMBA" run -n cb311 pip install -q \
68+
torch==2.5.1+cu121 torchaudio==2.5.1+cu121 torchvision==0.20.1+cu121 \
69+
--index-url https://download.pytorch.org/whl/cu121
70+
71+
echo "📦 Installing ONNX (prebuilt wheel)..."
72+
"$MICROMAMBA" run -n cb311 pip install -q onnx==1.16.0
73+
74+
echo "🎙️ Installing Chatterbox package (from GitHub)..."
75+
"$MICROMAMBA" run -n cb311 pip uninstall -y chatterbox-tts chatterbox || true
76+
"$MICROMAMBA" run -n cb311 pip install --no-cache-dir --upgrade -q \
77+
"chatterbox-tts @ git+https://github.com/devnen/chatterbox-v2.git@master"
78+
79+
echo "✅ Installation complete!"
80+
```
81+
82+
---
83+
84+
### Cell 3 — Verify GPU + verify Turbo won’t require Hugging Face tokens
85+
86+
This checks CUDA visibility and prints the installed `from_pretrained()` source so you can confirm it does **not** force Hugging Face auth via a `token=True` fallback (in `huggingface_hub`, `token=True` means “read token from local config,” and errors if none exists). [web:65]
87+
88+
```
89+
%%bash
90+
set -e
91+
92+
/content/bin/micromamba run -n cb311 python - <<'PY'
93+
import inspect, torch
94+
import chatterbox.tts_turbo as t
95+
96+
print("✅ torch:", torch.__version__)
97+
print("✅ cuda available:", torch.cuda.is_available())
98+
if torch.cuda.is_available():
99+
print("✅ gpu:", torch.cuda.get_device_name(0))
100+
101+
print("✅ chatterbox.tts_turbo path:", t.__file__)
102+
103+
src = inspect.getsource(t.ChatterboxTurboTTS.from_pretrained)
104+
print("\n--- from_pretrained() (first ~80 lines) ---")
105+
print("\n".join(src.splitlines()[:80]))
106+
107+
# Heuristic check for the common buggy pattern that forces token=True semantics
108+
markers = [" or True", "token=True", "token = True", "use_auth_token=True"]
109+
hits = [m for m in markers if m in src]
110+
print("\nHeuristic auth-forcing markers found:", hits)
111+
112+
if hits:
113+
raise SystemExit(
114+
"\n❌ This install still appears to force HF auth.\n"
115+
"Re-run Cell 2 (it already uses --no-cache-dir --upgrade).\n"
116+
)
117+
118+
print("\n✅ Looks good: Turbo should download without requiring user tokens.")
119+
PY
120+
```
121+
122+
---
123+
124+
### Cell 4 — Clone server + run with full live logs (recommended)
125+
126+
What this cell does:
127+
- Clones the server repo.
128+
- Installs server dependencies inside `cb311`.
129+
- Runs `server.py` in the **foreground** and prints all logs live.
130+
- Writes a full log file to: `/content/chatterbox_server_stdout.log`
131+
- Prints a Colab proxy link when port 8004 is reachable; Colab will show it as `https://localhost:8004/` but it resolves to a `*.colab.*` URL that opens in a new tab. [web:146]
132+
- Queries `/api/model-info` to confirm the model is loaded. [file:21]
133+
134+
```
135+
# @title 4. Install Server + Run With Full Live Logs (foreground)
136+
import os, time, subprocess, socket, requests
137+
from pathlib import Path
138+
139+
PORT = 8004
140+
REPO_DIR = "/content/Chatterbox-TTS-Server"
141+
LOG_STDOUT = "/content/chatterbox_server_stdout.log"
142+
143+
def sh(cmd, check=False):
144+
return subprocess.run(["bash", "-lc", cmd], check=check)
145+
146+
def port_open(host="127.0.0.1", port=PORT, timeout=0.25):
147+
try:
148+
with socket.create_connection((host, port), timeout=timeout):
149+
return True
150+
except OSError:
151+
return False
152+
153+
os.chdir("/content")
154+
155+
# Fresh clone
156+
sh("rm -rf /content/Chatterbox-TTS-Server", check=False)
157+
sh("git clone https://github.com/devnen/Chatterbox-TTS-Server.git", check=True)
158+
os.chdir(REPO_DIR)
159+
160+
print("=== Quick system checks ===")
161+
sh("nvidia-smi || true", check=False)
162+
163+
print("\n=== Installing server requirements (prefer repo pins if present) ===")
164+
if Path("requirements-nvidia.txt").exists():
165+
sh("/content/bin/micromamba run -n cb311 pip install -U pip setuptools wheel", check=False)
166+
sh("/content/bin/micromamba run -n cb311 pip install -r requirements-nvidia.txt", check=False)
167+
else:
168+
sh(
169+
"/content/bin/micromamba run -n cb311 pip install -U pip setuptools wheel && "
170+
"/content/bin/micromamba run -n cb311 pip install "
171+
"fastapi 'uvicorn[standard]' pyyaml soundfile librosa safetensors "
172+
"python-multipart requests jinja2 watchdog aiofiles unidecode inflect tqdm "
173+
"pydub audiotsm praat-parselmouth",
174+
check=False
175+
)
176+
177+
print("\n=== Removing old stdout log ===")
178+
Path(LOG_STDOUT).unlink(missing_ok=True)
179+
180+
print("\n=== Starting server with LIVE logs ===")
181+
print("Log file:", LOG_STDOUT)
182+
print("To stop the server, run Cell 5.\n")
183+
184+
env = os.environ.copy()
185+
env["PYTHONUNBUFFERED"] = "1"
186+
187+
# Put HF cache somewhere inspectable/persistent for this runtime
188+
env["HF_HOME"] = "/content/hf_home"
189+
env["TRANSFORMERS_CACHE"] = "/content/hf_home/transformers"
190+
env["HF_HUB_CACHE"] = "/content/hf_home/hub"
191+
Path(env["HF_HOME"]).mkdir(parents=True, exist_ok=True)
192+
193+
proc = subprocess.Popen(
194+
["/content/bin/micromamba", "run", "-n", "cb311", "python", "-u", "server.py"],
195+
stdout=subprocess.PIPE,
196+
stderr=subprocess.STDOUT,
197+
text=True,
198+
bufsize=1,
199+
env=env,
200+
)
201+
202+
with open(LOG_STDOUT, "w", encoding="utf-8", errors="replace") as f:
203+
shown_link = False
204+
while True:
205+
line = proc.stdout.readline()
206+
if line:
207+
print(line, end="")
208+
f.write(line)
209+
f.flush()
210+
211+
if (not shown_link) and port_open():
212+
shown_link = True
213+
print("\n=== Server port is reachable ===")
214+
print("Click the Colab proxy link below to open the Web UI.")
215+
from google.colab.output import serve_kernel_port_as_window
216+
serve_kernel_port_as_window(PORT)
217+
218+
# Verify model load status via server endpoint
219+
try:
220+
mi = requests.get(f"http://127.0.0.1:{PORT}/api/model-info", timeout=2).json()
221+
print("\n/api/model-info:", mi)
222+
except Exception as e:
223+
print("\n/api/model-info query failed:", repr(e))
224+
225+
if proc.poll() is not None:
226+
print("\n=== Server process exited with code", proc.returncode, "===")
227+
break
228+
```
229+
230+
**First run note:** model downloads can take a while; watch the progress output in Cell 4.
231+
232+
---
233+
234+
## Stopping / restarting (Cell 5)
235+
236+
### Cell 5 — Stop the server (free port 8004)
237+
238+
Run this any time you want to stop the server process and free the port.
239+
240+
```
241+
%%bash
242+
PORT=8004
243+
244+
echo "PIDs listening on port $PORT:"
245+
sudo lsof -t -i:$PORT || true
246+
247+
echo "Killing..."
248+
sudo kill -9 $(sudo lsof -t -i:$PORT) 2>/dev/null || true
249+
250+
echo "Verify nothing is listening:"
251+
sudo lsof -i:$PORT || true
252+
```
253+
254+
---
255+
256+
## What to run when…
257+
258+
### Start it the first time
259+
Run: **Cell 1 → Cell 2 → Cell 3 → Cell 4** (in order, one-by-one).
260+
261+
### Stop the server
262+
Run: **Cell 5**
263+
264+
### Start the server again (same runtime, nothing changed)
265+
Run: **Cell 4**
266+
If you get “address already in use,” run **Cell 5** and then rerun **Cell 4**.
267+
268+
### After changing / updating packages
269+
Run:
270+
1) **Cell 5** (stop server)
271+
2) **Cell 2** (reinstall packages)
272+
3) **Cell 3** (verify install)
273+
4) **Cell 4** (start server)
274+
275+
---
276+
277+
## Troubleshooting
278+
279+
### Web UI opens but TTS doesn’t work
280+
The server can be reachable even if the model didn’t load; always check `/api/model-info` (Cell 4 prints it) to confirm `loaded: True`. [file:21]
281+
282+
### “Token is required (`token=True`), but no token found”
283+
Something in your installed Turbo code is still requesting `huggingface_hub` to read a local token (token=True semantics). [web:65]
284+
Fix: re-run **Cell 2** and then confirm **Cell 3** does not show any auth-forcing markers.
285+
286+
### “/content/bin/micromamba: No such file or directory”
287+
You likely restarted the runtime or Cell 1 didn’t complete; rerun **Cell 1**.
288+
289+
### Where are model files cached?
290+
During Cell 4, downloads are stored under `/content/hf_home` (set by `HF_HOME` in Cell 4). [web:65]
291+
292+
---
293+
294+
## Notes
295+
- If Colab warns that `serve_kernel_port_as_window` might stop working, it still usually provides a working link; click the link that Colab prints (it looks like `https://localhost:8004/` but maps to a `*.colab.*` URL). [web:146]
296+
- For bug reports, attach `/content/chatterbox_server_stdout.log` and the `/api/model-info` output. [file:21]

0 commit comments

Comments
 (0)