Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
2bd2fb4
chore: update VSCode configuration files and tasks
lumpinif Feb 5, 2026
4cf3c72
feat(auth): implement getSessionWithAPIKey route
lumpinif Feb 5, 2026
ec6c347
feat(db): add isNull function to index export
lumpinif Feb 5, 2026
213808f
feat(auth): add type definitions for AuthWorkerRpc
lumpinif Feb 5, 2026
4593b35
feat(auth): enhance authentication modes and JWT support
lumpinif Feb 5, 2026
daf85c7
feat(jwt): add JWT minting script and CLI command
lumpinif Feb 5, 2026
41ee950
feat(jwt): enhance JWT minting script with environment variable updates
lumpinif Feb 5, 2026
e9ecce3
ref(auth): add example environment variable file for local development
lumpinif Feb 5, 2026
caca878
feat(auth): implement jwt auth mode and enhance navigation
lumpinif Feb 5, 2026
4a18208
feat(playground): update references from "Dashboard" to "Playground" …
lumpinif Feb 5, 2026
a020a16
feat(logging): enable activity logs and add local migration command
lumpinif Feb 5, 2026
c24dcc8
fix(auth): simplify getAuthMode function by removing redundant JWT check
lumpinif Feb 5, 2026
5f04f55
chore(vscode): update workspace and task configurations for Playground
lumpinif Feb 5, 2026
ff86ffe
chore(ui): update component configurations and formatting settings
lumpinif Feb 6, 2026
eb9e468
fix(logs): update default log limit to 10 and adjust card styling
lumpinif Feb 6, 2026
6582e0e
feat(env): introduce centralized runtime configuration and environmen…
lumpinif Feb 6, 2026
19c49db
ref(auth): add comprehensive authentication modes documentation and r…
lumpinif Feb 6, 2026
60db796
ref(auth): update environment variable configurations and improve aut…
lumpinif Feb 6, 2026
a806ab6
chore(workspace): update workspace configuration and add environment …
lumpinif Feb 7, 2026
960ee7e
pkg:auth:chore converge url/origin config
lumpinif Feb 7, 2026
30c483f
workers:v0:chore harden cors + urls
lumpinif Feb 7, 2026
7b00e4d
pkg:sdk:fix map legacy unauthorized codes
lumpinif Feb 7, 2026
ef9e23d
fix(logs): enhance error handling in log processing and export functions
lumpinif Feb 7, 2026
f96c2e9
ref(env): update environment variable configurations and remove depre…
lumpinif Feb 7, 2026
f499cdd
ref(cors): streamline CORS configuration and enhance security
lumpinif Feb 7, 2026
4bc64c9
ref(auth): update worker configuration and environment variable defin…
lumpinif Feb 7, 2026
99c2a2d
ref(vscode): enhance task configurations for environment management a…
lumpinif Feb 7, 2026
f9767b3
ref(urls): centralize URL handling and improve absolute URL resolution
lumpinif Feb 7, 2026
241d28a
ref(env): update environment variable examples and sync process
lumpinif Feb 7, 2026
5948e43
ref(auth): reorder dependencies in package.json for better clarity
lumpinif Feb 7, 2026
cb6288b
ref(dependencies): add '@deepcrawl/runtime' dependency to pnpm-lock.yaml
lumpinif Feb 7, 2026
787c60f
ref(auth): update worker configuration and environment variable defin…
lumpinif Feb 7, 2026
9923b1d
ref(config): update Biome configuration and workspace settings
lumpinif Feb 7, 2026
851ee92
ref(playground): implement API key management and fallback authentica…
lumpinif Feb 7, 2026
fbb89e7
ref(branding): implement dynamic brand name configuration
lumpinif Feb 7, 2026
f6d765c
ref(auth): enhance API key extraction and improve authorization handling
lumpinif Feb 7, 2026
9b32a17
ref(auth): enhance cookie-based authentication and API key management
lumpinif Feb 7, 2026
9bc9609
ref(env): enhance environment variable management and synchronization
lumpinif Feb 7, 2026
01493eb
ref(tasks): streamline CI checks and update package scripts
lumpinif Feb 7, 2026
d39e4b6
ref(lint): enhance lint-staged configuration with absolute path handl…
lumpinif Feb 7, 2026
ad5b4c8
ref(formatting): add default formatter for JavaScript in workspace an…
lumpinif Feb 7, 2026
5294db5
ref(husky): enhance pre-push script for changeset checks and CI valid…
lumpinif Feb 8, 2026
449fd39
chore: add changeset for '@deepcrawl/contracts' with no release
lumpinif Feb 8, 2026
d42c8da
ref(rate-limiting): implement Upstash Redis for API rate limiting
lumpinif Feb 8, 2026
c43601d
ref(rate-limiting): refactor rate limit middleware and introduce API …
lumpinif Feb 8, 2026
e1f7377
chore(deps): update package dependencies across multiple packages
lumpinif Feb 8, 2026
bc741ec
ref(auth): integrate new rate limiting configuration for authenticati…
lumpinif Feb 8, 2026
a3704b9
ref(playground): improve validation handling for playground operations
lumpinif Feb 8, 2026
e8fb9a1
feat(deploy-attribution): add Deploy Attribution Banner component and…
lumpinif Feb 8, 2026
855fbc5
chore(deps): update package dependencies to use catalog references
lumpinif Feb 8, 2026
ac1820a
feat(auth): enhance API key handling and improve logging in middleware
lumpinif Feb 8, 2026
eb6dfbc
feat(jwt): update JWT handling and configuration documentation
lumpinif Feb 8, 2026
0d4e3b2
refactor: update layout and component structures for improved clarity…
lumpinif Feb 8, 2026
32d4301
feat(jwt): enhance JWT minting process with environment key management
lumpinif Feb 8, 2026
6920b6c
refactor: update component structures and improve error handling
lumpinif Feb 8, 2026
4d36087
fix(jwt-mint): improve argument parsing and error handling
lumpinif Feb 8, 2026
9cecf3b
refactor(cors, jwt-auth): simplify trusted origins and enhance sessio…
lumpinif Feb 8, 2026
db6e9a7
feat(github-actions): add workflow to sync template branch with main
lumpinif Feb 8, 2026
cde09d7
chore(github-actions): enhance sync workflow with concurrency control
lumpinif Feb 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ dist/
.vscode/*
!.vscode/launch.json
!.vscode/*.code-snippets
!.vscode/extensions.json
!.vscode/settings.json
.vscode/settings.local.json
!.vscode/tasks.json
.idea/workspace.xml
.idea/usage.statistics.xml
.idea/shelf
Expand Down
3 changes: 3 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"recommendations": ["biomejs.biome"]
}
11 changes: 11 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"biome.enabled": true,
"biome.requireConfiguration": true,
"editor.defaultFormatter": "biomejs.biome",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.action.useSortedAttributes.biome": "explicit",
"source.organizeImports.biome": "explicit",
"source.fixAll.biome": "explicit"
}
}
130 changes: 130 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Start Frontend & Backend",
"detail": "Start dashboard (Next.js) and workers (Cloudflare Workers) in parallel terminal windows",
"dependsOn": ["Start Dashboard", "Start Workers"],
"dependsOrder": "parallel",
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
},
"group": {
"isDefault": true
}
},
{
"label": "Start All Dev Services",
"detail": "Start all dev services in one terminal window with turbo dev",
"type": "shell",
"command": "pnpm dev",
"options": {
"cwd": "${workspaceFolder}"
},
"isBackground": true,
"problemMatcher": [],
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
}
},
{
"label": "Start Dashboard",
"detail": "Start dashboard dev server (Next.js)",
"type": "shell",
"command": "pnpm dev",
"options": {
"cwd": "${workspaceFolder}/apps/app"
},
"isBackground": true,
"problemMatcher": [],
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
}
},
{
"label": "Start Auth Worker",
"detail": "Start auth worker dev server (Wrangler)",
"type": "shell",
"command": "pnpm dev",
"options": {
"cwd": "${workspaceFolder}/apps/workers/auth"
},
"isBackground": true,
"problemMatcher": [],
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
}
},
{
"label": "Start V0 Worker",
"detail": "Start v0 worker dev server (Wrangler)",
"type": "shell",
"command": "pnpm dev",
"options": {
"cwd": "${workspaceFolder}/apps/workers/v0"
},
"isBackground": true,
"problemMatcher": [],
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
}
},
{
"label": "Start Workers",
"detail": "Start auth + v0 workers in parallel",
"dependsOn": ["Start Auth Worker", "Start V0 Worker"],
"dependsOrder": "parallel",
"presentation": {
"panel": "new",
"group": "dev",
"reveal": "always",
"clear": true
}
},
{
"label": "Generate Worker Auth Types",
"detail": "Generate Cloudflare types for auth worker",
"type": "shell",
"command": "pnpm cf-typegen",
"options": {
"cwd": "${workspaceFolder}/apps/workers/auth"
},
"presentation": {
"panel": "new",
"group": "tools",
"reveal": "always",
"clear": true
}
},
{
"label": "Generate Worker V0 Types",
"detail": "Generate Cloudflare types for v0 worker",
"type": "shell",
"command": "pnpm cf-typegen",
"options": {
"cwd": "${workspaceFolder}/apps/workers/v0"
},
"presentation": {
"panel": "new",
"group": "tools",
"reveal": "always",
"clear": true
}
}
]
}
26 changes: 26 additions & 0 deletions apps/app/app/api/auth/getSessionWithAPIKey/route.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { getAuth } from '@deepcrawl/auth/lib/auth';
import { NextResponse } from 'next/server';

export async function POST(request: Request) {
const body = await request.json().catch(() => ({}));
const apiKey =
body?.apiKey ??
request.headers.get('x-api-key') ??
request.headers.get('authorization')?.replace('Bearer ', '');

if (!apiKey) {
return NextResponse.json(
{ error: 'Missing API key' },
{
status: 400,
},
);
}

const auth = getAuth();
const headers = new Headers(request.headers);
headers.set('x-api-key', apiKey);

const session = await auth.api.getSession({ headers });
return NextResponse.json(session);
}
11 changes: 11 additions & 0 deletions apps/workers/auth/src/rpc.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { WorkerEntrypoint } from 'cloudflare:workers';
import type { Session } from '@deepcrawl/auth/types';

/**
* Type-only RPC surface for the auth worker service binding.
* Keep this file free of runtime code or worker env dependencies.
*/
export default class AuthWorkerRpc extends WorkerEntrypoint {
getSessionWithAPIKey(apiKey: string): Promise<Session | undefined>;
clearApiKeyCache(apiKey: string): Promise<void>;
}
6 changes: 6 additions & 0 deletions apps/workers/v0/.dev.vars.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
BETTER_AUTH_URL=http://localhost:8787
BETTER_AUTH_SECRET=

AUTH_MODE=better-auth

JWT_SECRET=
JWT_ISSUER=
JWT_AUDIENCE=

DATABASE_URL=

# Github
Expand Down
117 changes: 117 additions & 0 deletions apps/workers/v0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,123 @@ The worker is configured via `wrangler.jsonc` with:
- **KV Storage** - Caching and data persistence
- **Custom Domain** - `api.deepcrawl.dev`

### Authentication Modes

Deepcrawl supports multiple authentication modes. Default is `better-auth`.

```bash
# Auth mode: better-auth (default) | jwt | none
AUTH_MODE=better-auth
```

- `better-auth` (default): API key + cookie auth via Better Auth.
- `jwt`: Verify JWT tokens from `Authorization: Bearer <token>`.
- `none`: No authentication, all operations are open. Use with caution.

### JWT Configuration

Required when `AUTH_MODE=jwt`:

```bash
JWT_SECRET=your_jwt_secret
JWT_ISSUER=deepcrawl # optional
JWT_AUDIENCE=deepcrawl-api # optional
```

JWT payload expectations:
- `sub` is required and maps to `user.id` and `session.userId`
- Optional fields: `email`, `name`, `picture`, `email_verified`, `exp`

Notes on issuer/audience:
- `iss` (issuer) identifies who minted the token.
- `aud` (audience) identifies which service the token is intended for.
- If unset, they are not validated. If set, tokens must match to be accepted.

Generate a JWT secret:

1. Cross-platform (Node.js):
```bash
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
```

2. macOS/Linux (OpenSSL):
```bash
openssl rand -hex 32
```

### JWT Example (Node.js)

Use the simplest and most common library: `jsonwebtoken`.

```bash
pnpm add jsonwebtoken
```

```ts
import jwt from 'jsonwebtoken';

const token = jwt.sign(
{
sub: 'user_123',
email: 'alice@example.com',
name: 'Alice',
},
process.env.JWT_SECRET as string,
{
expiresIn: '24h',
issuer: 'deepcrawl',
audience: 'deepcrawl-api',
},
);

// Send as: Authorization: Bearer <token>
```

### JWT Quick Mint (CLI)

If you only deploy the API worker and want a quick token for simple auth,
use the built-in script in this repo:

```bash
# From the repo root
JWT_SECRET=your_jwt_secret pnpm jwt:mint
```

This is an interactive Node.js script (not Bash), so it works on macOS and
Windows. It will prompt you for required fields if you don’t pass flags.
If you leave the JWT secret blank, the script can generate one for you.
It can also write the secret into local env files for you.

You can also pass flags to skip prompts:

```bash
pnpm jwt:mint -- --sub user_123 --email alice@example.com --expires-in 24
```

Write the secret into env files (optional):

```bash
pnpm jwt:mint -- --write-dev-vars
pnpm jwt:mint -- --write-dev-vars-production
```

Required inputs:
- `JWT_SECRET` (env var or prompt)
- `sub` (user id)

Optional inputs:
- `email`
- `name`
- `issuer` (`JWT_ISSUER`)
- `audience` (`JWT_AUDIENCE`)
- `expires-in` (hours, default 24)

The command prints a JWT you can attach as:

```
Authorization: Bearer <token>
```

## 🔌 API Endpoints

### **oRPC Endpoints** (Type-safe)
Expand Down
Loading