TEXTtoPBR

A web-based tool capable of generating seamless, photorealistic PBR (Physically Based Rendering) textures from simple text prompts. It automatically generates Albedo, Roughness, Normal, and Displacement maps using server-side AI generation combined with client-side processing.

Demo

Overview

This project simplifies the 3D texture creation workflow:

1. Input: User provides a text description (e.g., "mossy stone wall").
2. Generation: The app leverages Pollinations.AI (Flux model) to generate a high-quality base texture.
3. Processing: Custom JavaScript algorithms run locally to:
   • Make the texture seamless/tileable.
   • Generate secondary maps (Normal, Roughness, Displacement) from pixel intensity data.
4. Preview: A real-time React Three Fiber viewer allows you to inspect the material on various 3D shapes.
5. Export: Download the complete PBR map set as a ZIP.

Tech Stack

Core Pipeline (Generation and Processing)

• Pollinations.AI (Flux): The generative AI model used to create the initial diffuse texture from text.
• HTML5 Canvas API: Used for fast, client-side image processing. We implement:
  • Seamless Tiling: A mirror-blending algorithm to smooth edges.
  • Sobel Filters: To calculate normal maps based on luminance gradients.
  • Blur and Contrast: To derive roughness and displacement maps.

Frontend (Viewer)

• React.js: UI and state management.
• React Three Fiber (R3F): Renders the 3D preview scene.
• @react-three/drei: Provides helpers like <Stage>, <OrbitControls>, and <useTexture>.

Tools and Utilities

• Vite: Fast frontend tooling.
• JSZip: Bundles the generated maps into a downloadable ZIP.

---

How to Run

1. Clone the repository

   git clone https://github.com/Vasco888888/text-to-pbr.git
   cd text-to-pbr

2. Install dependencies

   npm install

3. Set up Environment Variables Create a .env file in the root directory and add your Pollinations API key:

   VITE_POLLINATIONS_API_KEY=your_api_key_here

4. Run the local server

   npm run dev

   Open http://localhost:3000 to start generating textures.

---

Implementation Details

Seamless Tiling Algorithm

To ensure textures repeat without visible seams, we use a custom blending function (makeSeamless in TextureProcessor.js). It takes the edges of the generated image and blends them with the opposite side using a Cubic Hermite Interpolation (Smoothstep) function (3t² - 2t³). This creates an S-curve transition that eliminates harsh gradients and visible seams at the tiling boundaries.

Map Generation

• Albedo: The base generated image (processed for seamlessness).
• Roughness: Derived from an inverted and offset luminance scale. Darker areas (crevices) are assumed to be rougher/less shiny.
• Displacement (Height): Derived from luminance. Brighter pixels represent higher points. We apply a Box Blur as a post-processing step to smooth out AI artifacts and reduce high-frequency noise.
• Normal: Generated via a Sobel Operator. We calculate horizontal and vertical gradients ($dX$ and $dY$), combine them with a configurable strength vector ($dZ$), and normalize the resulting 3D vector. This "bakes" surface direction into RGB colors that Three.js uses for real-time lighting.

Memory Optimization

The pipeline is designed for high-performance handling of 4K textures:

• Blob URLs over Base64: Instead of heavy Data URLs (strings), we use Binary Large Objects (Blobs). This reduces memory usage by ~33% and improves GPU upload speeds.
• Manual Cleanup: The app proactively calls URL.revokeObjectURL() for every generated map and temporary AI blob to prevent memory leaks during long sessions.

---

Project Structure

├── assets/                     
│   └── demo.gif                # Demo for the README
├── src/
│   ├── components/
│   │   ├── CanvasView.jsx      # The 3D preview scene with adjustable shapes
│   │   ├── Controls.jsx        # Sidebar for prompts and download
│   │   └── Controls.css        # Styling for the sidebar
│   ├── hooks/
│   │   └── usePBRGenerator.js  # Logic for fetching AI images and managing state
│   ├── TextureProcessor.js     # Low-level pixel manipulation (Normal/Roughness calc)
│   ├── App.jsx                 # Main layout
│   └── index.jsx               # Entry point
├── package.json
├── vite.config.js
└── README.md