CLAUDE.md

# AI Comic Factory - Project Documentation

## Project Overview

**AI Comic Factory** is a Next.js application that generates AI-powered comic strips using Large Language Models (LLMs) and image generation APIs. Users input a prompt, select a comic style, and the system generates a complete comic with panels, dialog, and artwork.

**Key Features:**
- Generate complete comics from a single text prompt
- Multiple comic art styles and fonts
- Support for multiple LLM providers (OpenAI, Anthropic, Groq, Hugging Face)
- Multiple image generation engines (SDXL, OpenAI DALL-E, Replicate)
- Interactive comic editor with speech bubbles and captions
- Export to CLAP format (Cinematic Language and Audio Protocol)
- Community sharing features (optional)
- OAuth integration with Hugging Face

## Technology Stack

**Frontend:**
- Next.js 14.2.7 with App Router
- React 18.3.1 with TypeScript 5.4.5
- Tailwind CSS 3.4.1 with custom comic fonts
- shadcn/ui component library (Radix UI primitives)
- Zustand for state management
- React Konva for canvas-based comic editing
- Framer Motion alternatives via Tailwind animations

**Backend/API:**
- Next.js Server Actions (9 server functions identified)
- Multiple LLM integrations: OpenAI, Anthropic Claude, Groq, Hugging Face
- Multiple rendering engines: SDXL, Replicate, VideoChain API, OpenAI DALL-E
- Image processing with Sharp, HTML2Canvas
- Docker containerization

**Key Dependencies:**
- `@aitube/clap` - CLAP format support for multimedia projects
- `@anthropic-ai/sdk` - Claude AI integration
- `@huggingface/inference` - Hugging Face model access
- `groq-sdk` - Groq API integration
- `openai` - OpenAI API integration
- `replicate` - Replicate.com API integration
- Custom font handling with 13 different comic fonts

## Project Structure

```
src/
├── app/                      # Next.js app router
│   ├── engine/              # Core business logic
│   │   ├── presets.ts       # Comic style presets (678 lines, 4 main presets)
│   │   ├── render.ts        # Image generation engine
│   │   ├── caption.ts       # Caption generation
│   │   └── censorship.ts    # Content filtering
│   ├── interface/           # UI components (22 directories)
│   │   ├── page/           # Comic page layout
│   │   ├── panel/          # Individual comic panels
│   │   ├── bottom-bar/     # Controls and actions
│   │   ├── settings-dialog/ # Configuration UI
│   │   └── ...
│   ├── queries/            # Server-side data fetching (13 files)
│   │   ├── predict.ts      # LLM prediction orchestration
│   │   ├── predictNextPanels.ts # Panel generation logic
│   │   ├── predictWith*.ts # Provider-specific implementations
│   │   └── ...
│   ├── store/              # Zustand state management
│   │   └── index.ts        # Main app state (21KB)
│   ├── layouts/            # Comic layout definitions
│   └── main.tsx           # Main application component
├── components/
│   ├── ui/                 # shadcn/ui components (27 components)
│   └── icons/             # Custom icons
├── lib/                    # Utility functions (49 files)
│   ├── fonts.ts           # Comic font definitions
│   ├── bubble/            # Speech bubble utilities
│   └── [various utilities for image processing, parsing, etc.]
├── fonts/                  # 13 custom comic fonts
└── types.ts               # TypeScript type definitions (217 lines)
```

## Development Commands

```bash
# Development
npm run dev          # Start development server
npm run build        # Production build
npm run start        # Start production server
npm run lint         # ESLint checking

# Node version
nvm use             # Uses Node v20.17.0 (specified in .nvmrc)
```

## Environment Configuration

The application requires extensive environment configuration in `.env.local`:

**Core Engines:**
- `LLM_ENGINE`: "INFERENCE_API" | "INFERENCE_ENDPOINT" | "OPENAI" | "GROQ" | "ANTHROPIC"
- `RENDERING_ENGINE`: "INFERENCE_API" | "INFERENCE_ENDPOINT" | "REPLICATE" | "VIDEOCHAIN" | "OPENAI"

**Authentication (configure only what you use):**
- `AUTH_HF_API_TOKEN` - Hugging Face API token
- `AUTH_OPENAI_API_KEY` - OpenAI API key
- `AUTH_GROQ_API_KEY` - Groq API key
- `AUTH_ANTHROPIC_API_KEY` - Anthropic/Claude API key
- `AUTH_REPLICATE_API_TOKEN` - Replicate.com token
- `AUTH_VIDEOCHAIN_API_TOKEN` - VideoChain API token

**LLM Configuration:**
- `LLM_HF_INFERENCE_API_MODEL` - Default: "HuggingFaceH4/zephyr-7b-beta"
- `LLM_OPENAI_API_MODEL` - Default: "gpt-4-turbo"
- `LLM_GROQ_API_MODEL` - Default: "mixtral-8x7b-32768"
- `LLM_ANTHROPIC_API_MODEL` - Default: "claude-3-opus-20240229"

**Rendering Configuration:**
- `RENDERING_HF_INFERENCE_API_BASE_MODEL` - Default: "stabilityai/stable-diffusion-xl-base-1.0"
- `RENDERING_REPLICATE_API_MODEL` - Default: "stabilityai/sdxl"
- `MAX_NB_PAGES` - Default: 6

## Architecture Patterns

**State Management:**
- Zustand store with typed selectors and actions
- Complex state includes: panels, speeches, captions, renderedScenes, layouts
- Real-time panel generation status tracking

**LLM Integration Pattern:**
- Abstracted provider interface through `predict()` function
- Provider-specific implementations in separate files
- Standardized prompt templates and response parsing
- Support for multiple prompt formats (Zephyr, Llama, etc.)

**Image Generation Flow:**
1. User provides prompt + selects preset
2. LLM generates panel descriptions, speech, and captions
3. Each panel description is sent to rendering engine
4. Images are generated and cached
5. User can edit speech bubbles and captions
6. Final comic can be exported as image or CLAP file

**Server Actions Architecture:**
- 9 server actions for LLM predictions and rendering
- Clean separation between UI and server logic
- Error handling and fallbacks for API failures

**Comic Preset System:**
- 4 main preset categories with 678 lines of configuration
- Each preset defines: art style, color scheme, font, LLM prompts, image prompts
- Extensible system for adding new comic styles

**Font System:**
- 13 custom comic fonts loaded as local fonts
- Includes both Google Fonts (Indie Flower, The Girl Next Door) and custom fonts
- Proper CSS variable integration for consistent typography

## Key Business Logic

**Panel Generation (`predictNextPanels`):**
- Generates multiple comic panels from a single prompt
- Handles continuation of existing stories
- Parses LLM responses into structured panel data (instructions, speech, captions)
- Error recovery and retry logic

**Rendering Pipeline (`render.ts`):**
- Multi-provider image generation (Replicate, HF, OpenAI, VideoChain)
- Automatic fallbacks between providers
- Image caching and optimization
- Support for different aspect ratios and resolutions

**State Persistence:**
- LocalStorage integration for user settings
- CLAP file format support for project serialization
- OAuth state management with Hugging Face

## Development Patterns

**Component Organization:**
- Feature-based component structure in `app/interface/`
- Reusable UI components in `components/ui/`
- Custom hooks in `lib/` for complex logic

**Type Safety:**
- Comprehensive TypeScript definitions in `types.ts`
- Strict typing for LLM engines, rendering engines, and data flows
- Generic interfaces for extensible provider support

**Error Handling:**
- Graceful degradation for API failures
- User feedback through toast notifications
- Fallback content for missing images/data

**Performance Considerations:**
- Image optimization with Sharp
- Lazy loading of comic panels
- Efficient state updates with Zustand
- Canvas-based rendering for complex layouts

## Testing & Quality

- **Linting**: ESLint with Next.js configuration
- **No test files found** - this is an area for improvement
- **Type checking**: Strict TypeScript configuration
- **Docker**: Production containerization available

## Deployment

- Designed for Hugging Face Spaces deployment
- Docker containerization with Node.js Alpine
- Standalone Next.js output for containerized deployment
- Environment-based configuration for different deployment targets

## Community & Contributions

- Open source project on Hugging Face
- Community contributions documented in `CONTRIBUTORS.md`
- Optional community sharing features
- OAuth integration for user management

## Development Notes

- **No API routes found** - uses Server Actions exclusively
- **Canvas-based editing** with React Konva for interactive panels
- **Multi-provider architecture** allows switching between AI services
- **Extensive font library** for authentic comic typography
- **CLAP format integration** for multimedia project export
- **Rate limiting** configurable for production usage

## Quick Start for Developers

1. Copy `.env` to `.env.local` and configure your API keys
2. Choose your LLM_ENGINE and RENDERING_ENGINE
3. Install dependencies: `npm install`
4. Run development server: `npm run dev`
5. The app will guide you through first-time setup

Most common development setup:
- LLM_ENGINE: "OPENAI" with OpenAI API key
- RENDERING_ENGINE: "REPLICATE" with Replicate token
- This provides reliable, high-quality results for testing