327 lines
18 KiB
Markdown
327 lines
18 KiB
Markdown
<!-- GSD:project-start source:PROJECT.md -->
|
||
## Project
|
||
|
||
**Unified Media Manager (UMM)**
|
||
|
||
A single Go + React service that replaces the entire \*arr stack (Sonarr, Radarr, Lidarr, Readarr, Prowlarr, Jellyseerr, Bazarr, Recyclarr, Analyzarr — 12 containers) with one unified media management platform. UMM handles media acquisition, download management, quality tracking, and library organization for movies, TV series, music, books, audiobooks, and podcasts.
|
||
|
||
**Core Value:** Media downloads and imports correctly — searching indexers, grabbing releases, downloading via SABnzbd/qBittorrent, and importing completed files to the library with proper naming and hardlinks. If this works, the arr stack is replaceable.
|
||
|
||
### Constraints
|
||
|
||
- **Tech stack**: Go 1.22+ (Echo v4), PostgreSQL 16, React 18 + TypeScript + Tailwind — already chosen and in use
|
||
- **Network**: Backend must share Gluetun's network namespace for VPN-routed indexer/download traffic
|
||
- **Filesystem**: Hardlinking requires download dir and media library on same filesystem (TrueNAS NFS)
|
||
- **Compatibility**: File naming conventions must match existing library structure so Jellyfin/Navidrome/Calibre continue working
|
||
- **Protocol**: Must support Torznab/Newznab XML APIs and Cardigann YAML definitions for indexer compatibility
|
||
- **Existing services preserved**: Gluetun, SABnzbd, qBittorrent, FlareSolverr, Tdarr — UMM replaces acquisition/management layer only
|
||
<!-- GSD:project-end -->
|
||
|
||
<!-- GSD:stack-start source:codebase/STACK.md -->
|
||
## Technology Stack
|
||
|
||
## Languages
|
||
- Go 1.22 — Backend API server, all `internal/` and `cmd/` packages
|
||
- TypeScript 5.4+ — Frontend React application in `frontend/src/`
|
||
- SQL (PostgreSQL dialect) — Database schema in `internal/db/migrations/`
|
||
- YAML — Docker Compose config (`docker-compose.yml`)
|
||
- Nginx config — Frontend reverse proxy (`frontend/nginx.conf`)
|
||
## Runtime
|
||
- Backend: Linux container (Alpine 3.19), compiled with `CGO_ENABLED=0`
|
||
- Frontend: Node.js 20-alpine (build), Nginx Alpine (serve)
|
||
- Docker + Docker Compose v3.8
|
||
- Go Modules — `go.mod` + `go.sum` present
|
||
- npm — `frontend/package-lock.json` present (lockfile committed)
|
||
## Frameworks
|
||
- Echo v4.12.0 — Go HTTP framework for REST API (`github.com/labstack/echo/v4`)
|
||
- React 18.3.1 — Frontend UI library
|
||
- React Router DOM 6.23.1 — Client-side routing
|
||
- Tailwind CSS 3.4.4 — Utility-first CSS framework
|
||
- No testing framework detected — no test files, no test config
|
||
- Vite 5.3.1 — Frontend build tool and dev server
|
||
- @vitejs/plugin-react 4.3.0 — Vite React plugin
|
||
- PostCSS 8.4.38 + Autoprefixer 10.4.19 — CSS processing pipeline
|
||
- Multi-stage Docker builds — `backend/Dockerfile` and `frontend/Dockerfile`
|
||
## Key Dependencies
|
||
- `github.com/labstack/echo/v4` v4.12.0 — HTTP routing, middleware, JSON binding
|
||
- `github.com/jackc/pgx/v5` v5.5.5 — PostgreSQL driver and connection pool
|
||
- `github.com/jackc/pgx/v5/pgxpool` — Connection pool (used in `internal/db/db.go`)
|
||
- `log/slog` (stdlib) — Structured logging
|
||
- `react` ^18.3.1 + `react-dom` ^18.3.1 — UI rendering
|
||
- `react-router-dom` ^6.23.1 — SPA routing
|
||
- PostgreSQL 16 (external) — Primary database, shared instance at `postgres-shared:5432`
|
||
- Nginx (Alpine) — Frontend static file server + API reverse proxy
|
||
- Traefik (external) — TLS termination and routing at `umm.local.tophermayor.com`
|
||
## Configuration
|
||
- Config loaded from env vars in `internal/config/config.go`
|
||
- `DATABASE_URL` — PostgreSQL connection string (default: `postgres://localhost:5432/unified_media_manager?sslmode=disable`)
|
||
- `QDRANT_URL` — Qdrant vector DB URL (default: `http://localhost:6333`)
|
||
- `OLLAMA_URL` — Ollama AI service URL (default: `http://localhost:11434`)
|
||
- `PORT` — Backend HTTP port (default: `8084`)
|
||
- `VITE_API_URL` — Frontend API base URL (build-time, default: `http://umm.local.tophermayor.com`)
|
||
- Backend: `go build -o /umm ./cmd/server/` via multi-stage Dockerfile
|
||
- Frontend: `tsc && vite build` → output to `frontend/dist/`
|
||
- Backend currently copies `frontend/dist/` into binary container (see `backend/Dockerfile:13`)
|
||
## Platform Requirements
|
||
- Go 1.22+ SDK
|
||
- Node.js 20+
|
||
- Docker + Docker Compose
|
||
- PostgreSQL 16 accessible (or use Docker default)
|
||
- Docker host with external network `proxy-net`
|
||
- PostgreSQL instance (shared, `postgres-shared:5432`)
|
||
- Traefik reverse proxy on same Docker network
|
||
- Gluetun VPN container (planned: backend will use `network_mode: service:gluetun`)
|
||
- NFS mount from TrueNAS for media files (`/mnt/truenas/mediadata:/data`)
|
||
## Database Schema
|
||
- 3 custom enum types: `MEDIA_TYPE`, `MEDIA_STATUS`, `QUEUE_STATUS`
|
||
- 24 tables total across single `umm` database
|
||
- Partitioned `media` table (9 partitions by media type)
|
||
- Partitioned `download_history` (time-range partitioning, 90-day retention)
|
||
- Full-text search via GIN index on `media.title`
|
||
- JSONB columns for flexible metadata (`external_ids`, `metadata`, `images`, `quality`)
|
||
- Migration system: embedded SQL files in `internal/db/migrations/`, tracked via `schema_migrations` table
|
||
<!-- GSD:stack-end -->
|
||
|
||
<!-- GSD:conventions-start source:CONVENTIONS.md -->
|
||
## Conventions
|
||
|
||
## Project Overview
|
||
## Naming Patterns
|
||
### Go (Backend)
|
||
- Use short, lowercase, single-word package names: `api`, `db`, `config`
|
||
- Package `internal/api/` contains all HTTP handlers in a single package (no sub-packages)
|
||
- Package `internal/db/` contains database pool, migrations, and embedded SQL files
|
||
- Package `internal/config/` contains environment config loading
|
||
- One domain entity per file: `media.go`, `queue.go`, `blocklist.go`, `indexers.go`, `dashboard.go`, `health.go`
|
||
- File names are lowercase, singular: `router.go`, not `routers.go`
|
||
- Use `_test.go` suffix for tests (none exist yet)
|
||
- Handler factory functions use the pattern `func <verb><Entity>(database *db.DB) echo.HandlerFunc` — e.g., `listMedia`, `createMedia`, `updateMedia`, `deleteMedia`
|
||
- Private helper functions use camelCase: `parsePagination`, `buildMediaWhere`, `scanMedia`, `scanMediaRows`
|
||
- Scanner functions accept an interface: `func scanQueueItem(scanner interface{ Scan(...interface{}) error })`
|
||
- Constants for column lists: `const queueColumns = \`id, media_id, ...\``
|
||
- Local variables: camelCase — `pageSize`, `totalPages`, `setClauses`
|
||
- Struct fields: PascalCase — `TotalPages`, `PageSize`, `ReleaseTitle`
|
||
- Abbreviations stay uppercase: `ID`, `URL`, `API` — e.g., `MediaID`, `ReleaseURL`, `APIKey`
|
||
- Exported structs: PascalCase — `Media`, `QueueItem`, `BlocklistItem`, `Indexer`, `DashboardStats`
|
||
- Request DTOs: PascalCase with `Request` suffix — `createMediaRequest`, `updateMediaRequest`
|
||
- Response DTOs: PascalCase with `Response` suffix — `mediaDetailResponse`, `paginatedResponse`
|
||
- Use unexported structs for request/response types that don't leave the package
|
||
- JSON tags use snake_case: `` `json:"media_type"` ``, `` `json:"release_title"` ``
|
||
- Use `omitempty` on nullable/optional fields: `` `json:"original_title,omitempty"` ``
|
||
- Required fields never have `omitempty`
|
||
### TypeScript (Frontend)
|
||
- PascalCase for components: `Dashboard.tsx`, `Library.tsx`, `Queue.tsx`
|
||
- camelCase for utilities: `client.ts`
|
||
- Directories: lowercase — `pages/`, `api/`, `components/`
|
||
- Default exports for page components: `export default function Dashboard()`
|
||
- Functional components only (no class components)
|
||
- One component per file
|
||
- camelCase: `search`, `typeFilter`, `fetchAPI`
|
||
- PascalCase for types/interfaces: `DashboardData`
|
||
- Generic type parameter: `<T>` in API functions
|
||
- Generic fetch wrappers in `frontend/src/api/client.ts`
|
||
- Three functions: `fetchAPI<T>`, `postAPI<T>`, `deleteAPI<T>`
|
||
## Code Style
|
||
### Go Formatting & Linting
|
||
- Use `gofmt` / `goimports` — standard Go formatting
|
||
- Tabs for indentation (Go standard)
|
||
- No linter config detected (no `.golangci.yml`)
|
||
- Follow standard Go conventions
|
||
- Grouped imports: stdlib first, then project imports, then third-party
|
||
- Error wrapping with `fmt.Errorf("verb: %w", err)` — never raw error return
|
||
- Struct literal field alignment not enforced but generally consistent
|
||
### TypeScript Formatting & Linting
|
||
- No Prettier config detected
|
||
- 2-space indentation
|
||
- Single quotes for strings (JSX attributes use double quotes in some places)
|
||
- Trailing commas not enforced
|
||
- No ESLint config detected in the project (only in node_modules)
|
||
- `strict: true` enabled
|
||
- `noUncheckedIndexedAccess: true` — must check array/object access
|
||
- `exactOptionalPropertyTypes: true` — optional properties must be explicitly `undefined`
|
||
- `verbatimModuleSyntax: true` — use `import type` for type-only imports
|
||
- `jsx: "react-jsx"` — automatic JSX runtime (no need to import React in every file)
|
||
- `isolatedModules: true` — Vite compatibility
|
||
## Import Organization
|
||
### Go
|
||
### TypeScript
|
||
- None configured — use relative paths: `'../api/client'`, `'./pages/Dashboard'`
|
||
## Error Handling
|
||
### Go Backend
|
||
- Always return `map[string]string{"error": "message"}` for error JSON
|
||
- Use appropriate HTTP status codes:
|
||
- Wrap errors with context using `fmt.Errorf("verb: %w", err)`
|
||
- Log with `slog.Error("message", "key", value, "error", err)` before returning HTTP error
|
||
- Check `tag.RowsAffected()` after UPDATE/DELETE for not-found detection
|
||
- Every handler creates a context with timeout: `context.WithTimeout(c.Request().Context(), <duration>)`
|
||
- Read operations: 10-second timeout
|
||
- Write operations: 5-second timeout
|
||
- Indexer test: 15-second timeout
|
||
- Always `defer cancel()` immediately after creating context
|
||
### Frontend
|
||
- `fetchAPI`/`postAPI`/`deleteAPI` throw `Error` on non-OK responses: `throw new Error(\`API error: ${res.status}\`)`
|
||
- Currently swallowing errors in components: `.catch(() => {})` — this needs improvement
|
||
- No global error boundary or toast notification system
|
||
## Logging
|
||
- Use `slog.Info` for startup, shutdown, migration events
|
||
- Use `slog.Error` for all failures before returning HTTP errors
|
||
- Use `slog.Warn` for recoverable issues
|
||
- Key-value pairs use string keys: `"error"`, `"id"`, `"port"`, `"query"`
|
||
- Never use `fmt.Println` or `log.Println` — always use `slog`
|
||
## Comments
|
||
- Public functions: no doc comments in current codebase (should be added)
|
||
- Inline comments for SQL query construction: `// addCol helper for dynamic UPDATE`
|
||
- Column constant definitions explain the SQL: `const mediaColumns = \`...\``
|
||
- No file-level comments
|
||
- No Go doc comments on exported types or functions — this is a gap
|
||
- No JSDoc/TSDoc on frontend functions
|
||
- No inline documentation for complex SQL queries
|
||
## Function Design
|
||
### Go Handlers
|
||
- Always return `c.JSON(statusCode, body)` — never return raw values
|
||
- Success responses use typed structs or `map[string]string{"status": "updated"}`
|
||
- Creation responses return `map[string]int64{"id": id}` with `http.StatusCreated`
|
||
### Frontend Components
|
||
## Database Query Conventions
|
||
### Query Building
|
||
- Define column list constants at file scope: `const queueColumns = \`id, media_id, ...\``
|
||
- Use `fmt.Sprintf("SELECT %s FROM table", columns)` to compose queries
|
||
- Use `deleted_at IS NULL` in WHERE clauses for soft-deleted tables (`media`, `media_files`)
|
||
- Hard delete for `blocklist`, `download_queue` (use status changes instead)
|
||
- Use `parsePagination(c)` to extract `page` and `page_size` from query params
|
||
- Default: page 1, page_size 50, max 100
|
||
- Return `paginatedResponse` struct with `data`, `total`, `page`, `page_size`, `total_pages`
|
||
### NULL Handling
|
||
- Use `sql.NullString`, `sql.NullInt64`, `sql.NullTime` for nullable columns
|
||
- Check `.Valid` then dereference: `if origTitle.Valid { m.OriginalTitle = &origTitle.String }`
|
||
- For `json.RawMessage` nullable fields, use `[]byte` scanner then convert
|
||
## CSS/Styling Conventions
|
||
- Background: `bg-gray-950` (page), `bg-gray-900` (nav/containers), `bg-gray-800` (inputs)
|
||
- Text: `text-white` (primary), `text-gray-400` (secondary), `text-gray-500` (placeholder)
|
||
- Accent: `text-indigo-400`, `focus:border-indigo-500`
|
||
- Status colors: `bg-blue-600`, `bg-green-600`, `bg-red-600`, `bg-orange-600`, etc.
|
||
## Module Design
|
||
### Go Exports
|
||
- `internal/config` → `Config` struct, `Load()` function
|
||
- `internal/db` → `DB` struct, `New()` constructor, `MigrationsFS` embedded FS
|
||
- `internal/api` → `NewRouter()` function (the only entry point into the API layer)
|
||
### Frontend Exports
|
||
## Configuration Conventions
|
||
- Read from environment variables via `os.Getenv()`
|
||
- Provide sensible defaults when env vars are unset
|
||
- Return a `*Config` struct — no global state, passed explicitly
|
||
- Use Vite env: `import.meta.env.VITE_API_URL`
|
||
- Empty string fallback means same-origin API calls
|
||
<!-- GSD:conventions-end -->
|
||
|
||
<!-- GSD:architecture-start source:ARCHITECTURE.md -->
|
||
## Architecture
|
||
|
||
## Pattern Overview
|
||
- Backend serves both API (`/api/*`) and static frontend assets from the same Echo server
|
||
- Frontend is a separate Docker container with nginx that proxies `/api/*` to backend — but backend also has `e.Static("/", "frontend/dist")` as a fallback
|
||
- Handler-per-function pattern: each API endpoint is a standalone function returning `echo.HandlerFunc` via closure over `*db.DB`
|
||
- No service/repository layer — handlers directly execute SQL against `pgxpool.Pool`
|
||
- Database migrations embedded at compile time via `go:embed`
|
||
- PostgreSQL partitioned tables used for multi-tenant media types and time-bounded history
|
||
## Layers
|
||
- Purpose: Route registration, middleware, request parsing, response serialization
|
||
- Location: `internal/api/router.go`
|
||
- Contains: Echo route group definitions, CORS config, pagination helpers
|
||
- Depends on: Echo v4 framework, `internal/config`, `internal/db`
|
||
- Used by: `cmd/server/main.go` which calls `api.NewRouter()`
|
||
- Purpose: Per-resource CRUD handlers — business logic and SQL queries are inline
|
||
- Location: `internal/api/*.go` (one file per domain: `media.go`, `queue.go`, `blocklist.go`, `indexers.go`, `dashboard.go`, `health.go`)
|
||
- Contains: Type structs (request/response), scan helpers, handler functions
|
||
- Depends on: `internal/db` (for `*db.DB` pool access), `pgx/v5` for scanning
|
||
- Used by: Router layer via closure: `g.GET("/media", listMedia(database))`
|
||
- Purpose: Connection pool management, migration runner
|
||
- Location: `internal/db/db.go`
|
||
- Contains: `DB` struct wrapping `pgxpool.Pool`, `New()` constructor, `RunMigrations()` method
|
||
- Depends on: `pgx/v5/pgxpool`, embedded `migrations/*.sql` files
|
||
- Used by: All handler functions, `cmd/server/main.go`
|
||
- Purpose: Environment variable loading with defaults
|
||
- Location: `internal/config/config.go`
|
||
- Contains: `Config` struct with `DatabaseURL`, `QdrantURL`, `OllamaURL`, `Port`
|
||
- Depends on: `os` standard library only
|
||
- Used by: `cmd/server/main.go`, `internal/api/health.go`
|
||
- Purpose: User interface for media management
|
||
- Location: `frontend/src/`
|
||
- Contains: React components, API client wrapper, Tailwind-styled pages
|
||
- Depends on: React 18, react-router-dom 6, Tailwind CSS
|
||
- Used by: End users via browser; served by nginx container or backend static fallback
|
||
## Data Flow
|
||
- Backend: Stateless — all state in PostgreSQL. No in-memory caching, no sessions.
|
||
- Frontend: Local `useState` per component. No global state store (no Redux/Zustand). Each page fetches data independently via `useEffect`.
|
||
## Key Abstractions
|
||
- Purpose: Inject `*db.DB` dependency into handlers without global state
|
||
- Examples: `listMedia(database *db.DB) echo.HandlerFunc`, `dashboard(database *db.DB) echo.HandlerFunc`
|
||
- Pattern: Every handler is a function that takes `*db.DB` and returns a closure
|
||
```go
|
||
```
|
||
- Purpose: Standardized envelope for all list endpoints
|
||
- Examples: Used in `listMedia`, `searchMedia`, `searchMissing`, `searchUpgrades`, `listQueue`, `listBlocklist`
|
||
- Pattern: `{data: [], total: N, page: N, page_size: N, total_pages: N}`
|
||
- Purpose: Reusable row scanning for types that appear in both single and multi-row queries
|
||
- Examples: `scanBlocklistItem(scanner interface{ Scan(...interface{}) error })`, `scanQueueItem(scanner)`
|
||
- Pattern: Accept `interface{ Scan(...interface{}) error }` so both `pgx.Row` and `pgx.Rows` can be passed
|
||
- Purpose: Construct parameterized SQL with variable WHERE clauses
|
||
- Examples: `buildMediaWhere()` in `media.go`, inline in `searchMedia`, `listQueue`
|
||
- Pattern: Maintain `idx` counter, build `[]string` conditions, `[]interface{}` args, format with `$N` placeholders
|
||
```go
|
||
```
|
||
- Purpose: Ship SQL migrations inside the compiled binary
|
||
- Example: `internal/db/db.go:17` — `//go:embed migrations/*.sql`
|
||
- Pattern: `embed.FS` read at runtime, sorted lexicographically, applied transactionally
|
||
## Entry Points
|
||
- Location: `cmd/server/main.go`
|
||
- Triggers: Docker container start (`CMD ["./umm"]`)
|
||
- Responsibilities:
|
||
- Location: `frontend/vite.config.ts`
|
||
- Triggers: `npm run dev` (development only)
|
||
- Responsibilities: Vite dev server on port 3000 with HMR
|
||
- Location: `frontend/Dockerfile`
|
||
- Triggers: Docker container start (nginx)
|
||
- Responsibilities: Serves built SPA from `/usr/share/nginx/html`, proxies `/api/*` to backend
|
||
## Error Handling
|
||
- Handlers return `c.JSON(http.StatusN, map[string]string{"error": "message"})` for errors
|
||
- Database errors logged via `slog.Error()` with context fields then returned as HTTP 500
|
||
- Validation errors return HTTP 400 with descriptive message
|
||
- Not-found returns HTTP 404 after checking `tag.RowsAffected() == 0`
|
||
- Scan errors in row iterators are logged and skipped (`continue`) rather than failing the whole request
|
||
- Migration failures are logged as warnings and don't stop server startup (`main.go:33`)
|
||
- All handlers use `context.WithTimeout` (5–15s) with deferred cancel
|
||
- No centralized error middleware
|
||
- No structured error types or error codes
|
||
- No request-level error logging correlation (no request IDs)
|
||
## Cross-Cutting Concerns
|
||
<!-- GSD:architecture-end -->
|
||
|
||
<!-- GSD:skills-start source:skills/ -->
|
||
## Project Skills
|
||
|
||
No project skills found. Add skills to any of: `.OpenCode/skills/`, `.agents/skills/`, `.cursor/skills/`, or `.github/skills/` with a `SKILL.md` index file.
|
||
<!-- GSD:skills-end -->
|
||
|
||
<!-- GSD:workflow-start source:GSD defaults -->
|
||
## GSD Workflow Enforcement
|
||
|
||
Before using edit, write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
|
||
|
||
Use these entry points:
|
||
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
|
||
- `/gsd-debug` for investigation and bug fixing
|
||
- `/gsd-execute-phase` for planned phase work
|
||
|
||
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
|
||
<!-- GSD:workflow-end -->
|
||
|
||
|
||
|
||
<!-- GSD:profile-start -->
|
||
## Developer Profile
|
||
|
||
> Profile not yet configured. Run `/gsd-profile-user` to generate your developer profile.
|
||
> This section is managed by `generate-OpenCode-profile` -- do not edit manually.
|
||
<!-- GSD:profile-end -->
|