diff --git a/README.md b/README.md new file mode 100644 index 0000000..2b7eabf --- /dev/null +++ b/README.md @@ -0,0 +1,243 @@ +# Unified Media Manager (UMM) + +> One platform to rule them all — replaces 12 Docker containers with one unified Go + React media management system. + +[![Build Status](https://github.com/TopherMayor/unified-media-manager/actions/workflows/test.yml/badge.svg)](https://github.com/TopherMayor/unified-media-manager) +[![Go Version](https://img.shields.io/badge/Go-1.25-blue)](https://golang.org/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) + +## Resume Summary + +**Unified Media Manager** — Designed and built a production-grade media management platform that consolidated 12 separate Docker containers (Sonarr, Radarr, Lidarr, Readarr, Prowlarr, Jellyseerr, Bazarr, Recyclarr, and their anime variants) into a single Go + React service. + +- Reduced infrastructure from **12 containers to 1**, eliminating 11 redundant databases, 11 log schemas, and duplicated configuration +- Built a multi-protocol indexer engine supporting Torznab/Newznab XML and Cardigann YAML definitions for compatibility with 50+ indexers +- Implemented hardlink-based acquisition so downloads and library files share blocks on disk (zero extra storage) +- Designed a PostgreSQL schema with 24 tables using partitioning, JSONB, and GIN indexes — replacing 82+ tables from the arr stack +- Built a full-stack React UI with 12 pages: Dashboard, Library, Search, Discover, Queue, Activity, Blocklist, Calendar, Requests, Settings, MediaDetail, and Semantic Search +- Implemented AI-powered semantic search using Qdrant vector database and Ollama embeddings for natural-language media discovery +- Tech stack: Go 1.25, Echo v4, PostgreSQL 16, React 18 + TypeScript + TailwindCSS, Qdrant, Ollama, Docker + +## What It Does + +UMM is a self-hosted media acquisition and management platform. You add movies, TV shows, music, books, or podcasts you want to watch/read/listen to, and UMM automatically: + +1. **Searches** your configured indexers (via Cardigann or Torznab) for matching releases +2. **Evaluates** quality, seeders, and snatch speed to pick the best release +3. **Downloads** via qBittorrent or SABnzbd — routed through a VPN (Gluetun) for privacy +4. **Imports** completed downloads to your library using hardlinks (no extra disk space) +5. **Enriches** metadata from TMDB, TVDB, MusicBrainz, and OpenLibrary +6. **Tracks** everything in PostgreSQL with full-text search across your entire library +7. **Notifies** you via webhook/Telegram when media is ready +8. **Searches semantically** — "quirky indie documentaries about AI" returns relevant results + +## Architecture + +``` + Internet + │ + ┌──────┴──────┐ + │ Traefik │ TLS termination (existing infra) + └──────┬──────┘ + │ + ┌─────────────┴─────────────┐ + │ │ + ┌──────┴──────┐ ┌───────┴───────┐ + │ Nginx │ │ React UI │ + │ :3000 │ │ (SPA) │ + └──────┬──────┘ └───────▲────────┘ + │ │ + │ /api/* proxy_pass │ fetch() + │ │ + ┌─────────▼──────────────────────────┴───────┐ + │ Go Backend :8084 │ + │ │ + │ ┌──────────┐ ┌────────────┐ ┌────────┐│ + │ │ REST API │ │ Workers │ │Schedu- ││ + │ │ (Echo v4)│ │ (queuemgr) │ │ler(cron││ + │ └────┬─────┘ └─────┬──────┘ └────────┘│ + │ │ │ │ + └───────┼──────────────┼─────────────────────┘ + │ │ + ┌─────────┼──────────────┼────────────────────────┐ + │ │ │ UM M Network │ + │ ┌──────▼──────┐ ┌────▼─────┐ ┌─────────────┐│ + │ │ PostgreSQL │ │ Qdrant │ │ Ollama ││ + │ │ :5432 │ │ :6333 │ │ :11434 ││ + │ │ (primary) │ │ (vectors)│ │ (AI/LLM) ││ + │ └─────────────┘ └───────────┘ └─────────────┘│ + │ │ + │ ┌──────────────────────────────────────────┐ │ + │ │ Gluetun VPN Network │ │ + │ │ SABnzbd :8080 │ qBittorrent :8085 │ │ + │ └──────────────────────────────────────────┘ │ + └─────────────────────────────────────────────────┘ + +Media files flow: + Indexers → UMM (evaluate) → Download Client (via VPN) + → Import (hardlink) → Library (NFS share) → Jellyfin/Navidrome/Calibre +``` + +## Key Features + +| Feature | Description | +|---------|-------------| +| **Unified Search** | Search across all media types (movies, TV, music, books, podcasts) in a single query | +| **Indexer Integration** | Cardigann YAML + Torznab/Newznab XML support — works with 50+ indexers | +| **Smart Downloads** | Quality-first release selection with seeder/speed scoring | +| **Hardlinks** | Completed downloads hard-linked to library — no extra storage | +| **Quality Tracking** | Current vs. desired quality per item with auto-upgrade rules | +| **Request System** | Built-in request/approval workflow — no separate Jellyseerr needed | +| **Subtitle Search** | Automatic subtitle search across multiple sources | +| **RSS Sync** | Monitor indexer RSS feeds for new releases automatically | +| **Activity Feed** | Real-time event log: searches, downloads, imports, failures | +| **Blocklist** | Block specific releases or indexers to prevent re-snatch | +| **Semantic Search** | Natural language queries ("dark comedies from the 90s") via Qdrant + Ollama | +| **Calendar View** | See upcoming releases by air date or your watchlist | +| **Analytics Dashboard** | Disk usage, download stats, library breakdown by media type | + +## Tech Stack + +| Layer | Technology | +|-------|-----------| +| Backend | Go 1.25 (Echo v4 HTTP framework) | +| Frontend | React 18 + TypeScript + TailwindCSS + Vite | +| Database | PostgreSQL 16 (partitioned tables, JSONB, GIN indexes) | +| Vector DB | Qdrant (semantic search) | +| AI | Ollama (nomic-embed-text embeddings, LLaVA vision) | +| Download | qBittorrent, SABnzbd (via VPN tunnel) | +| Indexers | Cardigann YAML + Torznab/Newznab XML | +| Container | Docker + Docker Compose | +| Reverse Proxy | Traefik (existing) | + +## Getting Started + +### Prerequisites + +- Docker + Docker Compose v3.8 +- PostgreSQL 16 instance (or use the Docker default) +- qBittorrent or SABnzbd (with VPN via Gluetun recommended) +- TMDB API key (free at https://www.themoviedb.org/) + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/TopherMayor/unified-media-manager.git +cd unified-media-manager + +# Copy and edit environment +cp .env.example .env +$EDITOR .env + +# Start the stack +docker compose up -d + +# Run database migrations +docker compose exec umm /umm migrate + +# Open the UI +open http://umm.local.tophermayor.com +``` + +### Configuration + +Environment variables (see `.env.example`): + +```bash +DATABASE_URL=postgres://user:pass@postgres-shared:5432/unified_media_manager?sslmode=disable +QDRANT_URL=http://qdrant:6333 +OLLAMA_URL=http://ollama:11434 +TMDB_API_KEY=your_tmdb_api_key +PORT=8084 +``` + +### Docker Compose + +The stack uses Traefik for routing. Add this label to your `docker-compose.yml`: + +```yaml +labels: + - "traefik.enable=true" + - "traefik.http.routers.umm.rule=Host(`umm.local.tophermayor.com`)" + - "traefik.http.routers.umm.tls=true" +``` + +## Project Structure + +``` +unified-media-manager/ +├── cmd/ +│ └── migrate/ # Database migration CLI +├── internal/ +│ ├── api/ # Echo HTTP handlers (12 resource files) +│ │ ├── activity.go +│ │ ├── blocklist.go +│ │ ├── calendar.go +│ │ ├── dashboard.go +│ │ ├── discover.go +│ │ ├── download_clients.go +│ │ ├── health.go +│ │ ├── import.go +│ │ ├── indexers.go +│ │ ├── media.go +│ │ ├── metadata.go +│ │ ├── notifications.go +│ │ ├── quality.go +│ │ ├── queue.go +│ │ ├── requests.go +│ │ ├── root_folder.go +│ │ ├── router.go # Echo router setup +│ │ ├── search.go +│ │ ├── subtitle.go +│ │ ├── tag.go +│ │ └── workers.go +│ ├── cardigann/ # Cardigann YAML indexer engine +│ ├── config/ # Environment config loading +│ ├── db/ # PostgreSQL pool + migrations +│ │ └── migrations/ # 13 SQL migration files +│ ├── download/ # qBittorrent + SABnzbd clients +│ ├── migrate/ # arr migration tool (import from Sonarr/Radarr) +│ ├── service/ # Business logic (30+ files) +│ └── worker/ # Background workers (scheduler, queue, scanner...) +├── frontend/ +│ └── src/ +│ ├── api/ # TypeScript API client +│ ├── components/ # Reusable UI components +│ └── pages/ # 12 page components +├── scripts/ +│ └── migrate-arrs.sh # Batch import from arr stack +├── docker-compose.yml +├── SPEC.md # Full technical specification +└── AGENTS.md # AI agent coding conventions +``` + +## API Endpoints (selected) + +| Method | Path | Description | +|--------|------|-------------| +| `GET` | `/api/search?q=...&type=...` | Full-text search across library | +| `GET` | `/api/search/semantic?q=...&k=5` | Semantic search via Qdrant + Ollama | +| `GET` | `/api/media` | Paginated media list with filters | +| `GET` | `/api/media/:id` | Media detail with releases | +| `POST` | `/api/media/:id/download` | Trigger download for specific release | +| `GET` | `/api/queue` | Active download queue | +| `DELETE` | `/api/queue/:id` | Cancel queued download | +| `GET` | `/api/discover/recommended` | AI-recommended media | +| `POST` | `/api/indexers/search` | Search all configured indexers | +| `GET` | `/api/activity` | Recent activity events | +| `GET` | `/api/dashboard/stats` | Dashboard metrics | +| `POST` | `/api/requests` | Create media request | +| `GET` | `/api/health` | Health check | + +## Contributing + +1. Fork the repository +2. Create a feature branch (`git checkout -b feature/amazing-feature`) +3. Commit your changes (`git commit -m 'Add amazing feature'`) +4. Push to the branch (`git push origin feature/amazing-feature`) +5. Open a Pull Request + +## License + +MIT License — see [LICENSE](LICENSE) for details.