AI Rack
AI-powered live coding music environment integrating Strudel, Hydra visuals, and Claude AI into a unified creative performance tool.
Overview
AI Rack is an AI-powered live coding music environment that brings together three creative systems into a cohesive 4-panel application: Strudel for functional live music patterns, Hydra for GPU-accelerated reactive visuals, and Claude as an AI collaborator that can generate, transform, and explain music in real time. The app runs as both a web application and a cross-platform Electron desktop app.
The core idea is that you can describe what you want musically in natural language, and the AI generates working Strudel code that plays immediately — or you can code patterns by hand and let the AI riff on them, shift moods, adjust energy, or layer in complementary parts.
Architecture
The frontend is built with React 18 + TypeScript and bundled with Vite 6. State management uses Zustand across four stores (pattern, chat, audio, viz), chosen for its minimal boilerplate and ability to be accessed from imperative audio callbacks outside the React tree. The code editor is CodeMirror 6 — the same editor Strudel uses internally — with custom syntax highlighting and per-tab undo history.
The backend is an Express + TypeScript server that proxies Claude API calls via SSE streaming and hosts a music generation engine. In Electron mode, the Express server runs as a child process alongside the renderer.
Audio runs through the full Strudel runtime (@strudel/web), with pre-warmed AudioWorklets and sample loading from both a local WAV library and CDN-hosted packs (Dirt-Samples, Tidal Drum Machines, EmuSP12, and others). Real-time audio analysis feeds through a Meyda feature extractor connected via an AudioNode tap, driving beat-reactive CSS custom properties that animate the entire UI.
Key Features
Multi-Tab Code Editor with Mix System
A tabbed CodeMirror editor where each tab holds an independent pattern. The MixBar lets you arm multiple tabs for simultaneous playback — armed tabs are concatenated with variable collision detection. Active notes flash green in the editor in real time via Strudel’s location tracking.
AI Chat with 10 Music Tools
Claude has access to specialized tool calls: update_pattern (inject code into the REPL), generate_drums / generate_bassline / generate_melody (server-side generation with genre and style parameters), shift_mood (transform patterns across 7 moods like dark, euphoric, dreamy), and visualization tools for both Canvas 2D and Hydra shaders.
Server-Side Music Engine
A music theory and pattern generation system built in TypeScript covering 17 scales, 9 chord progression styles, drum patterns for 9 genres at multiple complexity levels, bassline styles, euclidean rhythms, and transforms for mood shifting and energy scaling (0–10).
Dual-Mode Visualizations
Events Mode draws Canvas 2D graphics driven by Strudel sound events — kicks flash red, snares cyan, hats yellow. Hydra Mode runs GPU shader programs that react to audio features (RMS, spectral centroid, beat detection) via window.audio bindings, with 12+ built-in presets.
Beat-Reactive UI
Real-time audio analysis (RMS, spectral centroid, beat detection) drives CSS custom properties (--beat-intensity, --rms) through a requestAnimationFrame loop, creating pulsing borders, text glow, and background animations synced to the music. A toggleable CRT scanline effect adds retro character.
Knowledge-Augmented AI
The system prompt includes 13 curated knowledge files covering Strudel syntax, available samples, genre templates, advanced techniques, common gotchas, and Hydra shader patterns. These are injected as cached content to keep the AI informed without burning context on every request.
Design Decisions
Error recovery keeps the previous pattern playing if a new evaluation fails — no silence mid-performance. AI-triggered errors get up to 2 automatic correction retries before reverting to the last working code.
File operations use the File System Access API for native save/open dialogs with per-tab file handle caching, falling back to download/upload for browsers without support.
Strudel deduplication in the Vite config forces single instances of core Strudel modules to prevent “loaded more than once” runtime errors that break audio scheduling.
Tech Stack
- Frontend: React 18, TypeScript, Vite 6, CodeMirror 6, Zustand 5, Framer Motion, React Virtuoso
- Backend: Express 4, TypeScript, Anthropic SDK (SSE streaming)
- Audio: Strudel runtime, Meyda, Web Audio API
- Visuals: Hydra Synth (WebGL), Canvas 2D
- Desktop: Electron 33 (macOS, Windows, Linux)
- Music Engine: Custom MusicTheory, PatternGenerator, and Transforms modules