Fully automated setup: downloads GGUF, loads model, tests inference

Setup script now:
1. Generates API key
2. Starts Docker containers
3. Downloads GGUF from mortdec.ai automatically (~5.3GB)
4. Creates Ollama model with correct chat template
5. Runs test inference
6. Prints connection details for Seth

Matt just runs ./setup.sh — no manual file copying.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-20 19:33:39 -04:00
parent c5865feb35
commit df9f623943
2 changed files with 75 additions and 30 deletions
+1 -1
View File
@@ -22,7 +22,7 @@ services:
- "127.0.0.1:11434:11434" # Only accessible to gateway, not exposed
volumes:
- ollama-data:/root/.ollama
- ./models:/models
- ./models:/models:ro
devices:
- /dev/kfd:/dev/kfd
- /dev/dri:/dev/dri