Mortdecai/docs/superpowers/specs/2026-03-22-oracle-bot-design.md

Oracle Bot — Mortdecai Mind's Eye

Date: 2026-03-22 Status: Approved design, pending implementation Public URL: mind.mortdec.ai

Summary

A live HTML5 viewport that renders what the Mortdecai AI model "sees" during Minecraft server interactions. An invisible spectator bot (mineflayer) maintains real-time world state, the gateway streams tool traces to it, and browsers connect via WebSocket to watch the AI think.

Architecture

Browser (mind.mortdec.ai)
    ↕ WebSocket (ws://CT644:3333)
Oracle Bot (Node.js, single process)
    ├── MC Client (mineflayer, spectator mode)
    ├── Vision Server (Express + ws, port 3333)
    ├── Trace Receiver (POST /trace from gateway)
    └── Command API (POST /command, future tool integration)
        ↕ MC Protocol (offline auth)
Paper Server (1.21, CT 644:25568 dev)

Approach: Smart Bot is the Server (Approach 2). One Node.js process handles MC connection, WebSocket streaming, and HTTP endpoints. Designed to evolve into a gateway tool (Approach 3) where the AI controls the bot directly.

Bot Core

Three roles in one process:

1. MC Client — mineflayer bot in spectator mode. Maintains live chunk cache, entity list, player positions. Username: OracleBot. Connects to dev server (port 25568, offline auth).

2. Vision Server — Express + ws. Serves the HTML5 frontend. Streams world state and tool traces to connected browsers via WebSocket on port 3333.

3. Trace Receiver — POST /trace endpoint. Gateway calls this (fire-and-forget) on every tool invocation during the model-driven tool loop.

Future-ready command API:

• POST /command — accepts instructions: {action: "follow", target: "player"}, {action: "scan", center: {x,y,z}, radius: 20}
• Day one: only follow and scan implemented
• Endpoint exists so the gateway can later call it as a tool (oracle.scan, oracle.look)

Bot behavior:

• On connect: spectator mode, fly to first online player
• Follows the player the AI is currently interacting with (switches on trace events)
• On idle: parks at last active player or world spawn

Data Flow & States

Two modes:

Idle Mode (no active trace):

• Bot parks at last active player position
• Low-frequency heartbeat to browsers: player list, positions, time, weather
• Update rate: ~5 seconds
• Frontend: calm ambient view, slow-updating minimap, player dots

Active Mode (trace incoming):

• Gateway fires POST /trace with tool call data
• Bot teleports to relevant player
• Scans burst of chunk data around player
• High-frequency updates: blocks, entities, tool trace overlay
• God mode: dramatic visual (golden glow, Sethian orange accents)
• Sudo mode: clinical/technical (grid overlay, command syntax)
• Persists 10s after last trace, then fades to idle

WebSocket message types:

// Heartbeat (idle)
{type: "heartbeat", players: [{name, x, y, z}], time: 6000, weather: "clear"}

// World snapshot (active)
{type: "world", center: {x, y, z}, blocks: [{x, y, z, type}], entities: [{type, x, y, z, count}]}

// Tool trace event (active)
{type: "trace", tool: "world.scan_area", input: {...}, result: {...}, step: 2, mode: "god"}

// Mode change
{type: "mode", mode: "god"|"sudo"|"idle", player: "slingshooter08"}

Frontend (HTML5 Canvas)

Single page, no build step. Pure HTML5 Canvas + vanilla JS.

Layout:

┌─────────────────────────────────┬──────────────────┐
│                                 │   TOOL TRACE     │
│         WORLD MAP               │                  │
│     (2D top-down tiles)         │  [scan_area]  ●  │
│                                 │  [rcon.exec]  ●  │
│     ○ player dots               │  [journal]    ●  │
│     █ blocks colored by type    │                  │
│     ◇ entities                  │  step 3/8        │
│                                 │                  │
├─────────────────────────────────┤                  │
│  STATUS BAR                     │                  │
│  Mode: GOD | Player: sling...   │                  │
│  HP: 20 | Pos: (12, -60, 15)   │                  │
└─────────────────────────────────┴──────────────────┘

Visual modes:

• Idle: Dark muted palette, slow pulse animation. Sleeping eye aesthetic.
• God active: Sethian orange (#D35400), golden particles on commands, dramatic god message text. Blocks glow where AI acts.
• Sudo active: Cool blue/green terminal aesthetic, monospace overlays, precise grid. Clinical.

Block rendering:

• Each block type → color (stone=gray, dirt=brown, water=blue, redstone=red, air=transparent)
• Top-down slice at player Y level (configurable)
• Entities as icons/dots with distance rings
• Scanned areas pulse/highlight as tool traces arrive ("AI is looking here")

Branding:

• Font: Rajdhani Bold
• Primary accent: Sethian orange (#D35400)
• Background: dark (#1a1a2e)
• Title: "MORTDECAI — MIND'S EYE"
• Subtle eye/pyramid motif

Security & Resilience

Public vs internal endpoints:

• Public (via Caddy): WebSocket /ws, static files /, /index.html
• Internal only (localhost): POST /trace, POST /command — Caddy must NOT proxy these. Gateway calls them on localhost:3333 directly.
• WebSocket: max 100 concurrent connections, per-IP cap of 5. Excess connections get 429.

Caddy config:

mind.mortdec.ai {
    reverse_proxy /ws localhost:3333
    reverse_proxy / localhost:3333 {
        # Only serve static files and WebSocket, not /trace or /command
    }
    @blocked path /trace /command
    respond @blocked 404
}

Chunk loading after teleport:

• After bot teleports to a player, wait 2 seconds for chunk packets before scanning
• world-state.js tracks chunk load events and exposes awaitChunksLoaded(center, radius, timeoutMs)
• If timeout expires, scan with whatever chunks are loaded (partial data is better than no data)

Spectator mode enforcement:

• Add to mc_aigod_paper.py PlayerJoinEvent: if player name is OracleBot, set gamemode spectator before spawn
• Fallback: bot self-executes /gamemode spectator OracleBot via chat on spawn event

Bot reconnection:

• On kicked or end event: exponential backoff reconnect (1s, 2s, 4s, 8s, max 30s)
• Broadcast {type: "status", connected: false} to all browsers on disconnect
• Frontend shows "Bot offline — reconnecting..." overlay with pulse animation
• On reconnect: broadcast {type: "status", connected: true}, resume normal flow

Payload limits:

• World snapshots: max 32x32x1 top-down slice (1,024 blocks). Air blocks excluded.
• Delta compression: after initial snapshot, only send changed blocks
• Max WebSocket frame: 64KB. If payload exceeds, chunk into multiple messages.

Multiple simultaneous sessions:

• Trace events include session_id and player fields
• Bot follows the most recent trace's player
• Frontend tool trace panel shows all active sessions, color-coded by player
• If two sessions overlap, traces interleave in the timeline (both visible)

Message versioning:

• All WebSocket messages include v: 1 field
• Frontend ignores messages with unknown v values gracefully

Deployment

Location: CT 644 (same container as MC servers + gateway). Lowest latency.

Public access:

mind.mortdec.ai → Caddy (CT 600) → CT 644:3333 (WebSocket upgrade)

• No Authelia — fully public
• DNS: CNAME to Caddy ingress

Gateway integration (minimal):

• One addition to langgraph_gateway.py: fire-and-forget POST to http://localhost:3333/trace after each tool call
• Also POST on session start (mode + player) and session end (final response)
• Non-blocking: try/except with 1s timeout. If bot is down, gateway doesn't care.

Process management:

• systemd service: oracle-bot.service
• Auto-restart on crash
• Logs: /var/log/oracle-bot.log

Server-side setup:

• Paper server needs to /gamemode spectator OracleBot on join (command block or plugin event)

Future Evolution

Phase 2: Gateway tool integration

• oracle.scan tool — model queries bot's chunk cache instead of RCON. Faster, richer.
• oracle.look tool — bot teleports to coords and returns what it sees.
• POST /command endpoint (built day one) becomes the tool backend.

Phase 3: Multi-bot fleet

• Oracle spawns additional bots on command
• Model dispatches: oracle.dispatch({task: "watch", target: "SwiftWolf"})
• All bots feed into same vision server → same frontend

Phase 4: Multimodal training capture

• Frontend frames captured as screenshots paired with model decisions
• Builds (visual_state, model_action) dataset for multimodal fine-tuning
• Mind's Eye becomes training data for visual understanding

Tech Stack

• Bot + Server: Node.js, mineflayer, express, ws
• Frontend: HTML5 Canvas, vanilla JS, WebSocket API
• Integration: HTTP POST (gateway → bot), WebSocket (bot → browser)
• Deployment: systemd on CT 644, Caddy reverse proxy

File Structure

oracle-bot/
├── package.json        # entrypoint: "main": "server.js"
├── server.js           # ENTRYPOINT — express + ws, requires bot.js, serves frontend
├── bot.js              # mineflayer connection, spectator, chunk tracking
├── world-state.js      # abstracted world state (blocks, entities, players)
├── public/
│   └── index.html      # single-file frontend (HTML + Canvas + JS + CSS)
└── README.md