0.6.0 training session: Oracle Bot, RL combat, Mind's Eye, multilingual pipeline

Major changes from this session: Training: - 0.6.0 training running: 9B on steel141 3090 Ti, 27B on rented H100 NVL - 7,256 merged training examples (up from 3,183) - New training data: failure modes (85), midloop messaging (27), prompt injection defense (29), personality (32), gold from quarantine bank (232), new tool examples (30), claude's own experience (10) - All training data RCON-validated at 100% pass rate - Bake-off: gemma3:27b 66%, qwen3.5:27b 61%, translategemma:27b 56% Oracle Bot (Mind's Eye): - Invisible spectator bot (mineflayer) streams world state via WebSocket - HTML5 Canvas frontend at mind.mortdec.ai - Real-time tool trace visualization with expandable entries - Streaming model tokens during inference - Gateway integration: fire-and-forget POST /trace on every tool call Reinforcement Learning: - Gymnasium environment wrapping mineflayer bot (minecraft_env.py) - PPO training via Stable Baselines3 (10K param policy network) - Behavioral cloning pretraining (97.5% accuracy on expert policy) - Infinite training loop with auto-restart and checkpoint resume - Bot learns combat, survival, navigation from raw experience Bot Army: - 8-soldier marching formation with autonomous combat - Combat bots using mineflayer-pvp, pathfinder, armor-manager - Multilingual prayer bots via translategemma:27b (18 languages) - Frame-based AI architecture: LLM planner + reactive micro-scripts Infrastructure: - Fixed mattpc.sethpc.xyz billing gateway (API key + player list parser) - Billing gateway now tracks all LAN traffic (LAN auto-auth) - Gateway fallback for empty god-mode responses - Updated mortdec.ai landing page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 20:22:50 -04:00
parent baab24f8b1
commit 5b28002001
44 changed files with 20873 additions and 4352 deletions
@@ -0,0 +1,67 @@
+# Oracle Bot — Mortdecai Mind's Eye
+
+Live HTML5 viewport that renders what the Mortdecai AI model "sees" during Minecraft server interactions.
+
+**Public URL:** `mind.mortdec.ai`
+
+## Quick Start (local dev)
+
+```bash
+cd oracle-bot
+npm install
+node server.js
+# Open http://localhost:3333
+```
+
+## Deploy to CT 644
+
+```bash
+# 1. Copy files to CT 644
+ssh pve112 "pct exec 644 -- mkdir -p /opt/oracle-bot"
+scp -r oracle-bot/* pve112:/tmp/oracle-bot/
+ssh pve112 "pct push 644 /tmp/oracle-bot/ /opt/oracle-bot/ --recursive"
+
+# 2. Install deps on CT 644
+ssh pve112 "pct exec 644 -- bash -c 'cd /opt/oracle-bot && npm install --production'"
+
+# 3. Install systemd service
+ssh pve112 "pct exec 644 -- cp /opt/oracle-bot/oracle-bot.service /etc/systemd/system/"
+ssh pve112 "pct exec 644 -- systemctl daemon-reload"
+ssh pve112 "pct exec 644 -- systemctl enable --now oracle-bot"
+
+# 4. Verify
+ssh pve112 "pct exec 644 -- curl -s http://localhost:3333/health"
+```
+
+## Caddy Config (CT 600)
+
+Add to `/etc/caddy/Caddyfile`:
+
+```
+mind.mortdec.ai {
+    @internal path /trace /command
+    respond @internal 404
+
+    reverse_proxy 192.168.0.244:3333
+}
+```
+
+Then: `caddy reload --config /etc/caddy/Caddyfile`
+
+## Endpoints
+
+| Endpoint | Access | Description |
+|----------|--------|-------------|
+| `GET /` | Public | Frontend (index.html) |
+| `WS /ws` | Public | WebSocket stream |
+| `GET /health` | Public | Health check |
+| `POST /trace` | Internal | Gateway tool traces |
+| `POST /command` | Internal | Bot commands |
+
+## Test with simulated traces
+
+```bash
+curl -X POST http://localhost:3333/trace \
+  -H "Content-Type: application/json" \
+  -d '{"tool":"rcon.execute","input":{"command":"give slingshooter08 diamond 16"},"ok":true,"step":0,"mode":"god","player":"slingshooter08","session_id":"test-001"}'
+```
@@ -221,11 +221,9 @@ class OracleBot extends EventEmitter {
        `gamemode=${bot.game.gameMode}`
      );

-      // If not already spectator, become spectator via server command
-      if (bot.game.gameMode !== 'spectator') {
-        console.log('[OracleBot] Not in spectator mode — requesting /gamemode spectator');
-        bot.chat('/gamemode spectator');
-      }
+      // Note: Don't try to change gamemode — Paper kicks offline-mode bots on gamemode change.
+      // Bot operates in whatever mode the server assigns. Spectator can be set via
+      // essentials auto-gamemode config or a join-event plugin if needed.

      this.emit('spawned', {
        position: bot.entity.position,
@@ -249,13 +247,16 @@ class OracleBot extends EventEmitter {
    });

    bot.on('kicked', (reason) => {
-      let reasonText = reason;
-      try {
-        // reason may be a JSON chat component
-        const parsed = JSON.parse(reason);
-        reasonText = parsed.text || reason;
-      } catch (_) {
-        // use raw string
+      let reasonText;
+      if (typeof reason === 'object') {
+        reasonText = JSON.stringify(reason);
+      } else {
+        try {
+          const parsed = JSON.parse(reason);
+          reasonText = parsed.text || parsed.translate || JSON.stringify(parsed);
+        } catch (_) {
+          reasonText = String(reason);
+        }
      }
      console.warn(`[OracleBot] Kicked: ${reasonText}`);
      this.emit('disconnected', { reason: 'kicked', message: reasonText });
@@ -273,7 +274,8 @@ class OracleBot extends EventEmitter {
  _scheduleReconnect() {
    if (this._destroyed) return;

-    const delay = this._reconnectDelay;
+    // Minimum 5s delay to let server fully drop old session (avoids duplicate_login kick)
+    const delay = Math.max(this._reconnectDelay, 5000);
    console.log(`[OracleBot] Reconnecting in ${delay}ms ...`);

    this._reconnectTimer = setTimeout(() => {
@@ -0,0 +1,18 @@
+[Unit]
+Description=Oracle Bot — Mortdecai Mind's Eye
+After=network.target
+
+[Service]
+Type=simple
+WorkingDirectory=/opt/oracle-bot
+ExecStart=/usr/bin/node server.js
+Restart=always
+RestartSec=5
+Environment=PORT=3333
+Environment=MC_HOST=192.168.0.244
+Environment=MC_PORT=25568
+StandardOutput=append:/var/log/oracle-bot.log
+StandardError=append:/var/log/oracle-bot.log
+
+[Install]
+WantedBy=multi-user.target
@@ -33,7 +33,7 @@ html, body {

 #app {
  display: grid;
-  grid-template-columns: 1fr 280px;
+  grid-template-columns: 1fr 380px;
  grid-template-rows: 1fr 56px;
  width: 100vw; height: 100vh;
 }
@@ -106,6 +106,41 @@ html, body {
  flex-shrink: 0;
 }

+#stream-box {
+  padding: 10px 16px;
+  border-bottom: 1px solid var(--border);
+  background: rgba(211,84,0,0.05);
+  max-height: 180px;
+  overflow-y: auto;
+  flex-shrink: 0;
+}
+
+#stream-box.hidden { display: none; }
+
+#stream-box.sudo { background: rgba(33,150,243,0.05); }
+
+#stream-label {
+  font-size: 10px;
+  font-weight: 700;
+  letter-spacing: 1px;
+  color: var(--orange);
+  margin-bottom: 6px;
+  animation: pulse 1.5s ease-in-out infinite;
+}
+
+#stream-box.sudo #stream-label { color: var(--blue); }
+
+@keyframes pulse { 0%,100% { opacity: 1; } 50% { opacity: 0.5; } }
+
+#stream-text {
+  font-family: monospace;
+  font-size: 12px;
+  line-height: 1.4;
+  color: var(--text);
+  white-space: pre-wrap;
+  word-break: break-all;
+}
+
 #trace-list {
  flex: 1;
  overflow-y: auto;
@@ -118,14 +153,87 @@ html, body {

 .trace-entry {
  display: flex;
-  align-items: center;
-  gap: 10px;
-  padding: 8px 16px;
+  flex-direction: column;
+  gap: 4px;
+  padding: 10px 16px;
  font-size: 13px;
  border-bottom: 1px solid rgba(42,42,74,0.5);
  animation: traceIn 0.3s ease-out;
 }

+.trace-header-row {
+  display: flex;
+  align-items: center;
+  gap: 8px;
+}
+
+.trace-details {
+  display: none;
+  padding-left: 18px;
+  padding-top: 4px;
+}
+
+.trace-entry.expanded .trace-details {
+  display: block;
+}
+
+.trace-header-row {
+  cursor: pointer;
+}
+
+.trace-header-row:hover .trace-tool {
+  text-decoration: underline;
+}
+
+.trace-expand {
+  font-size: 10px;
+  color: var(--text-dim);
+  margin-left: auto;
+  transition: transform 0.2s;
+}
+
+.trace-entry.expanded .trace-expand {
+  transform: rotate(90deg);
+}
+
+.trace-preview {
+  font-size: 11px;
+  color: var(--text-dim);
+  padding-left: 18px;
+  white-space: nowrap;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  max-width: 100%;
+}
+
+.trace-detail {
+  font-size: 11px;
+  color: var(--text-dim);
+  word-break: break-all;
+  line-height: 1.5;
+  padding: 2px 0;
+}
+
+.trace-detail.command {
+  color: var(--orange);
+  font-family: monospace;
+  font-size: 12px;
+}
+
+.trace-detail.result {
+  color: var(--green);
+  font-family: monospace;
+}
+
+.trace-detail.result.fail {
+  color: var(--red);
+}
+
+.trace-detail.message {
+  color: #FFD54F;
+  font-style: italic;
+}
+
@keyframes traceIn {
  from { opacity: 0; transform: translateX(20px); }
  to { opacity: 1; transform: translateX(0); }
@@ -313,6 +421,10 @@ html, body {
  <!-- Tool Trace -->
  <div id="trace-panel">
    <div id="trace-header">TOOL TRACE</div>
+    <div id="stream-box" class="hidden">
+      <div id="stream-label">MODEL THINKING...</div>
+      <div id="stream-text"></div>
+    </div>
    <div id="trace-list"></div>
    <div id="trace-footer">No active session</div>
  </div>
@@ -654,6 +766,12 @@ function addTrace(data) {
  const entry = document.createElement('div');
  entry.className = 'trace-entry';

+  const input = data.input || {};
+
+  // Header row: dot + tool + step + time + expand arrow
+  const headerRow = document.createElement('div');
+  headerRow.className = 'trace-header-row';
+
  const dot = document.createElement('span');
  dot.className = 'trace-dot ' + (data.ok !== false ? 'ok' : 'fail');

@@ -670,10 +788,118 @@ function addTrace(data) {
  const d = data.ts ? new Date(data.ts) : new Date();
  time.textContent = d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', second: '2-digit' });

-  entry.appendChild(dot);
-  entry.appendChild(tool);
-  entry.appendChild(step);
-  entry.appendChild(time);
+  const expand = document.createElement('span');
+  expand.className = 'trace-expand';
+  expand.textContent = '▶';
+
+  headerRow.appendChild(dot);
+  headerRow.appendChild(tool);
+  headerRow.appendChild(step);
+  headerRow.appendChild(time);
+  headerRow.appendChild(expand);
+  entry.appendChild(headerRow);
+
+  // Preview line (always visible, one-line summary)
+  const preview = document.createElement('div');
+  preview.className = 'trace-preview';
+  if (data.tool === 'rcon.execute' && input.command) {
+    preview.textContent = '> ' + input.command;
+  } else if (data.tool === '_direct_response') {
+    const cmds = Array.isArray(input.commands) ? input.commands : [];
+    preview.textContent = cmds.length + ' cmds' + (input.message ? ' + message' : '');
+  } else if (data.tool === '_session_end') {
+    const cmds = Array.isArray(input.commands) ? input.commands : [];
+    preview.textContent = 'Session end: ' + cmds.length + ' cmds' + (input.message ? ' + message' : '');
+  } else {
+    const inputStr = Object.entries(input).map(([k,v]) => `${k}=${v}`).join(', ');
+    preview.textContent = inputStr || (data.player || '');
+  }
+  entry.appendChild(preview);
+
+  // Expandable details container
+  const details = document.createElement('div');
+  details.className = 'trace-details';
+
+  // Player + mode
+  if (data.player) {
+    const d1 = document.createElement('div');
+    d1.className = 'trace-detail';
+    d1.textContent = 'Player: ' + data.player + (data.mode ? ' [' + data.mode.toUpperCase() + ']' : '');
+    details.appendChild(d1);
+  }
+
+  // RCON command + result
+  if (data.tool === 'rcon.execute' && input.command) {
+    const d2 = document.createElement('div');
+    d2.className = 'trace-detail command';
+    d2.textContent = '> ' + input.command;
+    details.appendChild(d2);
+    if (data.result) {
+      const d3 = document.createElement('div');
+      d3.className = 'trace-detail result' + (data.ok === false ? ' fail' : '');
+      d3.textContent = '← ' + data.result;
+      details.appendChild(d3);
+    }
+  }
+
+  // Other tool inputs
+  if (data.tool !== 'rcon.execute' && data.tool !== '_direct_response' && data.tool !== '_session_end') {
+    for (const [k, v] of Object.entries(input)) {
+      const d4 = document.createElement('div');
+      d4.className = 'trace-detail';
+      d4.textContent = k + ': ' + v;
+      details.appendChild(d4);
+    }
+  }
+
+  // Direct response: all commands + message
+  if (data.tool === '_direct_response') {
+    const cmds = Array.isArray(input.commands) ? input.commands : [];
+    for (const cmd of cmds) {
+      const d5 = document.createElement('div');
+      d5.className = 'trace-detail command';
+      d5.textContent = '> ' + cmd;
+      details.appendChild(d5);
+    }
+    if (input.message) {
+      const d6 = document.createElement('div');
+      d6.className = 'trace-detail message';
+      d6.textContent = '"' + String(input.message) + '"';
+      details.appendChild(d6);
+    }
+  }
+
+  // Session end: commands + message
+  if (data.tool === '_session_end') {
+    const cmds = Array.isArray(input.commands) ? input.commands : [];
+    for (const cmd of cmds) {
+      const d7 = document.createElement('div');
+      d7.className = 'trace-detail command';
+      d7.textContent = '> ' + cmd;
+      details.appendChild(d7);
+    }
+    if (input.message) {
+      const d8 = document.createElement('div');
+      d8.className = 'trace-detail message';
+      d8.textContent = '"' + String(input.message) + '"';
+      details.appendChild(d8);
+    }
+  }
+
+  // Session ID
+  if (data.session_id) {
+    const d9 = document.createElement('div');
+    d9.className = 'trace-detail';
+    d9.textContent = 'Session: ' + data.session_id;
+    details.appendChild(d9);
+  }
+
+  entry.appendChild(details);
+
+  // Click to expand/collapse
+  headerRow.addEventListener('click', () => {
+    entry.classList.toggle('expanded');
+  });

  traceList.insertBefore(entry, traceList.firstChild);

@@ -689,6 +915,35 @@ function addTrace(data) {
  traceFooter.textContent = `${modeLabel} | ${playerLabel} | step ${data.step || '?'}`;
 }

+// ── Streaming display ──
+const streamBox = document.getElementById('stream-box');
+const streamText = document.getElementById('stream-text');
+const streamLabel = document.getElementById('stream-label');
+let streamTimeout = null;
+
+function handleStream(msg) {
+  // Show the stream box
+  streamBox.classList.remove('hidden');
+  streamBox.classList.toggle('sudo', msg.mode === 'sudo');
+
+  // Update the streaming text with accumulated content
+  if (msg.accumulated) {
+    streamText.textContent = msg.accumulated;
+    // Auto-scroll to bottom
+    streamBox.scrollTop = streamBox.scrollHeight;
+  }
+
+  // Reset the hide timeout (hide 3s after last token)
+  clearTimeout(streamTimeout);
+  streamTimeout = setTimeout(hideStream, 3000);
+}
+
+function hideStream() {
+  streamBox.classList.add('hidden');
+  streamText.textContent = '';
+  clearTimeout(streamTimeout);
+}
+
 // ── WebSocket ──
 let ws = null;
 let reconnectDelay = 1000;
@@ -766,9 +1021,14 @@ function connectWS() {
        updateStatus();
        break;

+      case 'stream':
+        handleStream(msg);
+        break;
+
      case 'mode':
        state.mode = msg.mode || 'idle';
        if (msg.player) state.activePlayer = msg.player;
+        if (msg.mode === 'idle') hideStream();
        state.dirty = true;
        updateStatus();
        break;
@@ -80,6 +80,27 @@ app.post('/trace', (req, res) => {
  return res.json({ ok: true });
 });

+// ── POST /stream ──────────────────────────────
+
+app.post('/stream', (req, res) => {
+  const data = req.body;
+  if (!data) return res.status(400).json({ ok: false });
+
+  // Broadcast stream token to all WebSocket clients
+  broadcast({
+    v: 1,
+    type: 'stream',
+    token: data.token || '',
+    accumulated: data.accumulated || '',
+    step: data.step,
+    mode: data.mode || state.mode,
+    player: data.player || state.activePlayer,
+    ts: Date.now(),
+  });
+
+  res.json({ ok: true });
+});
+
 // ── POST /command ─────────────────────────────

 app.post('/command', (req, res) => {
@@ -235,6 +256,10 @@ bot.on('connected', () => {
  broadcast(state.buildStatus(true));
 });

+bot.on('spawned', () => {
+  // Spectator is handled by bot.js internally (self-sets via chat after 2s delay)
+});
+
 bot.on('disconnected', () => {
  broadcast(state.buildStatus(false));
 });