Seth 13debc8a59 Add audit log ingestion pipeline with language/leak filtering ...

data/ingest_audit.py:
- Pulls training audit logs from CT 644 (dev + prod)
- Filters: language mismatch (Chinese output for English input), system
  prompt leaks, empty responses, duplicates
- Keeps multilingual examples where input/output languages match
- Converts to dataset schema, appends to seed_dataset.jsonl
- --dry-run to preview, --source dev/prod/both

Tested: 237 entries → 112 kept (16 lang mismatch, 10 prompt leak, 86 dupe, 13 empty dropped)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-18 17:58:52 -04:00

..

processed

Risk gradient (0-5), updated system prompts, 233 examples

2026-03-18 16:14:54 -04:00

raw

Risk gradient (0-5), updated system prompts, 233 examples

2026-03-18 16:14:54 -04:00

ingest_audit.py

Add audit log ingestion pipeline with language/leak filtering

2026-03-18 17:58:52 -04:00

schema.json

Risk gradient (0-5), updated system prompts, 233 examples

2026-03-18 16:14:54 -04:00

validate_dataset.py

Initial project scaffold: dataset schema, 31 seed training examples, Mineflayer bot framework, and 7-phase roadmap

2026-03-18 01:51:28 -04:00