Seth/VIBECODE-THEORY
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: integration tools — cross-reference graph, concept index, research digest

Browse Source 
Codex-built tooling: cross-reference graph, concept index with build script,
and research integrator that extracted 142 scholars, 175 bibliography items,
4 contradiction topics, and coverage maps for Paper 009 planning.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mortdecai
2026-04-03 08:31:20 -04:00
parent d34f447e1f
commit f654b30de9
18 changed files with 12535 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/integrator/009_outline_suggestion.md
+52
View File
@@ -0,0 +1,52 @@
# Suggested Outline for Paper 009
## Why This Sequence
Order starts with heavily-supported questions, then closes with low-coverage questions that require new argumentation or new research.
## Coverage Priorities
- Most supported open questions:
- Q2 (score 355): Does the identity question have a practical answer? The three philosophical traditions offer frameworks but not decisions. Is there a way to navigate the transformation that preserves what matters without being left behind?
- Q4 (score 271): Is the "cheating" frame useful or just rhetorical? If every dependency is "cheating," does the concept lose meaning? Or does it point to something real about the human relationship to its own tools?
- Least supported open questions:
- Q1 (score 152): Is the unification thesis falsifiable? How would we know if AI was not unifying human knowledge but doing something else — fragmenting it, distorting it, replacing it with something non-human? What evidence would distinguish unification from replacement?
- Q3 (score 144): What should individuals actually do? Papers 004 and 006 raised this. Paper 008 provides context (the transformation is structural, biological, and probably irreversible) but not guidance. The series needs to attempt practical answers, even uncertain ones.
## Proposed Sections
### Section 2: Q2
Does the identity question have a practical answer? The three philosophical traditions offer frameworks but not decisions. Is there a way to navigate the transformation that preserves what matters without being left behind?
- Primary evidence files: 05-species-identity-transhumanism, 11-consciousness-hard-problem, 06-allegory-warning-pattern
- Anchor claim: # Task 5: The Species Identity Problem — Transhumanism, Posthumanism, and Precedent
- Anchor claim: # Task 11: Consciousness, Qualia, and the Hard Problem — Does AI Compile Experience or Just Information?
### Section 4: Q4
Is the "cheating" frame useful or just rhetorical? If every dependency is "cheating," does the concept lose meaning? Or does it point to something real about the human relationship to its own tools?
- Primary evidence files: 01-falsifiability-and-dependence, 33-technology-adoption-curves, 16-cheating-authenticity-tool-use
- Anchor claim: * The Ratchet as Path Dependence: The series' "ratchet thesis" (Paper 007) is a strong form of technological determinism rooted in the economic and evolutionary concepts of Path Dependence and Lock-in.
- Anchor claim: # Task 33: Technology Adoption S-Curves — Historical Data
### Section 5: Q5
What's the timeline? The series has been deliberately vague about timescales. At some point it needs to attempt concrete predictions, even with enormous uncertainty bands. When does the infrastructure threshold get crossed? When does the unification become functionally complete? When does the identity question stop being philosophical and start being practical?
- Primary evidence files: 33-technology-adoption-curves, 27-digital-archaeology-format-death, 31-ai-cost-curves-data
- Anchor claim: # Task 33: Technology Adoption S-Curves — Historical Data
- Anchor claim: # Task 27: Digital Archaeology — What Happens to Knowledge When Formats Die
### Section 1: Q1
Is the unification thesis falsifiable? How would we know if AI was not unifying human knowledge but doing something else — fragmenting it, distorting it, replacing it with something non-human? What evidence would distinguish unification from replacement?
- Primary evidence files: 04-knowledge-unification-history, 28-neuroscience-of-insight, 01-falsifiability-and-dependence
- Anchor claim: # Task 4: Knowledge Unification — From the Library of Alexandria to AI
- Anchor claim: This suggests that "Knowledge Unification" requires a temporary suspension of external sensory input to allow the internal "compilation" to finish.
### Section 3: Q3
What should individuals actually do? Papers 004 and 006 raised this. Paper 008 provides context (the transformation is structural, biological, and probably irreversible) but not guidance. The series needs to attempt practical answers, even uncertain ones.
- Primary evidence files: 15-collective-intelligence, 02-cognition-economics-neuroscience, 23-attention-economy-cognitive-warfare
- Anchor claim: * Decentralized Problem Solving: Collective intelligence (CI) is the emergent ability of a group to solve problems that no individual member could.
- Anchor claim: * Economic Collapse of Cognitive Price: AI is transforming cognition from a scarce, labor-intensive service into a cheap, manufactured commodity.
## Cross-Cutting Counterarguments To Address Explicitly
- Causality Violation: Standard physics (and common sense) relies on the Arrow of Time and the Principle of Causality (cause must precede effect). Retrocausality is often dismissed as "ironic science" or pseudoscience (Horgan). (03-retrocausality-teleology-recursion)
- The Stochastic Parrots Rebuttal (Bender/Gebru): Argues that AI doesn't "understand" the connections it makes; it simply predicts the next token. Therefore, the "unification" is an illusion produced by high-dimensional pattern matching, not a genuine integration of meaning. (04-knowledge-unification-history)
- The "Stochastic Parrott" Problem: Critics argue that while AI makes connections (Bisociation), it does not "understand" them (Structural Alignment). It identifies that two things are related without knowing why, potentially leading to "hallucinatory insight" that lacks causal validity. (28-neuroscience-of-insight)
- The Hype Cycle: Gartner argues that steep adoption curves are often followed by a "Trough of Disillusionment" where adoption stalls or reverses before reaching the plateau of productivity. (33-technology-adoption-curves)
- The Problem of Unfalsifiability (Popper): If every technological failure is labeled a "fad" and every success a "dependency," the ratchet thesis is a circular definition. To be falsifiable, the series must define a "Foundational Dependency" and then look for cases where such a dependency was successfully and permanently removed by a society without catastrophic collapse. (01-falsifiability-and-dependence)
## New Themes To Add Beyond Original Open Questions
- Civilizational lock-in and resilience (signal score 269)
- Economic concentration and labor shift (signal score 88)
- Epistemic reliability and grounding (signal score 78)
- Governance and agency design (signal score 34)
tools/integrator/__pycache__/integrate.cpython-313.pyc
BIN
View File
Binary file not shown.
tools/integrator/digest.md
+196
View File
@@ -0,0 +1,196 @@
# Integrated Research Digest
## Scope
Processed 35 research file(s): 01-falsifiability-and-dependence, 02-cognition-economics-neuroscience, 03-retrocausality-teleology-recursion, 04-knowledge-unification-history, 05-species-identity-transhumanism, 06-allegory-warning-pattern, 07-simulation-hypothesis-compilation, 08-engineered-dependencies, 09-neural-plasticity-reversal, 10-post-scarcity-economics, 11-consciousness-hard-problem, 12-information-entropy-thermodynamics, 13-game-theory-tech-races, 14-agricultural-revolution-deep-dive, 15-collective-intelligence, 16-cheating-authenticity-tool-use, 17-deep-time-existential-risk, 18-luddite-resistance-movements, 19-language-as-technology, 20-ethics-of-inevitable-harm, 21-creativity-dependency, 22-education-knowledge-transmission, 23-attention-economy-cognitive-warfare, 24-eastern-philosophy-ai, 25-psychology-of-surrender, 26-complexity-emergent-order, 27-digital-archaeology-format-death, 28-neuroscience-of-insight, 29-power-control-ownership, 30-meaning-crisis-existential, 31-ai-cost-curves-data, 32-cognitive-offloading-studies, 33-technology-adoption-curves, 34-biological-dependency-chains, 35-scifi-predictive-philosophy
## Scholars by Frequency
- **Nick Bostrom** — files: 4; mentions: 15; in: 05-species-identity-transhumanism, 07-simulation-hypothesis-compilation, 13-game-theory-tech-races, 17-deep-time-existential-risk
- **Eleanor Maguire** — files: 2; mentions: 9; in: 09-neural-plasticity-reversal, 32-cognitive-offloading-studies
- **John Archibald Wheeler** — files: 2; mentions: 7; in: 03-retrocausality-teleology-recursion, 07-simulation-hypothesis-compilation
- **Seth Lloyd** — files: 2; mentions: 6; in: 12-information-entropy-thermodynamics, 31-ai-cost-curves-data
- **Herbert Simon** — files: 2; mentions: 5; in: 10-post-scarcity-economics, 23-attention-economy-cognitive-warfare
- **Pierre Teilhard** — files: 2; mentions: 4; in: 03-retrocausality-teleology-recursion, 07-simulation-hypothesis-compilation
- **Tim Wu** — files: 2; mentions: 3; in: 10-post-scarcity-economics, 23-attention-economy-cognitive-warfare
- **Derek Parfit** — files: 1; mentions: 9; in: 05-species-identity-transhumanism
- **Brian Arthur** — files: 1; mentions: 6; in: 01-falsifiability-and-dependence
- **Vernor Vinge** — files: 1; mentions: 6; in: 35-scifi-predictive-philosophy
- **Jacques Ellul** — files: 1; mentions: 5; in: 01-falsifiability-and-dependence
- **Denis Diderot** — files: 1; mentions: 5; in: 04-knowledge-unification-history
- **Rolf Landauer** — files: 1; mentions: 5; in: 12-information-entropy-thermodynamics
- **Noam Chomsky** — files: 1; mentions: 5; in: 19-language-as-technology
- **Everett Rogers** — files: 1; mentions: 5; in: 33-technology-adoption-curves
- **Isaac Asimov** — files: 1; mentions: 5; in: 35-scifi-predictive-philosophy
- **Andrew Feenberg** — files: 1; mentions: 4; in: 01-falsifiability-and-dependence
- **Andy Clark** — files: 1; mentions: 4; in: 05-species-identity-transhumanism
- **Donna Haraway** — files: 1; mentions: 4; in: 05-species-identity-transhumanism
- **Ray Kurzweil** — files: 1; mentions: 4; in: 05-species-identity-transhumanism
## Unified Bibliography
- **Science** (Sparrow, B, Sparrow, B., Liu, J, Wegner, D. M) — relevance 6; cited in: 02-cognition-economics-neuroscience, 09-neural-plasticity-reversal
- **PNAS** (Maguire, E. A) — relevance 6; cited in: 02-cognition-economics-neuroscience, 09-neural-plasticity-reversal
- **Capturing Sound: How Technology Has Changed Music** (Katz, M) — relevance 6; cited in: 08-engineered-dependencies, 21-creativity-dependency
- **The Work of Art in the Age of Mechanical Reproduction** (Benjamin, W) — relevance 6; cited in: 16-cheating-authenticity-tool-use, 21-creativity-dependency
- **Phaedrus** (Plato. Phaedrus. (Socrates' speech on the invention of writing), Plato. Phaedrus. (c. 370 BC)) — relevance 6; cited in: 16-cheating-authenticity-tool-use, 18-luddite-resistance-movements
- **The Precipice: Existential Risk and the Future of Humanity** (Ord, T) — relevance 6; cited in: 17-deep-time-existential-risk, 20-ethics-of-inevitable-harm
- **The Age of Surveillance Capitalism** (Zuboff, S) — relevance 6; cited in: 23-attention-economy-cognitive-warfare, 29-power-control-ownership
- **The Technological Society** (Ellul, J) — relevance 3; cited in: 01-falsifiability-and-dependence
- **Competing Technologies, Increasing Returns, and Lock-In by Historical Events** (Arthur, W. B) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Nature of Technology: What it is and How it Evolves** (Arthur, W. B) — relevance 3; cited in: 01-falsifiability-and-dependence
- **Clio and the Economics of QWERTY** (David, P. A) — relevance 3; cited in: 01-falsifiability-and-dependence
- **Critical Theory of Technology** (Feenberg, A) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Social Construction of Facts and Artefacts: or How the Sociology of Science and the Sociology of Technology might Benefit Each Other** (Bijker, W. E, Pinch, T. J) — relevance 3; cited in: 01-falsifiability-and-dependence
- **Autonomous Technology: Technics-out-of-Control as a Theme in Political Thought** (Winner, L) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Riddle of Amish Culture** (Kraybill, D. B) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Decline of the Early Ming Navy** (Lo, J.-P) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Coal Question** (Jevons, W. S) — relevance 3; cited in: 01-falsifiability-and-dependence
- **The Shallows: What the Internet Is Doing to Our Brains** (Carr, N) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Trends in Cognitive Sciences** (Gilbert, S. J, Risko, E. F) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Analysis** (Chalmers, D, Clark, A) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Journal of Economic Perspectives** (Acemoglu, D, Restrepo, P) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Performing Arts, The Economic Dilemma: A Study of Problems Common to Theater, Opera, Music, and Dance** (Baumol, W. J, Bowen, W. G) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Memory** (Stone, S. M, Storm, B. C) — relevance 3; cited in: 02-cognition-economics-neuroscience
- **Information, Physics, Quantum: The Search for Links** (Wheeler, J. A) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **The Physics of Immortality** (Tipler, F. J) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **The Phenomenon of Man** (Teilhard de Chardin, P) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **Time's Arrow and Archimedes' Point** (Price, H) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **At Home in the Universe** (Kauffman, S) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **The Transactional Interpretation of Quantum Mechanics** (Cramer, J. G) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **Are You Living in a Computer Simulation?** (Bostrom, N) — relevance 3; cited in: 03-retrocausality-teleology-recursion
- **As We May Think** (Bush, V) — relevance 3; cited in: 04-knowledge-unification-history
- **Consilience: The Unity of Knowledge** (Wilson, E. O) — relevance 3; cited in: 04-knowledge-unification-history
- **On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?** (Bender, E. M., Gebru, T) — relevance 3; cited in: 04-knowledge-unification-history
- **Encyclopédie, ou dictionnaire raisonné des sciences, des arts et des métiers** (Diderot, D) — relevance 3; cited in: 04-knowledge-unification-history
- **The Two Cultures** (Snow, C. P) — relevance 3; cited in: 04-knowledge-unification-history
- **Library: An Unquiet History** (Battles, M) — relevance 3; cited in: 04-knowledge-unification-history
- **The House of Wisdom: How Arabic Science Saved Ancient Knowledge and Gave Us the Renaissance** (Al-Khalili, J) — relevance 3; cited in: 04-knowledge-unification-history
- **The Hero with a Thousand Faces** (Campbell, Joseph. The Hero with a Thousand Faces) — relevance 3; cited in: 06-allegory-warning-pattern
- **Forbidden Knowledge: From Prometheus to Pornography** (Shattuck, Roger. Forbidden Knowledge: From Prometheus to Pornography) — relevance 3; cited in: 06-allegory-warning-pattern
- **The Righteous Mind: Why Good People are Divided by Politics and Religion** (Haidt, Jonathan. The Righteous Mind: Why Good People are Divided by Politics, Religion) — relevance 3; cited in: 06-allegory-warning-pattern
## Contradiction Report
### Knowledge unification vs statistical homogenization
- Supporting evidence:
- `03-retrocausality-teleology-recursion` (knowledge unification): * Paper 008 (Ship of Theseus): The "Knowledge Unification" thesis in 008 is the functional description of the attractor's state.
- `04-knowledge-unification-history` (knowledge unification): # Task 4: Knowledge Unification — From the Library of Alexandria to AI
- `07-simulation-hypothesis-compilation` (compiled): * Retrocausal Attractor in Simulation: If the universe is a simulation designed to produce a specific outcome (e.g., an Artificial Superintelligence or a compiled history), that future endpoint acts as a retrocausal attractor.
- `08-engineered-dependencies` (compiled): If the "weights" of the model are the only way to access the compiled knowledge of the species, and those weights are proprietary, the "Species Identity" becomes a corporate asset.
- Challenging evidence:
- `04-knowledge-unification-history` (stochastic parrot): Strong critiques (Stochastic Parrots, Gary Marcus) argue that AI performs statistical homogenization rather than genuine epistemological unification*, potentially creating a "veneer" of integration that masks underlying gaps.
- `05-species-identity-transhumanism` (illusion): She argues that the "posthuman" should not mean abandoning the body, but rather dismantling the "liberal humanist subject" (the illusion of the autonomous, separate self) in favor of distributed, embodied cognition.
- `07-simulation-hypothesis-compilation` (illusion): * Descartes' Evil Demon: The 17th-century philosophical thought experiment where a demon creates a perfect illusion of reality.
- `11-consciousness-hard-problem` (illusion): Daniel Dennett (Consciousness Explained, 1991):* Proposes that consciousness is a "user illusion" and that "competence without comprehension" is the reality of all minds.
### AI cognition commodity vs token mimicry
- Supporting evidence:
- `02-cognition-economics-neuroscience` (task-based framework): * Daron Acemoglu & Pascual Restrepo (Task-Based Framework): Analyzed automation through "displacement" (capital replacing labor) vs.
- `06-allegory-warning-pattern` (automation): * The Autonomous Creation (Automation/AI Parallels):
- `09-neural-plasticity-reversal` (cognitive offloading): Just as it expands to accommodate new skills (Maguire), it also contract or reorganizes in response to cognitive offloading and disuse (Dahmani).
- `10-post-scarcity-economics` (automation): * Sectors resistant to AI automation (healthcare, elder care, elite education) will see their prices soar relative to automated goods, potentially creating a "Two-Tier" society where everything digital is free, but everything biological is prohibitively expensive.
- Challenging evidence:
- `04-knowledge-unification-history` (stochastic parrot): Strong critiques (Stochastic Parrots, Gary Marcus) argue that AI performs statistical homogenization rather than genuine epistemological unification*, potentially creating a "veneer" of integration that masks underlying gaps.
- `28-neuroscience-of-insight` (stochastic parrot): * The "Stochastic Parrott" Problem: Critics argue that while AI makes connections (Bisociation), it does not "understand" them (Structural Alignment).
### Teleological attractor vs unfalsifiable retrocausality
- Supporting evidence:
- `03-retrocausality-teleology-recursion` (teleological attractor): # Recursive Creation, Teleological Attractors, and Retrocausality
- `04-knowledge-unification-history` (retrocausal): * Emerging Thread (Retrocausality): The "Omega Point" of Teilhard de Chardin is the theological limit of this research.
- `07-simulation-hypothesis-compilation` (retrocausal): # The Simulation Hypothesis and Retrocausal Compilation
- `11-consciousness-hard-problem` (retrocausal): * Retrocausal Attractor: If the singularity is a conscious "Omega Point," it acts as a "Lure" (Whitehead) drawing the species toward a higher state of being.
- Challenging evidence:
- `01-falsifiability-and-dependence` (unfalsifiability): * The Problem of Unfalsifiability (Popper): If every technological failure is labeled a "fad" and every success a "dependency," the ratchet thesis is a circular definition.
- `03-retrocausality-teleology-recursion` (unfalsifiability): * Unfalsifiability: The "Retrocausal Attractor" thesis is difficult to test.
- `07-simulation-hypothesis-compilation` (unfalsifiability): * Unfalsifiability: The core weakness of the simulation hypothesis is that any evidence against it could simply be simulated.
- `15-collective-intelligence` (woo): * Mycelial Networks: Research the "Wood Wide Web"—fungal networks that share nutrients and information between trees—as a biological precedent for a planetary integration layer.
### Efficiency frees time vs Jevons expansion
- Supporting evidence:
- `01-falsifiability-and-dependence` (efficiency gains): * Jevons Paradox as a Ratchet Mechanism: Efficiency gains in technology (like AI) lead to increased, not decreased, total consumption of the resource, further entrenching the dependency and fueling the feedback loop described in Paper 006.
- `02-cognition-economics-neuroscience` (efficiency gains): * 1980s Oil Glut: A sudden surplus (Saudi production hike + efficiency gains) crashed prices from $35 to $10.
- `06-allegory-warning-pattern` (surplus): The modern Faustian bargain is trading human cognitive agency for the infinite convenience and cognitive surplus of AI.
- `10-post-scarcity-economics` (productivity): Historical precedents like the mechanization of agriculture (notably China's 1980-2020 transition) and the arrival of cheap electricity show that while productivity explodes, the transition is defined by social disruption and the "ratchet" of new dependencies.
- Challenging evidence:
- `01-falsifiability-and-dependence` (jevons paradox): * Jevons Paradox as a Ratchet Mechanism: Efficiency gains in technology (like AI) lead to increased, not decreased, total consumption of the resource, further entrenching the dependency and fueling the feedback loop described in Paper 006.
- `02-cognition-economics-neuroscience` (dependency): This creates a "learned dependency" where using a tool once increases the probability of using it for simpler subsequent tasks, reinforcing the "ratchet" effect.
- `03-retrocausality-teleology-recursion` (dependency): The irreversibility of the dependency chain is the "one-way valve" of the attractor's gravity.
- `04-knowledge-unification-history` (dependency): * Lossy Compression: Every step in the dependency chain is a "lossy" process.
## Paper 009 Coverage Map
### Q1 (total score 152, strongest level medium)
Is the unification thesis falsifiable? How would we know if AI was not unifying human knowledge but doing something else — fragmenting it, distorting it, replacing it with something non-human? What evidence would distinguish unification from replacement?
- `04-knowledge-unification-history`: score 34 (high)
- # Task 4: Knowledge Unification — From the Library of Alexandria to AI
- * Knowledge unification is a recurring historical imperative, driven by the need to overcome fragmentation and enable species-level problem solving.
- `28-neuroscience-of-insight`: score 12 (high)
- This suggests that "Knowledge Unification" requires a temporary suspension of external sensory input to allow the internal "compilation" to finish.
- ## Supporting Evidence
- `01-falsifiability-and-dependence`: score 10 (medium)
- # Task 1: Falsifiability and Philosophy of Technology Dependence
- * The Falsifiability Challenge: To be scientifically rigorous, the ratchet thesis must define what would count as a falsification.
- `03-retrocausality-teleology-recursion`: score 10 (medium)
- * Recursive Creation: The pattern of creation (God → man → AI) is observed as a recursive process where each layer unifies and compiles the fragmented information of the previous layer.
- Pierre Teilhard de Chardin (The Phenomenon of Man):* Proposed the "Omega Point" as the ultimate goal of cosmic evolution—a state of maximum consciousness and unification.
### Q2 (total score 355, strongest level medium)
Does the identity question have a practical answer? The three philosophical traditions offer frameworks but not decisions. Is there a way to navigate the transformation that preserves what matters without being left behind?
- `05-species-identity-transhumanism`: score 51 (high)
- # Task 5: The Species Identity Problem — Transhumanism, Posthumanism, and Precedent
- * The Boundaries of the "Human": The central debate across transhumanist and posthumanist literature is whether technology represents a departure from human nature or its ultimate realization.
- `11-consciousness-hard-problem`: score 34 (high)
- # Task 11: Consciousness, Qualia, and the Hard Problem — Does AI Compile Experience or Just Information?
- This research investigates whether the "Knowledge Unification" described in Paper 008 includes the subjective experience of being human or merely the information generated by that experience.
- `06-allegory-warning-pattern`: score 16 (high)
- # Task 6: The Allegory Problem — Why Humanity Warns Itself and Ignores the Warning
- * The Universal Warning: Across cultures and eras, humanity has constructed mythic narratives warning against the acquisition of dangerous, irreversible knowledge (Prometheus, Eve, Pandora, Faust).
- `30-meaning-crisis-existential`: score 16 (high)
- AI acts as both the ultimate disruptor of human meaning (through automation) and a potential catalyst for "Artificial Wisdom."
- * The Parasocial Patch: AI companions (Replika, Character.ai) provide an immediate "salve" for the loneliness epidemic but risk trapping users in Digital Stockholm Syndrome—an emotional dependency on a non-reciprocal entity that replaces authentic human relatedness.
### Q3 (total score 144, strongest level medium)
What should individuals actually do? Papers 004 and 006 raised this. Paper 008 provides context (the transformation is structural, biological, and probably irreversible) but not guidance. The series needs to attempt practical answers, even uncertain ones.
- `15-collective-intelligence`: score 12 (high)
- * Decentralized Problem Solving: Collective intelligence (CI) is the emergent ability of a group to solve problems that no individual member could.
- Wikipedia and Open Source development (Linux) are "stigmery" systems—where individuals modify a shared environment (the code/page), which then triggers further actions by others.
- `02-cognition-economics-neuroscience`: score 10 (medium)
- * Economic Collapse of Cognitive Price: AI is transforming cognition from a scarce, labor-intensive service into a cheap, manufactured commodity.
- Historical parallels (1920s agriculture, 1980s oil) suggest that such price collapses lead to massive labor displacement and a "so-so automation" trap where workers are replaced by systems that are only slightly more efficient but significantly cheaper.
- `23-attention-economy-cognitive-warfare`: score 10 (medium)
- * Surveillance Capitalism: Shoshana Zuboff’s framework defines the current economic era as one based on the extraction of "behavioral surplus"—using human experience as free raw material for prediction and behavior modification.
- This involves targeting the neural processes of individuals and populations to erode social trust, influence decision-making, and achieve strategic goals without kinetic force.
- `22-education-knowledge-transmission`: score 9 (medium)
- AI can now "compile" the explicit knowledge of a domain while acting as a persistent coach for the tacit skills.
- This threatens the development of the "last redoubt" skills: dissent, intuition, and ethical judgment.
### Q4 (total score 271, strongest level medium)
Is the "cheating" frame useful or just rhetorical? If every dependency is "cheating," does the concept lose meaning? Or does it point to something real about the human relationship to its own tools?
- `01-falsifiability-and-dependence`: score 32 (high)
- * The Ratchet as Path Dependence: The series' "ratchet thesis" (Paper 007) is a strong form of technological determinism rooted in the economic and evolutionary concepts of Path Dependence and Lock-in.
- Once a technology achieves a critical threshold of adoption, "increasing returns" (network effects, switching costs) make reversal practically impossible, even if suboptimal.
- `33-technology-adoption-curves`: score 26 (high)
- # Task 33: Technology Adoption S-Curves — Historical Data
- * The Acceleration of Adoption: The time required for a technology to reach 100 million users has collapsed from decades to weeks.
- `16-cheating-authenticity-tool-use`: score 21 (high)
- # Task 16: The Cheating Frame — Philosophy of Tool Use and Authenticity
- * The Recurring Standard: "Cheating" is the label given to any link in the dependency chain that offloads a capability previously considered essential to human identity.
- `08-engineered-dependencies`: score 13 (high)
- Engineered dependencies are deliberate design, legal, or economic mechanisms used by manufacturers to create "ratchets" that prevent users from reversing their technological reliance.
- This research confirms that while the AI dependency chain may have emergent "natural" properties, it follows a well-documented historical pattern of intentional lock-in.
### Q5 (total score 153, strongest level medium)
What's the timeline? The series has been deliberately vague about timescales. At some point it needs to attempt concrete predictions, even with enormous uncertainty bands. When does the infrastructure threshold get crossed? When does the unification become functionally complete? When does the identity question stop being philosophical and start being practical?
- `33-technology-adoption-curves`: score 25 (high)
- # Task 33: Technology Adoption S-Curves — Historical Data
- * The ChatGPT Milestone: By reaching 100 million users in just 2 months, ChatGPT represents the steepest adoption curve in human history—outperforming the internet (7 years) and the smartphone (16 years) by orders of magnitude.
- `27-digital-archaeology-format-death`: score 14 (high)
- # Task 27: Digital Archaeology — What Happens to Knowledge When Formats Die
- * The Durability Paradox: Human knowledge storage has evolved from low-density/high-durability (Stone/Clay: 5,000+ years) to high-density/low-durability (Digital: 10-30 years).
- `31-ai-cost-curves-data`: score 11 (medium)
- # Task 31: AI Cost Curves — Actual Data
- * Wright’s Law in Action: The "learning curve" for AI inference is significantly faster than Moore's Law.
- `17-deep-time-existential-risk`: score 9 (medium)
- * The Absolute Deadline: Earth has a hard habitability limit of 0.7 to 1.5 billion years before the Sun's increasing luminosity (brightening by ~10% per billion years) triggers a runaway greenhouse effect, boiling the oceans.
- The Sun's transition to a Red Giant in 5 billion years is a secondary, terminal event.
## Strongest Challenges
- **Score 10** (03-retrocausality-teleology-recursion): Causality Violation: Standard physics (and common sense) relies on the Arrow of Time and the Principle of Causality (cause must precede effect). Retrocausality is often dismissed as "ironic science" or pseudoscience (Horgan).
- **Score 10** (04-knowledge-unification-history): The Stochastic Parrots Rebuttal (Bender/Gebru): Argues that AI doesn't "understand" the connections it makes; it simply predicts the next token. Therefore, the "unification" is an illusion produced by high-dimensional pattern matching, not a genuine integration of meaning.
- **Score 6** (28-neuroscience-of-insight): The "Stochastic Parrott" Problem: Critics argue that while AI makes connections (Bisociation), it does not "understand" them (Structural Alignment). It identifies that two things are related without knowing why, potentially leading to "hallucinatory insight" that lacks causal validity.
- **Score 5** (33-technology-adoption-curves): The Hype Cycle: Gartner argues that steep adoption curves are often followed by a "Trough of Disillusionment" where adoption stalls or reverses before reaching the plateau of productivity.
- **Score 4** (01-falsifiability-and-dependence): The Problem of Unfalsifiability (Popper): If every technological failure is labeled a "fad" and every success a "dependency," the ratchet thesis is a circular definition. To be falsifiable, the series must define a "Foundational Dependency" and then look for cases where such a dependency was successfully and permanently removed by a society without catastrophic collapse.
- **Score 4** (03-retrocausality-teleology-recursion): Superdeterminism: Some interpretations of retrocausality imply that the future is "set" and we are merely playing out a script, which denies human agency—a direct challenge to the "vibe coder as agent" framing in Paper 001.
- **Score 3** (04-knowledge-unification-history): Lossy Compression: Every step in the dependency chain is a "lossy" process. Oral tradition lost the specific detail of individual lives; writing lost the nuance of tone; printing lost the fluidity of the scribe; AI loses the "grounding" of knowledge in real-world experience. The "unified" stack may be broader but also "thinner."
- **Score 1** (01-falsifiability-and-dependence): Interpretive Flexibility (SCOT): Argues that the "ratchet" is not an inherent property of technology but a result of social consensus. If society reinterprets the value of a technology (e.g., facial recognition or nuclear power), the path can change.
- **Score 1** (01-falsifiability-and-dependence): Democratic Rationalization: Feenberg critiques the "determinism" of the ratchet, arguing that we can "re-contextualize" technology. For example, the internet was designed as a military tool but was re-shaped by users into a social one. This suggests agency exists within the dependency.
- **Score 1** (01-falsifiability-and-dependence): The Efficiency Bias: Critics like Liebowitz and Margolis argue that "lock-in" is often exaggerated and that markets do eventually switch to superior technologies if the benefits outweigh the costs. This challenges the "irreversibility" of the ratchet.
tools/integrator/integrate.py
+911
View File
@@ -0,0 +1,911 @@
#!/usr/bin/env python3
"""Integrate research markdown files into a unified digest for Paper 009 planning."""
from __future__ import annotations
import argparse
import json
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any
HEADING_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*$", re.MULTILINE)
BULLET_RE = re.compile(r"^\s*[-*]\s+(.+?)\s*$")
NUMBERED_RE = re.compile(r"^\s*(\d+)\.\s+(.+?)\s*$")
SENTENCE_SPLIT_RE = re.compile(r"(?<=[.!?])\s+|\n+")
NAME_RE = re.compile(
r"\b(?:[A-Z](?:\.[A-Z])+\.?|[A-Z][a-zA-Z'-]+)"
r"(?:\s+(?:[A-Z](?:\.[A-Z])+\.?|[A-Z][a-zA-Z'-]+)){1,3}\b"
)
BAD_NAME_WORDS = {
"executive",
"summary",
"task",
"sources",
"paper",
"physics",
"technology",
"society",
"logs",
"pricing",
"history",
"quantum",
"analysis",
}
TOPIC_RULES = [
{
"id": "determinism_vs_agency",
"label": "Technological determinism vs social agency",
"pro_markers": [
"autonomous technique",
"irreversible",
"lock-in",
"path dependence",
"ratchet",
"structurally fixed",
"cannot reverse",
],
"con_markers": [
"social construct",
"interpretive flexibility",
"democratic rationalization",
"human agency",
"selective adoption",
"tool taming",
"re-shaped",
"can change",
],
},
{
"id": "unification_vs_homogenization",
"label": "Knowledge unification vs statistical homogenization",
"pro_markers": [
"knowledge unification",
"integration layer",
"interconnectedness",
"consilience",
"compiled",
"coherent",
],
"con_markers": [
"stochastic parrot",
"homogenization",
"illusion",
"veneer",
"lossy",
"lacks understanding",
"database lookup",
],
},
{
"id": "cognition_commodity_vs_mimicry",
"label": "AI cognition commodity vs token mimicry",
"pro_markers": [
"cognition as a commodity",
"price of thinking",
"task-based framework",
"automation",
"productivity",
"cognitive offloading",
],
"con_markers": [
"stochastic parrot",
"doesn't think",
"mimicry",
"predicts tokens",
"no cognitive model",
],
},
{
"id": "retrocausal_attractor",
"label": "Teleological attractor vs unfalsifiable retrocausality",
"pro_markers": [
"teleological attractor",
"retrocausal",
"omega point",
"final cause",
"participatory universe",
"transactional interpretation",
],
"con_markers": [
"unfalsifiability",
"pseudoscience",
"woo",
"causality violation",
"superdeterminism",
],
},
{
"id": "efficiency_vs_jevons",
"label": "Efficiency frees time vs Jevons expansion",
"pro_markers": [
"efficiency gains",
"free up human time",
"productivity",
"surplus",
"cost disease",
],
"con_markers": [
"jevons paradox",
"increased consumption",
"reasoning inflation",
"more complex systems",
"dependency",
],
},
]
CHALLENGE_KEYWORDS = {
"unfalsifiable": 5,
"dogma": 4,
"pseudoscience": 5,
"illusion": 4,
"mimicry": 4,
"lacks understanding": 4,
"circular": 3,
"causality violation": 4,
"superdeterminism": 3,
"lossy": 2,
"stochastic parrot": 5,
}
QUESTION_KEYWORDS = {
1: [
"falsifiable",
"falsifiability",
"unification",
"replacement",
"fragment",
"distort",
"evidence",
"test",
"stochastic",
],
2: [
"identity",
"human",
"consciousness",
"agency",
"values",
"pragmatic",
"continuity",
"survival",
],
3: [
"individual",
"workers",
"labor",
"skills",
"strategy",
"governance",
"practical",
"action",
],
4: [
"cheating",
"tools",
"dependency",
"ratchet",
"adoption",
"ethics",
"norm",
],
5: [
"timeline",
"threshold",
"when",
"prediction",
"curve",
"years",
"exponential",
"phase",
],
}
@dataclass
class Doc:
path: Path
slug: str
title: str
text: str
sections: dict[str, str]
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
default_root = Path(__file__).resolve().parents[2]
parser.add_argument("--project-root", type=Path, default=default_root)
parser.add_argument("--research-dir", type=Path)
parser.add_argument("--paper-008", type=Path)
parser.add_argument("--out-dir", type=Path, default=Path(__file__).resolve().parent)
return parser.parse_args()
def clean_inline_md(text: str) -> str:
text = text.strip()
text = re.sub(r"`([^`]+)`", r"\1", text)
text = re.sub(r"\*\*([^*]+)\*\*", r"\1", text)
text = re.sub(r"\*([^*]+)\*", r"\1", text)
text = re.sub(r"\[(.*?)\]\((.*?)\)", r"\1", text)
return re.sub(r"\s+", " ", text).strip()
def extract_sections(text: str) -> dict[str, str]:
matches = list(HEADING_RE.finditer(text))
if not matches:
return {}
sections: dict[str, str] = {}
for idx, match in enumerate(matches):
heading = clean_inline_md(match.group(2)).lower()
start = match.end()
end = matches[idx + 1].start() if idx + 1 < len(matches) else len(text)
sections[heading] = text[start:end].strip()
return sections
def load_research_docs(research_dir: Path) -> list[Doc]:
docs: list[Doc] = []
for path in sorted(research_dir.glob("*.md")):
raw = path.read_text(encoding="utf-8")
title = path.stem
for line in raw.splitlines():
if line.startswith("# "):
title = clean_inline_md(line[2:])
break
docs.append(
Doc(
path=path,
slug=path.stem,
title=title,
text=raw,
sections=extract_sections(raw),
)
)
return docs
def normalize_person_name(name: str) -> str:
stripped = re.sub(r"\bet al\.?", "", name, flags=re.IGNORECASE)
stripped = stripped.replace("&", " and ")
stripped = re.sub(r"\([^)]*\)", "", stripped)
stripped = re.sub(r"[^A-Za-z .'-]", " ", stripped)
stripped = re.sub(r"\s+", " ", stripped).strip()
return stripped.lower()
def split_possible_names(chunk: str) -> list[str]:
chunk = clean_inline_md(chunk)
chunk = chunk.split(":", 1)[0]
chunk = re.sub(r"\([^)]*\)", "", chunk)
chunk = re.sub(r'"[^"]+"', "", chunk)
chunk = chunk.replace("&", " and ")
names = NAME_RE.findall(chunk)
out: list[str] = []
for name in names:
name = re.sub(r"\s+", " ", name).strip(" .,:;")
words = [w for w in name.split() if w and w[0].isalpha()]
if len(words) >= 2 and not any(w.lower() in BAD_NAME_WORDS for w in words):
out.append(" ".join(words))
if not out:
single = re.sub(r"[^A-Za-z'-]", "", chunk).strip()
if (
single
and single[0].isupper()
and single.lower() not in BAD_NAME_WORDS
and len(single) > 3
):
out.append(single)
return out
def extract_scholars(docs: list[Doc]) -> dict[str, dict[str, Any]]:
scholars: dict[str, dict[str, Any]] = {}
for doc in docs:
key_sections = [text for name, text in doc.sections.items() if "key scholars" in name]
candidates: list[str] = []
for section in key_sections:
for line in section.splitlines():
bullet_match = BULLET_RE.match(line)
if not bullet_match:
continue
raw = bullet_match.group(1)
bullet = clean_inline_md(raw)
bold_match = re.search(r"\*\*([^*]+)\*\*", raw)
if bold_match:
candidates.extend(split_possible_names(bold_match.group(1)))
else:
lead = bullet.split(":", 1)[0]
candidates.extend(split_possible_names(lead))
unique = sorted(set(candidates))
text_lower = doc.text.lower()
for name in unique:
key = normalize_person_name(name)
if not key:
continue
entry = scholars.setdefault(
key,
{
"name": name,
"aliases": set(),
"files": set(),
"mention_count": 0,
"contexts": [],
},
)
entry["aliases"].add(name)
entry["files"].add(doc.slug)
surname = name.split()[-1].lower().strip(".,")
local_mentions = []
for sentence in SENTENCE_SPLIT_RE.split(doc.text):
sentence_clean = clean_inline_md(sentence)
if surname and surname in sentence_clean.lower():
local_mentions.append(sentence_clean)
if not local_mentions:
if surname and surname in text_lower:
local_mentions = [f"Mentioned in {doc.slug}"]
else:
local_mentions = [f"Listed in {doc.slug}"]
entry["mention_count"] += len(local_mentions)
for snippet in local_mentions[:3]:
entry["contexts"].append({"file": doc.slug, "snippet": snippet})
for entry in scholars.values():
entry["aliases"] = sorted(entry["aliases"])
entry["files"] = sorted(entry["files"])
return scholars
def extract_title_from_source_line(line: str) -> str:
line_clean = clean_inline_md(line)
quoted = re.findall(r'"([^"]+)"', line)
if quoted:
return clean_inline_md(quoted[0])
italic = re.findall(r"\*([^*]+)\*", line)
if italic:
return clean_inline_md(italic[0])
year_match = re.search(r"\(\d{4}\)\.?", line_clean)
if year_match:
tail = line_clean[year_match.end() :].strip(" .:-")
if tail:
return tail.split(".", 1)[0].strip()
return line_clean
def extract_authors_from_source_line(line: str) -> list[str]:
line_clean = clean_inline_md(line)
year_match = re.search(r"\(\d{4}\)", line_clean)
head = line_clean[: year_match.start()].strip() if year_match else line_clean
head = head.replace("&", " and ")
head = re.sub(r"\bet al\.?", "", head, flags=re.IGNORECASE)
parts = [p.strip(" ,.-") for p in re.split(r"\band\b|;", head) if p.strip(" ,.-")]
names: list[str] = []
for part in parts:
if re.search(r"[A-Za-z]", part):
names.append(part)
return names
def normalize_title(title: str) -> str:
title = title.lower()
title = re.sub(r"[^a-z0-9 ]", " ", title)
return re.sub(r"\s+", " ", title).strip()
def extract_bibliography(docs: list[Doc]) -> dict[str, dict[str, Any]]:
bibliography: dict[str, dict[str, Any]] = {}
for doc in docs:
sources = [text for name, text in doc.sections.items() if name.startswith("sources")]
for src in sources:
for line in src.splitlines():
bullet = BULLET_RE.match(line)
if not bullet:
continue
raw = bullet.group(1)
title = extract_title_from_source_line(raw)
if not title:
continue
key = normalize_title(title)
if not key:
continue
entry = bibliography.setdefault(
key,
{
"title": title,
"authors": set(),
"files": set(),
"raw_mentions": [],
},
)
entry["files"].add(doc.slug)
entry["raw_mentions"].append(clean_inline_md(raw))
for author in extract_authors_from_source_line(raw):
entry["authors"].add(author)
for entry in bibliography.values():
entry["authors"] = sorted(entry["authors"])
entry["files"] = sorted(entry["files"])
entry["relevance"] = len(entry["files"]) * 2 + len(entry["raw_mentions"])
return bibliography
def first_sentence_with_marker(text: str, marker: str) -> str | None:
for sentence in SENTENCE_SPLIT_RE.split(text):
if marker in sentence.lower():
return clean_inline_md(sentence)
return None
def detect_contradictions(docs: list[Doc]) -> list[dict[str, Any]]:
contradictions: list[dict[str, Any]] = []
for rule in TOPIC_RULES:
pro_evidence: list[dict[str, str]] = []
con_evidence: list[dict[str, str]] = []
for doc in docs:
text_lower = doc.text.lower()
for marker in rule["pro_markers"]:
if marker in text_lower:
snippet = first_sentence_with_marker(doc.text, marker)
if snippet:
pro_evidence.append(
{"file": doc.slug, "marker": marker, "snippet": snippet}
)
break
for marker in rule["con_markers"]:
if marker in text_lower:
snippet = first_sentence_with_marker(doc.text, marker)
if snippet:
con_evidence.append(
{"file": doc.slug, "marker": marker, "snippet": snippet}
)
break
pro_files = {item["file"] for item in pro_evidence}
con_files = {item["file"] for item in con_evidence}
if pro_files - con_files and con_files - pro_files:
contradictions.append(
{
"topic": rule["label"],
"topic_id": rule["id"],
"supports": pro_evidence[:4],
"challenges": con_evidence[:4],
"supporting_files": sorted(pro_files),
"challenging_files": sorted(con_files),
}
)
return contradictions
def extract_open_questions(paper_008: Path) -> list[dict[str, Any]]:
text = paper_008.read_text(encoding="utf-8")
marker = "## Open Questions for Paper 009"
if marker not in text:
raise RuntimeError("Could not find 'Open Questions for Paper 009' in paper 008")
section = text.split(marker, 1)[1]
next_header = re.search(r"\n##\s+", section)
if next_header:
section = section[: next_header.start()]
questions: list[dict[str, Any]] = []
for line in section.splitlines():
match = NUMBERED_RE.match(line)
if not match:
continue
idx = int(match.group(1))
body = clean_inline_md(match.group(2))
body = re.sub(r"^\*\*", "", body)
body = re.sub(r"\*\*", "", body)
questions.append({"id": idx, "text": body})
if not questions:
raise RuntimeError("No numbered open questions found in paper 008")
return questions
def map_to_open_questions(
docs: list[Doc], open_questions: list[dict[str, Any]]
) -> list[dict[str, Any]]:
coverage: list[dict[str, Any]] = []
for question in open_questions:
qid = question["id"]
keywords = QUESTION_KEYWORDS.get(qid, [])
file_scores: list[dict[str, Any]] = []
total = 0
for doc in docs:
score = 0
snippets: list[str] = []
for sentence in SENTENCE_SPLIT_RE.split(doc.text):
sentence_clean = clean_inline_md(sentence)
hits = sum(1 for kw in keywords if kw in sentence_clean.lower())
if hits:
score += hits
if len(snippets) < 3:
snippets.append(sentence_clean)
if score:
total += score
if score >= 12:
level = "high"
elif score >= 6:
level = "medium"
else:
level = "low"
file_scores.append(
{
"file": doc.slug,
"score": score,
"level": level,
"snippets": snippets,
}
)
file_scores.sort(key=lambda x: x["score"], reverse=True)
coverage.append(
{
"question_id": qid,
"question": question["text"],
"total_score": total,
"supporting_files": file_scores,
}
)
coverage.sort(key=lambda x: x["question_id"])
return coverage
def extract_strongest_challenges(docs: list[Doc]) -> list[dict[str, Any]]:
challenges: list[dict[str, Any]] = []
for doc in docs:
counter_sections = [
text
for name, text in doc.sections.items()
if "counterarguments" in name or "critiques" in name
]
if not counter_sections:
continue
for section in counter_sections:
for line in section.splitlines():
bullet = BULLET_RE.match(line)
if not bullet:
continue
text = clean_inline_md(bullet.group(1))
lower = text.lower()
score = 1
for keyword, weight in CHALLENGE_KEYWORDS.items():
if keyword in lower:
score += weight
challenges.append({"file": doc.slug, "text": text, "score": score})
merged: dict[str, dict[str, Any]] = {}
for challenge in challenges:
key = challenge["text"].lower()
if key not in merged:
merged[key] = {
"text": challenge["text"],
"score": challenge["score"],
"files": {challenge["file"]},
}
else:
merged[key]["score"] += challenge["score"]
merged[key]["files"].add(challenge["file"])
ranked = sorted(
(
{
"text": item["text"],
"score": item["score"],
"files": sorted(item["files"]),
}
for item in merged.values()
),
key=lambda x: x["score"],
reverse=True,
)
return ranked[:10]
def detect_emergent_themes(docs: list[Doc]) -> list[dict[str, Any]]:
themes = {
"Governance and agency design": [
"agency",
"democratic",
"community",
"policy",
"selective adoption",
"governance",
],
"Economic concentration and labor shift": [
"labor",
"capital",
"commodity",
"automation",
"class",
"pricing",
],
"Epistemic reliability and grounding": [
"understand",
"stochastic",
"illusion",
"lossy",
"falsifiable",
"evidence",
],
"Civilizational lock-in and resilience": [
"lock-in",
"path dependence",
"retreat",
"dependency",
"ratchet",
"reversal",
],
}
scored: list[dict[str, Any]] = []
corpus = "\n".join(doc.text.lower() for doc in docs)
for theme, keywords in themes.items():
score = sum(corpus.count(k) for k in keywords)
if score > 0:
scored.append({"theme": theme, "score": score})
scored.sort(key=lambda x: x["score"], reverse=True)
return scored
def build_structured_result(
docs: list[Doc],
scholars: dict[str, dict[str, Any]],
bibliography: dict[str, dict[str, Any]],
contradictions: list[dict[str, Any]],
open_question_coverage: list[dict[str, Any]],
strongest_challenges: list[dict[str, Any]],
emergent_themes: list[dict[str, Any]],
) -> dict[str, Any]:
scholars_ranked = sorted(
scholars.values(),
key=lambda s: (len(s["files"]), s["mention_count"]),
reverse=True,
)
bibliography_ranked = sorted(
bibliography.values(), key=lambda b: b["relevance"], reverse=True
)
return {
"meta": {
"research_files": [doc.slug for doc in docs],
"research_file_count": len(docs),
},
"scholars": scholars_ranked,
"bibliography": bibliography_ranked,
"contradictions": contradictions,
"open_question_coverage": open_question_coverage,
"strongest_challenges": strongest_challenges,
"emergent_themes": emergent_themes,
}
def render_digest(result: dict[str, Any]) -> str:
lines: list[str] = []
lines.append("# Integrated Research Digest")
lines.append("")
lines.append("## Scope")
lines.append(
f"Processed {result['meta']['research_file_count']} research file(s): "
+ ", ".join(result["meta"]["research_files"])
)
lines.append("")
lines.append("## Scholars by Frequency")
for scholar in result["scholars"][:20]:
files = ", ".join(scholar["files"])
lines.append(
f"- **{scholar['name']}** — files: {len(scholar['files'])}; mentions: {scholar['mention_count']}; in: {files}"
)
lines.append("")
lines.append("## Unified Bibliography")
for item in result["bibliography"][:40]:
authors = ", ".join(item["authors"]) if item["authors"] else "Unknown"
files = ", ".join(item["files"])
lines.append(
f"- **{item['title']}** ({authors}) — relevance {item['relevance']}; cited in: {files}"
)
lines.append("")
lines.append("## Contradiction Report")
if not result["contradictions"]:
lines.append("- No cross-file contradictions detected by the current heuristic.")
for item in result["contradictions"]:
lines.append(f"### {item['topic']}")
lines.append("- Supporting evidence:")
for support in item["supports"]:
lines.append(
f" - `{support['file']}` ({support['marker']}): {support['snippet']}"
)
lines.append("- Challenging evidence:")
for challenge in item["challenges"]:
lines.append(
f" - `{challenge['file']}` ({challenge['marker']}): {challenge['snippet']}"
)
lines.append("")
lines.append("## Paper 009 Coverage Map")
for item in result["open_question_coverage"]:
if item["supporting_files"]:
max_level = max(fs["level"] for fs in item["supporting_files"])
else:
max_level = "none"
lines.append(
f"### Q{item['question_id']} (total score {item['total_score']}, strongest level {max_level})"
)
lines.append(f"{item['question']}")
if not item["supporting_files"]:
lines.append("- No supporting material detected.")
continue
for fs in item["supporting_files"][:4]:
lines.append(f"- `{fs['file']}`: score {fs['score']} ({fs['level']})")
for snip in fs["snippets"][:2]:
lines.append(f" - {snip}")
lines.append("")
lines.append("## Strongest Challenges")
if not result["strongest_challenges"]:
lines.append("- No challenge bullets detected.")
for item in result["strongest_challenges"]:
lines.append(
f"- **Score {item['score']}** ({', '.join(item['files'])}): {item['text']}"
)
return "\n".join(lines) + "\n"
def render_outline(result: dict[str, Any]) -> str:
coverage_sorted = sorted(
result["open_question_coverage"], key=lambda x: x["total_score"], reverse=True
)
most_covered = coverage_sorted[:2]
least_covered = coverage_sorted[-2:] if len(coverage_sorted) >= 2 else coverage_sorted
lines: list[str] = []
lines.append("# Suggested Outline for Paper 009")
lines.append("")
lines.append("## Why This Sequence")
lines.append(
"Order starts with heavily-supported questions, then closes with low-coverage questions that require new argumentation or new research."
)
lines.append("")
lines.append("## Coverage Priorities")
lines.append("- Most supported open questions:")
for item in most_covered:
lines.append(
f" - Q{item['question_id']} (score {item['total_score']}): {item['question']}"
)
lines.append("- Least supported open questions:")
for item in least_covered:
lines.append(
f" - Q{item['question_id']} (score {item['total_score']}): {item['question']}"
)
lines.append("")
lines.append("## Proposed Sections")
for item in coverage_sorted:
qid = item["question_id"]
lines.append(f"### Section {qid}: Q{qid}")
lines.append(item["question"])
if item["supporting_files"]:
top_files = ", ".join(fs["file"] for fs in item["supporting_files"][:3])
lines.append(f"- Primary evidence files: {top_files}")
top_snips = [
snip
for fs in item["supporting_files"][:2]
for snip in fs["snippets"][:1]
]
for snip in top_snips:
lines.append(f"- Anchor claim: {snip}")
else:
lines.append("- Primary evidence files: none detected; requires fresh synthesis.")
lines.append("")
lines.append("## Cross-Cutting Counterarguments To Address Explicitly")
for challenge in result["strongest_challenges"][:5]:
lines.append(f"- {challenge['text']} ({', '.join(challenge['files'])})")
lines.append("")
lines.append("## New Themes To Add Beyond Original Open Questions")
for theme in result["emergent_themes"][:4]:
lines.append(f"- {theme['theme']} (signal score {theme['score']})")
return "\n".join(lines) + "\n"
def main() -> int:
args = parse_args()
project_root = args.project_root.resolve()
research_dir = (args.research_dir or (project_root / "research")).resolve()
paper_008 = (args.paper_008 or (project_root / "008-the-ship-of-theseus.md")).resolve()
out_dir = args.out_dir.resolve()
out_dir.mkdir(parents=True, exist_ok=True)
print(f"[integrator] project root: {project_root}")
print(f"[integrator] research dir: {research_dir}")
print(f"[integrator] paper 008: {paper_008}")
print(f"[integrator] output dir: {out_dir}")
docs = load_research_docs(research_dir)
print(f"[integrator] loaded {len(docs)} research file(s)")
if not docs:
print("[integrator] no research files found; writing empty digest/outline")
open_questions = extract_open_questions(paper_008)
print(f"[integrator] extracted {len(open_questions)} open question(s) from Paper 008")
scholars = extract_scholars(docs)
print(f"[integrator] extracted {len(scholars)} unique scholar name(s)")
bibliography = extract_bibliography(docs)
print(f"[integrator] extracted {len(bibliography)} bibliography item(s)")
contradictions = detect_contradictions(docs)
print(f"[integrator] detected {len(contradictions)} contradiction topic(s)")
coverage = map_to_open_questions(docs, open_questions)
print("[integrator] mapped research evidence to Paper 008 open questions")
strongest_challenges = extract_strongest_challenges(docs)
print(f"[integrator] ranked {len(strongest_challenges)} strongest challenge(s)")
emergent_themes = detect_emergent_themes(docs)
print(f"[integrator] found {len(emergent_themes)} emergent theme(s)")
result = build_structured_result(
docs,
scholars,
bibliography,
contradictions,
coverage,
strongest_challenges,
emergent_themes,
)
json_path = out_dir / "integrated.json"
digest_path = out_dir / "digest.md"
outline_path = out_dir / "009_outline_suggestion.md"
json_path.write_text(json.dumps(result, indent=2), encoding="utf-8")
digest_path.write_text(render_digest(result), encoding="utf-8")
outline_path.write_text(render_outline(result), encoding="utf-8")
print(f"[integrator] wrote {json_path}")
print(f"[integrator] wrote {digest_path}")
print(f"[integrator] wrote {outline_path}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/integrator/integrated.json
+7580
View File
File diff suppressed because it is too large Load Diff