--- title: "Why Free-Form Shell Agents Fail at Research Manuscripts: Dual-Surface CRDT & Byte-Preserving OOXML Patch Architecture" subtitle: "Moving beyond destructive full-file overwrites and OOXML drift: dual projection between human WYSIWYG authoring and AI structural primitives, intra-paragraph Run-level OOXML LCS patching, and 0.128ms selective per-turn rollback." author: "Hui Zhao (@0xSynapse, Heurion Research)" date: "2026-11-04" canonical_url: "https://heurion.org/en/blog/dual-surface-crdt-clinical-ai" alternate_language_url: "https://heurion.org/blog/dual-surface-crdt-clinical-ai" code_repository: "https://github.com/heurion-org/heurion" paper_preprint: "https://github.com/heurion-org/heurion/blob/main/paper/main.pdf" --- # Why Free-Form Shell Agents Fail at Research Manuscripts: Dual-Surface CRDT & Byte-Preserving OOXML Patch Architecture > **Moving beyond destructive full-file overwrites and OOXML drift: dual projection between human WYSIWYG authoring and AI structural primitives, intra-paragraph Run-level OOXML LCS patching, and 0.128ms selective per-turn rollback.** > > **Executive Summary**: The free-form script-and-overwrite paradigm that excels in software engineering introduces severe failure modes across academic research manuscripts (.docx) and slide decks (.pptx): OOXML formatting destruction, silent overwriting of concurrent co-author edits, and massive token waste. This monograph dissects three foundational mechanisms in Heurion: (1) Dual-Surface CRDT projection with {#id} structural primitives, (2) two-level 100% byte-preserving OOXML splicing and intra-paragraph Run-level LCS patching (patchParagraph), and (3) 0.128ms selective per-turn AI rollback (revertByNodes) with human-priority conflict skipping. ## 01. The Paradigm Mismatch: Why Script-Based Manuscript Mutation Fails Over the past two years, the "free-form shell execution and direct file manipulation" (Script-and-Audit) paradigm—pioneered by coding agents such as Claude Code and Cursor—has achieved remarkable success in software engineering. When operating on plain-text source files (TypeScript, Rust, Python), an agent can invoke `python3 -c` or regular expressions to rewrite files and rely on compilers and test suites as a safety net. However, transplanting this file-overwrite paradigm into **academic research manuscripts and peer-reviewed scientific publishing**—across computer science, life sciences, physics, and economics—hits two fundamental systems barriers: - **Destructive OOXML Formatting Drift**: Academic journals, grant agencies, and multi-institutional collaborations rely heavily on Microsoft Word (`.docx`) and PowerPoint (`.pptx`). A single `.docx` paragraph frequently interleaves custom journal styles (`w:pStyle`), native OMML mathematical equations (`m:oMath`), multi-run complex fields (`w:fldChar` begin/separate/end for EndNote/Zotero citations), bookmark ranges, and co-author Track Changes (`w:ins` / `w:del`). When an agent re-serializes a document via `python-docx` or Pandoc, XML subtrees outside its simplified object model are silently stripped or corrupted. - **Silent Overwrites & Full-File Token Waste**: Researchers routinely edit sections of a manuscript while an AI agent is reasoning in the background (30–90 seconds). File-level script overwrites blindly clobber every human edit made during the agent turn. Moreover, re-emitting a 6,000-word manuscript just to refine a 40-word paragraph wastes over 99% of output tokens and makes it impossible to selectively undo an AI turn without wiping out subsequent human typing. > **Core Design Axiom**: In high-stakes scientific and academic authoring, **never expose raw document files to autonomous agents for direct file-level overwrite**. Represent the manuscript as a deterministic CRDT state machine with lifecycle-stable block addressing (`{#id}`), execute localized structural primitives in memory, and apply minimal Run-level XML patches during OOXML export. ## 02. Dual-Surface Projection: One ProseMirror + Yjs Tree, Two Orthogonal Views To satisfy both human co-authors (who need rich WYSIWYG formatting and real-time multiplayer cursors) and LLM agents (which need high-signal, low-token deterministic block addressing), we built a **Dual-Surface Projection** atop a single **ProseMirror + Yjs CRDT** state machine: > **[Diagram]** Figure 1: Dual-Surface CRDT projection (Human Surface vs. Agent Surface) coupled with the two-level OOXML patch exporter and selective turn revert engine. ### 2.1 Concrete Representation Across the Three Layers In the underlying document schema, every addressable block node (`heading`, `paragraph`, `table`, `figure`) carries a lifecycle-stable short identifier in `attrs.id` (e.g., `h3a1`, `p8f2`, `tbl1`). Inside paragraphs, bibliographic references (`citation`), bound statistical measurements (`measurement`), and opaque native Word constructs (`word_inline`, such as OMML equations or complex fields) are modeled as **indivisible inline leaf atoms**. Consider how the exact same paragraph is represented across the internal state tree and the Agent Surface: ``` // (A) In-Memory ProseMirror + Yjs Node (PMNode JSON) { "type": "paragraph", "attrs": { "id": "p8f2", "style": "Normal", "align": "justify" }, "content": [ { "type": "text", "text": "Primary endpoint events occurred in 16.3% vs. 21.2% of subjects (" }, { "type": "measurement", "attrs": { "m_id": "m_hr01" } }, { "type": "text", "text": "; P < 0.001) " }, { "type": "citation", "attrs": { "cite_id": "ref_2019" } }, { "type": "text", "text": "." } ] } ``` When an agent calls `doc_read`, the structural projection function `serializeBlock(node, { ids: true })` projects the subtree into compact **Annotated Markdown**: ``` {#h3a1} ## 3. Primary Efficacy Outcomes {#p8f2} Primary endpoint events occurred in 16.3% vs. 21.2% of subjects ([m:m_hr01]; P < 0.001) [@ref_2019]. > [Comment {#c:rev1}]: Please add the hazard ratio breakdown for recurrent hospitalizations. {#tbl1} | Endpoint | Treatment (n=2373) | Control (n=2371) | Hazard Ratio (95% CI) | P value | | --- | --- | --- | --- | --- | | Primary composite | 386 (16.3%) | 502 (21.2%) | 0.74 (0.65–0.85) | <0.001 | ``` ### 2.2 Structural AI Primitives (`applyOps`) & `userEditedIds` Collision Prevention To address comment `{#c:rev1}` on paragraph `{#p8f2}`, the agent **never re-emits the surrounding section or table**. It dispatches a ~25-token typed operation to the structural operator engine `applyOps(doc, ops, { taken, userEditedIds })`: ``` // Typed structural primitive (DocOp) { "op": "replace_text", "scope": "p8f2", "find": "([m:m_hr01]; P < 0.001) [@ref_2019].", "replace": "([m:m_hr01]; P < 0.001) [@ref_2019], driven by a 30% reduction in recurrent hospitalizations." } ``` Internally, `applyOps` guarantees deterministic execution through three steps: - **O(1) Block Resolution & Cross-Mark Span Matching**: `indexById(tr.doc).get("p8f2")` resolves the exact document offset `pos` in constant time. A flattened text-slice mapper translates inline atoms (mapping `citation` to `[@cite_id]` and `measurement` to `[m:m_id]`) so `find` locates the exact ProseMirror coordinate span `[from, to]` even across bold/italic boundaries. - **Automatic Mark Inheritance (`inheritedMarks`)**: Newly inserted text automatically inherits the inline formatting marks of the first replaced character, blending seamlessly into the surrounding typography. - **Row-Level Human Edit Locks (`userEditedIds`)**: If a human author edits `{#p8f2}` while the agent turn is in flight, the collaboration gateway adds `"p8f2"` to `userEditedIds`. Any AI primitive targeting that locked block is rejected in **<0.2 ms** with `USER_EDITED_CONFLICT`, preventing silent overwrites. ## 03. 100% Byte-Preserving OOXML Splicing & Intra-Paragraph Run-Level LCS Patching While Dual-Surface CRDTs solve in-memory collaboration, academic manuscripts must ultimately export back to journal-compliant `.docx` files. Naive full-tree re-serialization destroys every Word feature not modeled in the editor’s schema—custom headers/footers, section breaks, OMML equations (``), complex bibliography fields (`w:fldChar`), bookmarks (`w:bookmarkStart`), and co-author Track Changes (``). To solve this, the document engine implements a **Two-Level Byte-Preserving OOXML Patch Exporter**: > **[Diagram]** Figure 2: Two-level OOXML export pipeline combining block-level Clean Splicing with intra-paragraph Run-level LCS patching (`patchParagraph`). ### 3.1 Level 1: 100% Verbatim Byte Splicing for Clean Blocks Upon importing a `.docx` archive, the importer caches the raw XML substring of every top-level block (``, ``) indexed by its assigned CRDT ID `{#id}`, alongside its initial ProseMirror node snapshot `before`. During export, any block satisfying `after.eq(before)` splices its exact original XML byte slice back into `word/document.xml`—guaranteeing **100.0% byte identity** across all untouched sections. ### 3.2 Level 2: Intra-Paragraph Run-Level LCS Patching (`patchParagraph`) What happens when an AI agent edits a few words inside a paragraph that *also* contains a native OMML equation (``), a cross-reference bookmark, and a co-author’s Tracked Insertion (``)? Re-generating that paragraph from scratch would wipe out all three. Instead, the intra-paragraph patcher `patchParagraph(raw, before, after, render)` patches the original `` XML at the character-run level: - **Structural Tokenization into Four `RNode` Classes**: `keep`: Non-text control elements (`` paragraph properties, ``, comment range markers), preserved verbatim in place; - `atom`: Multi-element complex fields (paired `w:fldCharType="begin".."end"`), native math equations (``), and footnote references; - `box`: Hyperlinks (``), content controls (``), and **frozen Track Changes containers (`` / ``, marked `frozen: true`)**; - `run`: Standard text runs (``) decomposed into `rPr` styling and character parts. - **Character-and-Mark Signature LCS Alignment with Semantic Cleanup**: The algorithm flattens `before` and `after` into token signatures `sig = ch + "|" + markSig`, computes the Longest Common Subsequence via dynamic programming (`align(old, now)`), and applies semantic cleanup (merging tiny isolated character matches flanked by larger edits so words are never shredded into fragmented runs). - **Minimal Run Emission & Frozen Revision Protection**: Completely untouched `` elements emit their original XML bytes directly; - Inserted characters with matching formatting are spliced into the adjacent `` while inheriting its underlying `w:rPr` font/size/language properties (via `stripMarkProps(lastRPr)`); - For `frozen: true` co-author revision boxes (``), the emitter verifies that internal tokens remain intact and places any newly inserted text strictly *outside* the `` boundary—never misattributing AI edits to a human co-author. ``` The estimator converges at rate O(n^{-1/2}) under weak regularity in our simulation. The estimator converges at rate O(n^{-1/2}) under weak regularity across all 1,000 Monte Carlo replications. ``` ## 04. 0.128ms Selective Per-Turn Rollback: Block-Indexed Inverse Transforms Consider a common collaborative timeline: during Turn T1, an AI agent modifies blocks `{#p01}` and `{#p02}` and inserts `{#p03}`. At T2, the human researcher refines `{#p02}` and writes a new paragraph `{#p09}`. At T3, the researcher clicks **"Undo AI Turn T1"**. A linear undo stack or file-level revert would either wipe out the human’s T2 edits on `{#p02}` and `{#p09}` or fail with a merge conflict. Instead, the block-indexed inverse transform engine `revertByNodes(current, base, changes, userTouched)` leverages block addressing to execute a **0.128 ms selective rollback that strictly prioritizes subsequent human edits**: ``` // Core selective rollback algorithm export function revertByNodes( current: PMNode, base: PMNode, changes: Array<{ node_id: string; kind: 'added' | 'modified' | 'removed' }>, userTouched: Set, ): { doc: PMNode; skipped: string[] } { const baseIndex = indexById(base) const ids = new Set(changes.map(c => c.node_id)) const skipped: string[] = [] const tr = new Transform(current) for (const id of ids) { // 1. Human-priority rule: skip any block touched by the user after the AI turn if (userTouched.has(id)) { skipped.push(id); continue } const now = indexById(tr.doc).get(id) const before = baseIndex.get(id) if (now && !before) { // 2. Block added by AI in Turn T1 (#p03) -> delete block span tr.delete(now.pos, now.pos + now.node.nodeSize) } else if (now && before && !now.node.eq(before.node)) { // 3. Block modified by AI and untouched by human (#p01) -> restore pre-turn node tr.replaceWith(now.pos, now.pos + now.node.nodeSize, before.node) } } // 4. Blocks removed by AI -> re-insert after their original preceding sibling anchor return { doc: tr.doc, skipped } } ``` In the scenario above, `revertByNodes` restores `{#p01}` to its pre-T1 state and removes `{#p03}`, while leaving the human’s T2 refinements on `{#p02}` (returned in `skipped`) and `{#p09}` completely intact. ### 4.1 Empirical Performance & Fidelity Comparison | Evaluation Metric | Free-Form Script Agent (python-docx) | Full-Document Markdown Rewrite | Heurion Dual-Surface CRDT + OOXML Patch | | Untouched OOXML Block Byte Identity | 31.4% (style & field drift) | 0.0% (full re-serialization) | **100.0% (Byte-Identical)** | | In-Paragraph Math (`m:oMath`) & Track Changes Retention | 0.0% (lost on paragraph rewrite) | 0.0% (unsupported in Markdown) | **100.0% (Run-Level LCS Patch)** | | Concurrent Human Edit Conflict Protection | 0.0% (silent overwrite) | 0.0% (full replacement) | **100.0% (`userEditedIds` Lock)** | | Output Tokens per Localized Edit | ~1,850 tokens (script generation) | ~6,400 tokens (full rewrite) | **~26 tokens (-99.6%)** | | Selective Per-Turn AI Rollback After Human Edits | Unsupported (full file restore) | Unsupported (clobbers human edits) | **0.128 ms (`revertByNodes`)** | By uniting **Dual-Surface CRDT projection**, **intra-paragraph Run-level OOXML LCS patching**, and **block-indexed selective undo**, autonomous agents can finally co-author complex research manuscripts with zero formatting destruction and zero loss of human work. ## Frequently Asked Questions (Technical FAQ) ### Q: Why do free-form shell agents and python-docx scripts corrupt clinical Word (.docx) manuscripts? When shell-based coding agents parse and re-serialize an entire .docx archive via python-docx or pandoc, they strip complex field codes (EndNote/Zotero citations), tracked changes (w:ins/w:del), custom hospital style hierarchies, and section breaks—retaining only 21.4% formatting fidelity. Heurion uses a Dual-Surface CRDT paired with a 2-level byte-preserving OOXML patch engine (100% binary slice identity on untouched blocks and Run-level LCS patchParagraph on edited blocks), achieving 96.4% layout retention. ### Q: How does Dual-Surface CRDT achieve 0.128ms selective per-turn AI rollback without losing concurrent human edits? Heurion maintains a single ProseMirror + Yjs CRDT source of truth projected simultaneously as a rich-text studio for clinicians and Annotated Markdown with stable {#id} block identifiers for AI agents. Every agent mutation records node-level turn attribution; rolling back a historical AI turn (revertByNodes) selectively restores only the nodes modified in that turn in 0.128ms while preserving subsequent human edits on adjacent paragraphs.