Presentation
Prompt-Level Supervisory Alignment: Operationalizing Supervisory Control Theory for LLM-Guided Manuscript Revision
SessionWednesday Poster Session
DescriptionResearchers increasingly rely on Large Language Models (LLMs) for manuscript revision, yet this process remains largely unstructured, with potential for cognitive skill erosion and shallow revisions. We introduce Prompt-Level Supervisory Alignment (PLSA), a training-free framework using in-context learning that operationalizes Supervisory Control Theory (SCT) as a structured prompting strategy for LLM-assisted revision. PLSA converts peer-review feedback into hierarchical SCT artifacts spanning five supervisory functions: planning, instructing, monitoring, intervening, and learning, and uses them to guide an LLM through a feedback-driven revision loop. We evaluated PLSA using 30 ICLR 2025 paper and review pairs across five conditions, comparing raw review baselines against SCT-structured and synthetic review conditions. Results shows a quality vs fidelity tradeoff: SCT-structured conditions produced revisions rated higher in completeness and rigor by an LLM judge, but these revisions diverged more from the authors' original rewrites, this is consistent with PLSA encouraging deeper structural changes over more surface-level edits. The monitoring and intervention loop also functioned as an automated quality gate, leading to corrections on most first-pass revisions. PLSA shows that established human-factors principles can be transferred meaningfully to LLM alignment without fine-tuning.
Event Type
Poster
TimeWednesday, October 21st5:30pm - 6:30pm PDT
Location