Presentation Schedule
From Calculator to Signpost? Assessing Human-AI Collaborative Feedback in Political Science Writing (103519)
Session Chair: Erdem Aksoy
Sunday, 8 February 2026 15:30
Session: Session 3
Room: Opal 101 (Level 1)
Presentation Type: Oral Presentation
Generative artificial intelligence (GenAI) tools, powered by large language models (LLMs), are reshaping how instructors design and deliver formative feedback in writing-intensive courses. Yet most empirical studies remain confined to second-language (L2) or composition contexts, leaving their effectiveness in disciplinary writing—particularly in political science—largely unexamined. This study bridges this gap by translating AI capability research into evidence-based classroom practice, focusing on how LLM-assisted feedback affects student writing performance, academic integrity, and AI literacy in a political science setting. Using a randomized crossover design in an Introduction to Politics course (N=35), we compare three feedback conditions—Human-only, AI-only, and Human+AI collaboration—across two authentic writing tasks: an argumentative essay and a policy memo. Key outcome measures include writing quality (double-rated rubric assessing argument clarity, evidence accuracy, coherence, and audience adaptation), citation verification accuracy, AI literacy (four-dimension scale: understanding, evaluation, ethics, and use), and workload efficiency (time-on-feedback for both students and instructors). In addition, all AI interactions, prompt iterations, and evidence verification rates are systematically logged to assess transparency and ethical compliance. By integrating pedagogical, ethical, and governance perspectives, this study moves beyond testing what AI can write to examining how AI feedback reshapes the learning process itself. Preliminary findings are expected to reveal whether LLMs function more as “calculators”—performing surface-level refinements—or as “signposts”—supporting higher-order reasoning and argument development. The project contributes a replicable classroom model for responsible AI-assisted formative feedback in political science education and offers policy-relevant governance indicators—including evidence verification and feedback adoption rates—to guide institutional GenAI policy design and implementation.
Authors:
Shaoshuang Wen, Wenzhou-Kean University, China
Hong Pan, University of Nottingham Ningbo, China
About the Presenter(s)
Dr. Shaoshuang Wen is an Assistant Professor of Political Science at Wenzhou-Kean University in China.
See this presentation on the full schedule – Sunday Schedule
Conference Comments & Feedback
Place a comment using your LinkedIn profileShare this Presentation
Comments
Powered by WP LinkPress