From Structural Completion to Analytical Performance: Using Artificial Intelligence-Generated Artefacts to Diagnose Gaps in Student Reasoning

Abstract

This paper presents a structured pedagogical method that shows the gap between structural completion and analytical performance by using deliberately misleading artificial intelligence (AI)-style submissions as evaluation artefacts. The approach responds to an emerging challenge in AI-assisted student work, where students increasingly produce outputs that are fluent and structurally complete but fail to demonstrate the analytical processes required to meet learning outcomes. In such contexts, surface quality can obscure underlying weaknesses in reasoning. The method operates by combining task-function clarification, evaluation of deliberately misleading AI-generated artefacts, and explicit mapping between observed outputs and assessment criteria. Students are anchored to the functional purpose of each component of the task, then evaluate a structurally complete but analytically weak submission, and link observed deficiencies to marking expectations. The approach was developed within a postgraduate marketing module, where AI use is already embedded in student workflows and permitted for idea development and structuring, but not for substantive analysis, and implemented as a first-iteration, partially integrated teaching intervention. This work reframes AI-generated outputs not as shortcuts or risks, but as diagnostic artefacts that surface gaps in analytical reasoning against assessment criteria. It offers a transferable approach for supporting students in developing higher-level analytical capabilities in AI-enabled learning environments.