EducationOriginal

From AI Debate to Learning Design: When Does GenAI Support Student Learning?

From AI Debate to Learning Design: When Does GenAI Support Student Learning?

Yanling Liu

What happens when a student asks an AI chatbot to solve a problem, explain a concept, or draft an answer? The answer may depend less on the technology itself than on what happens after the response is generated. Research on generative artificial intelligence (GenAI) in higher education tells two seemingly different stories. Some studies show that students achieve better learning outcomes when they use GenAI. Others suggest that students may become less engaged, less reflective, or overly dependent on the technology. At first glance, these findings appear contradictory. However, a closer look reveals that they may be describing different learning environments rather than conflicting results. Instead of asking whether GenAI improves learning, a more useful question is: Under what conditions does GenAI support learning? Emerging research suggests that learning outcomes depend on instructional design, teacher support, student engagement, and opportunities for reflection and critical evaluation.

—Under what conditions does GenAI support learning?—

One of the clearest findings concerns the role of the instructor. A meta-analysis of GenAI interventions and academic performance found a statistically significant positive overall effect of GenAI-supported instruction on students’ academic performance. However, teacher support was the strongest moderator examined, meaning that the impact of GenAI varied depending on the guidance students received. Simply giving students access to GenAI did not guarantee better outcomes. Another meta-analysis reached a similar conclusion. While it found a medium positive effect of GenAI on learning, the strongest results occurred in collaborative and blended learning environments. That said, teaching method was the only statistically significant moderator. Together, these studies suggest that GenAI’s educational value is shaped not only by the technology but also by instructional design. What instructors ask students to do and the level of support they provide play a critical role in shaping learning outcomes.

The next question is what students actually do with GenAI. Research suggests that learning benefits are strongest when students actively engage with AI-generated content rather than simply accept it. One study found improved learning outcomes when students used GenAI to construct and extend knowledge. These students connected AI responses to course concepts, compared them with their own understanding, justified their decisions, and revised their work accordingly. In contrast, learning gains were weaker when students primarily used AI to reproduce answers. Research on critical thinking shows a similar pattern. A systematic review of empirical studies on GenAI and critical thinking found that students were more likely to develop critical-thinking skills when they need to reflect on AI-generated responses and evaluate and explain the information they received. The implication is straightforward: GenAI should not do all of the intellectual work. Instead, learning activities should encourage students to question GenAI outputs, verify important claims, compare responses with their own thinking, and decide what to accept, revise, or reject.

These findings point to several recurring conditions that appear to support effective GenAI-assisted learning. The six conditions summarized below emerged repeatedly across the studies reviewed. While they should not be viewed as a formula for success, they provide a useful framework for thinking about when GenAI is most likely to enhance learning.

Six Conditions That Can Shape GenAI-Supported Learning

ConditionWhat it involvesWhy it matters
Faculty supportGuidance, scaffolding, modeling, feedback, and evaluationHelps students use GenAI purposefully
High challengeComplex, meaningful, non-routine tasksEncourages deeper thinking
Constructive useComparing, evaluating, explaining, revising, and reflectingPositions GenAI as a learning partner
Critical literacyEvaluating accuracy, sources, and limitationsSupports informed judgment
Intentional designDeliberate choices about tasks, prompts, pedagogy, and assessmentKeeps students engaged
Student agencyMaking decisions, contributing their own reasoning, and retaining responsibility for the workHelps prevent overreliance and cognitive offloading

Although many studies identify benefits, the evidence also highlights important risks. GenAI does not automatically lead to improved learning, and under some conditions it may produce the opposite effect. For example, one study found that greater dependency on GenAI was associated with false self-efficacy and lower academic achievement. Interestingly, perceived teacher caring reduced part of this negative relationship, again underscoring the importance of human support. Similarly, a large survey found that GenAI use was generally associated with higher cognitive and emotional engagement, but the relationship varied across learning environments. The strongest positive effects were observed in courses that combined high challenge with high levels of support. In contrast, in lower-challenge courses, GenAI use was associated with less active learning and lower student motivation. These findings suggest that access to GenAI alone is insufficient; students also need meaningful intellectual challenges and instructional guidance.

Research in STEM disciplines illustrates this tension particularly well. A meta-analysis examining GenAI across academic disciplines found that disciplinary context influenced higher-order cognitive outcomes and identified small-group STEM instruction as a promising setting for deeper learning. At the same time, an emerging study of routine GenAI use among STEM students found that habitual, trust-driven reliance on AI was associated with lower levels of reflection, critical thinking, and understanding. Together, these findings suggest that discipline creates opportunities for learning, but it does not determine learning outcomes. What matters most is how students engage with GenAI within those disciplinary contexts.

These insights have important implications for information literacy. Students increasingly use GenAI to discover information, summarize sources, interpret evidence, and support research. As a result, they need skills that extend beyond prompt writing. They must be able to evaluate reliability, identify evidence, recognize omissions, and verify claims using credible sources. This need creates opportunities for collaboration between faculty and librarians. Faculty contribute disciplinary expertise, learning goals, and assessment strategies, while librarians provide expertise in information evaluation, authority, provenance, source comparison, and research practices. Together, they can design learning activities that encourage students to use GenAI critically rather than passively. For example, an AI-generated response can serve as the starting point for a research exercise in which students trace claims to scholarly sources, evaluate supporting evidence, and explain where the response is incomplete, unsupported, or inaccurate. In this approach, GenAI becomes a tool for inquiry rather than a substitute for it.

It is also important to recognize that student success extends beyond academic performance. Learning quality, engagement, agency, confidence, and well-being matter as well. Research suggests that GenAI may provide personalized support while also introducing risks such as digital fatigue, loneliness, technostress, and reduced fact-to-face interactions. Educators should therefore evaluate AI-enhanced learning not only by what students know, but also by how they learn and who they are becoming as learners.

The evidence points toward a practical path forward: establish learning goals before introducing GenAI, maintain meaningful cognitive challenge, and require students to verify, compare, reflect, and revise. By helping students connect AI-generated information to evidence and informed judgment, faculty and librarians can create learning environments in which GenAI enhances, rather than replaces, the learning process. Ultimately, the question is not whether GenAI belongs in higher education. It is already here. The more important question is: What kind of learning are we designing when GenAI becomes part of the process?

Cite this article in APA as: Liu, Y. (2026, October 21). From AI debate to learning design: When does GenAI support student learning? Information Matters. https://informationmatters.org/2026/10/from-ai-debate-to-learning-design-when-does-genai-support-student-learning/

Author

  • Yanling Liu is the Coordinator of Information Literacy and Research Services at Indiana University Kokomo Library, where she provides strategic leadership for initiatives that advance information literacy, research support, and student engagement. Her work focuses on gamification, accessibility, and the integration of generative AI in teaching and learning.

    View all posts
Subscribe to IM weekly updates