When Artificial Intelligence Closes the Thinking Loop

Artificial intelligence is not the problem. AI cannot decide to use a human being, but a human being can decide how to use AI. The responsibility therefore remains with the person operating the tool.

My concern is not that AI has entered education, business or professional life. Used correctly, it can assist with research, organization, interrogation, refinement and productivity. My concern is that it is increasingly being used before the individual has conceptualized an idea independently. Instead of supporting thought, it is beginning to substitute for the very process through which thought is developed.

There is a natural loop in how human beings develop and apply ideas. We experience something, form thoughts around it, conceptualize those thoughts, implement the resulting idea, and then learn through its success or failure. That experience returns to inform our future thinking. It is not always a perfectly linear process, but every part of the loop contributes to the development of judgement, competence and confidence.

When AI is introduced to support this process after meaningful conceptualization has taken place, it can be extraordinarily useful. However, when it is permitted to complete the conceptualization on behalf of the individual, an essential link in the thinking loop is weakened.

Improved performance is not necessarily learning

This distinction is especially important in education because completing a task and learning how to complete it are not the same thing.

Bastani et al. (2025) demonstrated this through a field experiment involving nearly 1,000 high school mathematics learners. During supported practice, learners using a standard GPT-4 interface achieved results 48% higher than the control group. Learners using a specially designed AI tutor with educational safeguards performed even better during practice.

However, when the standard AI tool was removed and learners had to complete an assessment independently, they performed 17% worse than learners who had never received AI assistance.

The researchers found that many learners had used the unrestricted version as a crutch by requesting and copying solutions rather than working through the problems themselves. The result is significant because the immediate performance created the appearance of progress. The learners completed the work more successfully while the tool was available, but some had not developed the independent capacity required to reproduce that performance.

This is the danger of confusing output with understanding.

An assignment may be completed. A report may be professionally written. A policy may appear coherent. A learner or employee may seem more productive. Yet none of these outcomes proves that the person has developed the knowledge, reasoning or judgement reflected in the final product.

The hurdle itself matters.

A difficult intellectual task is not necessarily an obstruction that should be removed as quickly as possible. The struggle to verbalize a thought, place it on paper, test its logic and develop it into something implementable is part of how intellectual capacity is formed.

In overcoming that hurdle, a person strengthens not only a particular skill but also their belief in their ability to reason and solve problems independently. When AI repeatedly removes that resistance, the hurdle does not simply disappear. It can become a pitfall. Beneath the growing reliance may develop a subconscious sense of inadequacy: the knowledge that the person may not have been able to produce, explain or defend the work independently.

Productivity without judgement

The same tension is visible in the workplace.

Research by Noy and Zhang (2023), published in Science, found that professionals using generative AI completed certain writing tasks approximately 40% faster while producing work rated 18% higher in quality. These findings demonstrate that AI can offer real productivity benefits. It would therefore be intellectually dishonest to argue that AI has no legitimate place in education or business.

The important question is what happens when the tool moves beyond assistance and begins substituting for professional judgement.

Dell’Acqua et al. (2026) studied highly skilled management consultants and described AI capability as a “jagged technological frontier.” Within this frontier, AI improved performance substantially. Consultants using AI completed more tasks and produced higher-quality work. Outside that frontier, however, AI generated inaccurate or less useful information and reduced human performance.

Perhaps the most concerning finding was that AI-assisted answers could remain more persuasive and internally coherent even when the underlying recommendation was wrong. In other words, AI did not merely produce an incorrect answer. It could help produce an incorrect answer that sounded more convincing.

This has serious implications for educational leadership, organizational governance and policy implementation. A polished document can create an illusion of competence. The structure, language and professional presentation may discourage others from questioning whether the proposal is accurate, contextualized or implementable.

The ability to generate persuasive language is not the same as the ability to understand a living system.

AI has not lived inside the system

A school is composed of learners, parents, teachers, colleagues, leaders and communities. The people inside that system experience its pressures, contradictions, limitations and possibilities every day. They possess contextual knowledge that is not always written down and cannot simply be captured through a generic prompt.

AI has not lived inside that system. It does not know the learner whose behavior changed after a disruption at home. It does not understand the parent whose apparent disengagement may be shaped by transport, employment, financial pressure or language barriers. It does not experience the professional dynamics between colleagues, the institutional history behind a policy, or the realities of the community surrounding the school.

The person using AI must therefore already understand the environment well enough to provide relevant context, interrogate the response and recognize when the output does not correspond with reality.

Without that human judgement, AI can produce an articulate but inaccurate interpretation of a problem. That interpretation may then become an educational intervention, business strategy, policy or governance decision that appears impressive on paper but is fundamentally mismatched with the environment in which it must operate.

Many potentially valuable initiatives are lost in precisely this way. The language is polished, the proposal is structured and the implementation plan appears complete, yet the thinking has become detached from the people who must live with its consequences.

This is not a limitation that can be solved only by learning to write better prompts. A detailed prompt can provide more information, but the user must first possess the contextual understanding required to know which information matters. If the individual does not understand the system, they may be unable to recognize what has been excluded, misinterpreted or incorrectly prioritized.

Confidence in the tool and confidence in oneself

Lee et al. (2025) examined 319 knowledge workers and collected 936 examples of how they used generative AI in professional tasks. Their findings showed that greater confidence in AI was associated with less critical thinking, while greater confidence in one’s own task-specific ability was associated with more critical engagement.

This closely reflects the concern at the center of the thinking loop. A person who understands the work is better positioned to question, refine, reject and contextualize AI output. A person who lacks confidence in their own ability may be more likely to accept the output because it sounds more authoritative than their own developing thoughts.

AI also changes the nature of professional thinking. Instead of gathering information, solving the problem and constructing the response, the user increasingly becomes responsible for verifying, integrating and overseeing an AI-generated response.

That is not necessarily harmful. Verification and integration are themselves important cognitive activities. The danger arises when the user lacks the knowledge, motivation or confidence required to perform them properly.

One cannot critically evaluate an answer in a field one does not understand merely because one has access to the answer.

Mentorship is not the same as substitution

This argument should not be interpreted as suggesting that people must think in isolation.

Seeking human mentorship is not evidence of intellectual inadequacy. It is an expression of humility, openness and a commitment to lifelong learning. A mentor introduces new information, challenges existing assumptions and invites the learner to reconsider established patterns of thought.

The learner must still listen, interpret, accept, reject, integrate and rethink. The mentor cannot complete that internal process on the learner’s behalf.

This is important from a neuroplasticity perspective. Learning requires engagement with new information and the integration of experience into existing cognitive structures. Human interaction can provide a particularly meaningful learning environment because it involves feedback, social context, interpretation and adaptation. Li and Jeong (2020), for example, emphasized the importance of socially interactive learning and its relationship with behavioural and neural changes.

Mentorship therefore participates in the development of thought. AI, when misused, can substitute for that development.

A recent commentary published in npj Artificial Intelligence develops a related neurological argument. Rossi, Fraccaro and Manzotti (2026) propose that passive and uncritical reliance on AI may weaken activity-dependent neuroplasticity and contribute to cognitive erosion. In contrast, active engagement; questioning, refining, verifying and co-creating, may sustain or potentially strengthen cognitive functioning.

The authors correctly acknowledge that these propositions require further prospective research. Nevertheless, the distinction is important: the cognitive effect of AI is likely influenced not merely by whether it is used, but by how it is used.

The value of human judgement remains

Further evidence from business reinforces this point.

In a preregistered field experiment involving 791 professionals at Procter & Gamble, Dell’Acqua et al. (2026) found that individuals using AI could match the performance of human teams working without AI. AI also helped professionals produce ideas that crossed traditional functional boundaries.

However, the researchers found that AI primarily improved the generation of ideas, while human judgement retained particular value in evaluating and selecting between those ideas.

This is where AI should sit within the thinking loop. It can broaden possibilities, interrogate assumptions, organize information and refine expression. It can contribute to the development of an idea. It should not automatically be trusted to determine which idea is contextually appropriate, ethically responsible or practically implementable.

Those decisions remain human responsibilities.

AI should not be rejected, nor should education attempt to return to a world in which it does not exist. That would be both unrealistic and intellectually dishonest. Instead, we must teach learners, educators, leaders and professionals where AI belongs within the thinking process.

It should be used after ideas have begun to take shape. It can help interrogate an argument, identify gaps, test alternatives and improve communication. It can challenge the thinker, but it should not replace the formation of thought. It can assist in developing an idea, but it should not be allowed to close the thinking loop on the individual’s behalf.

The question is therefore not whether we should use artificial intelligence.

The more important question is whether we are using it to strengthen the thinker, or to avoid the difficult work through which a thinker is formed.

References

1. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122

2. Dell’Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S., & Lakhani, K. R. (2026). The cybernetic teammate: A field experiment on generative AI and teamwork. Organization Science, 37(4), 1217–1242. https://doi.org/10.1287/orsc.2025.20702

3. Dell’Acqua, F., McFowland, E., III, Mollick, E., Lifshitz, H., Kellogg, K. C., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organization Science, 37(2), 403–423. https://doi.org/10.1287/orsc.2025.21838

4. Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Article 1121, 1–22. https://doi.org/10.1145/3706598.3713778

5. Li, P., & Jeong, H. (2020). The social brain of language: Grounding second language learning in social interaction. npj Science of Learning, 5, Article 8. https://doi.org/10.1038/s41539-020-0068-7

6. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586

7. Rossi, S., Fraccaro, V., & Manzotti, R. (2026). The brain side of human-AI interactions in the long-term: The “3R principle.” npj Artificial Intelligence, 2, Article 15. https://doi.org/10.1038/s44387-025-00063-1