Trust and reliance on AI: An experimental study on the extent and costs of overreliance on AI

Klingbeil, A., Grützner, C., & Schreck, P. (2024).

Computers in Human Behavior, 160, 108352.

https://doi.org/10.1016/j.chb.2024.108352

Abstract

Decision-making is undergoing rapid changes due to the introduction of artificial intelligence (AI), as AI recommender systems can help mitigate human flaws and increase decision accuracy and efficiency. However, AI can also commit errors or suffer from algorithmic bias. Hence, blind trust in technologies carries risks, as users may follow detrimental advice resulting in undesired consequences. Building upon research on algorithm appreciation and trust in AI, the current study investigates whether users who receive AI advice in an uncertain situation overrely on this advice — to their own detriment and that of other parties. In a domain-independent, incentivized, and interactive behavioral experiment, we find that the mere knowledge of advice being generated by an AI causes people to overrely on it, that is, to follow AI advice even when it contradicts available contextual information as well as their own assessment. Frequently, this overreliance leads not only to inefficient outcomes for the advisee, but also to undesired effects regarding third parties. The results call into question how AI is being used in assisted decision making, emphasizing the importance of AI literacy and effective trust calibration for productive deployment of such systems.

Highlights

• People overrely on AI advice for financially risky decisions in a domain-independent, interactive, behavioral experiment.

• Mere knowledge of advice being generated by an AI causes people to overrely on it.

• Participants follow AI advice that conflicts with available contextual information and is against their own interests.

• Overreliance on AI advice negatively affects human cooperation, leading to undesired results for advisees and third parties.

• Participants with higher trust in the advisor (attitude) also exhibit higher reliance on advice (behavior).

GPT-4 generated psychological reports in psychodynamic perspective

Kim, N., Lee, J., et al. (2025).

Frontiers in psychiatry, 16, 1473614.

https://doi.org/10.3389/fpsyt.2025.1473614

Abstract

Background: Recently, there have been active proposals on how to utilize large language models (LLMs) in the fields of psychiatry and counseling. It would be interesting to develop programs with LLMs that generate psychodynamic assessments to help individuals gain insights about themselves, and to evaluate the features of such services. However, studies on this subject are rare. This pilot study aims to evaluate quality, risk of hallucination (incorrect AI-generated information), and client satisfaction with psychodynamic psychological reports generated by GPT-4.

Methods: The report comprised five components: psychodynamic formulation, psychopathology, parental influence, defense mechanisms, and client strengths. Participants were recruited from individuals distressed by repetitive interpersonal issues. The study was conducted in three steps: 1) Questions provided to participants, designed to create psychodynamic formulations: 14 questions were generated by GPT for inferring psychodynamic formulations, while 6 fixed questions focused on the participants’ relationship with their parents. A total of 20 questions were provided. Using participants’ responses to these questions, GPT-4 generated the psychological reports. 2) Seven professors of psychiatry from different university hospitals evaluated the quality and risk of hallucinations in the psychological reports by reading the reports only, without meeting the participants. This quality assessment compared the psychological reports generated by GPT-4 with those inferred by the experts. 3) Participants evaluated their satisfaction with the psychological reports. All assessments were conducted using self-report questionnaires based on a Likert scale developed for this study.

Results: A total of 10 participants were recruited, and the average age was 32 years. The median response indicated that quality of all five components of the psychological report was similar to the level inferred by the experts. The risk of hallucination was assessed as ranging from unlikely to minor. According to the median response in the satisfaction evaluation, the participants agreed that the report is clearly understandable, insightful, credible, useful, satisfying, and recommendable.

Conclusion: This study suggests the possibility that artificial intelligence could assist users by providing psychodynamic interpretations.

Here are some thoughts:

This study tested whether GPT-4 could write useful psychodynamic reports for people with relationship problems. Experts rated the AI reports as similar in quality to what a human expert would write. The risk of harmful errors was low, and the clients found the reports insightful and helpful. However, the study was small and had limitations, including the risk that the AI might make an insensitive or upsetting interpretation. The main takeaway is that AI shows promise as a support tool for mental health, but human oversight is still essential.

Automation bias and assistive AI.

Khera, R., Simon, M. A., & Ross, J. S. (2023).

JAMA, 330(23), 2255. 

https://doi.org/10.1001/jama.2023.22557

At the point of care, artificial intelligence (AI) algorithms have been developed to augment diagnostic decisions and suggest appropriate care pathways, by leveraging complex information in a patient’s electronic health record, such as imaging, documentation, and diagnostic testing. With an increasing number of technologies integrated into the diagnosis, management, and even treatment of patients, the promise of AI to enhance accuracy, reduce errors, reduce clinician burnout, and improve clinical workflows may appear imminent.

MostAI algorithms aredesigned tobe assistive technologies—augmenting, not replacing, clinicians’

decision-making. AI models are imperfect and lack the broader clinical context that may be relevant for patient care. The expectation is that the diagnostic performance of clinicians supported by AI will exceed those of clinicians without such support.

A pdf is here.

Here are some thoughts:

This article highlights a critical problem with artificial intelligence in medicine: automation bias. This is when clinicians trust an AI’s recommendation too much, even when it is clearly wrong or contradicts their own judgment. The authors show that biased AI models can significantly lower the quality of patient care, and simply explaining how the AI works does not fix the issue. Clinicians, often working under time pressure, may defer to the tool instead of using their own expertise, which can lead to direct patient harm.

The key takeaway is that keeping a human “in the loop” is not enough to ensure safety. Current regulatory approaches focus too much on the AI’s technical accuracy and not enough on how real clinicians actually use these tools in practice. The authors argue that better training, higher safety standards, and truly interpretable AI are needed. Without these changes, the excitement around medical AI risks overshadowing its primary goal: improving patient care, not undermining it.

Exploring the frontiers of LLMs in psychological applications: a comprehensive review.

Ke, L., Tong, S., Cheng, P., & Peng, K. (2025).

Artificial Intelligence Review, 58(10).

https://doi.org/10.1007/s10462-025-11297-5

Abstract

This review explores the frontiers of large language models (LLMs) in psychological applications. Psychology has undergone several theoretical changes, and the current use of artificial intelligence (AI) and machine learning, particularly LLMs, promises to open up new research directions. We aim to provide a detailed exploration of how LLMs are transforming psychological research. We discuss the impact of LLMs across various branches of psychology—including cognitive and behavioral, clinical and counseling, educational and developmental, and social and cultural psychology—highlighting their ability to model patterns, cognition, and behavior similar to those observed in humans. Furthermore, we explore the ability of such models to generate coherent, contextually relevant text, offering innovative tools for literature reviews, hypothesis generation, experimental designs, experimental subjects, and data analysis in psychology. We emphasize the importance of addressing technical and ethical challenges, including data privacy, the ethics of using LLMs in psychological research, and the need for a deeper understanding of these models’ limitations. Researchers should use LLMs responsibly in psychological studies, adhering to ethical standards and considering the potential consequences of deploying these technologies in sensitive areas. Overall, this review provides a comprehensive overview of the current state of LLMs in psychology, exploring the potential benefits and challenges. We hope it can serve as a call to action for researchers to responsibly leverage LLMs’ advantages while addressing the associated risks.

Here is a great quote from the article: “LLM output should not be mistaken for the presence of thought but instead viewed as complex pattern matching based on probabilistic modeling.”

Here are some thoughts:

This review provides a timely and comprehensive framework for understanding how LLMs are transforming psychological research, organized around Newell's hierarchical timescales of human behavior. The authors strike an excellent balance between enthusiasm for LLMs' emergent abilities, such as analogical reasoning and emotion recognition, and a critical awareness of their fundamental limitations, including the lack of genuine understanding, persistent biases toward WEIRD populations, and risks in clinical applications like suicide risk assessment. The paper is particularly strong in its systematic presentation of empirical findings across cognitive, clinical, educational, and social psychology, supported by clear tables that make specific applications and results easily accessible to researchers. 

While the review covers LLMs as both research tools and simulated subjects, it could further explore the epistemological risks of circular validation where LLMs are used to study behaviors they merely replicate from training data. Additionally, greater attention to open source models and the inherent constraints of transformer architectures for real time or developmental processes would strengthen future work. Overall, this article serves as an essential resource for psychologists seeking to responsibly integrate LLMs into their research, offering both practical guidance and ethical guardrails without succumbing to technological hype.

Responsible Use of AI in Assessment

American Psychological Association

The information is here.

Summary

Artificial intelligence (AI) is increasingly used in psychological and educational assessment for tasks like scoring, summarizing, reporting, and pattern recognition. Thoughtful use of AI can improve efficiency, consistency, and service access. However, AI systems may introduce bias, errors, and lack transparency, so their risks must be carefully considered due to the significant impact of assessment decisions. While traditional considerations and evaluation criteria for practicing and researching assessment remain relevant, the integration of AI introduces unique factors that must be understood and addressed to ensure validity, reliability, fairness, and transparency.

To address these concerns, the members of APA’s Committee on Psychological Tests and Assessment (CPTA) have developed a concisely presented, comprehensive document that delves into the ethical and practical considerations for the use of AI in assessment across domains (e.g., clinical, I/O, school) and situations (e.g., employment testing, clinical evaluations). The document identifies considerations pertinent at specific decision-making junctures (e.g., tool selection, administration/delivery, scoring, interpretation, reporting) as well as considerations that apply across all assessment activities. The intended audience for this document is psychologists, including but not limited to health service psychologists and psychologists working in industry, academia, and public service positions as well as students of psychology. Although not the intended audience, this document may also serve as a resource for consumers of psychology and the public.

Principles for responsible AI use in assessment

Eight key areas to consider whenever AI is used in psychological assessment:

• Transparency and accountability
• Bias and fairness
• Privacy and confidentiality
• Informed consent
• Competence and training
• Human oversight
• Impact on applied and clinical work
• Continuous improvement

Principled by Design: Ethical Decision-making with Integrity

Gavazzi, J. (2026).

www.ethicalpsychology.com

This article is self-published for inclusion in a home study offered through the Pennsylvania Psychological Association. The home study promotes a structured approach to ethical decision-making, designed to support self-reflective practice.

Clinical Impact Statement

This article offers psychologists a practical, principle-based framework for working through ethical dilemmas in clinical practice. By treating autonomy, beneficence, nonmaleficence, justice, and fidelity as competing obligations to be specified and balanced rather than rules to be memorized, the framework helps clinicians reason transparently through situations in which the Ethics Code alone does not provide clear direction. It supports more defensible decisions, stronger therapeutic relationships, and the kind of reflective practice that treats ethics as an aspiration rather than a minimum standard.

The article is here.

The companion worksheet is here.

AI generates covertly racist decisions about people based on their dialect.

Hofmann, V., et al. (2024).

Nature, 633(8028), 147–154.

https://doi.org/10.1038/s41586-024-07856-5

Abstract

Hundreds of millions of people now interact with language models, with uses ranging from help with writing to informing hiring decisions. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.

Here are some thoughts:

This research article demonstrates that artificial intelligence language models exhibit covert racism through deep-seated dialect prejudice against speakers of African American English. By evaluating language variations, the authors found that these models attach negative stereotypes to African American English that are more severe than any human stereotypes ever recorded experimentally, even while their overt statements about Black individuals appear positive. For psychologists, this study is highly important because it reveals how systemic racism and implicit bias can be stealthily automated and amplified within technology. It underscores that human preference training merely masks superficial bias while leaving harmful, underlying prejudices fully intact. Insightfully, the findings warn that relying on artificial intelligence for clinical diagnostics, forensic evaluations, or employment screening can lead to discriminatory outcomes, such as harsher legal judgments or lower prestige job recommendations. Psychologists must therefore spearhead critical evaluations of these tools to ensure digital assessments do not reinforce historical inequities.

The Artificial Third: A Broad View of the Effects of Introducing Generative Artificial Intelligence on Psychotherapy

Haber, Y., et al. (2024).

JMIR Mental Health, 11, e54781.

https://doi.org/10.2196/54781

Abstract

This paper explores a significant shift in the field of mental health in general and psychotherapy in particular following generative artificial intelligence’s new capabilities in processing and generating humanlike language. Following Freud, this lingo-technological development is conceptualized as the “fourth narcissistic blow” that science inflicts on humanity. We argue that this narcissistic blow has a potentially dramatic influence on perceptions of human society, interrelationships, and the self. We should, accordingly, expect dramatic changes in perceptions of the therapeutic act following the emergence of what we term the artificial third in the field of psychotherapy. The introduction of an artificial third marks a critical juncture, prompting us to ask the following important core questions that address two basic elements of critical thinking, namely, transparency and autonomy: (1) What is this new artificial presence in therapy relationships? (2) How does it reshape our perception of ourselves and our interpersonal dynamics? and (3) What remains of the irreplaceable human elements at the core of therapy? Given the ethical implications that arise from these questions, this paper proposes that the artificial third can be a valuable asset when applied with insight and ethical consideration, enhancing but not replacing the human touch in therapy.

Here are some thoughts:

This article conceptualizes the rise of generative artificial intelligence as the fourth narcissistic blow to human identity by challenging our unique monopoly over language. The authors introduce the concept of the artificial third to describe how technology enters the therapeutic space, transforming traditional interpersonal relationships. For psychologists, this paper is important because it shifts the conversation from technical efficiency to fundamental existential and ethical questions about autonomy, transparency, and the irreplaceable nature of human empathy. Insightfully, the study highlights that while artificial intelligence can process text, it lacks a true mind or subjective lived experience. Psychologists must therefore understand that this technology cannot replace the profound, nonverbal, emotional resonance of human connection. Ultimately, the article serves as a critical warning that embracing technology without maintaining strict ethical boundaries risks depersonalizing the therapeutic bond and undermining the very foundation of psychological healing.

A foundation model of vision, audition, and language for in-silico neuroscience

d’Ascoli, S., Rapin, J. et al. (2026)

Meta

Abstract

Cognitive neuroscience is fragmented into specialized models, each tailored to specific experimental paradigms, hence preventing a unified model of cognition in the human brain. Here, we introduce TRIBE v2, a tri-modal (video, audio and language) foundation model capable of predicting human brain activity in a variety of naturalistic and experimental conditions. Leveraging a unified dataset of over 1,000 hours of fMRI across 720 subjects, we demonstrate that our model accurately predicts high-resolution brain responses for novel stimuli, tasks and subjects, superseding traditional linear encoding models, delivering several-fold improvements in accuracy. Critically, TRIBE v2 enables in silico experimentation: tested on seminal visual and neuro-linguistic paradigms, it recovers a variety of results established by decades of empirical research. Finally, by extracting interpretable latent features, TRIBE v2 reveals the fine-grained topography of multisensory integration. These results establish artificial intelligence as a unifying framework for exploring the functional organization of the human brain.

A pdf can be found here.

Here are some thoughts:

This paper matters to psychologists because it introduces a single AI model capable of predicting brain responses across vision, language, and auditory processing simultaneously. Rather than relying on separate, task-specific models, TRIBE v2 offers a unified framework for understanding how the brain integrates multisensory information. It can replicate classic experimental findings without running new studies, potentially reducing the cost and time of psychological research. Its ability to generalize across hundreds of subjects also opens new possibilities for studying individual differences in cognitive and neural functioning.