Event Details:
Join us on March 4th 12-1pm PST to learn about the results of Gaus et al.'s scoping review on the quality of studies on LLMs for mental health support!
Results of a Scoping Review on Quality of LLM Mental Health Studies
Millions of people around the world are using large language models for mental health support in a de-facto, unregulated public health intervention, and many specialized LLM applications are being developed for this use. Amid reports of both therapeutic benefit and significant harms, the scientific evidence supporting these applications has remained inconclusive.
To create an up-to-date snapshot of the field, Gaus and colleagues conducted a PRISMA-ScR scoping review of 132 peer-reviewed studies on transformer-based LLMs used to deliver, augment, or analyze mental health support and psychotherapy. Data were extracted on sample composition, study design, the adoption of responsible evaluation practices, and model and dataset choices.
The review identified a pronounced gap between widespread public adoption and a limited evidence base. The authors call for more robust methodological standards, including rigorous clinical trials, a focus on safety and implementation, and standardized, clinically meaningful automated benchmarks. Stronger evidence is needed to document that generative AI systems can safely and meaningfully improve access to mental health treatment.
About the Speaker:
Richard Gaus, MD - Dr. Richard Gaus is a physician with a strong foundation in AI and computer science. Currently he is finishing his master's thesis on robustness and reliability of LLMs for clinical reasoning at the Machine Intelligence for Medical Imaging Lab at Stanford University. Before that, he worked as a resident at the Dpt. of Psychiatry, LMU Hospital while simultaneously studying full-time in the Robotics, Cognition, Intelligence master's program at TUM. Beyond clinical work and research, Dr. Gaus built a track record of founding and leading projects at the healthcare/tech intersection, such as med-dev, the industry phase at TUM.ai, and Support Groups for Change.
Moderator
Betsy Stade, PhD is a research scientist and associate director of the Stanford ALACRITY CREATE Center for Advancing Therapy with AI. As a computational clinical psychologist, Betsy focuses her research on how AI and large language models can be used for evidence-based psychological practice. Betsy did her graduate work at the University of Pennsylvania and her clinical residency at the VA Palo Alto Health Care System, and is a licensed psychologist in California. Her research has been supported by the National Science Foundation.
Register here:
https://stanford.zoom.us/webinar/register/WN_4LLoWma0SwGMb3rLtrxMfA
CME credits: You may be eligible to receive CME credits for attending this webinar. More information will be provided at the start of the event. If you have questions, please reach out to CREATE (create-alacrity-center@stanford.edu).
Recording: This event will be recorded, and a Vimeo recording will be shared with our CREATE community. We ask that participants do not use AI note taking or recording tools unless they are needed for personal accessibility purposes.