This student story was published as part of the 2026 NASW Perlman Virtual Mentoring Program organized by the NASW Education Committee, providing science journalism experience for undergraduate and graduate students.
Story by Lea Nagelschmied
Mentored and edited by Christine Perdan Curran
When Mirela Fila was barely an adult, she fled from Romania, dreaming of making it to the U.S. but ended up in an Austrian refugee center. Now, 36 years later, she faces a new challenge as a hard-working nurse and manager in an Austrian retirement home. “Currently, we’ve got 52 inhabitants, and only one registered nurse is responsible for them,” she recounted. “That’s preparing medication, drug administration, bandages, organization, ward rounds, appointments, emails and more.” Mistakes with medication can happen, explains Fila, especially in stressful situations. Ten more residents will be coming soon.
Healthcare systems are overloaded in many countries, and there are well-documented problems with inaccurate drug dosing in institutions like residential care facilities for the elderly. In Fila’s retirement home, drug dosing is mostly handled by a doctor who is neither always available nor working directly in the institution. Fila believes that good AI support could improve decision-making and significantly improve care. “AI can be a great support for us,” she said.
Emerging uses of emerging AI technologies
Would you turn to AI if you had a concern about potential drug interactions? Many patients are doing so already based on AI input. Credit: Stux on pixabay (CC0 license) Patient scenario text added by L. Nagelschmied. (Select image to enlarge)
The novel technology is already widespread. In 2024, one in five general practitioners reported using general-purpose AI models. A 2026 preprint of another study suggests that 67% of clinicians across American hospitals are now using such AI models in clinical practice. Surveys also report that high proportions of patients utilize AI for health information and questions about medication, and often subsequently modify their drug usage.
To AI or not to AI?
When Fila is asked if she already uses AI for work, she hesitated. “Well, officially, no. It is not intended that we use AI. But honestly, I am using it for documentation and information.” Still, she would not use AI for drug or dosing recommendations. “I am using AI, but I don’t know if I may, and if the information I get is correct or not. We are working with humans, so we cannot take guesses.”
Fila is right to be hesitant. A new study from Leipzig University published by the Journal of Medical Internet Research (JMIR) shows that general-purpose AIs may give inaccurate dosing recommendations. The researchers built a scoring system to evaluate AI dosage recommendation quality and used real patient data to assess potential harm. Among GPT-4, Microsoft Copilot Business, Google Gemini 1-5 Flash, and Research Solutions Scite, no model reached a perfect score for answer quality, and all models caused potential harm to patients, especially under certain conditions.
One such condition was decreased renal function: kidneys filter bodily fluids, changing the concentration of a drug in the body. Reduced kidney function can magnify a drug’s effect leading to toxicity. Chronic kidney disease affects over 800 million people worldwide and more than one-third of adults over 70 in the U.S. Even with complete information regarding patients’ renal function, AI recommendations caused potential harm.
The most challenging cases
Potential drug interactions and language affected the quality of AI answers as well. The more drugs a patient takes, the worse AI performs at dosing recommendations. German language queries also led to less accurate recommendations. This is in line with other findings showing that English-language prompts usually get better responses than non-English prompts.
For her work in the retirement home, Fila uses AI in German, since this is how she learned her profession. Her colleagues speak Serbo-Croatian, Turkish, Ukrainian, and more. Very few speak any English. AI would provide them all with less reliable information.
The researchers only reported mild potential harm to patients. “Mild harm refers to effects that are clinically relevant but reversible without long-term consequences,” explained Barbonus. But their input questions were structured and formulated in a way that optimizes AI potential. What happens when the questions are not perfectly formulated, or when information is incomplete?
This is largely unresearched, but a recent study showed that patients trying to identify medical conditions and relevant treatments with AI did much worse than professionals – in fact, they did no better than patients using traditional search engines. Yet as AI answers are usually very confident and authoritative in tone, a user may feel more secure about the information.
Chat GPT was prompted to create an illustration of a pharmacist using AI to calculate the correct dosing for the prescription of an elderly patient with kidney disease. Credit: Lea Nagelschmied via ChatGPT. (Select image to enlarge)
Overall, an uncomfortable image arises. The quality of AI drug dosing recommendations depends on user proficiency in language and wording, patient representation in training data, and drug interaction complexity. Therefore, people with less education, underrepresented groups and minorities, and those with more health problems would be most likely to suffer harm from inappropriate drug dosing. Barbonus worries that increased risk for patients with more complex conditions and worse access to medical institutions might reinforce existing health disparities. And with an aging population, the number of vulnerable patients needing multiple drugs will increase.
But there is hope. A study found that clinicians collaborating with GPT-4 showed better diagnostic accuracy than clinicians restricted to traditional resources. Fila dreams of something even more trustworthy: “It would be great if there were a specific medical AI we could use for reliable information.” And indeed, models that equip general-purpose AIs with specialized tools, so-called “agentic models,” could be tailored to clinicians’ needs and improve the AI’s research and problem-solving skills.
In the end, AI-human collaboration does have the potential to improve our healthcare systems if carefully implemented. Barbonus and her co-authors suggest that an important step to improve safety would be tackling the language issue. This would support healthcare professionals all around the world, but professional oversight remains crucial. “The aim is not to replace healthcare professionals, and it should not be,” stressed Barbonus.
Main Header Image Caption: The number of medications a person takes increases the risk of adverse reactions. This is a challenge for health care professionals who are learning AI tools may do more harm than good as presently designed. Credit. Stevepb on pixabay (CC0 license).
Lea Nagelschmied is a postgraduate student of Applied Neuroscience at King’s College London. Even though Stanislaw Lem’s Washing Machine Tragedy is one of her favourite short stories, she believes in working towards a future where Artificial Intelligence supports, rather than eradicates humans. You can find her on LinkedIn or reach out via leasophienag@gmail.com.
Christine Perdan Curran has been an NASW member since the late 1980s and is delighted to help support the next generation of science writers.
The NASW Perlman Virtual Mentoring program is named for longtime science writer and past NASW President David Perlman. Dave, who died in 2020 at the age of 101 only three years after his retirement from the San Francisco Chronicle, was a mentor to countless members of the science writing community and always made time for kind and supportive words, especially for early career writers.
You can contact the NASW Education Committee at education@nasw.org. Thank you to the many NASW member volunteers who lead our #SciWriStudent programming year after year.
Founded in 1934 with a mission to fight for the free flow of science news, NASW is an organization of ~ 2,300 professional journalists, authors, editors, producers, public information officers, students and people who write and produce material intended to inform the public about science, health, engineering, and technology. To learn more, visit www.nasw.org
