Hospital AI System Goes Wrong & Results in Death
Sad story coming out of Brazil.
If AI can hallucinate by making up fictional stories and facts, tell troubled teens to ‘unalive’ themselves, as well as misidentify individuals to law enforcement that have resulted in wrongful arrest, this latest story is unfortunately, following a troubling pattern. A pattern of damaging impact on humans, as a result of AI operating without proper human oversight. In my view, this isn’t an AI accuracy versus human accuracy issue. It’s about ethical boundaries. While some tasks are well suited for AI, there are some tasks that AI has no business performing - especially in an emergency setting where life and death are in the balance. These ethical questions are now in the spotlight, where one mistake tipped the scales and resulted in tragedy for a 32 year old woman.
I included the article by Futurism below, for your reading convenience.
Futurism, June 14, 2026
By Joe Wilkins
‘AUTO-REJECT’
Woman’s Death Blamed on Hospital’s AI System
“What we saw was that doctors lost the autonomy to decide if a patient is very seriously ill.”
As AI finds its way into hospital systems around the world, the case of one Brazilian woman makes it clear that the tech doesn’t have to diagnose patients to determine whether they live or die.
According to Brazilian news publication MG1, 32-year old Rebeca Cardoso Tenente Molina died after a state-run AI system for assigning hospital beds forced her to wait five days to transfer to an intensive care unit.
In interviews with MG1, Molina’s family members explained that the AI system delayed her access to the care she needed. Prior to her death, Molina was hospitalized in the small municipality of São João Nepomuceno after seeking medical treatment for gallstones. As her health rapidly deteriorated, Molina waited for an ICU bed in the state of Oliveira to open up, at a hospital about 186 miles away.
Though the family went as far as pursuing emergency legal action against the hospital system to get her transferred, the move was significantly delayed. The family now believes that five-day wait was fatal, MG1 reported.
According to Molina’s sister and family lawyer Sâmela Cardoso Tenente Furtado, the AI hospital-management system assigned the patient a much lower “score” than the one her condition actually reflected. This automatic scoring, completed by Brazil’s State Regulation Operations Center (Core-MG) using AI tools, is alleged to have played the deciding role on delaying transfer to an ICU bed.
“What we saw was that doctors lost the autonomy to decide if a patient is very seriously ill,” Furtado told MG1. “The one who has to accept whether a patient is seriously ill is no longer the doctor who is there experiencing that reality with the patient, it’s the Core.”
In her interview, Furtado describes a flawed and inflexible system which refused to budge, even as test data showed her sister’s condition taking a turn for the worst.
“She would have been a 10, and the system only accepted her as a 6.8,” Furtado continued. “So she couldn’t progress properly in the system because a patient at 8, a patient at 6.9 would jump ahead of her. And the system wouldn’t accept increasing her severity level within the system because of the tests that were constantly feeding it data.”
“My sister, other people, are not just numbers, they are not just protocols, they are not just a CPF [Brazilian tax ID number] thrown into the system,” Molina’s sister told MG1. “They have families, they had dreams, they had a whole life ahead of them.”
In an official statement shared after the AI-system’s launch on May 19th, Minas Gerais’ Deputy Secretary of Health, Poliana Cardoso Lopes said that “Core provides a bed map that is updated three times a day. With this, it will be possible to have much more control over the process and generate better data on the clinical condition and needs of each person waiting for a bed.”
Responding to Molina’s death, the state health department told MG1 that transfers are determined based on the availability of beds that fit a patient’s clinical needs, and added that Core-MG has not fundamentally altered the protocol for transferring patients to other facilities.
Conclusion
It sounds like the entire triage process was running on an AI algorithm with no human override options. Best case scenario, the algorithm was developed with medical input, worst case scenario, it was developed by another AI or a software engineer. Either way, it took the power away from the providers in the hospital to lead in decision making. Truly a tragic outcome for this woman and her family, who mourns in the wake of her passing.
Sound very similar to Mercy movie but more real. This a warning and the Bible speaks very clearly how AI will be use to determine who can sell and buy, also even life and death. Revelation 13:15-16
"He was granted power to give breath to the image of the beast, that the image of the beast should both speak and cause as many as would not worship the image of the beast to be killed. He causes all, both small and great, rich and poor, free and slave, to receive a mark on their right hand or on their foreheads, and that no one may buy or sell except one who has the mark or the name of the beast, or the number of his name."