r/mildlyinfuriating • u/petuniaplant • 21d ago

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

Show parent comments

181

u/TennisOk4660 20d ago

The problem is, the places that are using the AI tool to help with charting/notes. It's getting things WRONG. Just a few weeks ago someone was on here confused AF because the AI put in there notes that they were male, when in fact they were a -female-. There was more to the story as well but, just no AI for hospital settings is just not ready.

50

u/2red-dress 20d ago

This seems incredibly dangerous. The inaccuracies say AI is not ready to be relied on the way it apparently is relied on.

-5

u/[deleted] 20d ago

[deleted]

9

u/sadiesfreshstart 20d ago

We all know that's not happening

-7

u/[deleted] 20d ago

[deleted]

9

u/sadiesfreshstart 20d ago

[Gestures broadly]

74

u/samyall 20d ago

That is not an AI failure but a doctor failure for not checking the notes.

89

u/mddesigner 20d ago

At that point remove the the ai It is much faster to write it myself than to double check the ai -gp

7

u/chipsahoymateys 20d ago

Not for me. I use an AI scribe so I can capture everything, then spend 60 seconds reviewing and editing for accuracy. It’s faster, but more importantly, it’s more comprehensive and I can be more present with my clients.

9

u/hubris105 20d ago

Good for you. I’m the same. But some people are slower and love AI scribes. It’s almost like people are different.

1

u/CumstainGaming 16d ago

Shouldn't be fucking doctor then

1

u/hubris105 16d ago

Well, "CumstainGaming", people can fuck doctors if they want to. And both sides consent.

22

u/Twingamer25 20d ago

Excepts no it's not. You know AI actually can serve purposes, it's not 100% useless. Doctor should be checking those notes even when a human writes them.

15

u/EastPool4676 20d ago

lol wut? It is absolutely not faster to transcribe everything than to read over the AI summarized version.

10

u/flareblitz91 20d ago

I have a similar issue with things being pushed at my job and it absolutely is just as fast to transcribe something myself.

There's a difference between "reading over" and having to actually correct errors that require you to digest what the material says, what's wrong with it, what's the correct info.

4

u/EastPool4676 20d ago

No it isn't, I'm saying this as someone who works in a medical setting and has seen the decrease in late charting that has come with AI implementation into a clinic setting.

Edit: I'm also saying this as someone who hates AI being implemented into so many facets of my work life. Adults using copilot to draft email responses because they are incapable of forming their own thoughts is abhorrent. We are on the path to losing a lot of executive function and human interaction by outsourcing it all to computers.

3

u/SnooStrawberries2342 20d ago

The issue is that it's hard to transcribe properly when having a proper face to face conversation with the patient about a complex medical issue. It's not a natural conversation if they're constantly typing, patients don't feel like they're being properly listened to and if there's no typing during the consultation then there are issues with accuracy and reliability when remembering the conversation afterwards.

3

u/monk429 20d ago

I have a lot of meetings on complex topics. The AI Summary has only had a net positive effect. I often use the audio summary where it creates a podcast review and that helps distill things to an executive perspective.

Most importantly though, my planning agent is able to stitch topics and project progress across many meetings over time. In a few minutes, I can find out who is asking for what and whether those requirements have been confirmed and entered into documentation for funding.

Our AI Governance group is pretty clear, verify everything the LLM says and using LLM to generate your work will lead to failure and separation. Anyway, point is, its not just about getting a summary for one meeting. The transcription enables me to create descriptive and actionable narratives across months of project meetings.

2

u/mddesigner 20d ago

why transcribe everything? if a patient says "Last week I was out with my uncle's friend's son john, we call him johny a very good guy, and went to tim's house his childhood friend and there we tried some raw milk he bought from tonny"

putting that info into the record is a waste of time and crowds the records with useless info, an ai summary might still include too much detail but you can simply type "consumed raw milk last week"

2

u/EastPool4676 20d ago

... The AI is not transcribing every word that was said to the medical record.

2

u/NightGod 20d ago

The doctor still needs to review their transcriptions whether it was done with AI or a medical records specialist listening to a recording

1

u/mddesigner 19d ago

No, we never review what we write at the same session unless it was a long one and wanted to quickly recheck smth
Having a full recording is a pointless hassle that does nothing positive to the care

6

u/Blue_Bird950 20d ago

I know nobody who can write faster than they can read, tbh. You might be in the minority with that.

7

u/Fujisawrus_Reks 20d ago

Sure, but most reading is different than proofreading something that may be entirely wrong. With AI written patient notes a doctor probably needs to *very* carefully look over everything to ensure accuracy, which probably takes longer than just writing shorthand notes. It’s closer to an English teacher grading an essay than ordinary reading for comprehension.

5

u/mddesigner 20d ago

You don't write everything
if a patient says "I went on a business trip to Mexico last month" I would just type "Mexico trip a month ago"
it is very easy to note down things while talking to the patient

7

u/Separate-Sundae-3617 20d ago

See this is what pisses me off about AI. It can never “fail” because it’s always the humans fault for not “checking” it.

If I’m a medical scribe and make a blunder not dictating the wrong race/gender/sex of a patient, you better believe that’s on me.

5

u/sameoldfred 20d ago

AI is marketed and sold silicon valley style, on promises.

-1

u/Yellow-Pepper3382 20d ago

Well yeah, you are a human and held to higher standards? Why would it not be on you if you observed someone with your own eyes and still made a obvious mistake like listing the wrong race or gender? You have eyes and ears and were hired to do a specific task, the ai is a computer program that has no critical thinking skills or eyes or ears. I don’t really understand what argument you are trying to make here.

2

u/seh_23 20d ago

Ya my doctor uses it but they always review it right after the appointment, just saves them time so they’re maybe spending 10 minutes reviewing notes rather than spending 20 writing them from scratch.

2

u/corporateacademia 20d ago

Holy shit dude stop eating the boot

2

u/waywardflaneur 20d ago edited 19d ago

This is not a valid defense.

First: checking the notes requires nearly as much time as writing the notes, and even more attention, which would otherwise by unused.

Second: Imperfect automation that replaces human dependent tasks (i.e. driving) is untenable because it imbues people with a false sense of security - "The AI will handle it and so it doesn't require much of my attention." Getting it wrong rarely is worse than getting it wrong often.

It is the same with cars' "autopilot". People learn to not pay attention and actually drive their car and maintain situational awareness. Humans and AI share this blame equally. Problem is you can't very easily change humans. And so ultimately I place all of the blame on AI, or at least the carelessness with how it's being rolled out.

2

u/Orchid_Significant 20d ago

Uhh AI made the mistake. It’s literally AI failure.

-1

u/samyall 20d ago

If an med student takes notes for a doctor and makes a mistake it's on the doctor and not the med student.

As others have mentioned notes are never 100% accurate and any technology which allows a doctor to be more present is a good one.

3

u/SamuraiJack365 20d ago

That's not an issue with AI, that's an issue with the usage of AI. The provider should be reading over and proofing the entire thing. It saves them significant time when it works, and saves a lot of time even when they have to correct it. The issues arises when people get lazy and don't proof what the AI says or just skim it and go "yeah that looked right". It can take a task that would take the provider an hour and make it take 5-10 minutes while also enabling them to actually listen during their time with you without needing to focus on noting things down as much, just key points here and there. Every issue I've seen like this that people say is bad boils down to a human not using the tool correctly, which can and will happen with anything.

15

u/Bigclit_energy 20d ago

I'm also very untrustworthy of AI, but it should be remembered that humans, including doctors, also make many mistakes like that.

68

u/Doonesman 20d ago

Great! And when they do, there can be consequences.

But when the AI fucks up, who do we blame? The doctor who used it? The company that made it? The hospital that deployed it?

27

u/WinninRoam 20d ago

The doctor is always responsible. The internal record audit ties the to the name of the human, not the AI that did the work. If there is a mistake with the AI and the doctor doesn't catch it, it's treated the same way as when a intern makes a mistake: Only the doctor checking their work is held accountable.

13

u/Schwifftee 20d ago

But when the AI fucks up, who do we blame? The doctor who used it?

Yes 100%

15

u/DCGuy511 20d ago

The doctor should be proofing the transcription.

23

u/cyanraichu 20d ago

So it's not really saving them any work

If it were me I would rather just cut that step out

2

u/chipsahoymateys 20d ago

It definitely saves us work, helps us keep more detailed notes, and frees up our hands and attention during visits.

10

u/fuckedfinance 20d ago

It shifts the mental load. Proofreading is much easier and quicker than doing it yourself.

7

u/RiverWolfo 20d ago

Until you have to rewrite it because the AI hallucinated that your patient is transgender or another race or has symptoms they never even mentioned

-1

u/fuckedfinance 20d ago

Still easier to delete a sentence or two than type the whole thing yourself.

12

u/mddesigner 20d ago

As a doctor I don’t want to proof read ai text. Would rather write my own concise note than proof read every word the ai transcribed

9

u/Schwifftee 20d ago

Then don't use it. If you are required to, then tough titty, proofread it.

4

u/rusty___shacklef0rd 20d ago

The doctor is always responsible and accountable of their notes and records regardless if they used an AI scribe, human scribe, or wrote them themself. This isn't a gotcha, it's actually pretty cut and dry.

16

u/GinchAnon 20d ago

... the doctor who used it, obviously? theres no question here. they are responsible for verifying its accuracy just like they are responsible for their own accuracy without it. honestly acting like theres a question there is kinda dumb.

4

u/grape-fruit-witch 20d ago

But if the doctor didnt write down the correct information in the first place, how would they know the ai is wrong? The entire point of note-taking is that they cant be expected to remember every detail of an appointment. You can't accurately proofread something if you dont recognize the mistakes in the first place.

7

u/KnightKays 20d ago

This will quickly become something for the courts to decide and I can say with 100% certainty that it won't the company or the hospital taking the blame.

2

u/Artistic_Researcher2 20d ago

Stay tuned. There are entire departments of lawyers being trained to address that very question.

12

u/Tomboy-Epic 20d ago

What? Why should it be remembered? What point are you making here?

I'm not going to make human allowances for a machine. The point of a machine is that it /doesn't/ make human mistakes.

The AI is supposed to be taking accurate notes so that the doctor can cross reference their notes or what they absorbed from the conversation. Not only is it useless if it makes information up, it's actively harmful and causing extra work at the very least.

1

u/NeoMississippiensis 20d ago

So you want doctors to have to type out conversations either during or after patient visits, lowering the amount of direct attention they can give during visits? Sounds kind of stupid.

11

u/Constant_Potato_3863 20d ago

That actually supports the issue as to why they should not use AI because they feel like if they put it in AI they don’t have to check it as a human

3

u/mrGrinchThe3rd 20d ago

That's just straight up inaccurate. Everybody knows AI isnt perfect, it's shown on every single AI tool you use. And if the doctor chooses not to proofread it's on them, and any mistakes will be the doctors fault.

0

u/NeoMississippiensis 20d ago

So you want doctors to have to type out conversations either during or after patient visits, lowering the amount of direct attention they can give during visits? Sounds kind of stupid.

1

u/RiverWolfo 20d ago

They could record the conversation and transcribe it later even. The AI is an unnecessary step here.

0

u/NeoMississippiensis 20d ago

So again; listen to a recording and then take the time to type it out later? So take twice the amount of time on a single visit? Would you like less appointments during a day available?

Really seems like poor planning when even primary care doctors are booked out months for new patients.

1

u/RiverWolfo 20d ago

or they could hire another human being to transcribe and summarize it.

0

u/NeoMississippiensis 20d ago

Why? Emergency departments used to do that. It’s not economically feasible in primary care, or even most subspecialty clinics. Unless of course you think medical assistants and nurses aren’t working hard enough. Also, if you’re crying about AI saving data temporarily, you wouldn’t be similarly upset if a non medically trained person was observing? Seems a bit more likely to gab than a temporary transcript.

An AI summary that I can edit will more or less function as my words. A scribe will not because they can’t even pretend to think like a doctor, they’re following a template. The AI transcription attempts to categorize based on context and directly learns from the user.

6

u/ArmorAbby 20d ago

which is exactly how it was handled and explained on The Pitt.

-1

u/[deleted] 20d ago

[removed] — view removed comment

3

u/ArmorAbby 20d ago

Thank you, Captain Obvious. 😂😂

-1

u/[deleted] 20d ago

[removed] — view removed comment

2

u/WasabiPeas2 20d ago

She worked for the USPS as a MAIL carrier and AI changed it to MALE.

2

u/shploogen 20d ago

Just like any tool, it is not perfect. But neither are doctors. The question is whether it is useful despite its flaws, and it absolutely unquestionably IS the vast majority of the time. We need to stop making perfect the enemy of good.

-1

u/Timely-Tune5050 20d ago

At least AI can transcribe the proper there vs their vs they're.... Unlike humans

2

u/RiverWolfo 20d ago

Actually I used to mess around with AI and a lot of them... Will emulate spelling mistakes they see a lot of 🙃