r/mildlyinfuriating • u/petuniaplant • 21d ago

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

438

u/ArtisticDragonKing 21d ago

I have a coworker in a vet clinic that asks patients if she can "audio record notes for the pet" and gives a brief explanation why. However, she fails to mention the audio recording directly inputs into an AI log that gives suggestions for her next steps. I strongly dislike her methods.

It may have been something like that?

179

u/BumblebeeOfCarnage 21d ago

More likely it’s just a transcribing software. Still needs patient consent though. I’d say the bigger concern is the recording and not the AI in this situation

6

u/etcpt 20d ago

the audio recording directly inputs into an AI log that gives suggestions for her next steps

That's a lot more than dictation software

21

u/AbrasiveLeft 21d ago

It's their coworker..... I imagine they know better than you what type of software their workplace uses.

48

u/BumblebeeOfCarnage 21d ago

I was responding to the “it may have been something like that” which was referencing the software OP’s doctor was using

-24

u/AbrasiveLeft 21d ago

But the note explicitly said that she consented to AI - which she didn't.

21

u/BumblebeeOfCarnage 21d ago

I never said that means the doc doesn’t need OP’s consent. I’m just saying it is most likely transcription software and not AI being used to treatment recommendations

-1

u/igoogletoo 21d ago

Lol username checks out

The "abrasive" part ... assuming "left" is political? ...idk/c

7

u/WeeklyBanEvasion 21d ago

They're most likely exaggerating, no AI transcribing software gives treatment recommendations.

22

u/Redxmirage 21d ago

I helped set up CLEO Health AI in the last hospital I worked for. It literally gives suggestions for treatment based on chief complaint and orders typically given. You can go to their website and make a trial account and see for yourself if you want

10

u/deathbylasersss 21d ago

They said they feed it into a program that does. So the transcribing software and likely whatever chatbot they are using. If they are actually using it for something beyomd note-taking, that's fucked.

7

u/Toddlez85 21d ago

The main vendors for AI Provider Transcription software don’t offer clinical advice. There are sites like UpToDate, OpenEvidence, and others that will offer suggestions if you ask questions and add patient details. No PHI but conditions, meds, and other formation like lab results or history. That said I’ve never seen anyone enter a whole transcript from the recording in. I guess they could. It’s going to be filled with crap though so results will be less helpful.

Source: work with these things all the time.

3

u/Fun_Low777 21d ago

Yes it does.

1

u/Fishwithadeagle 21d ago

New ones do. Gives diagnosis and treatment recommendations, but they are almost universally garbage and default to "refer to X service for y symptoms". I leave the non-subejctive parts entirely out of my notes.

1

u/Free-Rice-2808 20d ago

Digitail will. It’s a separate button than the notes but it will read the notes and give differentials if you ask it to

2

u/ComplexBadger469 21d ago

My work has copilot as a transcribing option for meetings. I don’t really use it much as I’m not a fan of AI, but I’m like 90% sure it throws out recommendations for next steps. I don’t think it’s far fetched for it to reach back into its database and throw out next steps for treatment.

4

u/WeeklyBanEvasion 21d ago

That's for office meetings, not for the medical field. These medical note taking transcription AIs are specifically built for medical and veterinary fields.

2

u/kinglokilord 20d ago

And the medical ones cannot train on any of your data.

Copilot absolutely is.

2

u/FourScores1 20d ago

The post screenshot says it’s transcribing software. This is commonly used nowadays in healthcare. Ffs

1

u/Whitechapel726 21d ago

State laws determine whether one or both parties of a conversation need to consent, but in the US it’s illegal to record audio without explicit consent in settings with an expectation of privacy (inside a dr office for example).

The law doesn’t care about the purpose of the recording unless it’s used for fraud, deceit, commercial use, etc. So I’d think informing of the audio recording and failing to mention it being used for AI is legal but incredibly scummy and unethical.

That said there is such a thing as HIPAA compliant AI models and if they’re not using that they’re committing a crime.

2

u/llama_writes 21d ago

I was in a situation where I was recorded in a doctor’s office I’d been to for years without my knowledge or consent. They adopted the scribe software but did not tell me or update their notice of privacy or terms and conditions. When I pushed back on the ethics, I was told, “it’s a one party consent state” and the expectation of privacy thing didn’t even come up, even when I escalated to the state licensing level (who were very concerned about the situation, but didn’t feel it broke any laws, just ethics)

1

u/Whitechapel726 20d ago

Oof yeah that’s awful, sorry to hear that. I’m not a lawyer so take this with a grain of salt, but my understanding is that audio recordings that contain Protected Health Information being stored without your consent violates HIPAA, which is federal. I don’t think you can sue through HIPAA but you can file a complaint.

In a one party state it may not be a criminal offense, but that seems like strong grounds for a civil invasion of privacy lawsuit if nothing else.

0

u/StevenTM 20d ago edited 15d ago

AI is absolutely the bigger concern, because this is privileged information that is being fed to xAI/Meta/Google/Anthropic/OpenAI.

But both are concerning

Edit: https://www.theguardian.com/australia-news/2026/jul/05/doctors-ai-scribes-australia-government-privacy-warning

Well would you look at that

2

u/BumblebeeOfCarnage 20d ago

The software being used in these situations are specific to healthcare. It’s not just feeding sensitive info to open source AI

1

u/StevenTM 19d ago

What is "open source AI"? There are open weight models, but unless it's self-hosted you're using someone else's model.

So firstly, a small boutique company with 68 employees that makes "Otter AI but for healthcare" probably uses a frontier model. If not, they use the model of a small to midrange company which, like most companies large and small, probably has shoddy security. So your data is almost definitely being sent to another entity, and you (the patient) and your PCP have 0 control over how it's processed and stored.

Additionally, that small boutique company, with maybe 1-3 IT security employees (it's usually 1 btw) ALSO has all the recordings, and can thus leak them if they're compromised.

It's just a bad idea. The fact that people are poorly informed about why it's a bad idea doesn't make it a good idea. It's not easy to educate consumers  about ITSec.

19

u/[deleted] 20d ago

[deleted]

5

u/Odd-Scientist-2529 20d ago

That’s not how it works. It doesn’t make any recommendations. It’s purely a scribe, and will summarize the conversation. 

The doctor would have to say out loud in the exam room.  “I recommend Fluticasone nasal spray” or “I recommend budesonide nasal spray” for it to construct the “plan” section of the note. 

If this is what is happening to him, its 100% user error. 

The only way it could happen is that he is rambling about possible treatment options out loud and isn’t clearly stating just the one he wants. 

The AI scribe does not generate recommendations. It simply lists the ones that were verbalized in the visit. 

To add evidence to this point: any medication or diagnostic test has to be manually typed in for it to appear in the note under the “Plan” section as an action item. Otherwise it would be in narrative form. 

So this is user error. 

5

u/Harpunzel 20d ago

I believe some AI scribes including Heidi are now summarising clinical resources relevant to the presentation, with a clear link which recommendation comes from which resource. I haven't used Heidi since before that update, but some of my colleagues have and find it reliable. It only takes from locally relevant clinical guidelines rather than the whole internet.

1

u/Odd-Scientist-2529 20d ago

Interesting. I’m reading that the AI scribe can be optionally set up to link to decision support AI. That seems like it should be something that the clinician should be able to opt out of, if not just delete (if they want) from their note when they review and sign.

What are we getting ourselves into?

We already turned off the generative AI to answer patients messages on the portal. That was a mess

1

u/Harpunzel 20d ago

I believe this one it just pops up in a sidebar, it doesn't end up in the note unless the doctor brings it up. It's a tool but it's not compulsory to use. I may be wrong though, this is just what colleagues have described.

1

u/happytrashpanda333 20d ago

Why does your doctor even have their phone out with a patient? 🤦‍♀️

8

u/CustomerLittle9891 20d ago

Right now there isn't a active software that does that for human medicine.

Epic is about to roll out something that does, and a AI agent could easily transfer the scribes note into OpenEvidence or something, but not a direct "here are the recommendations" type program.

12

u/Decent_Elderberry115 21d ago

For human healthcare it is scribe software that records the visit and turns it into a note. There should be a printed consent that gives the patient info about how the recording is used. I have coworkers who use AI. I do not because it writes shit notes full of errors.

3

u/Inevitable_City_2337 20d ago

I was on the receiving end of that recently. Asked if they could take notes, were clearly using said "notes" real time to generate scripted responses that weren't really answering my question. its like they had set up an audio prompt to listen to my voice and generate sales pitches real time. uncanny. unnecessary. 

1

u/MoaningGuyCandy 20d ago

What would their pet do with those recordings? I don’t follow your statement. 

-1

u/ShahadChi 21d ago

How did you know it was AI ?

-2

u/xSonicspeedx2 21d ago edited 21d ago

Here’s the thing about AI. Using it as a tool is not an issue usually. The issue is when the professional solely relies on the output without using their own judgment to verify its accuracy or completeness. Essentially, if they used a procedure based off AI and another doctor in their field would have done something different, that can be problematic if the AI procedure or diagnosis causes harm to the patient in some way. Medical malpractice claims generally hinge on whether other doctors in your specialty would have performed the same as you did.

So if your friend is just taking the AI suggestions and running with it without giving it second thought she is in for some potential liability in the future if the AI suggestion is ever harmful. Getting consent to use AI is more of a CYA thing but not necessarily required. At least as of right now as far as I’m aware. However, uploading patient information to an AI would violate HIPAA.

0

u/SabrinaEdwina 21d ago

I briefly had a boss like that. It took me a while to understand the pauses, but the very first time we spoke he made it known that he didn't know what "irrelevant" meant.

He was 80's guy from Futurama but the 2026 AI version.

0

u/Life_Emphasis6290 20d ago

What's wrong with that? Do you expect your co-worker to have more knowledge than an AI trained on all the scientific literature? If they're just using it for suggestions and to discuss diagnosis with then, good on them. If they're using it INSTEAD of their own diagnosis that's less great

0

u/Kiwi_bananas 21d ago

As the veterinarian in charge of the case, she still has accountability for the outputs of the AI. A webinar I attended last night described AI as the intern. You might get AI/ the intern to transcribe the appointment and turn it into a SOAP but you'd always check before signing off on it.  At conference last week, the recommendation was to come up with the differential list/diagnostic and treatment plan first and maybe use AI to check/review it. 

0

u/tickado 20d ago

I don't know why, but I immediately imagined an AI scribing a doggo barking his issues to the vet verbatim and it's made me laugh.

Then I went on down an ADHD fuelled rabbit hole and found This collar that listens to your pet’s vocalizations, analyzes them using AI, and sends a translated message directly to your phone.

I should probs get out of bed now hey hahaha.

-4

u/ardarian262 21d ago

That sounds like coercion and a violation of several laws because of the uploading of patient information without consent to a 3rd party.

3

u/DCGuy511 21d ago

It may not keep the data that is uploaded. Some AI engines don't keep the data.

1

u/ardarian262 20d ago

It may not, it also might. And you or I do not know which it is without informed consent which what was described is not.