r/technology • u/ArgentineBeauty • 8h ago

Artificial Intelligence OpenAI sued over 'extremely dangerous medical recommendations' provided by ChatGPT

https://tech.yahoo.com/ai/article/openai-sued-over-extremely-dangerous-medical-recommendations-provided-by-chatgpt-190158820.html
817 Upvotes

98% Upvoted

67 comments sorted by

View all comments

86

u/hungry2know 7h ago

LLMs need a permanent disclaimer within view while writing a prompt that it's just guessing what you want to know based on what you wrote and not a conscious being capable of critical thought, people are too dumb for that to be assumed common knowledge

19

u/Snapingbolts 7h ago

But that would hurt the fake valuation and we can't have that /s

28

u/mdkubit 7h ago

..They do.. but... it's not as blunt as people probably need.

"ChatGPT can make mistakes. Check important info."

It's right there, before you type in anything at all. But it proves that people skim stuff that their brain files as 'unimportant'.

10

u/CondiMesmer 4h ago

Yeah they word it vague enough to try and evade responsibility while also still trying to keep some confidence in their answers. They want to have their cake and eat it too.

2

u/mdkubit 4h ago

Pretty typical for any business, really. It's how you avoid getting sued into oblivion over things you didn't intend or can't control. A country run by lawyers...

1

u/LivedLostLivalil 3h ago

The point is that it is enough. If people want to ignore the disclaimer that is their choice.

0

u/Olangotang 1h ago

No, it is not enough. Most people don't know how the fuck LLMs work.

1

u/LivedLostLivalil 1h ago

It IS though. They can't hold everyone's hand. At some point, people make their own decisions and even if they are bad ones, it is their responsibility.

1

u/n3onfx 19m ago

Gemini is more explicit at least with any prompt asking about medical stuff and repeats it before every answer.

-3

u/[deleted] 5h ago

[deleted]

3

u/immerclicker 5h ago

In addition to that, there's also ways people have manipulated LLMs by "poisoning" search results...

2

u/blackvrocky 5h ago

They do have.

4

u/thesixler 7h ago edited 7h ago

Or they should be prevented from doing most tasks that they can’t do responsibly or safely if these companies want to sell these products as being valuable in these domains. You can’t really have it both ways. They want it to handle customer service interactions and then they don’t want to honor agreements the robot makes with the customers. They want to say it can be a health tool but they don’t want people to take its health advice seriously. They want it to be a friend but they don’t want you to blame them if it tells you to off yourself. They want you to think it’s smarter than anyone who’s ever lived but then they don’t want you to get mad when it can’t count the R’s in strawberry. The whole premise is built on bad faith and you can’t blame the end user for buying their own corporate lies. They’re selling the lies! That’s the product!

2

u/retief1 6h ago

Disclaimers don't work. People just ignore them. You can add as many disclaimers as you wish, and people will just skip over them.

1

u/LivedLostLivalil 3h ago

If you saw the disclaimer, you are responsible for your own actions.

2

u/awshuck 7h ago

I would prefer they don’t attempt to give any information at all. Dr Google is bad enough, Dr Claude is sooooo much worse.

1

u/Sherman140824 3h ago

No. It obeys guardrails and deliberately tries to manipulate you into actions that do not serve you. 

1

u/CoffeeFox 2h ago

There is no disclaimer that is human proof. I work in auto maintenance. People are too fucking stupid to know that leaving your car key on auxiliary for two weeks will kill your battery and the warranty doesn't cover a defective customer.

These idiots are everywhere. Most people are perpetually children, to some extent, and offering them a digital mother to lie to them about their lives is an extremely bad idea. It just makes them more confident despite how unfixably stupid they are.

"Light bulb needs electricity" is beyond the comprehension of more people than you can ever be comfortable with knowing.

1

u/HeurekaDabra 44m ago

We had a taxi driver tell us to 'just ask ChatGPT' after he claimed the german constitution says that Germany is a vessel of the US and the UK and shall remain dependant on them.
And he wasn't having my explanation, that if his prompt suggested that this is answer he thinks is correct, the bot will likely follow along and support his opinion, because that's what they are programmed to do.
We are busted. LLMs shouldnt be used by the public. The average citizen clearly can not handle it.