OpenAI worries its AI voice may charm users

OpenAI worries its AI voice may charm users

Actors Scarlett Johansson (R) and Joaquin Phoenix starred in the 2013 fiction film 'Her', in which a man falls in love with a human sounding AI assistant
Actors Scarlett Johansson (R) and Joaquin Phoenix starred in the 2013 fiction film 'Her', in which a man falls in love with a human sounding AI assistant. Photo: TIZIANA FABI / AFP
Source: AFP

PAY ATTENTION: Follow our WhatsApp channel to never miss out on the news that matters to you!

OpenAI says it is concerned that a realistic voice feature for its artificial intelligence might cause people to bond with the bot at the cost of human interactions.

The San Francisco-based company cited literature which it said indicates that chatting with AI as one might with a person can result in misplaced trust and that the high quality of the GPT-4o voice may exacerbate that effect.

"Anthropomorphization involves attributing human-like behaviors and characteristics to nonhuman entities, such as AI models," OpenAI said Thursday in a report on safety work it is doing on a ChatGPT-4o version of its AI.

"This risk may be heightened by the audio capabilities of GPT-4o, which facilitate more human-like interactions with the model."

OpenAI said it noticed testers speaking to the AI in ways that hinted at shared bonds, such as lamenting aloud that it was their last day together.

Read also

Musk's misleading election posts viewed 1.2 billion times: study

It said these instances appear benign but must be studied to see how they might evolve over longer periods of time.

Socializing with AI could also make users less adept or inclined when it comes to relationships with humans, OpenAI speculated.

"Extended interaction with the model might influence social norms," the report said.

"For example, our models are deferential, allowing users to interrupt and 'take the mic' at any time, which, while expected for an AI, would be anti-normative in human interactions."

The ability for AI to remember details while conversing and to tend to tasks could also make people over-reliant on the technology, according to OpenAI.

"The recent concerns shared by OpenAI around potential dependence on ChatGPT's voice mode indicate what many have already begun asking: Is it time to pause and consider how this technology affects human interaction and relationships?" said Alon Yamin, co-founder and CEO of AI anti-plagiarism detection platform Copyleaks.

Read also

Expect more product placement at Olympics, says IOC

He said AI should never be a replacement for actual human interaction.

OpenAI said it will further test how voice capabilities in its AI might cause people to become emotionally attached.

Teams testing ChatGPT-4o voice capabilities were also able to prompt it to repeat false information and produce conspiracy theories, raising concerns the AI model could be told to do so convincingly.

OpenAI was forced to apologize to actress Scarlett Johansson in June for using something very similar to her voice in its latest chatbot, throwing a spotlight on voice-cloning tech.

Although OpenAI denied the voice they used was Johansson's, their case was not helped by CEO Sam Altman flagging the new model with a one-word message on social media -- "Her".

Johansson voiced an AI character in the film "Her", which Altman has previously said is his favorite film about the technology.

The 2013 film stars Joaquin Phoenix as a man who falls in love with an AI assistant named Samantha.

Source: AFP

Authors:
AFP avatar

AFP AFP text, photo, graphic, audio or video material shall not be published, broadcast, rewritten for broadcast or publication or redistributed directly or indirectly in any medium. AFP news material may not be stored in whole or in part in a computer or otherwise except for personal and non-commercial use. AFP will not be held liable for any delays, inaccuracies, errors or omissions in any AFP news material or in transmission or delivery of all or any part thereof or for any damages whatsoever. As a newswire service, AFP does not obtain releases from subjects, individuals, groups or entities contained in its photographs, videos, graphics or quoted in its texts. Further, no clearance is obtained from the owners of any trademarks or copyrighted materials whose marks and materials are included in AFP material. Therefore you will be solely responsible for obtaining any and all necessary releases from whatever individuals and/or entities necessary for any uses of AFP material.