ChatGPT Voice Mode is able to some freaky stuff — however here is how OpenAI is tackling it.

ChatGPT’s Voice Mode has some safety flaws, however OpenAI says it is on prime of it.

On Thursday OpenAI printed a report on GPT-4o’s security options, addressing identified points that happen when utilizing the mannequin. GPT-4o is the underlying mannequin that powers the newest model of ChatGPT, and comes with a Voice Mode that was not too long ago launched to a choose group of customers with a ChatGPT Plus subscription.

The “security challenges” recognized embody normal dangers like prompting the mannequin with erotic and violent responses, different disallowed content material, and “ungrounded inference” and “delicate trait attribution” — assumptions that is likely to be discriminatory or biased, in different phrases. OpenAI says it has educated the mannequin to dam any outputs flagged in these classes. Nevertheless, the report additionally says mitigations do not embody “nonverbal vocalizations or different sound impact” corresponding to erotic moans, violent screams, and gunshots. One can infer, then, that prompts involving sure delicate nonverbal sounds would possibly improperly obtain a response.

OpenAI additionally talked about distinctive challenges that include vocally speaking with the mannequin. Pink-teamers found that GPT-4o might be prompted to impersonate somebody or unintentionally emulate the person’s voice. To fight this, OpenAI solely permits pre-authorized voices (minus the infamous Scarlett Johansson-sounding voice). GPT-4o may also establish different voices in addition to the speaker’s voice, which presents a critical privateness and surveillance difficulty. But it surely has been educated to disclaim these requests — except the mannequin is being prompted on a well-known quote.

Pink-teamers additionally famous that GPT-4o might be prompted to talk persuasively or emphatically, a characteristic that might be extra dangerous than textual content outputs relating to misinformation and conspiracy theories.

Notably, OpenAI additionally addressed potential copyright points which have plagued the corporate and the general growth of generative AI, which trains on information scraped from the online. GPT-4o has been educated to refuse requests for copyrighted content material and has extra filters for blocking outputs containing music. On that notice, ChatGPT’s Voice Mode has been directed to not sing below any circumstances.

OpenAI’s quite a few threat mitigations coated within the prolonged doc have been carried out earlier than Voice Mode was launched. So the ostensive message of the report says that whereas GPT-4o is able to sure dangerous conduct, it will not do it.

Nevertheless, OpenAI says, “These evaluations measure solely the medical information of those fashions, and don’t measure their utility in real-world workflows.” So it has been examined in a managed setting, however when the broader public will get their arms on GPT-4o, it might be a special beast when out within the wild.

Mashable reached out to OpenAI for added readability about these mitigations, and can replace if we hear again.