OpenAI has constructed a device that would doubtlessly catch college students who cheat by asking ChatGPT to put in writing their assignments — however in keeping with The Wall Road Journal, the corporate is debating whether or not to really launch it.
In a press release supplied to TechCrunch, an OpenAI spokesperson confirmed that the corporate is researching the textual content watermarking technique described within the Journal’s story, however mentioned it’s taking a “deliberate method” to as a consequence of “the complexities concerned and its doubtless influence on the broader ecosystem past OpenAI.”
“The textual content watermarking technique we’re creating is technically promising, however has necessary dangers we’re weighing whereas we analysis alternate options, together with susceptibility to circumvention by unhealthy actors and the potential to disproportionately influence teams like non-English audio system,” the spokesperson mentioned.
This might be a distinct method from most earlier efforts to detect AI-generated textual content, which have been largely ineffective. Even OpenAI itself shut down its earlier AI textual content detector final 12 months as a consequence of its “low charge of accuracy.”
With textual content watermarking, OpenAI would focus solely on detecting writing from ChatGPT, not from different corporations’ fashions. It will accomplish that by making small adjustments to how ChatGPT selects phrases, primarily creating an invisible watermark within the writing that would later be detected by a separate device.
Following the publication of the Journal’s story, OpenAI additionally up to date a Could weblog put up about its analysis round detecting AI-generated content material. The replace says textual content watermarking has confirmed “extremely correct and even efficient towards localized tampering, equivalent to paraphrasing,” however has confirmed “much less sturdy towards globalized tampering; like utilizing translation programs, rewording with one other generative mannequin, or asking the mannequin to insert a particular character in between each phrase after which deleting that character.”
In consequence, OpenAI writes that this technique is “trivial to circumvention by unhealthy actors.” OpenAI’s replace additionally echoes the spokesperson’s level about non-English audio system, writing that textual content watermarking might “stigmatize use of AI as a helpful writing device for non-native English audio system.”