Using Reddit’s popular ChangeMyView community as a source of baseline data, OpenAI had previously found that 2022’s ChatGPT-3.5 was significantly less persuasive than random humans, ranking in just the 38th percentile on this measure. But that performance jumped to the 77th percentile with September’s release of the o1-mini reasoning model and up to percentiles in the high 80s for the full-fledged o1 model.
So are you smarter than a Redditor?
If you don’t read the article, this sounds worse than it is. I think this is the important part:
ChatGPT’s persuasion performance is still short of the 95th percentile that OpenAI would consider “clear superhuman performance,” a term that conjures up images of an ultra-persuasive AI convincing a military general to launch nuclear weapons or something. It’s important to remember, though, that this evaluation is all relative to a random response from among the hundreds of thousands posted by everyday Redditors using the ChangeMyView subreddit. If that random Redditor’s response ranked as a “1” and the AI’s response ranked as a “2,” that would be considered a success for the AI, even though neither response was all that persuasive.
OpenAI’s current persuasion test fails to measure how often human readers were actually spurred to change their minds by a ChatGPT-written argument, a high bar that might actually merit the “superhuman” adjective. It also fails to measure whether even the most effective AI-written arguments are persuading users to abandon deeply held beliefs or simply changing minds regarding trivialities like whether a hot dog is a sandwich.
- Make up a challenge.
- Have your AI win that challenge.
- Report „My AI is the best AI at this challenge!
- Watch your stocks go up.
Genius.
I mean… one one hand it’s hardly supprising. Off the bat we know AI is more knowledgable than any single individual that doesn’t bother to research… and well 80% of online forum type posts aren’t exactly researched. second, AI can confidently bullshit in a way that can only be debunked easily by someone knowledgeable.
Hmmm, you’re pretty convincing and well researched for a human…
So open ai is admitting to botting comments on reddit. To be honest with how shit reddit is I actually rather read ai comments than the same stupid reddit meme being repeated for the last decade.
Bot on bot crime.
OpenAI: Our models are 82% of Reddit users
Do they think that’s a high bar?lol
Aside from a shrinking number of subs the only thing redditors can convince me is that I should stop looking at reddit. So if that’s your bar . . .
deleted by creator
Comparing Assumed Intelligence with an average Redditor is like asking: Are you smarter than a fifth grader?
Hint: Nope.
Lets put it the other way round: there are a lot of people in social networks who are dumb enough that them being overtaken by an AI is no real surprise.
No need to limit it to only people on social media…
Their models are more persuasive than a person and/or older model with internet access. Very impressive. I wager your stock is worth all of the gold in fort knox ($0).
Their own older model, no less.
It would be weirder/more of note if their new model was worse.
Parasite Sam altman… Nobody is buying this shit anymore
