Christ, my head is pounding. Three fingers of bourbon might help me make sense of this latest clusterfuck from our AI overlords. pours drink
You know what’s worse than being wrong? Being wrong with the absolute certainty of a tech bro explaining cryptocurrency to a bartender at 2 AM. That’s exactly what ChatGPT Search has been up to lately, according to some fine folks at Columbia’s Tow Center who probably don’t spend their afternoons testing AI systems with a bottle of Jack nearby like yours truly.
Here’s the deal: ChatGPT Search was supposed to be this game-changing thing that would make Google sweat bullets. Instead, it’s turned out to be about as reliable as my memory after a night at O’Malley’s. The researchers found this thing spewing wrong answers left and right, but - and here’s where it gets good - it only admitted to being uncertain seven times out of 160 tries.
Seven times. Let that sink in while I pour another drink.
That’s like being wrong 153 times at a bar trivia night but only saying “I’m not sure” seven times. Even the worst barroom know-it-all eventually shuts up after enough people call bullshit. Not our artificial friend though. It just keeps going, like that one guy who insists he invented Facebook after his sixth martini.
The researchers did something clever. They grabbed articles from three types of publishers: ones in bed with OpenAI, ones suing the pants off them, and ones just trying to mind their own damn business. Then they fed ChatGPT Search some quotes and watched it fumble around like me trying to find my keys at closing time.
And fumble it did. It started attributing Time magazine articles to the Orlando Sentinel, which is like claiming Hemingway wrote “Fifty Shades of Grey.” It even tried passing off stolen content as legitimate sources, which reminded me of every term paper I wrote in college.
takes long drag from cigarette
The real comedy here isn’t just that it’s wrong - it’s how goddamn confident it is while being wrong. At least when I’m spouting nonsense at the bar, I have the decency to slur my words so people know not to take me too seriously.
OpenAI’s response? Pure gold. They basically said, “You’re doing it wrong,” which is exactly what every tech company says when their shit hits the fan. It’s the equivalent of blaming your hangover on the glass you drank from rather than the bottle of tequila you emptied into it.
Here’s what keeps me up at night (besides the whiskey): We’re building a world where machines are programmed to lie with confidence. At least humans have tells when they’re bullshitting - they fidget, they avoid eye contact, they order another round. But AI? It just sits there, spewing wrong answers with the unwavering confidence of a drunk swearing they’re okay to drive.
The really twisted part? OpenAI is paying some publishers for the right to misquote them. That’s like me paying the bar to tell everyone I’m actually Brad Pitt. Sure, nobody’s buying it, but I paid for the privilege of trying to sell that lie.
Look, I get it. Technology moves fast. Mistakes happen. But there’s something deeply fucked up about building systems that are more confident when they’re wrong than when they’re right. It’s like we’ve created a digital version of every awful boss I’ve ever had.
ice clinks in empty glass
The truth used to be something you could find if you looked hard enough. Now it’s buried under layers of AI-generated bullshit, each layer more confident than the last. And the worst part? We’re just getting started.
Maybe I’m just an old drunk yelling at clouds, but I remember when being wrong meant something. When you had to own your mistakes instead of hiding behind algorithms and “enhanced search results.”
At least when I’m wrong, which is often, I have the dignity to blame it on the bourbon.
Time to close this laptop and head to O’Malley’s. The truth might not be at the bottom of this bottle, but at least I know what I’m getting myself into.
Stay human, stay skeptical, and keep your bullshit detectors well-calibrated.
P.S. If any AI tries to attribute this post to the Orlando Sentinel, you’ll find me at the bar, plotting my revenge.
Source: Researchers call ChatGPT Search answers ‘confidently wrong’