Post by @stefano@mastodon.bsd.cafe
From what I’ve read, they used AI to find the PHP vulnerability and then - hey! - used AI to fix it.
What if we’d just skipped the whole thing and left that bug sitting there for another twenty years? 😂
12 boosts
@nuintari I was going to post an harsh message. So I decided to promote positivity and make this message a little less…harsh
@stefano Nah man, you should always promote positivity, you are good at it!
I can pick up the negaslop. Let's face it, I'm prettty good at being angry.
@stefano Looking at the 11 thousand line fix, about 50% of which is newly added test cases... yeah... LLMs tend to test their own code quite a lot.
@stefano I mean, I'm a huge fan of tests really, and without tests my own software would suck and have a lot more bugs. It's just that the kind of tests cases LLMs produce often border on the absurd, either checking basic assumptions or completely irrelevant things.
One example I had recently was test cases added by LLMs checking whether a change from "string" to json.Marshal("string") actually did have the intended effect of escaping the string. If I'm a consumer of a library like that, I'm not gonna test that library's basic assumptions in my own program. I expect the library to work as advertised. It''s like checking whether opening a file without errors will indeed mean that the file is now open.
@stsp they usually test the simplest and correct things and forget to test what really needs to be tested. Cool, isn’t it? 😂
@stefano@mastodon.bsd.cafe
AI finding bugs...hmm, OK. Once they've found them all things should settle down a bit.
AI fixing bugs... Uh oh! 😬
But maybe we should just turn AI off while everyone is patching their gear. Once it's off... We could leave it off, as a treat.
@stefano@mastodon.bsd.cafe ah, I heard that but didn't look into it.
So specifically in 8.5.12? Or did it exist before? Does it exist in the 8.4, 8.3 and 8.2 latests as well?
Very interesting if fix by AI! Verrrry interesting.
@paul as far as I’ve read, only 8.5.x. The others should be ok. But I’ve read it somewhere and didn’t investigate further.
@stefano exactly, and the people who don't want to use "AI" to defend their networks, may have bigger issues in the future.
At this point, the attackers WILL use an AI agent to do this, are you going to have to go through log files this you're being attacked?
Probably not, if anyone wants a week free of Claude Code, let me know. I have three invites I can send out.
Anyone know of a stripped down, minimalist, replacement for PHP?
Asking for a friend. That friend is me.