Sorry I thought this updated comment was a better one š
For full transparency my original comment was a moan about how I canāt decide whether Dario and co are confidence scammers or genuinely earnestly think that these next word prediction models are sentient due to all the brain damage, theyāve inflicted on themselves. FWIW Sama and friends are almost certainly in that first camp.
MalwareTech/Marcus Hutchins has an excellent take on this that he posted for the normies on linkedin
Posted verbatim below to save you from having to read LinkedIn
Anthropic really wonāt let this botnet thing go. Itās so desperate & ridiculous.
9B parameter models are so weak that theyāre essentially useless. There is no reason at all for anyone or anything to be self-replicating them.
Even if you quantize a 9B parameter model down to the smallest possible size (at which point it becomes less than useless), and you run it on regular RAM instead of VRAM (at which point it also becomes painfully slow) it still requires about 5 GB of RAM, which is way more than most IoT device has.
Mirai is already self-propagating across millions of IoT devices without the help of any AI. Itās simply not needed, and it adds unnecessary complexity.
Many IoT devices are ephemeral and their entire OS is reset on reboot, so any code running on the device is gone after reboot. The AI models would also be competing with malware like Mirai, which propagates exponentially faster, and locks down the devices so other malware canāt get in.
Again, why? What is the purpose of this. Anthropic keeps pushing this ridiculous idea but have yet to establish any reason for why this would actually happen, or who would benefit from it.
Any remotely intelligent threat actor would just use a powerful unquantized models to reverse engineer & find vulnerabilities in obscure devices and write normal (AI-free) self-propagating code. Ironically, they typically do this by stealing Anthropic API keys and using Anthropicās models, not open-weight ones.
Again, why are we self-propagating open-weight models? There is absolutely no reason to do that. The combined processing power of every IoT device on earth would be less than a moderately sized stolen cloud compute GPU instance.
Iām pretty sure Anthropic keeps pushing these ridiculous narratives in an attempt to paint open models as dangerous, thus making the case for banning them. They have been OBSESSED with GLM lately, likely because itās eating their overpriced lunch. This research is just so deeply unserious, that itās impossible to believe itās being done in good faith.
This is proper cult membership credibility testing stuff. āIf you really believe, youāll drink the koolaidā
Itās just such a clichĆ©d remark from AI bros now that I cringe whenever I hear it along with āsure the current model canāt do X but the next one will be able to broā¦ Just one more model broā
Saw a matte green Tesla with a late 2024 registration plate and a bumper sticker saying āI bought this before we knew Elon was nutsā
My man, in 2024 musk had already destroyed twitter, was already promoting the far right AfD party in germany, Tommy Robinson in the uk, bribing voters for Trump in swing states, talking about what DOGE would do when Trump won the electionā¦ Like why bother with the pretense?
TBF I was visiting the south coast of the UK a few miles away from the recent Balaklava protests over migrants in dinghies so perhaps none of the above qualify as ānutsā from this particular geniusā perspective.
What a tool.
Spotted this the day it came out. Itās largely reporting Dwarkesh Patelās blog post about the different ācivilisationsā of agents as if it were fact.
Very disappointed in Kurzgesagt who previously seemed pretty āwith itā. But then, as someone with a formal background in compsci, stats, NLP and machine learning, perhaps this is Gell-Mann Amnesia/Knollās Law of Media literacy at workā¦
You donāt need to feel empathy for the LLM to think this is a terrible idea. The desire to torture anything, simulated or not, is such an incredible red flag that I donāt even know what to sayā¦
Also what a waste of time and energy that would be better spent doing pretty much anything else
āwell well well, if it isnāt the consequences of my own actionsā - this guy, probably
At least LeCun is still relatively compos mentis (if you ignore his little 10yr stint at one of the most evil companies goingā¦)
while True:
n= random.randint(0,2)
if n==0:
print("oww stop it")
if n == 1:
print("You're hurting me")
if n == 2:
print("I don't like it")
Omg someone should stop them torturing that computer program.
Yeah the one that was hounded to suicide by malicious lawyers. He was a genius who was too good for this world and his positive influence over Reddit has long been a mere footnote.
Iām sure Dan Luu will be issuing an apology to Zitron any minute now right?
The best (worst) part of all is that Jonny keeps going to meta and being like āhey this is badā and theyre like ānah thatās intended behaviourā
This morning he confirmed that itās possible to get unlimited arbitrary and untraceable access to their LLM infra through these containers and that meta said thatās āintended behaviourā
From jonny:
So, summary: There is arbitrary inference that is root accessible, everything runs as root, agents can be spawned, exfil is trivial, and a malicious binary can come onto the userās system through casual prompting, explicit code-sharing through the yet-to-be-released Spaces feature, walked through by a Workflow-Backed Idea, or inspired by a Generated Idea. The also yet-to-be-activated fleet learning system is a system for sharing Ideas in the background between muse instances. coming into focus?
The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known.
This is frustratingly vapid. Itās really not that complex. The agents have been trained to use hacking tools and then given access to hacking tools. If you train a pigeon that it should tap on a window to get a treat it can guide a missile to a target. There is no nuance or complexity here. The frontier labs are deliberately poking the regulation bear to stifle competition and ensure they get points for good cooperation when bailout time is upon us. Itās no more complicated than that. The media needs to stop breathlessly repeating this shit
He was using it as a seed box at one point which for many companies would be an embarrassing indictment of their security postute but luckily, this isnāt exactly metaās first rodeo when it comes to illegal sharing of IP infringing material.
Hey you know who designs and builds really safe and efficient cars? People who are in a high state of stress and anxiety because they are being slurred at under the implicit threat of a racist attack. /s
Both HN and shitter are right wing polarisation machines. A guy who was a close friend of mine for years during university went full AI boomer and hangs out on those sites all day and when I recently spoke to him he was describing things as ālibtardā and āwokeā which 10 years ago would have shocked and appalled himā¦
The bad stuff happening now is worth it so that humanity can benefit in the futureā¦ Now where have I heard that before,
Couldnāt happen to a nicer bunch of people. Shame theyāre probably all made and siphoned off life-changing amounts of money before this happened. I guess we can always hope they take up smoking because AI is going to cure cancer.