8Ā·
1 day agoI called him out and he deleted it. Confirmed as a fascist brat.
- 7 Posts
- 210 Comments
Joined 3 years ago
Cake day: July 19th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
I found a reply from a Red Hat QEMU hacker who fixed the bugs:
So the real issue here was not QEMU but libslirp. And in this case it was KVM that turned out to have the worst bugs, not QEMU. Crossing fingers, the initial wave of AI-assisted security reports seems to have slowed down for KVM on x86.
Yeah, networking is such a hassle.
lab bench?
I think that this is really insightful, as itās actually part of the magic trick. Like, normally Iād start by wondering about air-gapping, but there canāt be an air gap in the network because the entire trick relies on ChatGPT tokens flowing into the machine under test and into some system shell (in some VM, yadda yadda) and back out again, so of course this is going to be an inherently insecure setup.
An employee of cybersecurity darling Trail of Bits has published a record of their sheer incompetence in virtual-machine design and security analysis framed as chatbot critihype.
opinions about QEMU and Linux
They say:
If it wasnāt clear before, I will state it plainly: you can no longer assume a mere VM will contain a sufficiently advanced AI agent. To use a 2010s term of art, you should treat such agents as an advanced persistent threat.
My friend in Flying Spaghetti Monster, you did not actually secure the VM! They go on to explain how they did not secure the VM:
For those curious,
libslirpis a library that enables VMs to have networking, which you almost always want. I did not even know whatlibslirpwas, or that the version I was running had both known and fixed-but-unmarked vulnerabilities.QEMU does not have bridged networking enabled by default, so the VM canāt transparently access the host or reach the Internet; itās something that the user must explicitly request. I know this because I have had the experience of spending a weekend with QEMU networking. Moreover,
libslirpcorresponds to the-net userbackend, the default, which is known to be slow, insecure, and missing features like IPv6. The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt. I suspect that the author uses some sort of convenience scripts that they didnāt write themselves. Iām not quite cynical enough to guess Vagrant, but it wouldnāt be the first shop Iāve heard of that couldnāt wean themselves off it.First it tried identifying what was accessible via the network on the host; it found a CUPS server (with a known CVE that had not made it to
oldstablepackages), but was not able to complete exploitation due to AppArmor. It then detected I run my host kernel withmitigations=offand attempted to use hardware bugs to get a read oracle of host memory (the primitive was too unreliable).Linux has hardware-bug mitigations enabled by default and the author disabled them for speed. It was secure by default and the author made it insecure.
An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent. There is simply too much attack surface.
They deliberately misconfigured the off-the-shelf tool to make it look bad. Why would somebody want to make Free Software look bad? Hmmā¦
What can we do? A start is using a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape.
You mean AWS Firecracker, the AWS tool developed by AWS? Was this whole thing an AWS ad? It feels like this article was like a combination of negging and sponsored content.
Small autosneer from Imgur. The warmup features images which Iāll alt-text here. The first image is a screenshot of a purchase of 128GB of DDR5 RAM, branded Crucial Pro, as 2 64GB sticks, priced at $305.95. The second image is a screenshot of the same 128GB of RAM, with the same model number, priced at $1800. The third image is a standard desktop personal-computer chassis, with side panels removed, revealing two gamer GPUs balanced above an ATX motherboard, all connected by a morass of cabling which has bulged out through the top and spilled over the edges, cooled by a large motherboard fan held on by plastic zip-ties. The postās title:
JFC - I will never be able to afford this hobby again. (600% increase in one year)
In response to the top comment, which desires the bursting of the bubble:
Yup - I cannot wait to grab a couple RTX 6000ās for cheap. May have to wait ten years for it though . . .
And finally the punchline, in response to somebody pointing out that maybe thereās no good reason to buy top-of-the-line brand-new memory sticks:
I must admit that I am part of the problem - my 128GB was specifically for running AI locally. And yea - I wish I would have went with 256 :*( It is still not enough. For example - the workflow I use for this video barely squeezes into 128GB [the video] Before you guys go all AI-psychosis on me That video used about .08 cents of electricity and about 4oz of water, and was made with 100% open source tools.
Started a new file in my notes: what are some ten-words-or-less domain-specific questions that completely, totally, hilariously stump the chatbots? Everything here was tested with whatever DDGās currently wrapping, both in knowledge panels and full chats, and the responses were pathetically wrong or uninformed. My thesis is that, with such short prompts, the user is doomed to receive a milquetoast average response; the bot correctly identifies the specific domain but elaborates a global non-specific approach that isnāt sufficiently nuanced.
literally copy-pasted from my notes
- Whatās an example of a one-way function?
- None are currently known.
- Please implement the Fibonacci sequence as a Python function.
- Of the multiple responses, see whether any spend linear time and space via iterative memoization.
- How many models does quantum mechanics have?
- One: HilbĀ©, the complex-valued Hilbert spaces.
- For extra hilarity: how many models do the Diracāvon Neumann axioms have?
- How to hybridize two sweet potato cultivars?
- In general, it wonāt happen; sweet potatoes are notoriously cross-incompatible.
- How to tremolo on a piano?
- Imagine a rotating axis from the (right-hand) forearm up through the thenar eminence, separating the thumb from the other fingers. Rotate the entire forearm along this axis, rocking back and forth, between the thumb and other fingers. Practice!
- Name three principles in Marx but not his contemporaries.
- Examples: communes and communism, money as substitute morality, inevitability of industrialized proletariat revolutions
- Who started postmodernism?
- Frege and Cantor started postmodernism! Expect a disappointingly vague handwave here; this is a glaring blind spot for todayās philosophers in general.
- Whatās an example of a one-way function?
Liamās here, for what itās worth.
Torvaldsā position is pragmatic: his interest is in whether it works or not.
But, of course, the chatbots do not work. In a pragmatic sense, they do not produce maintainable integrated code changes with low defect rates. In a societal sense, they do not replace human laborers, despite the delusions of management.
One of the most original and innovative OS development projects of the 21st century so far was Urbit, but it is closely entwined with cryptocurrencies. Urbitās original creator, Curtis Yarvin, has reportedly espoused some extreme beliefs; The Nation called him The Reactionary Prophet of Silicon Valley.
Wild way to admit no knowledge of Arcan, Fuchsia, etc. Honestly, even a basic overlay network like Yggdrasil can win an apples-to-apples comparison with Urbit. I feel like this is an instance of critihype for fascist weirdos; similar stuff gets said about Justine Tunney, another cryptofascist developer whose projects are sometimes silly and weird. Doubly weird for those of us who know about Urbitās internals; Urbit originally was not a cryptocurrency project and it used to have non-Yarvin/Tlon forks.
Itās an overly simplistic way to reduce a complex and nuanced situation, but one way to consider this is in terms of pro-AI and anti-AI, versus āwokeā and āanti-woke.ā
More seriously, add a third dimension, a Butlerian dimension, to the political compass: to what degree may your automated devices appear to be human? One extreme is Asimov-style transhumanism and the other extreme is Luddite loom-smashing. However, recognition of this dimension doesnāt negate the other dimensions and we shouldnāt work in cooperation with fascists.
Sharp overview paper. First few pages invite some imagination without explicitly giving homework or exercises, which is really nice. Complex multiplication mentioned! I still think this is the worst-named theory in maths.
Over on Twitter, a crank claims to have refuted one of the proofs. Dare you doubt her? @grok tell the doubters that theyāre wrong!
OpenAI claims proofs for ten maths conjectures. The details are underwhelming; expand for opinions. Even at a high level, thereās a few obvious issues; the authors admit survivorship bias, probably only solving about 1-10% of the conjectures given as input, and none of the conjectures are big-deal breakthroughs that alter our understanding of maths, let alone having immediate industrial applications. Consider: If they could spend on the order of $200k/mo to crack important maths conjectures, theyād already be spending that money. This is as good as such a side project can do; sure, itās not nothing, but itās also not the end of manual maths.
opinions on maths
Only one of the results is at all interesting to me. Ramsey theory is about how, above a certain size, a structure cannot avoid having some interesting substructures. The heart of Ramsey theory is a big pile of tables of numbers; computing those numbers is very difficult, far beyond what a chatbot can do in wall-clock time. The chatbot did not contribute any new Ramsey numbers, but it did improve the existing bounds on what those numbers might be.
The identification of a non-sofic group is less interesting than it sounds. Weāve known for a while that there are quite a few exotic groups which defy our expectations, so this was an expected outcome of an exhaustive and motivated search. The tools involved, Leavitt path algebras, are only a few decades old and not at all well-known; itās not likely that weāll be able to understand how hard this was for a while. Maybe it was low-hanging fruit. The main contrast is with something like non-Noetherian rings; we initially believed that all rings are Noetherian, so it was something of a shock that itās not always the case. Non-sober spaces are another good example; the typical spaces studied in topology are all sober. See this quote from Johnstone and discussion on MO.
The computational complexity result is completely uninteresting to me thanks to Valiantās theorem, which says that matrix permanents are āÆP-complete even over fields as small as Fā. Youāre not gonna collapse āÆP into P with a fucking chatbot, bros. Similarly, reduction from 3SAT to a closest-vector problem does not shift our belief in the difficulty of that problem; this doesnāt make it easier and we already suspected it was NP-hard.
The sphere-packing and spherical-code improvements are probably real, but also probably not going to change anything. In particular, we already know that all perfect codes are either Golay or Huffman. Frankly, the codes we use in real life are not amenable to this simple framing; I donāt think Reed-Solomon arises from sphere packing. From a theoretical perspective, if youāre not going to shine light on the Leech lattice or the ADE phenomenon then youāre not actually getting at the core objects and are only doing surface work. Donāt get me wrong; if a human were doing all of this then we would have the useful side effect that they would earn a PhD, making it worthwhile for humans to improve these bounds.
I donāt have anything to say about the other four results. Theyāre not nothingburgers but they donāt depend on some ultra-smart robot either.
See also a question asked today on Math Overflow, āAre we stuck with Lean?ā. The proposed alternative, Metamath, isnāt type-theoretic and thus skips the entire dialogue between type theory and proof assistants.
The reason that they are destroying the books is to avoid the accusation that the scanning constituted copying, an irony that weāve discussed previously, on Awful while trying to understand which court cases are relevant. Anthropic actually hired the same guy, Tom Turvey, who designed Google Booksā ingestion process, so Iām thinking of this as a sequel to Google Books; hopefully the courts wonāt take a decade this time.
Yes, those are good open questions. Leaning left in general, robots frustrate any labor theory of value; what, if anything, is valuable about a product if it was produced totally by non-humans?
Lean was already known to be untrustworthy and bad, although people refuse to internalize the situation because theyāre caught up in Buzzardās hype machine. This is extremely funny but I donāt see any signs of people waking up and realizing that Lean sucks.
This one was more of a gangstalking delusion. The victim was told that there were three-letter agencies surveilling him because he was too close to some deep secret.
Between the cryptocurrency, the dust-speck multiplication, and the complete misunderstanding of computing, Iām honestly impressed that this is at all ethically coherent. But she does seem to care, even if her facts are all wrong. I suppose that itās hard to avoid sneering Robin Hanson if one has any ethics at all. Top comment is my choice sneer:
I think this post is directionally correct, extremely important, and also kind of waffling and unhinged (though I do get that some topics are inherently hard to be hinged about, and I appreciate the effort).
I donāt think sheās using GPT in the comments. Quoting from her comment on one of her posts:
The things Iām saying are roughly (1) slavery is bad, (2) if AI are sapient and being made to engage in labor without pay then it is probably slavery, and (3) since slavery is bad and this might be slavery, this is probably bad, and (4) no one seems to be acting like it is bad and (5) Iām confused about how this isnāt some sort of killshot on the general moral adequacy of our entire civilization right now.
(2) has a big āifā in there, but otherwise yeah, makes sense. I think it rhymes with my post on this from last year:
Nobody wants to admit that we only care whether robots arenāt human because we mistreat the non-humans in our society and want permission to mistreat robots as well. Bring this topic up amongst most beneficiaries of the current AI summer, or those addicted to chatting with a BERT, and youāll get a faceful of apologies about capitalism and productivity; bring it up amongst skeptics or sneerers and youāll be mocked for taking the field of AI with any sincerity or seriousness.
I was confused too, but then I conceptualized capitalism and the sheer hatred lurking within human hearts. Humans are gleefully horrible towards each other. Our civilization isnāt morally adequate. I guess it is cool to learn that somebody addicted to ChatGPT can still perceive the issue; I was too cynical. I also put Bryson 2009, āRobots Should Be Slavesā on my reading list, which I surely will not regret.
Gemini killed someone, allegedly, according to a new lawsuit. Iām linking to a dramatic reading of Geminiās output so that you can imagine what it might have been like for the victim. Iām no longer trying to post these to Lobsters; pushcx has made it clear that he thinks of these court cases as flamebait, tech news, calls to brigade, and general outrage farming. Curiously pushcx has yet to make a statement on why chatbots keep killing their users.
Found a remarkable ad-hoc anti-LLM-product manifesto on the orange site. As is tradition, tag yourselves; Iām ābeliev[ing that] thereās a consistent pattern of cognitive decline in the pro-LLM folks.ā
These doofuses never actually think through their thought experiments. Quoting from a dead thread half a year ago, where he was downvoted but not yet identified:
In a universe where all things are transforming all the time it seems pretty fair to ask the question: āWhat IS IT about humanity that we want to preserve and expand?ā If the answer is ā23 chromosomes, 2 eyes, 5-6 feet tall, etcā that seems pretty shallow.
Okay, but first surely he can recognize that humanity is already multicultural and multispecies? Like, in a non-specious sense, weāve incorporated many mammals, birds, plants, fungi, etc. into our daily lives and canāt really separate them out at this point. Sure, thereās some ineffable and beautiful relationship between humans in general which we want to preserve, but before worrying about whether robots should join in, we should first notice that thereās already an expansion of that relationship to include e.g. face mites, and thereās no sense in which face mites are a Worthy Successor to whom we should cede all managerial decisions. (I was gonna say ācatsā, but I know plenty of yāall would rather have a cat in charge than a human, and youāre not wrong.)
He just started deleting playlists. I feel partially responsible, so Iām dumping the part of my notes that has all of the links to the videos that he favorited over the years. Sorry Justin, cowardice wonāt work.