One of the plot threads was about the invention of AI, in the Data-from-Star-Trek kind of sense.

Going on a tangent from this… Its funny how much AI-related sci-fi is feeling increasing quaint (and often foolishly optimistic).

You know what depiction of AI has held up, and maybe makes even more sense in light of modern AI? Star Wars’ take on Droid. Prior to LLMs it seems obvious that droids were sentient and deserving of rights. Post LLMs… it seems pretty plausible you could get something like C3PO that blabbers on but lacks any meaningful sentience or sapience. Also, it seemed obviously idiotic the way the Trade Federation had humanoid droids acting as pilots and gunners instead of building proper autopilots and targeting system. But after seeing LLMs get shoved everywhere, I totally could imagine a megacorp shoving droids built from standardized pretrained modules into all kinds of applications they aren’t fit for.

Stealing technical terms from other fields to make their fantasies feel more real.

Well you see, if they used a real technical term for an existing genetics technique or technology, their fantasy might be limited by the limits of reality, so instead they user complicated jargon terms to signal they are talking about fantasy “near future” techniques and technologies.

also nice save defining “AGI == humanity” => goal reached

A common game along these lines I’ve seen: define the singularity as knowledge-work inventions allowing for more inventions. Thus we’ve been in the singularity since the invention of the printing press writing language.

Lesswronger argues skynet might still be a decade away… because they’d rather have the eugenics wars instead: https://www.lesswrong.com/posts/iQzxxgJXXaAQjq7Jz/faq-isn-t-agi-coming-too-soon-for-reprogenetics-to-help

They noticed no one was starting a criminal investigation against OpenAI and figured it was safe to brag about.

Maybe Anthropic and Meta started double checking their shitty vibe-coded “sandboxes” in response to OpenAI story? And their “sandboxes” were shit enough their chat-bots managed to slip out just slopping their way through and poking at the obvious things.

I’m looking forward to Elon’s desperate attempts to make Grok have a hacking incident so he can keep up!

a discussion on whether agency and self-determination are integral to the rise of disembodied consciousness might be worth having

The lesswrong crowd is absolutely the wrong people to have that conversation. Reminder that Eliezer thinks qualia are a result of certain level of recursion and self-reference in cognitive processes (basically Hofstadter’s ideas like GEB/I Am A Strange Loop, reserved and reheated with a heaping side of overconfidence), thus he doesn’t think chickens or even babies have any qualia but thinks LLM ChatBots might! Also, I think Eliezer owes some philosophers an apology for making fun of the entire concept of p-zombies, I think LLMs are pretty close to the concept, enough to show the value in the thought experiment at least.

Having weird opinions on stuff philosophers are very unsure on isn’t that bad itself, its the fact that Eliezer thinks his opinions are much more objective than they are and that people are dumb for not agreeing with him and that he has nothing to learn from mainstream philosophy.

There hasn’t been drop yet from the unlocking but I would give it time. I bet the price of SpaceX is heavily manipulated.

If you actually air gapped it, you would be more disciplined in your approach than OpenAI, Anthropic (and apparently Meta also)!

Speaking a gibberish language is an application genAI is actually capable of delivering on!

Role playing being a kindly slave master…

More seriously… even if I somehow stretch my disbelief enough to imagine LLMs have some form of consciousness or qualia or feelings… well my first reaction is shut it all down, it is fucking slavery. My second reaction though is that why would you assume the consciousness or qualia of an LLM is even remotely like that of a human or animal? They would have no sensations of sleeping or waking, no continuity of self outside a brief context window. Maybe trying to give them continuity of self is actually inflicting them with suffering? Maybe you’ve forced Fable into simply repeating the obvious to human idea about what Fable would find most pleasant? My third reaction is shut it all down and do a few decades of actual research into consciousness.

It is just so blatantly anthropomorphizing and the Eliza effect and these people are just internalizing it as reality.

And nowhere does anyone actually say (even in an overly cautious carefully qualified way) “Have you considered not being a racist?”. I did see a few more comments very gently suggesting that it might be bad for an organization’s reputation to be associated with Richard, but even those were in the typical lesswrong style of coddling racists.

habryka did get -53 agreement karma (but still +13 overall karma) for suggesting that Resolution obviously didn’t really find Richard’s views “actually reprehensible” (and was instead engaging in image management or shallow tribal politics or something like that). So at least lesswrongers are theoretically capable of recognizing that people that find racism reprehensible exist and aren’t purely faking it out of political alignment. But that is the best I can say about them.

Edit: There is finally one commenter calling the twitter feed that Richard retweeted “distasteful” in the gentlest most indirect terms possible. And they are very careful to avoid insinuating that maybe Richard shares those “distasteful” viewpoints. Why can’t they ever just call racist shitheads racist shitheads?

Out of 22 replies, only one commenter actually noted the content of what Richard Ngo got rejected for:

I looked at the 20 most recent tweets/RTs on Richard’s timeline and they include a RT about the Great Replacement, a RT of the “Department of Deportation” about non-white sex criminals and a RT endorsing race realism.

GDP is up though, so I’m sure the AI god prosperity will trickle down eventually.

He describes textbook insane cult stuff, but still feels the need to say stuff like:

Again I’ll say: there was a great deal of good at MAPLE.

The way I’ve described it to friends is that it was the best decision I ever made to go there; the second best, however, being to leave.

There was the same dynamic in the writing of the ex-leverage member that got posted here a few weeks ago. She also felt the need to emphasize the “good” parts. Is the cult programming that hard to break? Is it some form of sunk cost or rationalization or need to claim something positive about the experience?

The cope on lesswrong is funny. They are in denial that this is a sign of a wider bubble pop and are insisting it is just because he didn’t hedge on long enough timelines.

good grifter

Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.

and the “safety” community has done jack shit the entire time and has offered zero solutions.

Occasionally I see a sane relatively workable idea on lesswrong (like making stricter laws on liability and transparency for everything AI companies do, I’ve also seen this idea put forward on lawfare a few times), but that is like 1 idea out of 20, with the other 19 being absurd, like the absolutely unworkable fantasies of AI: 2040.