The AI 2027 guy is taking credit for being a good forecaster: https://www.lesswrong.com/posts/cxuzALcmucCndYv4a/daniel-kokotajlo-s-shortform?commentId=44KvqZEG2vwKCaFAu

Recently an interviewer asked me how I got to be such a good forecaster, and I replied by saying something humble. In retrospect it was a bad answer because I should have instead used the opportunity to give actual advice on how to forecast AI well.

Reminder, according to this guy (originally, he has already nudged the dates of his prophecies back a few years), we are supposed to be having agents replace human workers in mass this year, and be one year out from a superhuman coding agent and researcher that radically iteratively improves itself.

We joked about how the AI 2027 people would try to pivot as they were proven wrong… I think they are going to outright try to claim credit for being ‘right’ despite all their critical prophecies being wrong by claiming credit for secondary details.

When banning people takes several ten thousand word essays going back and forth, how can you trust that responsibility to lesser mortals?

Literally just started a discussion on European booster fanfic hype a few days ago: https://awful.systems/post/8591627/11736760

I wonder if Europe 2031 will get a boost out of this, it is really perfect timing for them, they can even claim an early prediction success on the US cutting Europe off! (But as with AI 2027, Europe 2031 assumes a much more competent US that can implement strategies like that in a competent fashion instead of some disorganized demands after 5pm on a Friday).

The move to block it for everyone does conveniently feed into Anthropic’s “it’s too powerful” narrative, but Anthropic is keen to demonstrate in this case that the issue they believe has been raised also applies to OpenAI, so I don’t think this was part of their original marketing strategy - even if it can easily be folded into it.

It also saves them on the cost of actually serving the model, and stalls the cycle of people gradually realizing the new model isn’t much better than the previous one.

Gary Marcus speculates they are upping their prices because they literally can’t afford to hold out. I’m wondering if it is because they need better numbers for their IPO. VC funding be circulated to create nice sounding statements, but IPO filings have a standard of rigor where trying that would be fraud. So they are trying to squeeze their customers to get a few good looking (i.e. revenues higher than operating costs) quarters for the IPO.

We can simplify this to what actually matters

I was initially drawn into his dry recounting of the details, and overlooked that he was false-equivocating (obnoxious but legal) “content creator” activity with police corruption and willful violation of some very clear and foundational laws.

The article collects a lot of information, and isn’t out right wrong, but I find the author under-sympathetic to someone that didn’t have the financial resources to challenge a corrupt corporation and decided going viral was their best bet. Also, I find the author’s language in a comment:

if you put yourself in the cops shoes (something I wouldn’t necessarily recommend doing) they show up and from their vantage point it’s a bunch of rowdy out of town youtuber influencer kids against local homeowners in the community.

Is grossly too sympathetic to cops. The author is basically rationalizing and portraying sympathetically the way cops side with wealth and capital over the actual law.

The comments are not just agi fears and sinophobia right?

Lol the OP was actually being contrarian (to the standard lesswrong attitude) by even vaguely half-assedly considering that the US media may have created a biased narrative that should be questioned.

lesswrong continues to mix sinophobia in with its AI crithype: https://www.lesswrong.com/posts/nmpzH6sLLtKsQhSPM/china-won-t-win-the-ai-race-but-would-it-be-much-worse-if-it

Previously, on awful.system: https://awful.systems/post/4103825

This article has the highlight of identifying the horrible cynical dystopian move of China’s government in response to AI and LLMs of… checks notes… protecting worker rights and workers from mass firings

“An arbitration panel ruled in favor of a map data collector whose entire department was laid off and replaced with artificial intelligence. The panel found that the company’s adoption of A.I. was a voluntary move to remain competitive and did not warrant the employee’s firing. Companies that benefit from technology must, at the same time, adopt “social responsibilities” and protect worker rights, the panel ruled.”

The author feels the need to emphasize how bad China is.

There are quite a few examples of the Chinese state punishing people for speaking out about true problems.

There is Li Wenliang, a doctor who posted to a group chat about COVID before it was officially acknowledged and was forced to sign a police document admitting he had broken a law by spreading false rumours. His reprimand was later withdrawn.

The advantage of the US system appears to be a greater ability to be transparent, in particular for a concerned person in the know to blow the whistle publicly.

Hahaha, no… For example, in Florida, DeSantis has the home of a fired state worker raided for her accessing her old work email (trying to collect accurate COVID numbers, iirc).

This lesswronger is so close to getting it but doesn’t quite make the leap to ‘are we the baddies’. They list out some bad ways the US has used AI and they do acknowledge

But I’m very aware that I’ve been inculcated in a media and cultural environment that says, in its most kind form, be suspicious of non-Western states.

But somehow hold out on actually changing there mind or overcoming their biases.

Anyone familiar with the IPO process have any guessestimates about how long until the public complete S-1 follows? Or odds that it leaks?

That was a nice detailed explanation. The description of the way the tests degenerated was really worrying. Even some boosters insist the tests need careful human oversight.

Relatedly, I think another part of the problem is the implicit assumption that ‘able to do one narrowly defined/narrowly constrained type of problem within a field’ = ‘expert in a field’.

my point is the Scott-A is massively overvaluing the societal worth of pure mathematicians.

I think pure mathematics is as valuable as the humanities. Unlike many stembros, I think the disconnect is that we vastly undervalue humanities, not that pure mathematics is overvalued. Agreed Scott is probably overvaluing them.

If they advertise themselves as a team of forecasters, but then pick a number that doesn’t line up with their forecasts because one team member has a gut feeling or vibes it should be sooner, then that is just another reason not to trust them and to treat them like the clowns they are. Of course, even that reading is pretty charitable, the real reason they picked 2027 is to balance urgency and hype generation with a bit of cushion for when the prediction doesn’t pan out.

I was surprised in a good way see nearly every single comment call him out. Of course, some of those comments (maybe even the majority) are probably boosters mad that he is skipping the slop emails in his inbox. I guess Paul Graham found an angle of hypocrisy that both AI boosters and realists can unite in mocking. Quite an accomplishment.

Not when what they want contradicts the basic limits of reality and logistics!

Ed Zitron has done a breakdown on building normal sized data centers vs. the current target size of AI data centers, and on the bigger end normal data centers are 10s of MWs up to 100s of MWs. After 2 years Stargate Abilene has only turned on its first 200-300 MW. So I think even if regulators roll over on using twice the power of the entire state this project would take 2-3 years just to turn on the first few hundred megawatts then stall out.

Every author named as writing a paper bears full responsibility for the paper.

This has the nice added bonus that it will likely catch PI’s that put their name on their grad students paper without actually doing the mentoring they were supposed to. It will also catch professors that coast (or at least inflate their citation index) by getting their name on papers they barely contributed to.

I am quite convinced that, under these arxive guidelines, every single major PI in the field will be banned within a few years.

Catching a lot of PIs that have allowed and even encouraged slop submission is a good thing in my book.

I was pretty happy about seeing that news about arXiv! So much news has been various organizations giving into LLM usage like some kind of inevitability, so it was a nice change of pace.

he just posted an entirely unnecessary amount of words

taking a quick look at it… it’s actually short by Scott’s standards, but still overly long, given that the only point he makes is claiming Lindy’s Law is applicable to predicting AI progress in absence of other information. Edit: glancing at it again… its not that short, I kinda skimmed until I got to Scott’s actual point my first time glancing at it. You can’t blame me for not reading it.

you-can’t-really-knows

Yeah, he straw-mans AI critics/skeptics as trying to make an argument from ignorance, then tries to argue against that strawman using Lindy’s Law (which assumes ignorance and a pareto distribution). He completely ignores that AI critics are actually making detailed arguments about LLM companies consuming all the good and novel training data, hitting the limits on what compute costs they can afford, running into problems of the long lead time for building datacenters, etc. Which is pretty ironic given his AI 2027 makes a nominal claim to accounting for all that stuff (in actuality it basically all rests on METR’s task horizons, and distorts even that already questionable dataset).

The plagiarism, massive expenditure of venture capital, and unreliable slop output are all intrinsic to the technology, and they hate to be reminded of that because there isn’t much they can do about it. From a technological standpoint, even locally run community fine-tuned open-weight models still originated from plagiarism and big corporate investments, and still output slop. From a social standpoint, the most the can do is try to claim legitimacy through consensus building and we are a threat to that.