Revolut says that in less than a year it’s managed to slash its [code generation] costs by roughly 90 percent by making self-hosted models its default instead of relying primarily on commercial services like Claude.
Oof, remind me never to get an account with Revolut.
This is going to be a trend, and why I think the IPO for Anthropic is ridiculous. Competition is massive, there is no moat.
I think the doomerism was all to get that moat. There’s no vendor lockin either, even if you’re using the API. You can switch between providers really easily.
deleted by creator
About time this hit enterprise.
Been saying the downfall of OpenAI and Anthropic will be local models that can accomplish the same thing without needing a dstacenter worth of GPUs to run.
Plus you have no total token limit on self hosted models. You can let a slop agent run for weeks without ballooning costs or getting rate limited.
And no model censoring, so you can go build that nuclear bomb you always wanted lol.
Both AI giants are already losing a ton of money because their pricing isn’t viable for the amount of training they’re doing trying to be on top. If they readjust, they’re gonna suffocate against much cheaper solutions.
The data center is still needed, just not the same scale. You don’t run “small” local models to compete with Claude’s size. They don’t publish numbers, but estimates were like 1-2T params for the models like early Opus. You need a lot of hardware to run those efficiently at a scale an enterprise would need.
Or they just spent enough to upgrade developer desktops with RTX Pro’s to run quantized ~100B param models at their desks. Those are very good for coding.
I’m not surprised it’s moving this direction, but I am surprised it’s moving this direction now.
Hardware is extremely difficult to find right now and token prices are falling off a cliff in terms of price per intelligence. About 6 months ago I was talking about how it’s only going to get cheaper from here and still seems to hold up today, we really don’t need anything better than Opus 5.5 and compute is only going to keep increasing (price decreasing) who knows what the field will look like next year.
Either way I’m happy to live in a world pushing towards that, just surprised.
Microsoft predicted this some time ago which is why they have begun pushing a Hybrid Model where tasks are run locally first and only going to the cloud for heavy lift.
If AI isnʻt going to go away, it needs to be cheap, sustainable, open source, and self hosted/ hostable.
And ethically trained. People should be financially compensated if their data was used to train AI without their consent.
Private copying levy was implemented to compensate greedy media industry. This was included in the price of every writable massmedia device in Finland until 2015.
Maybe something similar could be used with AI. Directly compensating for the victims of ip theft would be pretty impossible but the money could be used to fund art programs and such. AI companies would be the ones paying, based on the usage their models have, be it online or self hosted.
I don’t think it’s really possible to do this.
I mean speaking just logistically, how would you determine who gets how much? If it’s based on quality, how do you determine what is of high quality? Otherwise, does everyone who has every written anything that finds its way into a training dataset get like a tenth of a cent?
Alternative proposal: all AI models must be released as open weights, available to everyone.
AI, through its use of training data, has benefited from the sum total of human knowledge, therefore the results of this training should be given back for the use of all humanity.
That seems fair to me.
Not only open weight, but the dataset used should also be public.
This is also my opinion. I always advocate for putting GenAI models under UNESCO stewardship, because they are a snapshot of all digital, cultural output the world has to offer, and therefor they aren’t less important than physical cultural artifacts. Free personal use, licensed use for commercial usage, with lower prices for educational/nongovernmental organisations. Use the funding to provide grants to the arts in a large scale.
Holy shit! What an excellent idea.
I always advocate for putting AI models under UNESCO stewardship
As now will I.
You are right if we look at it short term, long term llm will be replaced with the next thing and who knows maybe it can be trained ethically, at least we should require research into it.
I don’t think it’s really possible to do this.
Why not? Both parties have to figure out how much they want to pay or to get payed. Similar like it works for any other license like on Netflix or Spotify. It is not the AI companies job to determine how much they will get payed but how much the data is worth to them and the data creator how much they want.
If i am at a bakery and the bread they offer does not look like its for that much to me then i have not to buy it.
How do you suggest an individual should go about this?
Do I go down to my local courthouse and file a suit against Anthropic so I can ask the judge to subpoena them to determine if my Reddit shitpost from 2012 is in their training set?
Class action lawsuits. How would something like this work for anything else? Like Netflix subscription violations?
And we are talking about meaningful data not your shitpost, like licensed code or a book you wrote and anthropic pirated.
And we are talking about meaningful data not your shitpost, like licensed code or a book you wrote and anthropic pirated.
Ah ha! You see, that’s the problem. My shitpost is meaningful data. This is something people seem to have a really hard time understanding. Scaling laws and the size of the search space in AI training means that the datum is not valuable. It’s only the data (in conglomerate) that is value for training, and my shitpost would be a part of that.
Once you understand this, you should see what I meant when I said it’s not feasible. How do you determine and then fairly distribute the payout to everyone who has ever written or recorded anything in digital format? This is not a simple supply vs demand equation, like your bakery example assumes.
Also I have no idea what Netflix has to do with this discussion.
Ah ha! You see, that’s the problem.
No. Meaningful in the sense of copyright. Data you get copyright
There you got your distinction.
That soup is already cooked. And obtaining consent is prohibitive, so your data set will be tiny.
Open weight models cannot do this, they do not have any funding. If you demand this, you are just demanding that there will never be any open models, that proprietary AI models controlled by massive trillion dollar corporations will be the only models available, and frankly the cure is worse than the disease.
It’s also impractical, like closing the gate after the horses have already gone. Open-weight, unethically trained models are already out there. They are not going away. You cannot put the genie back in the bottle. They will never earn any money, so they will never be able to compensate for the training that was done to create them, unless you consider the fact that they exist, and they are open to anybody, as at least some measure of compensation.
It’s not enough compensation, but it is something. The training was certainly unethical, but so is an economic system that requires intellectual property to be transactional. Why fix one unethical thing while keeping and reinforcing another?
The only sensible and reasonable way to resolve the issue in the long term is for significant government support and patronage of the arts and content creators (and ideally all people) to be established. People who do this for a living are providing valuable service to all of humanity and that deserves to be recognized not by transactional, metered capitalism that defines each piece of art or content in terms of its raw economic value (a system easily and frequently corrupted anyway if you look at the prevalence of theft, forgery, fraud and money-laundering in the high art world), but by guaranteeing a comfortable and dignified life that allows creative people to freely create the art or other works they love and feel passionate about without worrying that they’re going to starve to death or be forced into a dead-end job if they’re not economically successful.
It is, but the issue is the right time to buy hardware was 2 years ago.
Its like 1/ Mooreʻs law.
The right time to buy hardware was always 2 years ago.
You know what… I like this because it pisses off Sam Altman
tbf every business does use AI these days anyway, it’s vibe coding that’s the issue.
The issue with AI is not the tech itself, but the people controlling it, so I definitely agree with you!
That it’s in the hand of a few people is one major issue, but the externalized costs* are even worse one in my book and this is another post where it’s conveniently ignored mostly.
- Energy consumption, water usage, scraping ddos
Yeah, also how it’s used. If revolut knows what’s happening in their code, then should be grand
so they are paying -9x now?
Yep, but obviously with a big initial investment.
