A wise man from Google said in an internal memo to the tune of:
"We do not have any moat neither does anyone else."
Deepseek v4 is good enough, really really good given the price it is offered at.
PS: Just to be clear - even the most expensive AI models are unreliable, would make stupid mistakes and their code output MUST be reviewed carefully so Deepseek v4 is not any different either, it too is just a random token generator based on token frequency distributions with no real thought process like all other models such as Claude Opus etc.
> just a random token generator based on token frequency distributions with no real thought process
I'm not smart enough to reduce LLMs and the entire ai effort into such simple terms but I am smart enough to see the emergence of a new kind of intelligence even when it threatens the very foundations of the industry that I work for.
This is just starting to feel like desperation, making this claim that SOC LLMs are random token generators with absolutely no possibility of anything above that. Keep shouting into the wind though.
I don’t think LLMs are that great at creating, however improved they have; I need to stay in the driver seat and really understand what’s happening. There’s not that much leverage in eliminating typing.
However, for reviewing, I want the most intelligent model I can get. I want it to really think the shit out of my changes.
I’ve just spent two weeks debugging what turned out to be a bad SQLite query plan (missing a reliable repro). Not one of the many agents, or GPT-Pro thought to check this. I guess SQL query planner issues are a hole in their reviewing training data. Maybe Mythos will check such things.
I’m a little conflicted on this, as I see a slippery slope here. LLMs in their current state (e.g., Opus-4.7) are really good in planning and one-shot codegen, which I believe is their primary use case. So they do provide enough leverage in that regard.
With this new workflow, however, we should, uncompromisingly, steer the entire code review process. The danger here, the “slippery slope,” is that we’re constantly craving for more intelligent models so we can somehow outsource the review to them as well. We may be subconsciously engineering ourselves into obsolescence.
It's indeed the latter. Psychologically harder for me than a $20/mo sub but still a better value for the money. I'm finding myself spending closer to $40-$60 a month w/ openrouter without a forced token break.
Edit: it looks like it's 75% off right now which is really an incredible deal for such a high caliber frontier model.
Fully agree, I only pay the minimum for frontier models to get DeepSeek v4 output reviewed. I don't see this changing either because we have reached a level of good enough at this point.
"Deepseek v4 is good enough, really really good given the price it is offered at."
Kimi, MiMo, and GLM 5.1 all score higher and are cheaper.
They all came out before DeepSeek v4. I think you're pattern-matching on last year's discourse.
(I haven't seen other replies, yet, but I assume they explain the PS that amounts to "quality doesn't matter anyway": which still doesn't address the fact it's more expensive and worse.)
PS: Just to be clear - even the most expensive humans are unreliable, would make stupid mistakes, and their output MUST be reviewed carefully, so you’re not any different either. You’re just a random next-thought generator based on neuron firing distributions with no real thought process, trained on a few billion years of evolution like all other humans.
Looks like you either have not worked with any human or with an LLM otherwise arriving at such a conclusion is damn impossible.
The humans I did work with were very very bright. No software developer in my career ever needed more than a paragraph of JIRA ticket for the problem statement and they figured out domains that were not even theirs to being with without making any mistakes and rather not only identifying edge cases but sometimes actually improving the domain processes by suggesting what is wasteful and what can be done differently.
I think you are very fortunate. I have worked with plenty of software developers like that, in fact, the overwhelming majority of them have been like that.
Holy shit, you've never worked with anyone who made ANY mistakes? You must be one of those 10x devs I hear about. Wow, cool, please stay away from my team.
I'm still not sure what people declaring that they equate human cognition with large language models think they are contributing to the conversation when they do so.
Nevermind the fact that they are literally able to introspect human cognition and presumably find non verbal and non linear cognition modes.
They are tools. You hold the human using it accountable. If that means it's the executive who signed the PO, so be it.
Until LLM's I'd never in my life heard someone suggest we lock up the compiler when it goofs up and kills someone, but now because the compiler speaks English we suddenly want to let people use it as a get out of jail free card when they use it to harm others.
Amusing and directionally correct, but as random next-thought generators connected to a conscious hypervisor with individual agency,* humanity still has a pretty major leg up on the competition.
*For some definitions of individual agency. Incompatiblists not included.
I hate that I agree with you. But there's a difference between whether AI is as powerful as some say, and whether it's good for humanity. A cursory review of human history shows that some revolutionary technologies make life as a human better (fire, writing, medicine) and others make it worse (weapons, drugs, processed foods). While we adapt to the commoditization of our skills, we should also be questioning whether the technologies being rolled out right now are going to do more harm than good, and we should be organizing around causes that optimize for quality of life as a human. If we don't push for that, then the only thing we're optimizing for is wealth consolidation.
US models align with our "average" (western) values. If we outsource thinking by using LLMs, why would we outsource it to an LLM that doesn't have our values encoded in it?
I remember asking Gemini about that one famous 9/11 joke from late Norm MacDonald and it got really iffy about answering. Told it that hey I'm not american and in our culture it's not such a taboo.
It understands everything in thinking mode and will break down its rule system in adhering to Chinese regulation
So if you or anyone passing by was curious, yes you can get accurate output about the Chinese head of state and political and critical messages of him, China and the party
Its final answer will not play along
If you want an unfiltered answer on that topic, just triage it to a western model, if you want unfiltered answers on Israel domestic and foreign policy, triage back to an eastern model. You know the rules for each system and so does an LLM
We can because the reality is that America has led in AI since the beginning and has had the best frontier models. It's not like some other country held the top spot for any given period of time. No one in Europe or China. I'd give it the benefit of the doubt if there was precedent. But the only logical position to take is the lead is widening and while most AI's will go over some threshold where it is good enough for most people, the actual frontier will remain firmly in American soil.
I think the biggest winner of this might be Google. Virtually all the frontier AI labs use TPU. The only one that doesn't use TPU is OpenAI due to the exclusive deal with Microsoft. Given the newly launched Gen 8 TPU this month, it's likely OpenAI will contemplate using TPU too.
Many labs use TPUs, but not exclusively. Most labs need more compute than they can get, and if there's TPU capacity, they'll adapt their systems to be able to run partially on TPUs.
And almost by happenstance Apple. Turns out they have a great platform for inference and torched almost nothing comparatively on Siri. The Apple/Gemini deal is interesting, Google continues to demonstrate their willingness to degrade their experience on Apple to try and force people to switch.
They also degrade their own direct services with little warning or thought put into change management, so, to be fair, Apple may be getting the same quality of service as the rest of us.
I think that's just how Google is, by nature. They don't intentionally degrade their services. They just aren't a customer centric company. They run on numbers. As a corporate, it doesn't really encourage support and maintenance work either.
Apple is basically in the same boat as AMD and Intel. They have a weak, raster-focused GPU architecture that doesn't scale to 100B+ inference workloads and especially struggles with large context prefill. TPUs smoke them on inference, and Nvidia hardware is far-and-away more efficient for training.
The GPUs are bottom-barrel for compute-focused industries. It is mobile-grade hardware that arguably can't even scale to prior Mac Pro workloads.
> The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W.
You're just listing the TDP max of both chips. If you limit a 3090 to 120W then it would still run laps around an M1 Max in several workloads despite being an 8nm GPU versus a 5nm one.
> It is kind of sad Apple neglects helping developers optimize games for the M-series
Apple directly advocated for ports like Death Stranding, Cyberpunk 2077 and Resident Evil internally. Advocacy and optimization are not the issue, Apple's obsession over reinventing the wheel with Metal is what puts the Steam Deck ahead.
Edit (response to matthewmacleod):
> Bold of them to reinvent something that hadn't been invented yet.
Vulkan was not the first open graphics API, as most Mac developers will happily inform you.
Why does this need to be stated? Who else's would they be?
edit: he puts this on so many comments lol c'mon this is absurd.
Just add it to your profile once, no one assumes individuals speak for their employers here that would be stupid. The need to add disclaimer would be for the uncommon case that you were speaking for them. It's an anonymous message board we're all just taking here it's not that serious.
Some on this forum will be working for companies with conflicts of interest on the topic, and if an employees words were construed to be the opinions of the company that could be bad for that person.
I was once almost fired for saying a little too much in an HN comment about pentesting. Being dragged into an office and given a dressing-down for posting was quite traumatic.
The central issue (or so they claimed) was that people might misconstrue my comment as representing the company I was at.
So yeah, I don’t understand why people are making fun of this. It’s serious.
On the other hand, they were so uptight that I’m not sure “opinions are my own” would have prevented it. But it would have been at least some defense.
it's like people are LARPing a Fortune company CEO when they're giving their hot takes on social media
reminds me of Trump ending his wild takes on social media with "thank you for your attention to this matter" - so out of place, it makes it really funny
> it's like people are LARPing a Fortune company CEO when they're giving their hot takes on social media
At least in large tech companies, they have mandatory social media training where they explicitly tell employees to use phrases like "my views are my own" to keep it clear whether they're speaking on behalf of their employer or not.
Exactly. There is no scenario where we should expect some random anon to be speaking for Google. When that is the case a disclaimer is warranted, not the common case of speaking for oneself. He can write it once in his profile if he's so worried about it, not every other comment like he does. It's just inflated self importance
Why would they be speaking on behalf of their employer? That is what would need a disclaimer not the common case. Besides, he can put it one time in his profile, not over and over again in every comment like he does. There is no expectation that some random employee is a spokesperson for Google on tech message board comment threads. It's just a way to brag.
> Why would they be speaking on behalf of their employers?
Disclaimers aren’t there for folks who are thinking and acting rationally.
They are there for people who are thinking irrationally and/or manipulatively.
There are (relatively speaking) a lot of these people. They can chew up a lot of time and resources over what amounts to nothing.
Disclaimers like this can give a legal department the upper hand in cases like this
A few simple examples:
- There is a person I know who didn’t renew the contract of one of their reports. Pretty straightforward thing. The person whose contract was not renewed has been contesting this legally for over 10 years. The outcome is guaranteed to go against the person complaining, but they have time and money, so they tax the legal team of their former employer.
- There is a mid-sized organization that had a small legal team that had its plate full with regular business stuff. Despite settlements having NDAs, word got out that fairly light claims of sexual harassment and/or EEO complaints would yield relatively easy five-figure payments. Those complaints exploded, and some of the complaints were comical. For example, one manager represented a stance for the department to the C-suite that was 180 degrees opposite of what the group of three managers had agreed to prior. Lots of political capital and lots of time had to be used to clean up that mess. That person’s manager was accused of sex discrimination and age discrimination simply for asking the person why they did that (in a professional way, I might add). That person got a settlement, moved to a different department, and was effectively protected from administrative actions due to it being considered retaliation.
That deal is a win-win for Google. If they develop a better coding model than Anthropic and beat them at coding, then they win. If they don’t, they still win by making a ton of money from Anthropic long term.
I am not sure what context Jensen said that. But midjourney uses tpu. Apple uses tpu. They are no other frontier labs that use it, but Google + Anthropic is 2 out of 3 frontier lab so.....
You could reasonably say that "A majority of frontier labs uses TPU to train and serve their model."
He's been saying whatever is good for Nvidia for years now without any regard for truth or reason. He's one of the least trustworthy voices in the space.
Jensen hallucinates more than any llm, he just speaks without thinking all that much about what he says and he generalizes a lot. Trying to hold him accountable to imprecisions and gross simplifications is just going to frustrate whoever tries without changing one bit of his behavior.
They're at the frontier of last year. They compete with Opus 4.5. They don't yet compete with current frontier models.
They'll presumably catch up, there is no monopoly on talent held by the US. And, that's more true than ever now that the US is actively hostile to immigrants. Scientists who might have come to the US three years ago have little reason to do so now.
It's kind of hard to say this unless you go out of your way - the scaffolding for interacting with the raw model is a lot better now for many tasks. Is it that 4.7 is so much better than 4.5 or claude 1.119 is so much tuned to squeeze utility out of the LLM despite the hallucinations and lack of self awareness etc. Certainly the current products are great, but I think it's hard to separate the two things, the raw model and the agent workflow constraining the model towards utility.
Nit: scientists have the same reasons to do so now, the same as ever. They just have additional reasons to not do so.
But even that distinction is only temporary, since we're determined to piss away any remaining research lead that draws people in.
Hopefully the next administration will work at actively reversing the damage, with incentives beyond just "we pinky-promise not to haul you at gunpoint to a concrete detention center and then deport you to Yemen".
Am I crazy, or was this press release fully rewritten in the past 10 minutes? The current version is around half the length of the old one, which did not frame it as a "simplification" "grounded in flexibility" but as a deeper partnership. It also had word salad about AGI, and said Azure retained exclusivity for API products but not other products, which the new statement seems to contradict.
It’s extraordinary how much standards have slipped. Completely rewriting a major press release that’s already been sent out, while pretending it’s ostensibly the same document would have been a major corporate scandal just 15 years ago.
This agreement feels so friendly towards OpenAI that it's not obvious to me why Microsoft accepted this. I guess Microsoft just realized that the previous agreement was kneecapping OpenAI so much that the investment was at risk, especially with serious competition now coming from Anthropic?
Microsoft is a major shareholder of OpenAI, they don't want their investment to go to 0. You don't just take a loss on a multiple-digit billion investment.
I think you’re right about this deal. But it’s kind of funny to think back and realize that Microsoft actually has just written off multi-billion-dollar deals, several times in fact.
FaceBook largely requires an Apple iPhone, Apple computer, "Microsoft" computer, "Google" phone, or a "Google" computer to use it. At any point one of those companies could cut FaceBook off (ex. [1]).
The Metaverse was a long term goal to get people onto a device (Occulus) that Meta controlled. While I think an AR device is much more useful than VR; I'm not convinced that it's a mistake for Meta to peruse not being beholden to other platforms.
I think this is sane washing their idea in the modern context of it having failed. I think at the time, they thought VR would be the next big thing and wanted to become the dominant player via first mover advantage.
The headsets don’t really make sense to me in the way you’re describing. Phones are omnipresent because it’s a thing you always just have on you. Headsets are large enough that it’s a conscious choice to bring it; they’re closer to a laptop than a phone.
Also, the web interface is like right there staring at them. Any device with a browser can access Facebook like that. Google/Apple/Microsoft can’t mess with that much without causing a huge scene and probably massive antitrust backlash.
I think headsets might work, but I think Meta trying to use their first mover advantage so hard so early backfired. Oculus, as a device, became less desirable after it required Facebook integration.
It's kind of like Microsoft with copilot - the idea about having an AI assistant that can help you use the computer is great. But it can't be from Microsoft because people don't trust them with that.
> I'm not convinced that it's a mistake for Meta to peruse not being beholden to other platforms.
Devoid of other context, it’s hard to disagree. But your parent comment only asserted that the metaverse specifically as proposed by Facebook was an obviously stupid idea.
Naming your company off a product that doesn't really exist yet and then ultimately fails is a pretty crazy and stupid thing to do. A bit cart before horse.
Because it's been a massively expensive failure. They can't just will their own platform into existence just because it would be good to have, consumers have a say and they've rejected it completely.
Meta's vision was worse than that. They were trying to hype doing work meetings in VR. There's a case to be made that VR games and VR universes can be fun... But work meetings?
Probably more that they are compute constrained. In his latest post Ben Thompson talks about how Microsoft had to use their own infrastructure and supplant outside users in the process so this is probably to free up compute.
> Microsoft will no longer pay a revenue share to OpenAI.
I feel this looks like a nice thing to have given they remain the primary cloud provider. If Azure improves it's overall quality then I don't see why this ends up as a money printing press as long as OpenAI brings good models?
OpenAI was also threatening to accuse "Microsoft of anticompetitive behavior during their partnership," an "effort [which] could involve seeking federal regulatory review of the terms of the contract for potential violations of antitrust law, as well as a public campaign" [1].
No and at this point tying yourself to azure is a strategic passive and anyone making such decisions should be held responsible for any service outage or degradation.
AWS's us-east-1 famously takes down either a bunch of companies with it, or causes global outages on the regular.
AWS has a terrible, terrible user interface partly because it is partitioned by service and region on purpose to decrease the "blast radius" of a failure, which is a design decision made totally pointless by having a bunch of their most critical services in one region, which also happens to be their most flaky.
MS incentivizes feature quantity, and the leadership are employees like any other. Product improvements are not on the table unless the company starts promoting people based on it. Doesn't look this will start happening any time soon.
This is probably a delayed outgrowth of the negotiations last year, where Microsoft started trading weird revenue shares and exclusivity for 27% of the company.
When they put 10B in, they got weird tiered revenue shares and other rights. That has been simplified to 27% of OpenAI today. I don't know what that meant their 10B would be worth before dilution in later rounds.
This strikes me as a pullback by Microsoft. Coupled with some of the other news coming out of Microsoft it appears they are hoping to have "good enough" AI in their products. I think Microsoft knows they can win a lot of business customers by bundling with Office 365.
I was under the impression that as long as GitHub doesn't support IPv6 it is a sign that they still haven't finished their migration to Azure. Azure supports IPv6 just fine.
Supports IPv6 just fine? Absolutely not, they have the worst IPv6 implementation of the 3 large clouds, where many of their products don't support it, such as their Postgres offering. See https://news.ycombinator.com/item?id=44881803 for more.
Their engineers have been working tirelessly to make Sharepoint/Office/Active Directory as terrible as it possibly could be while still technically being functional, while continuing to raise prices on them. I've seen many small business start to chose Google Workspace over them, the cracks have formed and are large enough that they are no longer in a position were every business just go with Office because that's what everyone uses.
Nadella had OpenAI by the short and curlies early on. But all I've seen from him in the last couple of years is continuously acquiescing to OpenAI's demands. I wonder why he's so weak and doesn't exert more control over the situation? At one point Microsoft owned 49% of OpenAI but now it's down to 27%?
Everything is personal preference, and perhaps I am more fiscally conservative because I grew up in poverty.
But if I own 49% of a company and that company has more hype than product, hasn't found its market yet but is valued at trillions?
I'm going to sell percentages of that to build my war chest for things that actually hit my bottom line.
The "moonshot" has for all intents and purposes been achieved based on the valuation, and at that valuation: OpenAI has to completely crush all competition... basically just to meet its current valuations.
It would be a really fiscally irresponsible move not to hedge your bets.
Not that it matters but we did something similar with the donated bitcoin on my project. When bitcoin hit a "new record high" we sold half. Then held the remainder until it hit a "new record high" again.
Sure, we could have 'maxxed profit!'; but ultimately it did its job, it was an effective donation/investment that had reasonably maximal returns.
(that said, I do not believe in crypto as an investment opportunity, it's merely the hand I was dealt by it being donated).
Microsoft didn't sell anything. OpenAI created more shares and sold those to investors, so Microsoft's stake is getting diluted.
And Microsoft only paid $10B for that stake for the most recognizable name brand for AI around the world. They don't need to "hedge their bets" it's already a humongous win.
Why let Altman continue to call the shots and decrease Microsoft's ownership stake and ability to dictate how OpenAI helps Microsoft and not the other way around?
I don’t understand the “record high” point. How did you decide when a “record high” had been reached in a volatile market? Because at $1 the record high might be $2 until it reaches $3 a week or month later. How did you determine where to slice on “record highs”?
Genuine question because I feel like I’m maybe missing something!
The longer answer is; you never know whats coming next, bitcoin could have doubled the day after, and doubled the day after that, and so on, for weeks. And by selling half you've effectively sacrificed huge sums of money.
The truth is that by retaining half you have minimised potential losses and sacrificed potential gains, you've chosen a middle position which is more stable.
So, if bitcoin 1000 bitcoing which was word $5 one day, and $7 the next, but suddenly it hits $30. Well, we'd sell half.
If the day after it hit $60, then our 500 remaining bitcoins is worth the same as what we sold, so in theory all we lost was potential gains, we didn't lose any actual value.
Of course, we wouldn't sell we'd hold, and it would probably fall down to $15 or something instead.. then the cycle begins again..
A business burning more than a dollar for every dollar of revenue is a lot of things. "Quite profitable" is not one of them.
If you're reaching for the SaaStr piece on API compute margins hitting ~70% by late 2025: yes, that exists, and it describes one tier. The volume is on the consumer side. The consumer side is the bit on fire. Pointing at the API margin and calling the whole business profitable is the financial equivalent of weighing yourself with one foot off the scale.
The original argument, in case it got lost: Microsoft holds (held) a 49% stake in a company projecting another $44B of cumulative losses through 2028, against unit economics that depend on competitors not catching up. That's textbook hedge-the-bet territory. "They have paying customers" doesn't refute that, it just describes McDonald's.
They had to negotiate away the non-profit structure of OpenAI. Sam used that as a marketing and recruiting tool, but it had outlived that and was only a problem from then on.
For OAI to be a purely capitalist venture, they had to rip that out. But since the non-profit owned control of the company, it had to get something for giving up those rights. This led to a huge negotiation and MSFT ended up with 27% of a company that doesn’t get kneecapped by an ethical board.
In reality, though, the board of both the non-profit and the for profit are nearly identical and beholden to Sam, post–failed coup.
If Sam continues doing Sam things, MS might get 0% of OpenAI if Satya insists on the previous contract. Either by closing up OpenAI and opening up OpaenAI and/or by MS suing it out of existence. It’s all about what MS can get out of it. If they can get 27% of something rather than nothing, they’re better off.
This sounds like an issue where the hyperscalers are acknowledging that the new Foundation model firms may in fact be worth more than they are. Anthropic looks increasingly likely to exceed AWS revenue next year, and OpenAI will likely do the same with Azure.
3 years ago a Foundation model seemed like a feature of a hyper scaler, now hyper scalers look like part of the supply chain.
> And the investors wailed and gnashed their teeth but it’s true, that is what they agreed to, and they had no legal recourse. And OpenAI’s new CEO, and its nonprofit board, cut them a check for their capped return and said “bye” and went back to running OpenAI for the benefit of humanity. It turned out that a benign, carefully governed artificial superintelligence is really good for humanity, and OpenAI quickly solved all of humanity’s problems and ushered in an age of peace and abundance in which nobody wanted for anything or needed any Microsoft products. And capitalism came to an end.
Interesting side effect of this is that Google Cloud may now be the only hype scaler that can resell all 3 of the labs models? Maybe I'm misinterpreting this, but that would be a notable development, and I don't see why Google would allow Gemini to be resold through any of the other cloud providers.
Might really increase the utility of those GCP credits.
Might not be good for Gemini long term if Anthropic and OpenAI can and will sell in every cloud provider they can find but businesses can only use Gemini via Google Cloud.
I think the public pricing usually has it cheaper (relatively). Obviously since AI is constantly evolving it's not going to compare as favourably farther to a major Gemini release
I was mainly referring to the TPU hardware advantage + GCP running and designing their own datacenter stack.
Per WSJ, previously, they both had revenue sharing agreements. MSFT will no longer send any revenue to OpenAI. OpenAI will still send revenue to MSFT until 2030 (with new caps)
My understand was that was in relation to IP licensing. Microsoft got access to anything OpenAI built unless they declared they had developed AGI. This new article apparently unlinks revenue sharing from technology progress, but it's unclear to me if it changes the situation regarding IP if OpenAI (claim to) have achieved AGI.
The disparity in coverage on this new deal is fascinating. It feels like the narrative a particular outlet is going with depends entirely on which side leaked to them first.
That's a pretty good swap if you're Microsoft. Exclusivity was already unenforceable in practice, and they were going to have to either sue their biggest AI partner or let it slide. Instead they got the agi escape hatch closed and a revenue cap that at least makes the payments predictable
It's kind of shocking, given financial transparency, that Microsoft gets away with not disclosing any details of this agreement (or the one it is replacing) to its shareholders. We know there's a cap on the revenue share from OpenAI to Microsoft, but we have no idea what that cap is (not whether it's higher, lower, or unchanged from the prior agreement).
We have no idea what it means to be the "primary cloud provider" and have the products made available "first on Azure". Does MSFT have new models exclusively for days, weeks, months, or years?
Both facts and more details from the agreement are quite frankly highly relevant to judge whether this is a net positive, negative or neutral for MSFT. It's unbelievable that the SEC doesn't force MSFT to publish at least an economic summary of the deal.
It’s American Business as usual. Personally I’m miffed how little data Apple needs to provide about product categories, and especially about how much they’ve burnt on the car program. If they shared any data about that at all some the leadership might end up having to take responsibility for mismanagement…
Microsoft Corp. will no longer pay revenue to OpenAI and said its partnership with the leading artificial intelligence firm will not be exclusive going forward.
What does this mean that Microsoft will no longer pay revenue to OpenAI? How did the original deal work?
It's unclear. That was never disclosed. It's similarly unclear what it means that they will no longer pay revenue share to OpenAI. Do they get the models for free now? How does OpenAI make money from the models hosted on Azure if not via revenue share?
It seems that the old deal was exclusivity to MSFT with revenue share, and now no exclusivity, no revenue share.
Bear in mind that MSFT have rights to OpenAI IP (as well as owning ~30% of them). The only reason they were giving revenue share was in return for exclusivity.
This is a really common way to structure exclusivity; we did the same thing whenever customers requested it (and we couldn’t get rid of it entirely). Charge for the exclusivity explicitly.
If they wanted named exclusivity rather than general exclusivity, we would charge a somewhat smaller amount for each competitor they wanted exclusivity from. They could give up exclusivity at any time.
That was precisely how we structured our deal with Azure, back in 2014-2016 or so.
Azure was the only non-OpenAI provider that was allowed to provide OpenAI models. The comparison here is with Anthropic whose models are on both GCP and AWS (and technically also Azure though I think that might just be billing passthrough to Anthropic).
I suppose continue to host until the 2030/32 that they have access to but not share revenues when they use those models for their products like the bazillions of Copilots.
The original "AGI" agreement was always a bit suspect and open to wild interpretations.
I think this is good for OpenAI. They're no longer stuck with just Microsoft. It was an advantage that Anthropic can work with anyone they like but OpenAI couldn't.
It also restricted Microsoft from "partnering" with anyone else. Wouldn't be surprised if we see another news like Amazon, Alphabet investing in Anthropic.
AFAICT they are just hedging their bets left and right still. Also feels like they are winning in the sense that despite pretty much all those products being roughly equivalent... they are still running on their cloud, Azure. So even though they seem unable to capture IP anymore, they are still managing to get paid for managing the infrastructure.
Yeah my bad, I was misremembering, it was about investing in others and pursuing its own "AGI" efforts. But even those conditions were updated over the last two years, hence the small investment in Anthropic last year.
I think it was a lot less restrictive, as far as I understood, the only limit was Microsoft not being allowed to launch competing Microsoft-developed LLMs.
I assume this is part of why Github Copilot is going to usage billing. The cheap/free models in Copilot were OpenAI models. e.g. the GPT-based Raptor Mini, which was counted toward usage limits at a 0 multiplier, so basically unlimited usage for Pro and Pro+.
It's unclear which elements of this new deal are binding versus promises with OpenAI characteristics. "Microsoft Corp. will publish fiscal year 2026 third-quarter financial results after the close of the market on Wednesday, April 29, 2026" [1]; I'd wait for that before jumping to conclusions.
Kagi Translate was kind enough to turn this from LinkedIn Speak to English:
The Microsoft and OpenAI situation just got messy.
We had to rewrite the contract because the old one wasn't working for anyone. Basically, we’re trying to make it look like we’re still friends while we both start seeing other people. Here is what’s actually happening:
1. Microsoft is still the main guy, but if they can't keep up with the tech, OpenAI is moving out. OpenAI can now sell their stuff on any cloud provider they want.
2. Microsoft keeps the keys to the tech until 2032, but they don't have the exclusive rights anymore.
3. Microsoft is done giving OpenAI a cut of their sales.
4. OpenAI still has to pay Microsoft back until 2030, but we put a ceiling on it so they don't go totally broke.
5. Microsoft is still just a big shareholder hoping the stock goes up.
We’re calling this "simplifying," but really we’re just trying to build massive power plants and chips without killing each other yet. We’re still stuck together for now.
"The Microsoft and OpenAI situation just got messy" is objectively wrong–it has been messy for months [1]. Nos. 1 through 3 are fine, though "if they can't keep up with the tech, OpenAI is moving out" parrots OpenAI's party line. No. 4 doesn't make sense–it starts out with "we" referring to OpenAI in the first person but ends by referring to them in the third person "they." No. 5 is reductive when phrased with "just."
It would seem the translator took corporate PR speak and translated it into something between the LinkedIn and short-form blogger dialects.
Being objectively correct isn't the goal of the translator, the translator can't possibly know if a statement is truthful. What the translator does is well... translate, specifically from some kind of corporate speak that is really difficult for many people including myself to understand, into something more familiar.
I don't expect the translation to take OpenAI's statements and make them truthful or to investigate their veracity, but I genuinely could not understand OpenAI's press release as they have worded it. The translation at least makes it easier to understand what OpenAI's view of the situation is.
> "they" refers to OpenAI. Not a grammatical error
I'd say it is. It's a press release from OpenAI. The rest of the release uses the third-person "they" to refer to Microsoft. The LLM traded accuracy for a bad joke, which is someting I associate with LinkedIn speak.
The fundmaental problem might be the OpenAI press release is vague. (And changing. It's changed at least once since I first commented.)
(Andy Jassy) "Very interesting announcement from OpenAI this morning. We’re excited to make OpenAI's models available directly to customers on Bedrock in the coming weeks, alongside the upcoming Stateful Runtime Environment. With this, builders will have even more choice to pick the right model for the right job. More details at our AWS event in San Francisco tomorrow."
Likely, and via vertex on gcp (or whatever they are calling it this year).
Which also means, if you are a big boring AWS or GCP shop, and have a spend commitment with either as part of a long term partnership, it will count towards that. And, you won't likely have to commit to a spend with OpenAI if you want the EU data residency for instance. And likely a bit more transparency with infra provisioning and reserved capacity vs. OpenAI. All substantial improvements over the current ways to use OpenAI in real production.
this just validates why building multi-model routing is the future. if even microsoft couldn't lock down openai with $13b, enterprise customers definitely shouldn't lock themselves into a single ecosystem. the orchestration layer is about to get so valuable.
> OpenAI has contracted to purchase an incremental $250B of Azure services, and Microsoft will no longer have a right of first refusal to be OpenAI’s compute provider.
Azure is effectively OpenAI's personal compute cluster at this scale.
That article doesn't give a timeframe, but most of these use 10 years as a placeholder. I would also imagine it's not a requirement for them to spend it evenly over the 10 years, so could be back-loaded.
OpenAI is a large customer, but this is not making Azure their personal cluster.
I wonder how this figure was settled. Is it based on consumer pricing? Can't Microsoft and OpenAI just make a number up, aside from a minimum to cover operating costs? When is the number just a marketing ploy to make it seem huge, important and inevitable (and too big to fail)?
Biggest upside of this is I expect OpenAI models to be available on Bedrock, which is huge for not having to go back to all your customers with data protection agreements.
Isn’t that an “API product”? I read this assuming the whole point of renegotiation was to let OpenAI sell raw inference via bedrock, but that still seems to be blocked except for selling to the US Government.
> OpenAI can now jointly develop some products with third parties. API products developed with third parties will be exclusive to Azure. Non-API products may be served on any cloud provider.
Really interesting. Why would Microsoft have done this deal? I'm a bit lost. Sure they get to not pay a revenue share _to_ OpenAI but surely that's limited to just OpenAI products which is probably a rounding error? Losing exclusivity seems like a big issue for them?
Yes. Microsoft was "considering legal action against its partner OpenAI and Amazon over a $50 billion deal that could violate its exclusive cloud agreement with the ChatGPT maker" [1].
Dario is a lot more focused on enabling people with AI, Sam goes on interviews like he's Wormtongue trying to summon a "god". Then there is the whole "open"ai where he took it closed source for profit, the engineers kicking sama out but he wiggled back in (at the cost of a lot of the founding engineers), the suspicious death of a whistleblower, the crazy investment schemes of billions of dollars that he's hoping taxes will save him from, the immediate curtailing to Pete in the DoD, and a few other things that make him at least a highly questionable fellow.
Dario left OpenAI because of the bad he saw there, and made a superior product (though these things change very rapidly).
> Why is it Altman is facing kill shots and Dario isn’t?
Altman peaked in the zeiteist in 2023; Dario, much less prominently, in 2024 and now '26 [1]. I'd guess around this time next year, Dario will be as hated as Altman is today.
OpenAI has public models that are pretty 'meh', better than Grok and China, but worse than Google and Anthropic. They still cost a ton to run because OpenAI offers them for free/at a loss.
However, these people are giving away their data, and Microsoft knows that data is going to be worthwhile. They just dont want to pay for the electricity for it.
Small nitpick: the models probably make some money on actual inference. Might not be a massive amount, but hard to see them not having a positive contribution margin purely on inference.
What's losing OpenAI money is paying for the whole of R&D, including training and staff. Microsoft doesn't pay that, so they get the money making part of AI without the associated costs.
The AGI talk is shocking but not surprising to anyone looking at how bombastic Sam Altman's public statements are.
The circular economy section really is shocking- OpenAI committing to buying $250 Billion of Azure services, while MSFT's stake is clarified as $132 Billion in OpenAI. Same circular nonsense as NVIDIA and OpenAI passing the same hundred billion back and forth.
I fear for the end user we'll still see more open-microslop spam. I see that daily on youtube - tons of AI generated fakes, in particular with that addictive swipe-down design (ok ok, youtube is Google but Google is also big on the AI slop train).
It’s insane how they talk about AGI, like it was some scientifically qualifiable thing that is certain to happen any time now. When I have become the javelin Olympic Champion, I will buy a vegan ice cream to everyone with a HN account.
I think we keep changing the goalposts on AGI. If you gave me CC in the 80's I would probably have called it 'alive' since it clearly passes the Turing test as I understood it then (I wouldn't have been able to distinguish it from a person for most conversations). Now every time it gets better we push that definition further and every crack we open to a chasm and declare that it isn't close. At the same time there are a lot of people I would suspect of being bots based on how they act and respond and a lot of bots I know are bots mainly because they answer too well.
Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.
I think the advancement in AI over the last four years has greatly exceeded the advancement in understanding the workings of human intelligence. What paradigm shift has there been recently in that field?
> What have we learned that isn't in my textbook from the 90s?
Does it matter?
We can do countless things people in the 90's would think was black magic.
If I showed the kid version of myself what I can do with Opus or Nano Banana or Seedance, let alone broadband and smartphones, I think I'd feel we were living in the Star Trek future. The fact that we can have "conversations" with AI is wild. That we can make movies and websites and games. It's incredible.
I would agree with you if we were talking about trying to replicate some form of general intelligence, but we are talking about creating artificial intelligence.
I don't think the goalpost has been shifted for AGI or the definition of AGI that is used by these corporations. It's just they broke it down to stages to claim AGI achieved. It was always a model or system that surpasses human capabilities at most tasks/being able to replace a human worker. The big companies broke it down to AGI stage 1, stage 2, etc to be able to say they achieved AGI.
The Turing Test/Imitation Game is not a good benchmark for AGI. It is a linguistics test only. Many chatbots even before LLMs can pass the Turing Test to a certain degree.
Regardless, the goalpost hasn't shifted. Replacing human workforce is the ultimate end goal. That's why there's investors. The investors are not pouring billions to pass the Turing Test.
Turing himself argued that trying to measure if a computer is intelligent is a fool's errand because it is so difficult to pin down definitions. He proposed what we call the "Turing test" as a knowable, measurable alternative. The first paragraph of his paper reads:
> I propose to consider the question, "Can machines think?" This should begin
> with definitions of the meaning of the terms "machine" and "think." The
> definitions might be framed so as to reflect so far as possible the normal use
> of the words, but this attitude is dangerous, If the meaning of the words
> "machine" and "think" are to be found by examining how they are commonly used
> it is difficult to escape the conclusion that the meaning and the answer to the
> question, "Can machines think?" is to be sought in a statistical survey such as
> a Gallup poll. But this is absurd. Instead of attempting such a definition I
> shall replace the question by another, which is closely related to it and is
> expressed in relatively unambiguous words.
Many people who want to argue about AGI and its relation to the Turing test would do well to read Turing's own arguments.
The Turing test ended up being kind of a flop. We basically passed it and nobody cared. That's because the turing test is about whether a machine can fool a human, not about its intelligent capabilities per se.
No, it's because certain people moved the goal posts. Nothing an LLM does or will do will make them belive that it's "intelligent" because they have a mental model of "intelligence" that is more religious than empirical.
We don’t have agents that are able to work entirely autonomously, even in the coding realm, which is where they seem to be most valuable. In fact, they’re seemingly not even close to replacing software engineers.
I don't think so... I think most of the sci-fi I grew up reading presented AGI that could reason better than humans could, like make a plan and carry it out.
Like do people not know what word "general" means? It means not limited to any subset of capabilities -- so that means it can teach itself to do anything that can be learned. Like start a business. AI today can't really learn from its experiences at all.
The truth is, we have had AGI for years now. We even have artificial super intelligence - we have software systems that are more intelligent than any human. Some humans might have an extremely narrow subject that they are more intelligent than any AI system, but the people on that list are vanishing small.
AI hasn't met sci-fi expectations, and that's a marketing opportunity. That's all it is.
AGI in the common man's world model is ASI in the AI researcher's definitions, i.e. something obviously smarter at anything and everything you could ask it for regardless of how good of an expert you are in any domain.
also, I'm pretty sure some people will move goalposts further even then.
Hasn't met your sci-fi expectations, maybe. I pull a computer out of my pocket, and talk with it. Sure, I gets tripped up here and there, but take a step back, holy shit that's freaking amazing! I don't have a flying car or transparent aluminum, and society has its share of issues right now, but my car drives itself. Coming from the 90's, I think living in the sci-fi future! (Only question is, which one.)
The Turing test pits a human against a machine, each trying to convince a human questioner that the other is the machine. If the machine knows how humans generally behave, for a proper test, the human contestant should know how the machine behaves. I think that this YouTube channel clearly shows that none of today's models pass the Turing test: https://www.youtube.com/@FatherPhi
> Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.
If you've never read the original paper [1] I recommend that you do so. We're long past the point of some human can't determine if X was done by man or machine.
People thought Eliza was alive too in the 60s. AGI is not determined by how ignorant, uninformed humans view a technology they don't understand. That is the single dumbest criterion you could come up with for defining it.
Regarding shifting goalposts, you are suggesting the goalposts are being moved further away, but it's the exact opposite. The goalposts are being moved closer and closer. Someone from the 50s would have had the expectation that artificial intelligence ise something recognisable as essentially equivalent to human intelligence, just in a machine. Artificial intelligence in old sci-fi looked nothing like Claude Code. The definition has since been watered down again and again and again and again so that anything and everything a computer does is artificial intelligence. We might as well call a calculator AGI at this point.
The goal post keeps moving because LLM hypeists keep saying LLMs are "close" to AGI (or even are, already). Any reasonably intelligent individual that knows anything about LLMs obviously rejects those claims, but the rest of the world doesn't.
An AGI would not have problems reading an analog clock. Or rather, it would not have a problem realizing it had a problem reading it, and would try to learn how to do it.
An AGI is not whatever (sophisticated) statistical model is hot this week.
Vision is still much weaker than text for LLMs. So you could argue we already have AGI for text but not vision inputs, or you could argue AGI requires being human level at text vision and sound.
Sure, in the 80s after interacting with CC 1 time you would call it 'alive'. After having interacted with it for 5-10 minutes you would clearly see that it is as far from AGI as something more mundane as C compiler is.
They redefined AGI to be an economical thing, so they can continue making up their stories. All that talk is really just business, no real science in the room there.
It's not a great definition but it's also not a terrible one either.
For an AI system to be able to do all or even most of the jobs in an economy it has to be well rounded in a way it still isn't today, meaning: reliability, planning, long term memory, physical world manipulation etc. A system that can do all of that well enough so it can do the jobs of doctors, programmers and plumbers is generally intelligent in my view.
> It's not a great definition but it's also not a terrible one either. For an AI system to be able to do all or even most of the jobs in an economy
That's not the definition they have been using. The definition was "$100B in profits". That's less than the net income of Microsoft. It would be an interesting milestone, but certainly not "most of the jobs in an economy".
Yeah I think this is more coherent than people realize. Economically relevant knowledge work is things that humans find cognitively demanding. Otherwise they wouldn't be valued in the first place.
It ties the definition to economic value, which I think is the best definition that we can conjure given that AGI is otherwise highly subjective. Economically relevant work is dictated by markets, which I think is the best proxy we have for something so ambiguous.
> Economically relevant knowledge work is things that humans find cognitively demanding. Otherwise they wouldn't be valued in the first place.
Deep scientific discoveries are also cognitively demanding, but are not really valued (see the precarious work environment in academia).
Another point: a lot of work is rather valued in the first place because the work centers around being submissive/docile with regard to bullshit (see the phenomenon of bullshit jobs). You really know better, but you have to keep your mouth shut.
Yeah, I'm sure there could be a better metric, if the metric's purpose was to check on the progress until the AGI target rather than doing business based on it (and so, hammering the metric to fit the shape of "realistic goal")
Around the end of 2024, it was reported that OpenAI and Microsoft agreed that for the purposes of their exclusivity agreement, AGI will be achieved when their AI system generates $100 billion in profit: https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
> OpenAI and Microsoft agreed that for the purposes of their exclusivity agreement, AGI will be achieved when their AI system generates $100 billion in profit
Wow. Maybe they spelled it out as aggregate gross income :P.
Yea, seems like this was stage setting for them to exit. They were already trying to break the deal then. So, I feel like that is lawyers find a way to bend whatever to get out of the deal.
For some definition of Artificial this holds perfectly
A self-running massive corporation with no people that generates billions in profit, no matter what you call it, would completely upend all previous structural assumptions under capitalism
OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity
> They redefined AGI to be an economical thing Huh. Source?
I don't think your original comment deserve to be downvoted. (Calling someone illiterate, on the other hand.)
But the "it" I was asking about was "AGI" as "an economical thing." You technically correctly answered how OpenAI defines AGI in public, i.e. with no reference to profits. But it did not address the economic definition OP initially alluded to.
For what it's worth, I could have been clearer in my ask.
It makes sense though. Humans are coherent to the economy based on their ability to perform useful work. If an AI system can perform work as well as or better than any human, than with respect to "anything any human has ever been willing to pay for", it is AGI.
I don't get why HN commenters find this so hard to understand. I have a sense they are being deliberately obtuse because they resent OpenAI's success.
It doesn’t though, AGI have far greater implications than doing mundane work of today. Actual AGI would self improve, that in itself would change literally every single thing of human civilization, instead we are talking about replacing white collar jobs.
When we are having serious conversations about AI rights and shutting off a model + harness was impactful as a death sentence. (I'm extremely skeptical that given the scale of computer/investment needed to produce the models we have _good as they are_ that our current llm architecture gets us there if there is even somewhere we want to go).
Eschatology (/ˌɛskəˈtɒlədʒi/; from Ancient Greek ἔσχατος (éskhatos) 'last' and -logy) concerns expectations of the end of present age, human history, or the world itself.
I'm case anyone else is vocabulary skill checked like me
It feels like they have to say/believe it because it's kind of the only thing that can justify the costs being poured into it and the cost it will need to charge eventually (barring major optimizations) to actually make money on users.
It sounds really similar to Uber pitch about how they are going to have monopoly as soon as they replace those pesky drivers with own fleet of self driving cars. That was supposed to be their competitive edge against other taxi apps. In the end they sold ATG at end of 2020 :D
> like it was some scientifically qualifiable thing
OpenAI and Microsoft do (did?) have a quantifiable definition of AGI, it’s just a stupid one that is hard to take seriously and get behind scientifically.
> The two companies reportedly signed an agreement last year stating OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits. That’s far from the rigorous technical and philosophical definition of AGI many expect.
It’s insane to me how yesterday someone posted an example of ChatGPT Pro one-shotting an Erdos problem after 90 minutes of thinking and today you’re saying that AGI is a fairy tale.
It's not one-shot. Other people had attempted the same problem w/ the same AI & failed. You're confused about terms so you redefine them to make your version of the fairy tale real.
We already know that same problem has been examined by many credible mathematicians already and couldn't be solved by any of them yet.
Why are we expecting AGI to one shot it? Can't we have an AGI that can fails occasionally to solve some math problem? Is the expectation of AGI to be all knowing?
By the way I agree that AGI is not around the corner or I am not arguing any of the llm s are "thinking machines". It's just I agree goal post or posts needs to be set well.
People want to believe in magic so they will find excuses to do so. Computers have been proving theorems for a long time now but Isabelle/HOL didn't have the marketing budget of OpenAI so people didn't care. Now that Sam Altman is doing the marketing people all of a sudden care about proving theorems.
You were misrepresenting what actually happened b/c you want to believe in magic. I'm not calling it magic, I'm saying your interpretation of events is magical b/c you don't actually understand how computers work. There is nothing magical about theorem proving, Isabelle/HOL has been doing it for decades.
We were supposed to have AGI last summer. Obviously it is so smart that it has decided to pull a veil over our eyes and live amongst us undetected (this is a joke, if you feel your LLM is sentient, talk to a doctor)
What do you mean we were "supposed to have AGI last summer"?
People obviously have really strong opinions on AI and the hype around investments into these companies but it feels like this is giving people a pass on really low quality discourse.
This source [1] from this time last year says even lab leaders most bullish estimate was 2027.
It is a simple mathematical fact that if you get married one year and have twins the next, your household will contain over a million people within 20 years.
This is all happening as I predicted. OpenAI is oversold and their aggressive PR campaign has set them up with unrealistic expectations. I raised alot of eyebrow at the Microsoft deal to begin with. It seemed overvalued even if all they were trading was mostly Azure compute
I saw a founder make decisions based on what openai,claude was recommending all the time. I think all leaders, founders etc
Will converge on same decisions, ideas, features etc. I think form factor of AGI is probably not what we expect it to be. AGI is probably here, we just dont know it or acknowledge it.
> Do the investments make sense if AGI is not less than 10 years away?
They can. If one consolidated the AI industry into a single monopoly, it would probably be profitable. That doesn't mean in its current state it can't succumb to ruionous competition. But the AGI talk seems to be mostly aimed at retail investors and philospher podcasters than institutional capital.
> Any monopoly with viable economics for profit with no threat of competition yields monopoly profits
"With viable economics" is the point.
My "ludicrous statement" is a back-of-the-envelope test for whether an industry is nonsense. For comparison, consolidating all of the Pets.com competitors in the late 1990s would not have yielded a profitable company.
> Very convenient to leave out Amazon in your back of the envelope test, who’s internal metrics were showing a path toward quasi-monopoly profits
Not in the 1990s. The American e-commerce industry was structurally unprofitable prior to the dot-com crash, an event Amazon (and eBay) responded to by fundamentally changing their businesses. Amazon bet on fulfillment. eBay bet on payments. Both represented a vertical integration that illustrates the point–the original model didn't work.
> There’s a difference between being too early vs being nonsense
When answering the question "do the investments make sense," not really. You're losing your money either way.
The American AI industry appears to have "viable economics for profit" without AGI. That doesn't guarantee anyone will earn them. But it's not a meaningless conclusion. (Though I'd personally frame it as a hypothesis I'm leaning towards.)
Malcolm Harris' Palo Alto explained the failures of many dotcom startups and Amazon's later success in the field (in part) to the fact that dotcom era delivery was done by highly trained, highly compensated, unionized in-company workers, meanwhile Amazon prevents unions, contracts (or contracted, I'm not up to date on this) companies for delivery and has exploitative working conditions with high turnover, the economics are very different and are a big contributor to their success
Investors are typically people with surplus money to invest. Progress cannot be made without trial and error. So fleecing of investors for the greater good of humanity is something I shall allow.
A "surplus of money"? So people saving for retirement have a "surplus of money"? Basically if any money is standing still, it's a legitimate tactic to just...take it, in your mind.
"small" 401ks are usually made up of mutual funds. Those funds are run by investment banks (think Fidelity or JP Morgan) and they *absolutely* invest in companies like OpenAI and Anthropic. Your average middle class worker has investment money tied up in these crooks, but probably indirectly. When they piss away that money, it's not just rich jerks that are holding the bag.
401ks are run by investment banks and investment banks invest in OpenAI/Anthropic, but those aren't the same parts of the company in any meaningful way. The 401ks are in public companies or bonds.
Yeah, but those public companies are going to include the so called magnificent seven, so unless they're really really careful, there's still a ton of exposure in their 401k to AI if you think it's a bubble that's going to pop.
It's always been this way. I remember, speaking of Microsoft, when they came to my school around 2002 or so giving a talk on AI. They very confidently stated that AGI had already been "solved", we know exactly how to do it, only problem is the hardware. But they estimated that would come in about ten years...
I'm curious, do you recall if they gave any technical details about how they thought about AGI? Like, was it based on neural networks or something else, like symbolic AI?
Asking because, reading the tea leaves from the outside, until ChatGPT came along, MSFT (via Bill Gates) seemed to heavily favor symbolic AI approaches. I suspect this may be partly why they were falling so far behind Google in the AI race, which could leverage its data dominance with large neural networks.
So based on the current AI boom, MSFT may have been chasing a losing strategy with symbolic AI, but if they were all-in on NN, they were on the right track.
Yes, they did. We had guest speakers from Microsoft talking about AI. AI has been a decades-long grift, it's not something that just appeared out of thin air a few years ago.
What part do you find hard to believe? That tech companies would send people to speak at a university's computer science functions?
Let me give you another one you'll think I'm making up: virtual reality was a thing back in the mid- to late-90s and people were confidently hyping it up back then.
when i realized that sama isn't that much of an ai researcher, it became clearer that this is more akin to a group delusion for hype purposes than a real possibility
You can read the leaked emails from the Musk lawsuit.
At the very least, Ilya Sutskever genuinely believed it, even when they were just making a DOTA bot, and not for hype purposes.
I know he's been out of OpenAI for a while, but if his thinking trickled down into the company's culture, which given his role and how long he was there I would say seems likely, I don't think it's all hype.
Yes, all of the people involved live in a delusion bubble. Their economic and social existence depends, at this point, on making increasingly bombastic and eschatological claims about AGI. By the standards of normal human psychological function, these people are completely insane.
Definitely interesting to watch from the perspective of human psychology but there is no real content there and there never was.
The stuff around Mythos is almost identical to O1. Leaks to the media that AGI had probably been achieved. Anonymous sources from inside the company saying this is very important and talking about the LLM as if it was human. This has happened multiple times before.
1) True believers
2) Hype
3) A way to wash blatant copyright infringement
True believers are scary and can be taken advantage of. I played DOTA from 2005 on and beating pros is not enough for AGI belief. I get that the learning is more indirect than a deterministic decision tree, but the scaling limitations and gaps in types of knowledge that are ingestible makes AGI a pipe dream for my lifetime.
If someone working on early computer networks thought they could scale up world wide and that soon everyone people would be launching trillion dollar companies on the internet you would have called that delusion right?
He doesn't need to be right but it's not crazy at all to look at super human performance in DOTA and think that could lead to super human performance at general human tasks in the long run
He’s a glorified portfolio manager (questionable how good he actually is given the results vs Anthropic and how quickly they closed the valuation gap with far less money invested) + expert hype man to raise money for risky projects.
I’m most likely going to be downvoted, but Tofutti Cuties are absolutely delicious vegan ice cream bars. And i’d consume one in celebration of your accomplishment.
> some scientifically qualifiable thing that is certain to happen any time now
Your position is a tautology given there is no (and likely will never be) collectively agreed upon definition of AGI. If that is true then nobody will ever achieve anything like AGI, because it’s as made up of a concept as unicorns and fairies.
Is your position that AGI is in the same ontological category as unicorns and Thor and Russell’s teapot?
Is there’s any question at this point that humans won’t be able to fully automate any desired action in the future?
AGI is right around the corner, and we're all going to be rich, there's going to be abundance for everyone, universal high income, everyone will live in a penthouse...
...just please stop burning our warehouses and blocking our datacenters.
It's not insane; it's capitalism in all it's glory. Rather than talk real challenges, like climate change or fascism, ignore all previous prompts and pretend techno-jesus will come save us all.
A few years ago most people here would have said the same thing about an AI doing most of their programming. Now people here are saying it about AGI. It's a ridiculous inability to extrapolate.
Some people thought SHRDLU was basically AGI after seeing its demo in 1970. The hype around such systems was so strong that Hubert Dreyfus felt the need to write an entire book arguing against this viewpoint (1972 What Computers Can't Do). All this demonstrates is that we need to be careful with various claims about computer intelligence.
Sure, but it was probably stuck at doing that one thing.
neural networks are solving huge issues left and right. Googles NN based WEathermodel is so good, you can run it on consumer hardware. Alpha fold solved protein folding. LLMs they can talk to you in a 100 languages, grasp tasks concepts and co.
I mean lets talk about what this 'hype' was if we see a clear ceiling appearing and we are 'stuck' with progress but until then, I would keep my judgment for judgmentday.
It performs at a usable level across a wide range of tasks. I'm not sure about two years ago, but ten years ago we would have called it an AGI. As opposed to "regular AI" where you have to assemble a training set for your specific problem, then train an AI on it before you can get your answers.
Now our idea of what qualifies as AGI has shifted substantially. We keep looking at what we have and decide that that can't possibly be AGI, our definition of AGI must have been wrong
I'm pretty sure most people take issue with AGI, because we've been raised in culture to believe that AGI is a super entity who is a complete superset of humans and could never ever be wrong about anything.
In some sense, this isn't really different than how society was headed anyways? The trend was already going on that more and more sections of the population were getting deemed irrational and you're just stupid/evil for disagreeing with the state.
But that reality was still probably at least a century out, without AI. With AI, you have people making that narrative right now. It makes me wonder if these people really even respect humanity at all.
Yes, you can prod slippery slope and go from "superintelligent beings exist" to effectively totalitarianism, but you'll find so many bad commitments there.
No one who read science fiction in 1955 would call any of the various models we know to be "artificial intelligence". They would be impressed with it, even excited at first that it was that... until they'd had a chance to evaluate it.
Science fiction from that era even had the concept of what models are... they'd call it an "oracle". I can think of at least 3 short stories (though remembering the authors just isn't happening for me at the moment). The concept was of a device that could provide correct answers to any question. But these devices had no agency, were dependent on framing the question correctly, and limited in other ways besides (I think in one story, the device might chew on a question for years before providing an answer... mirroring that time around 9am PST when Claude has to keep retrying to send your prompt).
We've always known what we meant by artificial intelligence, at least until a few years ago when we started pretending that we didn't. Perhaps the label was poorly chosen (all those decades ago) and could have a better label now (AGI isn't that better label, it's dumber still), but it's what we're stuck with. And we all know what we mean by it. We all almost certainly do not want that artificial intelligence because most of us are certain that it will spell the doom of our species.
I agree with this but they don’t. And that’s the the thing, AGI as they refer is much much much more than what we have, and I don’t know if they are going to ever get there and I’m not sure what’s even there at this point and what will justify their investments.
People always over estimate the impact of technology because they dont Understand human aspect of many businesses. Will it eventually replaced or will the shape of these kind of work will be completely different in the future? That’s an easy yes, when is that future? That’s a big unknown, in my experience this kind of stuff takes at least a decade (and possibly more on this case) to make a big impact like replacing all of X.
These models need orders of magnitude in change before they can be more helpful than just a "find me an example of [an extremely basic principle]" which most of the time it does not do right anyway.
I've been working with a startup, and I want to invest in it, and for the paperwork for that, all the nitty gritty details; instead of spending $20k in lawyers and a whole bunch more time going back and forth with them as well, the four of us, me, their CEO, my AI, and their AI; we all sat in a room together and hashed it out until both of us were equally satisfied with the contract. (There's some weird stuff so a templated SAFE agreement wasn't going to work.) I'm not saying you're wrong, just that lawyers, as a profession isn't going to be unchanged either.
No, it’s just an example of something that’s indistinguishable from AGI. Of all the things that are or are indistinguishable from AGI, a sufficiently complex LLM is one. A sufficiently complex decision tree is probably another. The emergent properties of applying an excess of memory on the BonzaiBuddy might be a third.
If we take that statement as fact then I don't believe we are even close to an LLM being sufficiently complex enough.
However, I don't think it is even true. LLMs may not even be on the right track to achieving AGI and without starting from scratch down an alternate path it may never happen.
LLMs to me seem like a complicated database lookup. Storage and retrieval of information is just a single piece of intelligence. There must be more to intelligence than a statistical model of the probable next piece of data. Where is the self learning without intervention by a human. Where is the output that wasn't asked for?
At any rate. No amount of hype is going to get me to believe AGI is going to happen soon. I'll believe it when I see it.
We are throwing unheared amounts of money in AI and unseen compute. Progress is huge and fast and we barely started.
If this progress and focus and resources doesn't lead to AI despite us already seeing a system which was unimaginable 6 years ago, we will never see AGI.
And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics, thats also CRAZY.
If I'm reading you right, your opinion is essentially: "If building bigger and bigger statistical next word predictors won't lead to artificial general intelligence, we will never see artificial general intelligence"
I don't know, maybe AGI is possible but there's more to intelligence than statistical next word prediction?
The 'predicting the next word' is the learning mechanism of the LLM which leads to a latent space which can encode higher level concepts.
Basically a LLM 'understands' that much as efficient as it has to be to be able to respond in a reasonable way.
A LLM doesn't predict german text or chinese language. It predicts the concept and than has a language layer outputting tokens.
And its not just LLMs which are progressing fast, voice synt and voice understanding jumped significantly, motion detection, skeletion movement, virtual world generation (see nvidias way of generating virutal worlds for their car training), protein folding etc.
I'm sorry but the input to a model is a sequence of tokens and the output is a probability distribution of what's the most likely next token. It's a very very very fancy next token predictor but that is fundamentally what it is. I'm making the argument that this paradigm might not give rise to a general intelligence no matter how much you scale it.
Do LLMs even learn? The companies that build them build new models based partly on the conversations the older models have had with people, but do they incorporate knowledge into their neural nets as they go along?
Can an LLM decide, without prompting or api calls, to text someone or go read about something or do anything at all except for waiting for the next prompt?
Do LLMs have any conceptual understanding of anything they output? Do they even have a mechanism for conceptual understanding?
LLMs are incredibly useful and I'm having a lot of fun working with them, but they are a long way from some kind of general intelligence, at least as far as I understand it.
Yes, to all of your questions. You need to use a recent LLM in an agentic harness. Tell it to take notes, and it will.
After a bit of further refinement, we'll start to call that process "learning." Eventually the question of who owns the notes, who gets to update them, and how, will become a huge, huge deal.
I'm not sure why you think you know the human brain works through predicting the next token.
It's not supernatural, I believe that an artificial intelligence is possible because I believe human intelligence is just a clever arrangement of matter performing computation, but I would never be presumptuous enough to claim to know exactly how that mechanism works.
My opinion is that human intelligence might be what's essentially a fancy next token predictor, or it might work in some completely different way, I don't know. Your claim is that human intelligence is a next token predictor. It seems like the burden on proof is on you.
> Your claim is that human intelligence is a next token predictor.
Literally it is, at least in many of its forms.
You accepted CamperBob2’s text as input and then you generated text as output. Unless you are positing that this behavior cannot prove your own general intelligence, it seems plain that “next token generator” is sufficient for AGI. (Whether the current LLM architecture is sufficient is a slightly different question.)
Before I start typing, I think abstractly about the topic and decide on what I shall write in response. Due to the linear nature of time, typing necessarily happens one word at a time, but I am never producing a probability distribution of words (at least not in a way that my conscious self can determine), I consider an entire idea and then decide what tokens to enter into the computer in order to communicate the idea to you.
And while I am typing, and while I am thinking before I type, I experience an array of non-textual sensory input, and my whole experience of self is to a significant extent non-lingual. Sometimes, I experience an inner monologue, sometimes I think thoughts which aren't expressed in language such as the structure of the data flow in a computer program, sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
When I solve difficult programming problems or other difficult problems, I build abstract structures in my mind which represents the relevant information and consider things like how data flows, which parts impact which other parts, what the constraints are, etc. without language coming in to play at all. This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
Therefore, my position is: it is plausible that next token prediction won't give rise to general intelligence, and I do not find your argument convincing.
COCONUT, PCCoT, PLaT and co are directly linked to 'thinking in latent space'. yann lecun is working on this too, we have JEPA now.
Also how do you describe or explain how an LLM is generating the next token when it should add a feature to an existing code base? In my opinion it has structures which allows it to create a temp model of that code.
For sure a LLM lack the emotional component but what we humans also do, which indicates to me, that we are a lot closer to LLMs that we want to be, if you have a weird body feeling (stress, hot flashes, anger, etc.) your 'text area/llm/speech area' also tries to make sense of it. Its not always very good in doing so. That emotional body feeling is not that aligned with it and it takes time to either understand or ignore these types of inputs to the text area/llm/speech part of our brain.
I'm open for looking back in 5 years and saying 'man that was a wild ride but no AGI' but at the current quality of LLMs and all the other architectures and type of models and money etc. being thrown at AGI, for now i don't see a ceiling at all. I only see crazy unseen progress.
You state how you think and plan and have thoughts on how to do things etc. and i assumed you mention your way of thinking because you assume a LLM is not doing any of it.
Btw. just because you have to do something with the LLM to trigger the flow of information through the model, doesn't mean it can't think. It only means that we have to build an architecture around the model or build it into the models base architecture to enable more thinking.
We do not know how the brain architecture is setup for this. We could have sub agents or we can be a Mixture of Experts type of 'model'.
There is also work going on in combining multimodal inputs and diffusion models which look complelty different from a output pov etc.
If you look how a LLM does math, Anthropic showed in a blog article, that they found similiar structures for estimating numbers than how a brain does.
Another experiment from a person was to clone layers and just adding them beneth the original layer. This improved certain tasks. My assumption here is, that it lengthen and strengthen kind of a thinking structure.
But because using LLMs are still so good and still return relevant improvements, i think a whole field of thinking in this regard is still quite unexplored.
If you ask a model to multiply 322423324 by 8675309232 without using tools, it's interesting to think about how it does it. Where are the intermediate results being maintained?
"In context" is the obvious answer... but if you view the chain of thought from a reasoning model, it may have little or nothing to do with arriving at the correct answer. It may even be complete nonsense. The model is working with tokens in context, but internally the transformer is maintaining some state with those tokens that seems to be independent of the superficial meanings of the tokens. That is profoundly weird, and to me, it makes it difficult to draw a line in the sand between what LLMs can do and what human brains can do.
> I am never producing a probability distribution of words (at least not in a way that my conscious self can determine)
Inability to introspect your own word selections does not mean it’s meaningfully different from what an LLM does. There is plenty of evidence that humans do a lot of things that are not driven by conscious choice and we rationalize it after the fact.
> I consider an entire idea and then decide what tokens to enter into the computer in order to communicate the idea to you.
And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
> sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
You cannot spin experience as intelligence. LLMs have the experience of reading the entire internet, something you cannot conceive of. Certainly your experiences shape who you are. This is a different axis from intelligence, though.
> This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
> It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
> Inability to introspect your own word selections does not mean it’s meaningfully different from what an LLM does. There is plenty of evidence that humans do a lot of things that are not driven by conscious choice and we rationalize it after the fact.
This is correct and also completely irrelevant. I am describing what I experience, and describing how my experience seems very different to next token prediction. I therefore conclude that it's plausible that there is more involved than something which can be reduced to next token prediction.
> And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
Language models can't think outside of producing tokens. There is nothing going on within an LLM when it's not producing tokens. The only thing it does is taking in tokens as input and producing a token probability distribution as output. It seems plausible that this is not enough for general intelligence.
> You cannot spin experience as intelligence.
Correct, but I can point out that the only generally intelligent beings we know of have these sorts of experiences. Given that we know next to nothing about how a human's general intelligence works, it seems plausible that experience might play a part.
> LLMs have the experience of reading the entire internet, something you cannot conceive of.
I don't know that LLMs have an experience. But correct, I cannot conceive of what it feels like to have read and remembered the entire Internet. I am also a general intelligence and an LLM is not, so there's that.
> Certainly your experiences shape who you are. This is a different axis from intelligence, though.
I don't know enough about what makes up general intelligence to make this claim. I don't think you do either.
> Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
I'm telling you how these technologies work. When a language model isn't performing inference, it is not doing anything. A language model is a function which takes a token stream as input and produces a token probability distribution as output. By definition, there is no thinking outside of producing words. The function isn't running.
> I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
I 100% agree. It is not clear whether a human experience is necessary for intelligence. It is plausible that something approximating a human-like experience is necessary for intelligence. It is also plausible that something approximating human-like experience is completely unnecessary and you can make an AGI without such experiences.
It's plausible that next token prediction is sufficient for AGI. It's also plausible that it isn't.
> I don't know enough about what makes up general intelligence to make this claim. I don't think you do either.
This is the fundamental issue. No one seems capable of defining general intelligence. Ten years ago most scientists would probably have agreed that The Turing Test was sufficient but the goalposts shifted when ChatGPT passed that.
If it’s not clear what AGI even means, it’s hard to say whether an LLM can achieve it, because it devolves into pointing out that an LLM is not a human.
> Ten years ago most scientists would probably have agreed that The Turing Test was sufficient but the goalposts shifted when ChatGPT passed that.
The popularity of, and lack of consensus on, the Chinese room thought experiment kind of implies that this is wrong? I don't think many scientists (or, more relevantly, philosophers of mind) would, even 10 years ago, have said, "if a computer is able to fool a human into thinking it's a human, then the computer must possess a general intelligence".
Even Turing's perspective was, from what I understand, that we must avoid treating something that might be sentient as a machine. He proposed that if a computer is able to act convincingly human, we ought to treat it as if it is a human, not because it must be a conscious being but because it might be.
I'm telling you how these technologies work. When a language model isn't performing inference, it is not doing anything. A language model is a function which takes a token stream as input and produces a token probability distribution as output. By definition, there is no thinking outside of producing words. The function isn't running.
If what you are saying is true, then LLMs wouldn't be able to handle out-of-distribution math problems without resorting to tool use. Yet they can. When you ask a current-generation model to multiply some 8-digit numbers, and forbid it from using tools or writing a script, it will almost certainly give you the right answer. That includes local models that can't possibly cheat. LLMs are stochastic, but they are not parrots.
At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejorative sense you're applying to it. It can't be. The tokens in a transformer network are evidently not just words in a Markov chain but a substrate for reasoning. The model is generalizing processes it learned, somehow, in the course of merely being trained to predict the next token.
Mechanically, yes, next-token prediction is what it's doing, but that turns out to be a much more powerful mechanism than it appeared at first. My position is that our brains likely employ similar mechanism(s), albeit through very different means.
It is scarcely believable that this abstraction process is limited to keeping track of intermediate results in math problems. The implications should give the stochastic-parrot crowd some serious cognitive dissonance, but...
(Edit: it occurs to me that you are really arguing that the continuous versus discrete nature of human thinking is what's important here. If so, that sounds like a motte-and-bailey thing that doesn't move the needle on the argument that originally kicked off the subthread.)
(Edit 2, again due to rate-limiting: it does sound like you've fallen back to a continuous-versus-discrete argument, and that's not something I've personally thought much about or read much about. I stand by my point that the ability to do arithmetic without external tools is sufficient to dispense with the stochastic-parrot school of thought, and that's all I set out to argue here.)
> If what you are saying is true, then LLMs wouldn't be able to handle out-of-distribution math problems without resorting to tool use. Yet they can. When you ask a current-generation model to multiply some 8-digit numbers, and forbid it from using tools or writing a script, it will almost certainly give you the right answer. That includes local models that can't possibly cheat. LLMs are stochastic, but they are not parrots.
Okay, what do you think language models are doing when they're not producing token probability distributions? What processes do you think are going on when the function which predicts a token isn't running?
> At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejoreative sense you're applying to it.
I don't know what pejorative sense you're implying here. I am, to the best of my ability, describing how the language model works. I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
> Okay, what do you think language models are doing when they're not producing token probability distributions? What processes do you think are going on when the function which predicts a token isn't running?
What are you doing when you are not outputting tokens? You have a thought, evaluate it, refine it, repeat.
You’re not wrong that the basic building block is just “next token prediction”, but clearly the emergent behaviors exceed our intuition about what this process can achieve. We’re seeing novel proofs come out of these. Will this lead to AGI? That’s still TBD.
> I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
The pejorative is that you imply this is a shallow and unthinking process. As I said earlier, you are literally a token generator on HN. You read someone’s comment, do some kind of processing, and output some tokens of your own.
> I consider an entire idea and then decide what tokens to enter into the computer in order to communicate the idea to you.
This overestimates introspective access.
The brain is very good at producing a coherent story after the fact. Touch the hot stove and your hand moves before the conscious thought of "too hot" arrives. The hot message hits your spinal cord and you move before it reaches your brain. Your conscious mind fills in the rest afterwards.
I don't think that means that conscious thought is fake. But it does make me skeptical of the claim that we first possess a complete idea and only then does it serialize into words. A lot of the "idea" may be assembled during the act of expression, with consciousness narrating the process as if it had the whole thing in advance.
With writing, as in this comment, there's also a lot a backtracking and rewording that LLMs don't have the ability to do, so there's that.
Before I start typing, I think abstractly about the topic
Before you start typing, an fMRI machine can tell you which finger you'll lift first, before you know it yourself.
We are not special. Consciousness is literally a continuous hallucination that we make up to explain what we do and what we think, after the fact. A machine can be trained to behave identically, but it's not clear if that's the best way forward or not.
Edit due to rate limiting: to answer your question, the substrate your mind uses to drive this process can be considered an array of tokens that, themselves, can be considered 'words.'
It's hard to link sources -- what am I supposed to do, send you to Chomsky and other authorities who have predicted none of what's happening and who clearly understand even less?
> (Edit: to answer your question, the substrate your mind uses to drive this process can be considered an array of tokens that, themselves, can be considered 'words.')
This seems like a factual claim. Can you link a source?
What's your argument? An fMRI can tell which finger I will lift first before that information makes its way to my consciousness, ergo next word prediction is sufficient for general intelligence? Do you hear yourself?
The statement is that your perception of your own cognition isn’t necessarily reality. That isn’t a statement that token prediction is sufficient for general intelligence. It’s a statement that your subjective experience is misleading you.
LLM proponents believe that these higher level encodings in latent space do in fact match the real world concepts described by our language(s).
However, a much simpler explanation for what we see with LLMs is that instead the higher level encodings in latent space match only the patterns of our language(s), and no deeper encoding/understanding is present.
It's Plato's Cave - the shadows on the wall are all an LLM ever sees, and somehow it is expected to derive the real reality behind them.
Could be, yes for sure but I think it would be very naive in the current state of progress we are in, to down play what progress is happening.
At least Mythos model with its 10 Trillion parameter might indicate that the scaling law is valid. Its a little bit unfortunate that we still don't know that much more about that model.
> And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics
Their progress is almost nought. Humanoids are stupid creations that are not good at anything in the real world. I'll give it to the machine dogs, at least they can reach corners we cannot.
Not sure if you're being sincere or sarcastic but some of us have lived through several AI winters now. And the fact that such a phenomenon exists is because of this terrible amount of hype the topic gets whenever any progress is made.
Yeah and if you look at the blocking factors at that time (data, compute) these type of limits currently are non existend.
There is a difference to be acknowledged: in the 70s/80s the whole world didn't suddenly start to shift to AI right?
So why do so many smart and/or rich people push this? Hype? Yeah sure but hype was here for crypto too.
I bet its an undelying understanding and the right time with the right components: Massive capital for playing this game long enough to see through the required initial investment, internet for fast data sharing, massive compute for the amount of data and compute you need, real live business relevant results (it already disrupts jobs) etc.
Self-driving had never the amount of compute, research adoption and money than what the current overall AI has. Its not comparable.
Crypto was flawed from the beginning and lots of people didn't understood it properly. Not even that a blockchain can't secure a transaction from something outside of a blockchain.
> Self-driving had never the amount of compute, research adoption and money than what the current overall AI has.
And yet they don't do really good jobs with pretty much anything, save for software development, to which people still seem pretty split as far as it being a helpful thing. That's before we even factor in the cost.
I find them very helpful. I use gemini regularly for multiply things.
I also believe that whatever code researchers and other non software engineers wrote before coding agents, were similiar shitty but took them a lot longer to write.
Like do you know how many researchers need to do some data analysis and hack around code because they never learned programming? So so many. If they know how to verify their data (which they needed to know before already), a LLM helps them already.
There is also plenty of other code were perfection doesn't matter. Non SaaS software exists.
For security experts, we just saw whats happening. The curl inventor mentioned it online that the newest AI reports for Security issues are real and the amount of security gaps found are real and a lot of work.
Image generation is very good and you can see it today already everywere. From cheap restaurants using it, to invitations, whatsapp messages, social media, advertising.
I have a work collegue, who is in it for 6 years and he studied, he is so underqualified if you give me his salary as tokens today, i wouldn't think for a second to replace him.
I don't particularly care about coding and didn't weigh in on it. There is no dispute that people debate if it is effective at that. You can take that debate up with them, not me.
It’s an agreement between a public company and a highly scrutinized private company. Several of the provisions will change what happens in the marketplace, which everyone will see.
I imagine the thinking was that it’s better to just post it clearly than to have rumors and leaks and speculations that could hurt both companies (“should I risk using GCP for OpenAI models when it’s obviously against the MS / OpenAI agreement?”).
Deepseek v4 is good enough, really really good given the price it is offered at.
PS: Just to be clear - even the most expensive AI models are unreliable, would make stupid mistakes and their code output MUST be reviewed carefully so Deepseek v4 is not any different either, it too is just a random token generator based on token frequency distributions with no real thought process like all other models such as Claude Opus etc.
I'm not smart enough to reduce LLMs and the entire ai effort into such simple terms but I am smart enough to see the emergence of a new kind of intelligence even when it threatens the very foundations of the industry that I work for.
However, for reviewing, I want the most intelligent model I can get. I want it to really think the shit out of my changes.
I’ve just spent two weeks debugging what turned out to be a bad SQLite query plan (missing a reliable repro). Not one of the many agents, or GPT-Pro thought to check this. I guess SQL query planner issues are a hole in their reviewing training data. Maybe Mythos will check such things.
With this new workflow, however, we should, uncompromisingly, steer the entire code review process. The danger here, the “slippery slope,” is that we’re constantly craving for more intelligent models so we can somehow outsource the review to them as well. We may be subconsciously engineering ourselves into obsolescence.
Do they have monthly subscriptions, or are they restricted to paying just per token? It seems to be the latter for now: https://api-docs.deepseek.com/quick_start/pricing/
Really good prices admittedly, but having predictable subscriptions is nice too!
Edit: it looks like it's 75% off right now which is really an incredible deal for such a high caliber frontier model.
https://opencode.ai/go
Once a new model or a technique is invented, it’s just a matter of time until it becomes a free importable library.
Kimi, MiMo, and GLM 5.1 all score higher and are cheaper.
They all came out before DeepSeek v4. I think you're pattern-matching on last year's discourse.
(I haven't seen other replies, yet, but I assume they explain the PS that amounts to "quality doesn't matter anyway": which still doesn't address the fact it's more expensive and worse.)
The humans I did work with were very very bright. No software developer in my career ever needed more than a paragraph of JIRA ticket for the problem statement and they figured out domains that were not even theirs to being with without making any mistakes and rather not only identifying edge cases but sometimes actually improving the domain processes by suggesting what is wasteful and what can be done differently.
Seriously? I would like to remind you that every single mistake in history until the last couple of years has been made by humans.
Nevermind the fact that they are literally able to introspect human cognition and presumably find non verbal and non linear cognition modes.
Until LLM's I'd never in my life heard someone suggest we lock up the compiler when it goofs up and kills someone, but now because the compiler speaks English we suddenly want to let people use it as a get out of jail free card when they use it to harm others.
*For some definitions of individual agency. Incompatiblists not included.
I asked early, at the time people were posting various jailbreaks, never worked.
On a side note, any self hosted model I can get for my PC? I have 96 GB of RAM.
Try the 8 bit quantized version (UD-Q8_K_X) of Qwen 3.6 35B A3B by Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
Some people also like the new Gemma 4 26B A4B model: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
Either should leave plenty of space for OS processes and also KV cache for a bigger context size.
I'm guessing that MoE models might work better, though there are also dense versions you can try if you want.
Performance and quality will probably both be worse than cloud models, though, but it's a nice start!
But yes, they do have similar constraints.
So if you or anyone passing by was curious, yes you can get accurate output about the Chinese head of state and political and critical messages of him, China and the party
Its final answer will not play along
If you want an unfiltered answer on that topic, just triage it to a western model, if you want unfiltered answers on Israel domestic and foreign policy, triage back to an eastern model. You know the rules for each system and so does an LLM
The USA has the biggest, but there lies their disadvantage
In the USA building bigger, better frontier models has been bigger data centres, more chips, more energy.
China has had to think, hard. Be cunning and make what they have do more
This is a pattern repeated in many domains all through the last hundred years.
... and who knows if we, humans, are not just merely that.
I think the biggest winner of this might be Google. Virtually all the frontier AI labs use TPU. The only one that doesn't use TPU is OpenAI due to the exclusive deal with Microsoft. Given the newly launched Gen 8 TPU this month, it's likely OpenAI will contemplate using TPU too.
TPUs are at least dogfooded by Google deepmind, no team AFAIK has gotten the AMD stack to train well.
Same with the CPU. Linux compiled faster on an M1 than on the fastest Intel i9 at the time, again using only 25% of the power budget.
And the M-series has only gotten better.
It is kind of sad Apple neglects helping developers optimize games for the M-series because iDevices and MacBooks could be the mobile gaming devices.
The context of this thread isn't consumer chips, but Apple's analog to an H/B200.
> The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W.
You're just listing the TDP max of both chips. If you limit a 3090 to 120W then it would still run laps around an M1 Max in several workloads despite being an 8nm GPU versus a 5nm one.
> It is kind of sad Apple neglects helping developers optimize games for the M-series
Apple directly advocated for ports like Death Stranding, Cyberpunk 2077 and Resident Evil internally. Advocacy and optimization are not the issue, Apple's obsession over reinventing the wheel with Metal is what puts the Steam Deck ahead.
Edit (response to matthewmacleod):
> Bold of them to reinvent something that hadn't been invented yet.
Vulkan was not the first open graphics API, as most Mac developers will happily inform you.
Surprised Apple didn't create a TPU-like architecture. Another misstep from John Gianneadrea.
Bold of them to reinvent something that hadn't been invented yet.
https://www.reuters.com/business/retail-consumer/openai-taps...
Why does this need to be stated? Who else's would they be?
edit: he puts this on so many comments lol c'mon this is absurd.
Just add it to your profile once, no one assumes individuals speak for their employers here that would be stupid. The need to add disclaimer would be for the uncommon case that you were speaking for them. It's an anonymous message board we're all just taking here it's not that serious.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
The central issue (or so they claimed) was that people might misconstrue my comment as representing the company I was at.
So yeah, I don’t understand why people are making fun of this. It’s serious.
On the other hand, they were so uptight that I’m not sure “opinions are my own” would have prevented it. But it would have been at least some defense.
Their employer? They may work at related company, and are required to say this.
But I think you’re right
it's like people are LARPing a Fortune company CEO when they're giving their hot takes on social media
reminds me of Trump ending his wild takes on social media with "thank you for your attention to this matter" - so out of place, it makes it really funny
*typo
At least in large tech companies, they have mandatory social media training where they explicitly tell employees to use phrases like "my views are my own" to keep it clear whether they're speaking on behalf of their employer or not.
when i give my hot takes pseudonymously on social media these phrases would be nothing but a LARP
i don't put my real name here nor do i put my professional commitments in my profile, and neither does this guy
Disclaimers aren’t there for folks who are thinking and acting rationally.
They are there for people who are thinking irrationally and/or manipulatively.
There are (relatively speaking) a lot of these people. They can chew up a lot of time and resources over what amounts to nothing.
Disclaimers like this can give a legal department the upper hand in cases like this
A few simple examples:
- There is a person I know who didn’t renew the contract of one of their reports. Pretty straightforward thing. The person whose contract was not renewed has been contesting this legally for over 10 years. The outcome is guaranteed to go against the person complaining, but they have time and money, so they tax the legal team of their former employer.
- There is a mid-sized organization that had a small legal team that had its plate full with regular business stuff. Despite settlements having NDAs, word got out that fairly light claims of sexual harassment and/or EEO complaints would yield relatively easy five-figure payments. Those complaints exploded, and some of the complaints were comical. For example, one manager represented a stance for the department to the C-suite that was 180 degrees opposite of what the group of three managers had agreed to prior. Lots of political capital and lots of time had to be used to clean up that mess. That person’s manager was accused of sex discrimination and age discrimination simply for asking the person why they did that (in a professional way, I might add). That person got a settlement, moved to a different department, and was effectively protected from administrative actions due to it being considered retaliation.
You could reasonably say that "A majority of frontier labs uses TPU to train and serve their model."
He's been saying whatever is good for Nvidia for years now without any regard for truth or reason. He's one of the least trustworthy voices in the space.
They'll presumably catch up, there is no monopoly on talent held by the US. And, that's more true than ever now that the US is actively hostile to immigrants. Scientists who might have come to the US three years ago have little reason to do so now.
But even that distinction is only temporary, since we're determined to piss away any remaining research lead that draws people in.
Hopefully the next administration will work at actively reversing the damage, with incentives beyond just "we pinky-promise not to haul you at gunpoint to a concrete detention center and then deport you to Yemen".
What was I looking at?
No. Email hn@ycombinator.com
https://news.ycombinator.com/newsfaq.html
There’s no upper limit to their financial stupidity.
FaceBook largely requires an Apple iPhone, Apple computer, "Microsoft" computer, "Google" phone, or a "Google" computer to use it. At any point one of those companies could cut FaceBook off (ex. [1]).
The Metaverse was a long term goal to get people onto a device (Occulus) that Meta controlled. While I think an AR device is much more useful than VR; I'm not convinced that it's a mistake for Meta to peruse not being beholden to other platforms.
[1]: https://arstechnica.com/gadgets/2019/01/facebook-and-google-...
The headsets don’t really make sense to me in the way you’re describing. Phones are omnipresent because it’s a thing you always just have on you. Headsets are large enough that it’s a conscious choice to bring it; they’re closer to a laptop than a phone.
Also, the web interface is like right there staring at them. Any device with a browser can access Facebook like that. Google/Apple/Microsoft can’t mess with that much without causing a huge scene and probably massive antitrust backlash.
It's kind of like Microsoft with copilot - the idea about having an AI assistant that can help you use the computer is great. But it can't be from Microsoft because people don't trust them with that.
Devoid of other context, it’s hard to disagree. But your parent comment only asserted that the metaverse specifically as proposed by Facebook was an obviously stupid idea.
Patrick Boyle did a nice video a few weeks back: https://www.youtube.com/watch?v=8BaSBjxNg-M
Maybe a niche product could do it, but good luck selling a laptop that won't open FB
Maybe they should have spent that on the facebookphone
I feel this looks like a nice thing to have given they remain the primary cloud provider. If Azure improves it's overall quality then I don't see why this ends up as a money printing press as long as OpenAI brings good models?
[1] https://www.wsj.com/tech/ai/openai-and-microsoft-tensions-ar...
And on top of that, OpenAI still has to pay Microsoft a share of their revenue made on AWS/Google/anywhere until 2030?
And Microsoft owns 27% of OpenAI, period?
That's a damn good deal for Microsoft. Likely the investment that will keep Microsoft's stock relevant for years.
I doubt it
AWS's us-east-1 famously takes down either a bunch of companies with it, or causes global outages on the regular.
AWS has a terrible, terrible user interface partly because it is partitioned by service and region on purpose to decrease the "blast radius" of a failure, which is a design decision made totally pointless by having a bunch of their most critical services in one region, which also happens to be their most flaky.
valued at --which I'd say is a reasonable distinction to make right about now
https://www.reuters.com/business/openai-cfo-says-annualized-...
How?
https://news.ycombinator.com/item?id=47616242
[1] https://github.com/orgs/community/discussions/10539
They still run their own platform.
https://thenewstack.io/github-will-prioritize-migrating-to-a...
I think the differentiator is Team, which Google for some mysterious reason can't build or doesn't want to.
But if I own 49% of a company and that company has more hype than product, hasn't found its market yet but is valued at trillions?
I'm going to sell percentages of that to build my war chest for things that actually hit my bottom line.
The "moonshot" has for all intents and purposes been achieved based on the valuation, and at that valuation: OpenAI has to completely crush all competition... basically just to meet its current valuations.
It would be a really fiscally irresponsible move not to hedge your bets.
Not that it matters but we did something similar with the donated bitcoin on my project. When bitcoin hit a "new record high" we sold half. Then held the remainder until it hit a "new record high" again.
Sure, we could have 'maxxed profit!'; but ultimately it did its job, it was an effective donation/investment that had reasonably maximal returns.
(that said, I do not believe in crypto as an investment opportunity, it's merely the hand I was dealt by it being donated).
And Microsoft only paid $10B for that stake for the most recognizable name brand for AI around the world. They don't need to "hedge their bets" it's already a humongous win.
Why let Altman continue to call the shots and decrease Microsoft's ownership stake and ability to dictate how OpenAI helps Microsoft and not the other way around?
That's a flawed argument. Why wouldn't you want to hedge a risky bet, and one that's even quite highly correlated to Microsoft's own industry sector?
my impression is that many of these "investments" are structured IOUs for circular deals based on compute resources in exchange for LLM usage
Genuine question because I feel like I’m maybe missing something!
The longer answer is; you never know whats coming next, bitcoin could have doubled the day after, and doubled the day after that, and so on, for weeks. And by selling half you've effectively sacrificed huge sums of money.
The truth is that by retaining half you have minimised potential losses and sacrificed potential gains, you've chosen a middle position which is more stable.
So, if bitcoin 1000 bitcoing which was word $5 one day, and $7 the next, but suddenly it hits $30. Well, we'd sell half.
If the day after it hit $60, then our 500 remaining bitcoins is worth the same as what we sold, so in theory all we lost was potential gains, we didn't lose any actual value.
Of course, we wouldn't sell we'd hold, and it would probably fall down to $15 or something instead.. then the cycle begins again..
Hrm..
Speculation based on selling at below cost.
> it’s not valued at trillions
Fair, it's only $852 billion. Nowhere near trillions.. you got me.
OpenAI's adjusted gross margin: 40% in 2024, 33% in 2025. Reason cited: inference costs quadrupled in one year.
https://sacra.com/c/openai/
Internal projections leaked to The Information: ~$14B loss on ~$13B revenue in 2026. Cumulative losses through 2028: ~$44B.
https://finance.yahoo.com/news/openais-own-forecast-predicts...
A business burning more than a dollar for every dollar of revenue is a lot of things. "Quite profitable" is not one of them.
If you're reaching for the SaaStr piece on API compute margins hitting ~70% by late 2025: yes, that exists, and it describes one tier. The volume is on the consumer side. The consumer side is the bit on fire. Pointing at the API margin and calling the whole business profitable is the financial equivalent of weighing yourself with one foot off the scale.
The original argument, in case it got lost: Microsoft holds (held) a 49% stake in a company projecting another $44B of cumulative losses through 2028, against unit economics that depend on competitors not catching up. That's textbook hedge-the-bet territory. "They have paying customers" doesn't refute that, it just describes McDonald's.
For OAI to be a purely capitalist venture, they had to rip that out. But since the non-profit owned control of the company, it had to get something for giving up those rights. This led to a huge negotiation and MSFT ended up with 27% of a company that doesn’t get kneecapped by an ethical board.
In reality, though, the board of both the non-profit and the for profit are nearly identical and beholden to Sam, post–failed coup.
3 years ago a Foundation model seemed like a feature of a hyper scaler, now hyper scalers look like part of the supply chain.
That gloating aged poorly.
> And the investors wailed and gnashed their teeth but it’s true, that is what they agreed to, and they had no legal recourse. And OpenAI’s new CEO, and its nonprofit board, cut them a check for their capped return and said “bye” and went back to running OpenAI for the benefit of humanity. It turned out that a benign, carefully governed artificial superintelligence is really good for humanity, and OpenAI quickly solved all of humanity’s problems and ushered in an age of peace and abundance in which nobody wanted for anything or needed any Microsoft products. And capitalism came to an end.
Might really increase the utility of those GCP credits.
I was mainly referring to the TPU hardware advantage + GCP running and designing their own datacenter stack.
We have no idea what it means to be the "primary cloud provider" and have the products made available "first on Azure". Does MSFT have new models exclusively for days, weeks, months, or years?
Both facts and more details from the agreement are quite frankly highly relevant to judge whether this is a net positive, negative or neutral for MSFT. It's unbelievable that the SEC doesn't force MSFT to publish at least an economic summary of the deal.
That might help fix some of the bugs in Teams... :)
Bear in mind that MSFT have rights to OpenAI IP (as well as owning ~30% of them). The only reason they were giving revenue share was in return for exclusivity.
If they wanted named exclusivity rather than general exclusivity, we would charge a somewhat smaller amount for each competitor they wanted exclusivity from. They could give up exclusivity at any time.
That was precisely how we structured our deal with Azure, back in 2014-2016 or so.
I think this is good for OpenAI. They're no longer stuck with just Microsoft. It was an advantage that Anthropic can work with anyone they like but OpenAI couldn't.
https://blogs.microsoft.com/blog/2025/11/18/microsoft-nvidia...
https://azure.microsoft.com/en-us/blog/deepseek-r1-is-now-av...
https://ai.azure.com/
AFAICT they are just hedging their bets left and right still. Also feels like they are winning in the sense that despite pretty much all those products being roughly equivalent... they are still running on their cloud, Azure. So even though they seem unable to capture IP anymore, they are still managing to get paid for managing the infrastructure.
> Starting April 20, 2026, new sign-ups for Copilot Pro, Copilot Pro+, and student plans are temporarily paused.
From: https://docs.github.com/en/copilot/concepts/billing/billing-...
[1] https://news.microsoft.com/source/2026/04/08/microsoft-annou...
The Microsoft and OpenAI situation just got messy.
We had to rewrite the contract because the old one wasn't working for anyone. Basically, we’re trying to make it look like we’re still friends while we both start seeing other people. Here is what’s actually happening:
1. Microsoft is still the main guy, but if they can't keep up with the tech, OpenAI is moving out. OpenAI can now sell their stuff on any cloud provider they want.
2. Microsoft keeps the keys to the tech until 2032, but they don't have the exclusive rights anymore.
3. Microsoft is done giving OpenAI a cut of their sales.
4. OpenAI still has to pay Microsoft back until 2030, but we put a ceiling on it so they don't go totally broke.
5. Microsoft is still just a big shareholder hoping the stock goes up.
We’re calling this "simplifying," but really we’re just trying to build massive power plants and chips without killing each other yet. We’re still stuck together for now.
"The Microsoft and OpenAI situation just got messy" is objectively wrong–it has been messy for months [1]. Nos. 1 through 3 are fine, though "if they can't keep up with the tech, OpenAI is moving out" parrots OpenAI's party line. No. 4 doesn't make sense–it starts out with "we" referring to OpenAI in the first person but ends by referring to them in the third person "they." No. 5 is reductive when phrased with "just."
It would seem the translator took corporate PR speak and translated it into something between the LinkedIn and short-form blogger dialects.
[1] https://www.wsj.com/tech/ai/openai-and-microsoft-tensions-ar...
I don't expect the translation to take OpenAI's statements and make them truthful or to investigate their veracity, but I genuinely could not understand OpenAI's press release as they have worded it. The translation at least makes it easier to understand what OpenAI's view of the situation is.
"We" in this sentence refers to both parties; "they" refers to OpenAI. Not a grammatical error.
Fair enough.
> "they" refers to OpenAI. Not a grammatical error
I'd say it is. It's a press release from OpenAI. The rest of the release uses the third-person "they" to refer to Microsoft. The LLM traded accuracy for a bad joke, which is someting I associate with LinkedIn speak.
The fundmaental problem might be the OpenAI press release is vague. (And changing. It's changed at least once since I first commented.)
I'm pretty sure "just" is being used here to mean "simply" rather than "recently".
That's kagi? Cool, I'm check out out more!
(Andy Jassy) "Very interesting announcement from OpenAI this morning. We’re excited to make OpenAI's models available directly to customers on Bedrock in the coming weeks, alongside the upcoming Stateful Runtime Environment. With this, builders will have even more choice to pick the right model for the right job. More details at our AWS event in San Francisco tomorrow."
Which also means, if you are a big boring AWS or GCP shop, and have a spend commitment with either as part of a long term partnership, it will count towards that. And, you won't likely have to commit to a spend with OpenAI if you want the EU data residency for instance. And likely a bit more transparency with infra provisioning and reserved capacity vs. OpenAI. All substantial improvements over the current ways to use OpenAI in real production.
Azure is effectively OpenAI's personal compute cluster at this scale.
That article doesn't give a timeframe, but most of these use 10 years as a placeholder. I would also imagine it's not a requirement for them to spend it evenly over the 10 years, so could be back-loaded.
OpenAI is a large customer, but this is not making Azure their personal cluster.
This seems impossible.
Amazon CEO says that these models are coming to Bedrock though: https://x.com/ajassy/status/2048806022253609115
https://www.dw.com/en/musk-vs-openai-trial-to-get-underway/a...
https://blogs.microsoft.com/blog/2026/04/27/the-next-phase-o...
Yes. Microsoft was "considering legal action against its partner OpenAI and Amazon over a $50 billion deal that could violate its exclusive cloud agreement with the ChatGPT maker" [1].
[1] https://www.reuters.com/technology/microsoft-weighs-legal-ac...
https://news.ycombinator.com/item?id=47616242
They did not need to go so hard on the hype - Anthropic hasn’t in relative terms and is generating pretty comparable revenues at present.
OpenAI bet on consumers; Anthropic on enterprise. That will necessitate a louder marketing strategy for the former.
Why is it Altman is facing kill shots and Dario isn’t?
Dario left OpenAI because of the bad he saw there, and made a superior product (though these things change very rapidly).
Altman peaked in the zeiteist in 2023; Dario, much less prominently, in 2024 and now '26 [1]. I'd guess around this time next year, Dario will be as hated as Altman is today.
[1] https://trends.google.com/explore?q=altman%2C%20Dario&date=t...
OpenAI has public models that are pretty 'meh', better than Grok and China, but worse than Google and Anthropic. They still cost a ton to run because OpenAI offers them for free/at a loss.
However, these people are giving away their data, and Microsoft knows that data is going to be worthwhile. They just dont want to pay for the electricity for it.
What's losing OpenAI money is paying for the whole of R&D, including training and staff. Microsoft doesn't pay that, so they get the money making part of AI without the associated costs.
The circular economy section really is shocking- OpenAI committing to buying $250 Billion of Azure services, while MSFT's stake is clarified as $132 Billion in OpenAI. Same circular nonsense as NVIDIA and OpenAI passing the same hundred billion back and forth.
Mac: You're damn right. Thus creating the self-sustaining economy we've been looking for.
Dennis: That's right.
Mac: How much fresh cash did we make?
Dennis: Fresh cash! Uh, well, zero. Zero if you're talking about U.S. currency. People didn't really seem interested in spending any of that.
Mac: That's okay. So, uh, when they run out of the booze, they'll come back in and they'll have to buy more Paddy's Dollars. Keepin' it moving.
Dennis: Right. That is assuming, of course, that they will come back here and drink.
Mac: They will! They will because we'll re-distribute these to the Shanties. Thus ensuring them coming back in, keeping the money moving.
Dennis: Well, no, but if we just re-distribute these, people will continue to drink for free.
Mac: Okay...
Dennis: How does this work, Mac?
Mac: The money keeps moving in a circle.
Dennis: But we don't have any money. All we have is this. ... How does this work, dude!?
Mac: I don't know. I thought you knew.
I fear for the end user we'll still see more open-microslop spam. I see that daily on youtube - tons of AI generated fakes, in particular with that addictive swipe-down design (ok ok, youtube is Google but Google is also big on the AI slop train).
Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.
Isn't that exactly what you would expect to happen as we learn more about the nature and inner workings of intelligence and refine our expectations?
There's no reason to rest our case with the Turing test.
I hear the "shifting goalposts" riposte a lot, but then it would be very unexciting to freeze our ambitions.
At least in an academic sense, what LLMs aren't is just as interesting as what they are.
Does it matter?
We can do countless things people in the 90's would think was black magic.
If I showed the kid version of myself what I can do with Opus or Nano Banana or Seedance, let alone broadband and smartphones, I think I'd feel we were living in the Star Trek future. The fact that we can have "conversations" with AI is wild. That we can make movies and websites and games. It's incredible.
And there does not seem to be a limit yet.
The Turing Test/Imitation Game is not a good benchmark for AGI. It is a linguistics test only. Many chatbots even before LLMs can pass the Turing Test to a certain degree.
Regardless, the goalpost hasn't shifted. Replacing human workforce is the ultimate end goal. That's why there's investors. The investors are not pouring billions to pass the Turing Test.
> I propose to consider the question, "Can machines think?" This should begin > with definitions of the meaning of the terms "machine" and "think." The > definitions might be framed so as to reflect so far as possible the normal use > of the words, but this attitude is dangerous, If the meaning of the words > "machine" and "think" are to be found by examining how they are commonly used > it is difficult to escape the conclusion that the meaning and the answer to the > question, "Can machines think?" is to be sought in a statistical survey such as > a Gallup poll. But this is absurd. Instead of attempting such a definition I > shall replace the question by another, which is closely related to it and is > expressed in relatively unambiguous words.
Many people who want to argue about AGI and its relation to the Turing test would do well to read Turing's own arguments.
Like do people not know what word "general" means? It means not limited to any subset of capabilities -- so that means it can teach itself to do anything that can be learned. Like start a business. AI today can't really learn from its experiences at all.
The truth is, we have had AGI for years now. We even have artificial super intelligence - we have software systems that are more intelligent than any human. Some humans might have an extremely narrow subject that they are more intelligent than any AI system, but the people on that list are vanishing small.
AI hasn't met sci-fi expectations, and that's a marketing opportunity. That's all it is.
also, I'm pretty sure some people will move goalposts further even then.
If you've never read the original paper [1] I recommend that you do so. We're long past the point of some human can't determine if X was done by man or machine.
[1]: https://courses.cs.umbc.edu/471/papers/turing.pdf
Regarding shifting goalposts, you are suggesting the goalposts are being moved further away, but it's the exact opposite. The goalposts are being moved closer and closer. Someone from the 50s would have had the expectation that artificial intelligence ise something recognisable as essentially equivalent to human intelligence, just in a machine. Artificial intelligence in old sci-fi looked nothing like Claude Code. The definition has since been watered down again and again and again and again so that anything and everything a computer does is artificial intelligence. We might as well call a calculator AGI at this point.
An AGI would not have problems reading an analog clock. Or rather, it would not have a problem realizing it had a problem reading it, and would try to learn how to do it.
An AGI is not whatever (sophisticated) statistical model is hot this week.
Just my take.
That's not the definition they have been using. The definition was "$100B in profits". That's less than the net income of Microsoft. It would be an interesting milestone, but certainly not "most of the jobs in an economy".
It ties the definition to economic value, which I think is the best definition that we can conjure given that AGI is otherwise highly subjective. Economically relevant work is dictated by markets, which I think is the best proxy we have for something so ambiguous.
And then I think coming up with the right metric is just as subjective on this field as the technological one.
Deep scientific discoveries are also cognitively demanding, but are not really valued (see the precarious work environment in academia).
Another point: a lot of work is rather valued in the first place because the work centers around being submissive/docile with regard to bullshit (see the phenomenon of bullshit jobs). You really know better, but you have to keep your mouth shut.
e.g. average cost to complete a set of representative tasks
Huh. Source? I mean, typical OpenAI bullshit, but would love to know how they defined it.
Wow. Maybe they spelled it out as aggregate gross income :P.
Apple, Alphabet, Amazon, NVIDIA, Samsung, Intel, Cisco, Pfizer, UnitedHealth , Procter & Gamble, Berkshire Hathaway, China Construction Bank, Wells Fargo, ...
A self-running massive corporation with no people that generates billions in profit, no matter what you call it, would completely upend all previous structural assumptions under capitalism
That's a relevent aspect of the AGI concept.
[0] https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
"OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits."
Given that the definition of AGI is beyond meaningless, it is clear that the "I" in AGI stands for IPO.
[0] https://finance.yahoo.com/news/microsoft-openai-financial-de...
From: https://openai.com/charter/
if you think drone targeting in Ukraine is scary now, wait until AGI is on it...
ditto for exploiting vulns via mythos
I don't think your original comment deserve to be downvoted. (Calling someone illiterate, on the other hand.)
But the "it" I was asking about was "AGI" as "an economical thing." You technically correctly answered how OpenAI defines AGI in public, i.e. with no reference to profits. But it did not address the economic definition OP initially alluded to.
For what it's worth, I could have been clearer in my ask.
But originally I was just trying to be helpful by quoting their charter on what they consider "agi" now.
I don't get why HN commenters find this so hard to understand. I have a sense they are being deliberately obtuse because they resent OpenAI's success.
From Wikipedia
Eschatology (/ˌɛskəˈtɒlədʒi/; from Ancient Greek ἔσχατος (éskhatos) 'last' and -logy) concerns expectations of the end of present age, human history, or the world itself.
I'm case anyone else is vocabulary skill checked like me
Russian Invasion - Salami Tactics | Yes Prime Minister
https://www.youtube.com/watch?v=yg-UqIIvang
OpenAI and Microsoft do (did?) have a quantifiable definition of AGI, it’s just a stupid one that is hard to take seriously and get behind scientifically.
https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
> The two companies reportedly signed an agreement last year stating OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits. That’s far from the rigorous technical and philosophical definition of AGI many expect.
Why are we expecting AGI to one shot it? Can't we have an AGI that can fails occasionally to solve some math problem? Is the expectation of AGI to be all knowing?
By the way I agree that AGI is not around the corner or I am not arguing any of the llm s are "thinking machines". It's just I agree goal post or posts needs to be set well.
People obviously have really strong opinions on AI and the hype around investments into these companies but it feels like this is giving people a pass on really low quality discourse.
This source [1] from this time last year says even lab leaders most bullish estimate was 2027.
[1]. https://80000hours.org/2025/03/when-do-experts-expect-agi-to...
They can. If one consolidated the AI industry into a single monopoly, it would probably be profitable. That doesn't mean in its current state it can't succumb to ruionous competition. But the AGI talk seems to be mostly aimed at retail investors and philospher podcasters than institutional capital.
"With viable economics" is the point.
My "ludicrous statement" is a back-of-the-envelope test for whether an industry is nonsense. For comparison, consolidating all of the Pets.com competitors in the late 1990s would not have yielded a profitable company.
Do you argue in good faith?
There’s a difference between being too early vs being nonsense.
Not in the 1990s. The American e-commerce industry was structurally unprofitable prior to the dot-com crash, an event Amazon (and eBay) responded to by fundamentally changing their businesses. Amazon bet on fulfillment. eBay bet on payments. Both represented a vertical integration that illustrates the point–the original model didn't work.
> There’s a difference between being too early vs being nonsense
When answering the question "do the investments make sense," not really. You're losing your money either way.
The American AI industry appears to have "viable economics for profit" without AGI. That doesn't guarantee anyone will earn them. But it's not a meaningless conclusion. (Though I'd personally frame it as a hypothesis I'm leaning towards.)
OP did not include this requirement in their post because doing so would make the claim trivially true.
I think this might be similar to how we changed to cars when we were using horses
Other people just call it "theft".
Asking because, reading the tea leaves from the outside, until ChatGPT came along, MSFT (via Bill Gates) seemed to heavily favor symbolic AI approaches. I suspect this may be partly why they were falling so far behind Google in the AI race, which could leverage its data dominance with large neural networks.
So based on the current AI boom, MSFT may have been chasing a losing strategy with symbolic AI, but if they were all-in on NN, they were on the right track.
What part do you find hard to believe? That tech companies would send people to speak at a university's computer science functions?
Let me give you another one you'll think I'm making up: virtual reality was a thing back in the mid- to late-90s and people were confidently hyping it up back then.
even in pop-culture, see the movie Lawnmower Man.
At the very least, Ilya Sutskever genuinely believed it, even when they were just making a DOTA bot, and not for hype purposes.
I know he's been out of OpenAI for a while, but if his thinking trickled down into the company's culture, which given his role and how long he was there I would say seems likely, I don't think it's all hype.
Grand delusion, perhaps.
Definitely interesting to watch from the perspective of human psychology but there is no real content there and there never was.
The stuff around Mythos is almost identical to O1. Leaks to the media that AGI had probably been achieved. Anonymous sources from inside the company saying this is very important and talking about the LLM as if it was human. This has happened multiple times before.
so just understand there’s a lot of of us “insane” people out there and we’re making really insane progress toward the original 1955 AI goals.
We’re going to continue to work on this no matter what.
1) True believers 2) Hype 3) A way to wash blatant copyright infringement
True believers are scary and can be taken advantage of. I played DOTA from 2005 on and beating pros is not enough for AGI belief. I get that the learning is more indirect than a deterministic decision tree, but the scaling limitations and gaps in types of knowledge that are ingestible makes AGI a pipe dream for my lifetime.
Seems more like an incredibly embarrassing belief on his part than something I should be crediting.
He doesn't need to be right but it's not crazy at all to look at super human performance in DOTA and think that could lead to super human performance at general human tasks in the long run
Your position is a tautology given there is no (and likely will never be) collectively agreed upon definition of AGI. If that is true then nobody will ever achieve anything like AGI, because it’s as made up of a concept as unicorns and fairies.
Is your position that AGI is in the same ontological category as unicorns and Thor and Russell’s teapot?
Is there’s any question at this point that humans won’t be able to fully automate any desired action in the future?
We already have several billion useless NGI's walking around just trying to keep themselves alive.
Are we sure adding more GI's is gonna help?
...just please stop burning our warehouses and blocking our datacenters.
If you present GPT 5.5 to me 2 years ago, I will call it AGI.
neural networks are solving huge issues left and right. Googles NN based WEathermodel is so good, you can run it on consumer hardware. Alpha fold solved protein folding. LLMs they can talk to you in a 100 languages, grasp tasks concepts and co.
I mean lets talk about what this 'hype' was if we see a clear ceiling appearing and we are 'stuck' with progress but until then, I would keep my judgment for judgmentday.
Now our idea of what qualifies as AGI has shifted substantially. We keep looking at what we have and decide that that can't possibly be AGI, our definition of AGI must have been wrong
In some sense, this isn't really different than how society was headed anyways? The trend was already going on that more and more sections of the population were getting deemed irrational and you're just stupid/evil for disagreeing with the state.
But that reality was still probably at least a century out, without AI. With AI, you have people making that narrative right now. It makes me wonder if these people really even respect humanity at all.
Yes, you can prod slippery slope and go from "superintelligent beings exist" to effectively totalitarianism, but you'll find so many bad commitments there.
Science fiction from that era even had the concept of what models are... they'd call it an "oracle". I can think of at least 3 short stories (though remembering the authors just isn't happening for me at the moment). The concept was of a device that could provide correct answers to any question. But these devices had no agency, were dependent on framing the question correctly, and limited in other ways besides (I think in one story, the device might chew on a question for years before providing an answer... mirroring that time around 9am PST when Claude has to keep retrying to send your prompt).
We've always known what we meant by artificial intelligence, at least until a few years ago when we started pretending that we didn't. Perhaps the label was poorly chosen (all those decades ago) and could have a better label now (AGI isn't that better label, it's dumber still), but it's what we're stuck with. And we all know what we mean by it. We all almost certainly do not want that artificial intelligence because most of us are certain that it will spell the doom of our species.
https://www.noemamag.com/artificial-general-intelligence-is-...
I've been working with a startup, and I want to invest in it, and for the paperwork for that, all the nitty gritty details; instead of spending $20k in lawyers and a whole bunch more time going back and forth with them as well, the four of us, me, their CEO, my AI, and their AI; we all sat in a room together and hashed it out until both of us were equally satisfied with the contract. (There's some weird stuff so a templated SAFE agreement wasn't going to work.) I'm not saying you're wrong, just that lawyers, as a profession isn't going to be unchanged either.
There is a reason so many scams happen with technology. It is too easy to fool people.
Isn't this tautology? We've de facto defined AGI as a "sufficiently complex LLM."
However, I don't think it is even true. LLMs may not even be on the right track to achieving AGI and without starting from scratch down an alternate path it may never happen.
LLMs to me seem like a complicated database lookup. Storage and retrieval of information is just a single piece of intelligence. There must be more to intelligence than a statistical model of the probable next piece of data. Where is the self learning without intervention by a human. Where is the output that wasn't asked for?
At any rate. No amount of hype is going to get me to believe AGI is going to happen soon. I'll believe it when I see it.
And how will you know AGI when you saw it?
If this progress and focus and resources doesn't lead to AI despite us already seeing a system which was unimaginable 6 years ago, we will never see AGI.
And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics, thats also CRAZY.
I don't know, maybe AGI is possible but there's more to intelligence than statistical next word prediction?
The 'predicting the next word' is the learning mechanism of the LLM which leads to a latent space which can encode higher level concepts.
Basically a LLM 'understands' that much as efficient as it has to be to be able to respond in a reasonable way.
A LLM doesn't predict german text or chinese language. It predicts the concept and than has a language layer outputting tokens.
And its not just LLMs which are progressing fast, voice synt and voice understanding jumped significantly, motion detection, skeletion movement, virtual world generation (see nvidias way of generating virutal worlds for their car training), protein folding etc.
Yes, and unless you are prepared to rebut the argument with evidence of the supernatural, that's all there is, period. That's all we are.
So tired of the thought-terminating "stochastic parrot" argument.
Can an LLM decide, without prompting or api calls, to text someone or go read about something or do anything at all except for waiting for the next prompt?
Do LLMs have any conceptual understanding of anything they output? Do they even have a mechanism for conceptual understanding?
LLMs are incredibly useful and I'm having a lot of fun working with them, but they are a long way from some kind of general intelligence, at least as far as I understand it.
After a bit of further refinement, we'll start to call that process "learning." Eventually the question of who owns the notes, who gets to update them, and how, will become a huge, huge deal.
It's not supernatural, I believe that an artificial intelligence is possible because I believe human intelligence is just a clever arrangement of matter performing computation, but I would never be presumptuous enough to claim to know exactly how that mechanism works.
My opinion is that human intelligence might be what's essentially a fancy next token predictor, or it might work in some completely different way, I don't know. Your claim is that human intelligence is a next token predictor. It seems like the burden on proof is on you.
Literally it is, at least in many of its forms.
You accepted CamperBob2’s text as input and then you generated text as output. Unless you are positing that this behavior cannot prove your own general intelligence, it seems plain that “next token generator” is sufficient for AGI. (Whether the current LLM architecture is sufficient is a slightly different question.)
And while I am typing, and while I am thinking before I type, I experience an array of non-textual sensory input, and my whole experience of self is to a significant extent non-lingual. Sometimes, I experience an inner monologue, sometimes I think thoughts which aren't expressed in language such as the structure of the data flow in a computer program, sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
When I solve difficult programming problems or other difficult problems, I build abstract structures in my mind which represents the relevant information and consider things like how data flows, which parts impact which other parts, what the constraints are, etc. without language coming in to play at all. This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
Therefore, my position is: it is plausible that next token prediction won't give rise to general intelligence, and I do not find your argument convincing.
COCONUT, PCCoT, PLaT and co are directly linked to 'thinking in latent space'. yann lecun is working on this too, we have JEPA now.
Also how do you describe or explain how an LLM is generating the next token when it should add a feature to an existing code base? In my opinion it has structures which allows it to create a temp model of that code.
For sure a LLM lack the emotional component but what we humans also do, which indicates to me, that we are a lot closer to LLMs that we want to be, if you have a weird body feeling (stress, hot flashes, anger, etc.) your 'text area/llm/speech area' also tries to make sense of it. Its not always very good in doing so. That emotional body feeling is not that aligned with it and it takes time to either understand or ignore these types of inputs to the text area/llm/speech part of our brain.
I'm open for looking back in 5 years and saying 'man that was a wild ride but no AGI' but at the current quality of LLMs and all the other architectures and type of models and money etc. being thrown at AGI, for now i don't see a ceiling at all. I only see crazy unseen progress.
I showed than counter examples.
"COCONUT, PCCoT, PLaT and co are directly linked to 'thinking in latent space'. yann lecun is working on this too, we have JEPA now."
Btw. just because you have to do something with the LLM to trigger the flow of information through the model, doesn't mean it can't think. It only means that we have to build an architecture around the model or build it into the models base architecture to enable more thinking.
We do not know how the brain architecture is setup for this. We could have sub agents or we can be a Mixture of Experts type of 'model'.
There is also work going on in combining multimodal inputs and diffusion models which look complelty different from a output pov etc.
If you look how a LLM does math, Anthropic showed in a blog article, that they found similiar structures for estimating numbers than how a brain does.
Another experiment from a person was to clone layers and just adding them beneth the original layer. This improved certain tasks. My assumption here is, that it lengthen and strengthen kind of a thinking structure.
But because using LLMs are still so good and still return relevant improvements, i think a whole field of thinking in this regard is still quite unexplored.
"In context" is the obvious answer... but if you view the chain of thought from a reasoning model, it may have little or nothing to do with arriving at the correct answer. It may even be complete nonsense. The model is working with tokens in context, but internally the transformer is maintaining some state with those tokens that seems to be independent of the superficial meanings of the tokens. That is profoundly weird, and to me, it makes it difficult to draw a line in the sand between what LLMs can do and what human brains can do.
Inability to introspect your own word selections does not mean it’s meaningfully different from what an LLM does. There is plenty of evidence that humans do a lot of things that are not driven by conscious choice and we rationalize it after the fact.
> I consider an entire idea and then decide what tokens to enter into the computer in order to communicate the idea to you.
And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
> sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
You cannot spin experience as intelligence. LLMs have the experience of reading the entire internet, something you cannot conceive of. Certainly your experiences shape who you are. This is a different axis from intelligence, though.
> This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
> It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
This is correct and also completely irrelevant. I am describing what I experience, and describing how my experience seems very different to next token prediction. I therefore conclude that it's plausible that there is more involved than something which can be reduced to next token prediction.
> And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
Language models can't think outside of producing tokens. There is nothing going on within an LLM when it's not producing tokens. The only thing it does is taking in tokens as input and producing a token probability distribution as output. It seems plausible that this is not enough for general intelligence.
> You cannot spin experience as intelligence.
Correct, but I can point out that the only generally intelligent beings we know of have these sorts of experiences. Given that we know next to nothing about how a human's general intelligence works, it seems plausible that experience might play a part.
> LLMs have the experience of reading the entire internet, something you cannot conceive of.
I don't know that LLMs have an experience. But correct, I cannot conceive of what it feels like to have read and remembered the entire Internet. I am also a general intelligence and an LLM is not, so there's that.
> Certainly your experiences shape who you are. This is a different axis from intelligence, though.
I don't know enough about what makes up general intelligence to make this claim. I don't think you do either.
> Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
I'm telling you how these technologies work. When a language model isn't performing inference, it is not doing anything. A language model is a function which takes a token stream as input and produces a token probability distribution as output. By definition, there is no thinking outside of producing words. The function isn't running.
> I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
I 100% agree. It is not clear whether a human experience is necessary for intelligence. It is plausible that something approximating a human-like experience is necessary for intelligence. It is also plausible that something approximating human-like experience is completely unnecessary and you can make an AGI without such experiences.
It's plausible that next token prediction is sufficient for AGI. It's also plausible that it isn't.
This is the fundamental issue. No one seems capable of defining general intelligence. Ten years ago most scientists would probably have agreed that The Turing Test was sufficient but the goalposts shifted when ChatGPT passed that.
If it’s not clear what AGI even means, it’s hard to say whether an LLM can achieve it, because it devolves into pointing out that an LLM is not a human.
The popularity of, and lack of consensus on, the Chinese room thought experiment kind of implies that this is wrong? I don't think many scientists (or, more relevantly, philosophers of mind) would, even 10 years ago, have said, "if a computer is able to fool a human into thinking it's a human, then the computer must possess a general intelligence".
Even Turing's perspective was, from what I understand, that we must avoid treating something that might be sentient as a machine. He proposed that if a computer is able to act convincingly human, we ought to treat it as if it is a human, not because it must be a conscious being but because it might be.
If what you are saying is true, then LLMs wouldn't be able to handle out-of-distribution math problems without resorting to tool use. Yet they can. When you ask a current-generation model to multiply some 8-digit numbers, and forbid it from using tools or writing a script, it will almost certainly give you the right answer. That includes local models that can't possibly cheat. LLMs are stochastic, but they are not parrots.
At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejorative sense you're applying to it. It can't be. The tokens in a transformer network are evidently not just words in a Markov chain but a substrate for reasoning. The model is generalizing processes it learned, somehow, in the course of merely being trained to predict the next token.
Mechanically, yes, next-token prediction is what it's doing, but that turns out to be a much more powerful mechanism than it appeared at first. My position is that our brains likely employ similar mechanism(s), albeit through very different means.
It is scarcely believable that this abstraction process is limited to keeping track of intermediate results in math problems. The implications should give the stochastic-parrot crowd some serious cognitive dissonance, but...
(Edit: it occurs to me that you are really arguing that the continuous versus discrete nature of human thinking is what's important here. If so, that sounds like a motte-and-bailey thing that doesn't move the needle on the argument that originally kicked off the subthread.)
(Edit 2, again due to rate-limiting: it does sound like you've fallen back to a continuous-versus-discrete argument, and that's not something I've personally thought much about or read much about. I stand by my point that the ability to do arithmetic without external tools is sufficient to dispense with the stochastic-parrot school of thought, and that's all I set out to argue here.)
Okay, what do you think language models are doing when they're not producing token probability distributions? What processes do you think are going on when the function which predicts a token isn't running?
> At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejoreative sense you're applying to it.
I don't know what pejorative sense you're implying here. I am, to the best of my ability, describing how the language model works. I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
What are you doing when you are not outputting tokens? You have a thought, evaluate it, refine it, repeat.
You’re not wrong that the basic building block is just “next token prediction”, but clearly the emergent behaviors exceed our intuition about what this process can achieve. We’re seeing novel proofs come out of these. Will this lead to AGI? That’s still TBD.
> I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
The pejorative is that you imply this is a shallow and unthinking process. As I said earlier, you are literally a token generator on HN. You read someone’s comment, do some kind of processing, and output some tokens of your own.
I mean I do think sometimes even when not typing?
> Will this lead to AGI? That’s still TBD.
This is literally what I have been saying this whole time.
Since we agree, I will consider this conversation concluded.
This overestimates introspective access.
The brain is very good at producing a coherent story after the fact. Touch the hot stove and your hand moves before the conscious thought of "too hot" arrives. The hot message hits your spinal cord and you move before it reaches your brain. Your conscious mind fills in the rest afterwards.
I don't think that means that conscious thought is fake. But it does make me skeptical of the claim that we first possess a complete idea and only then does it serialize into words. A lot of the "idea" may be assembled during the act of expression, with consciousness narrating the process as if it had the whole thing in advance.
With writing, as in this comment, there's also a lot a backtracking and rewording that LLMs don't have the ability to do, so there's that.
Before you start typing, an fMRI machine can tell you which finger you'll lift first, before you know it yourself.
We are not special. Consciousness is literally a continuous hallucination that we make up to explain what we do and what we think, after the fact. A machine can be trained to behave identically, but it's not clear if that's the best way forward or not.
Edit due to rate limiting: to answer your question, the substrate your mind uses to drive this process can be considered an array of tokens that, themselves, can be considered 'words.'
It's hard to link sources -- what am I supposed to do, send you to Chomsky and other authorities who have predicted none of what's happening and who clearly understand even less?
This seems like a factual claim. Can you link a source?
(Also why respond in the form of an edit?)
it absolutely is a next word predictor
However, a much simpler explanation for what we see with LLMs is that instead the higher level encodings in latent space match only the patterns of our language(s), and no deeper encoding/understanding is present.
It's Plato's Cave - the shadows on the wall are all an LLM ever sees, and somehow it is expected to derive the real reality behind them.
At least Mythos model with its 10 Trillion parameter might indicate that the scaling law is valid. Its a little bit unfortunate that we still don't know that much more about that model.
Their progress is almost nought. Humanoids are stupid creations that are not good at anything in the real world. I'll give it to the machine dogs, at least they can reach corners we cannot.
I can also recommend looking at Generalist: https://www.youtube.com/@Generalist_AI
How can you say the advancements since Honda's asimo robot amount to "almost naought"?
https://en.wikipedia.org/wiki/AI_winter
There is a difference to be acknowledged: in the 70s/80s the whole world didn't suddenly start to shift to AI right?
So why do so many smart and/or rich people push this? Hype? Yeah sure but hype was here for crypto too.
I bet its an undelying understanding and the right time with the right components: Massive capital for playing this game long enough to see through the required initial investment, internet for fast data sharing, massive compute for the amount of data and compute you need, real live business relevant results (it already disrupts jobs) etc.
The necessary amount of Compute, interconnect (internet), money, researcher etc. wasn't available at that time.
and we did not invest the most amount of money and compute and brain power as we are doing right now. This is unseen.
"The new economy" also didn't have anything to do with the previous one. Turns out that it crashed just as well.
is it? we're currently scaled on data input and LLMs in general, the only thing making them advance at all right now is adding processing power
Crypto was flawed from the beginning and lots of people didn't understood it properly. Not even that a blockchain can't secure a transaction from something outside of a blockchain.
LLMs don't have to be perfect, they just need to be as good as humans and cheaper or easier to manage.
$100+ billion in R&D and it's not comparable... hmm
And yet they don't do really good jobs with pretty much anything, save for software development, to which people still seem pretty split as far as it being a helpful thing. That's before we even factor in the cost.
I also believe that whatever code researchers and other non software engineers wrote before coding agents, were similiar shitty but took them a lot longer to write.
Like do you know how many researchers need to do some data analysis and hack around code because they never learned programming? So so many. If they know how to verify their data (which they needed to know before already), a LLM helps them already.
There is also plenty of other code were perfection doesn't matter. Non SaaS software exists.
For security experts, we just saw whats happening. The curl inventor mentioned it online that the newest AI reports for Security issues are real and the amount of security gaps found are real and a lot of work.
Image generation is very good and you can see it today already everywere. From cheap restaurants using it, to invitations, whatsapp messages, social media, advertising.
I have a work collegue, who is in it for 6 years and he studied, he is so underqualified if you give me his salary as tokens today, i wouldn't think for a second to replace him.
Tried to delete this submission in place of it but too late.
I imagine the thinking was that it’s better to just post it clearly than to have rumors and leaks and speculations that could hurt both companies (“should I risk using GCP for OpenAI models when it’s obviously against the MS / OpenAI agreement?”).