Github has recently changed the way their status page tracks uptime in the name of "transparency"[0]. "Partial Outages" are now only worth 30% of their duration, and "Degraded Performance" is worth none, so their uptime values are now wildly inflated.
Cursed by mighty Redmond to roam the market wasteland until death, one of the seventy some odd beleaguered CoPilot products is now being lashed like a haggard burro to the dying light of a once prominent development platform that, upon itself, were pinned the hopes and dreams of a commercial software juggernaut to capture the hearts and minds of developers all around the world.
For all the hate it gets, I use them regularly with little to no complaints.
I have always found it as a pretty nice to have feature if I am already using GitHub. It’s far from perfect or robust but I can get a lot of use out of it with low to no friction.
Which is fair but inversely we do many builds throughout the day most business days and have not had an impact where we noticed it. Could also be that we deploy often and frequently and have setup our builds to be as quick as possible so any issues would likely go unnoticed.
Because it doesn't even come close to frontier models in intelligence/speed/price. I can run my 3090 nonstop and rack up an electricity bill that costs more than a subscription and get worse results that are slower. They are ok for simple/non complex things, but that's not really what I need AI for.
I feel the opposite. I do need AI for simple things. Complex things are usually so ill-defined that the actual bottleneck takes place in meatspace, not in my IDE.
The better your code is architected, the less powerful model you’ll need for it to make sense of it.
E.g. a well-designed deployment (infrastructure-as-code) repository doesn’t need a frontier model to be understood well-enough to create a new job / service using sibling jobs / services as templates.
And this already saves me dozens of minutes per week, although it’s not a 2x multiplier in my efficiency.
The issue is that local models are dumb and tend to make mistakes than look good at a first glance. So any "saving" is quickly ruined by having to do an extensive review. You might as well just write things yourself.
I use it as code scaffolding, which means in a way I’m often rewriting it. For me, writing from scratch isn’t the same amount of effort as using a code scaffolding tool.
Local models are nowhere near the performance of frontier models. Unless you can fork out like £100k to get something passable in terms of performance.
They never really get tight very long: the various states are way too busy flooding the world with endless money printing to kick the can of the public debt always further.
Covid financial crash? We went to new highs. 2022 tech flash crash (Meta and Netflix did -75% for example): we then went to new highs.
The only way for governments who ever spend way more than they bring in taxpayer dollars is to de-valuate the currency.
So "financial markets getting tighter": probably won't last.
As you said yourself: Quantitative easing did not solve anything. We keep kicking the can down the road, and the problems grow exponentially every time. This approach won’t work forever. In fact, we may be past the tipping point already.
is there any data on how many Actions minutes a single copilot review actually takes? the announcement doesn't mention it, and for a team doing 20+ PRs a day that number adds up fast.
It's gonna be interesting to see how this plays out. Usually a tech rugpull like this lasts a number of years. And this sort of has, but the agentic stuff has really only caught on like wildfire in the last, I dunno, six months or so. The rugpull would be way more effective if there could be several years of getting developers addicted to this development paradigm, but alas, the VC money burned was too great to subsidize for very long.
Its also a weird way to cede control of the market to the foreign model vendors, because I'm reasonably sure that DeepSeek et al aren't subsidising tokens to the same extent that the big 3's subscription models have been.
I think there's "sort of" a moat for non-Chinese vendors. As much as people distrust the US right now, I think deep down inside everyone knows that the second you let a Chinese provider do inference on your codebase they're gonna suck up every bit of it. But hey, cheap tokens, right?
So you'll probably never see government customers allow that and neither will a lot of commercial customers.
Why do we assume us providers aren't doing the same? Also all the Chinese providers are giving open weight models. Many you can run locally.
I don't see the risk. If your code is easily AI generated you don't have a moat anyways. A Chinese competitor probably won't have as easy of a time as a US one of you operate in the US
The US has robust IP and trademark law that allows companies some amount of chance to find a legal remedy to anyone who clones their business. China is notorious for protecting local companies from foreign IP suits.
Further, at a lot of companies, the risk has to be acceptable to shareholders and auditors. Perceived risk is often a more powerful motivator than actual risk.
Once your code, images, etc pass into the slop machine it is owned by whoever generated it later. Obviously they would need a new logo, llc, and some ui theme tweaks. Otherwise none of these AI coder products would exist.
Also, how long do you think openai, Microsoft, Google, anthropic, etc could delay a lawsuit while you pay hundreds of thousands in legal retainers? 5 years? 10?
From the perspective of someone currently living in the EU... I'd say thats pretty much a wash (or even slightly tilted in China's favour) for folks outside the US
Fortunately there’s plenty of open weight models that are just safetensors and you have a wide variety of providers to choose from, as well as just hosting it yourself.
If that is true, we’ve discovered that offering a product for $1 the $17, yields to dramatically shorter runway but possibly more addicted users. Can’t wait for products offered at $1 the $100.
My guess is that they're moving to a spot where they can pitch an LLM "doing something" as an action, and copilot is their first move. I don't see it as crazy to think of a "copilot code review" in a similar way to other build actions.
But also - enterprise accounts already have budget assigned to github actions, and this allows them to start billing right away without having to actually get (or allow) businesses to evaluate the return of having copilot do code reviews.
So seems like it's a mix of immediate incentives and long term architecture. I don't like it, though. If I were an enterprise my first response would be to turn it off.
> enterprise accounts already have budget assigned to github actions, and this allows them to start billing right away without having to actually get (or allow) businesses to evaluate the return of having copilot do code reviews
Hang on, I read this as copilot reviews with bill both actions minutes and AI credits. Did I miss something?
I'm assuming the running of the model is consuming the tokens, and the client coordinating and orchestrating the calls to the model to perform the review is happening in an action runner, thus using action minutes.
Agreed, especially weird since they just rolled out usage-based billing for Co-pilot. It would make a lot more sense to just re-use that usage instead IMHO
Yeah my mistake, I wasn't very clear in my comment.
Though actually the more I think about it, I think this change actually does make more sense. In the case of the AI running on GitHub side, that does feel pretty equivalent to CI minutes. I would hope that the number of minutes they bill for is pretty minimal though, since the vast majority of that will be I/O waiting on the agent to return
Good thing GitHub has plenty of built-up goodwill to spend down. If they didn’t this cascade of (probably necessary but nevertheless negative for customers) changes might be the tipping point to push a lot of companies to seek other options.
> Good thing GitHub has plenty of built-up goodwill to spend down
Do they, though? I don't know a single person who uses GitHub who actually likes it. It's far more often something like "it's fine, but I miss (GitLab|Gerrit)" or "I stopped using it for personal stuff and moved to (Codeberg|GitLab)."
The brand recognition among non-technical folks is really the strongest selling point in my eyes. And that's irrelevant to ~95% of software development.
GitLab is getting ensh*ttified as well. Rarely a day passes when they’re not trying to somehow push their AI features on me, even though I never asked for it. Thinking about moving to managed Forgejo.
Unclear why this is so shocking. Sounds like they have been making migrations on their underlying systems and this better aligns with the cost to run. I would be curious how many are using their code review system.
Interesting, I didn't know minutes where free before.
Stopped my recurring subscription at the end of last year when it started spinning up actions for review. Which as a side effect doubled the time (or so) to do a review. Whereas before that I would open a PR, wait at most a minute or two and the review was already done.
But I think the issue is that my situation (solo dev, mono repo) is just not right for a dedicated instance.
With only 1-2 runners, the pipeline is slow (low parallelism) and resource constrained.
And at least 50% of the time its idle (I'm not working/sleeping).
I guess what I'm really looking for is for some kind of aggressive autoscaling, and aggressive caching.
I tried a couple of things (GHA, Dagger + Hertzner, Buildkite)
And Im just not too sure theres going to be any out of the box solution since my priority is essentially to minimize cost and maximize efficiency. Not really a great customer for any providers.
Im tempted to just get agent to build something out quickly with cloudflare workers + spot instances.
I self-host Drone CI still. I think Harness is in the slow process of letting it rot (it still gets at least some updates, though), which is kind of a shame, but it still does just fine for my CI needs (solo usage as well).
Very easy to stand up, does just fine. Definitely doesn't have the "library" of prebuilt actions that GHA does, but for the most part... I consider that a plus.
Otherwise it's very similar in concept - define actions in a yaml file, run commands on an image, webhook integration with most repo providers.
I run it on some old hardware locally (k3s cluster on old machines) and it outperforms the 1000 minutes from GHA easily, and costs basically nothing but some maintenance and time.
I've been keeping my eyes open for something new in this space since Harness bought it, though - so if other folks have recommendations I'd be interested in alternatives.
Expect to see more of these kinds of announcements as companies need to start showing returns on their AI investments. It's hard to say how subsidized the current AI products are[1] but we're definitely getting a free lunch at VC's expense the moment.
[1] Ed Zitron speculates the actual prices with token based billing for heavy users will be something like 10x the subscription price, but this seems high.
Not that I give much credence to anything Zitron says, but the amount of inference you can get on a £200 a month OpenAI or Anthropic subscription is easily an order of magnitude more than what you'd get paying the same amount at subscription rate.
Although I would also point out that OpenAI recently tripled the amount of Codex inference you get per month for £200 (and to head off the suggestion, this is distinct from their current 2x promotion on £100/month plans)
> Not that I give much credence to anything Zitron says, but the amount of inference you can get on a £200 a month OpenAI or Anthropic subscription is easily an order of magnitude more than what you'd get paying the same amount at subscription rate.
Neither of those is how much it actually costs the company selling the service. And I have feeling they are running at loss here so the play is "get everything possible using LLMs then jack up the pricing"
There have been plenty of studies which indicate that inference considered by itself is almost certainly quite profitable at all the frontier labs. The problem is amortizing the cost of all the expensive training runs required to train new models into the revenue stream.
Inference is cheap but training is quite expensive. Plus all the money they've invested and keep investing on hardware, data centers, etc. And evidently they also need to make a profit at some point.
Maybe from the perspective of traditional, turn-based chat. But when you start having developers command an army of agents that work around the clock, those cheap tokens start adding up fast...
If the unit-economics work out and they can sell $0.99 of tokens for $1.00, doesn't matter how many agents you spin up. The flat rate subscriptions can't last though.
> If the unit-economics work out and they can sell $0.99 of tokens for $1.00
I think the margins have to be a lot higher than that in order to give investors the return they're expecting, to continue the never-ending training treadmill, and to build more and more datacenters to accommodate people basically DDOS'ing the GPUs in order to run their workloads.
Yes, in theory what you said makes sense. But the tightrope these companies have to walk is that the per-token costs still have to be low enough that developers and companies don't just say "ehhh I guess we can still do all this work the old-fashioned way" but ALSO high enough to cover the massive expenses AND astronomical returns everyone's expecting.
VC investment isn’t about margins, it’s about finding a unicorn. It doesn’t matter if margins are negative if your product is dominant in the market as you can fiddle with the margins after the fact. You just need to be invested long enough to see everyone else fail.
The problem with AI is that there doesn't seem to be a durable barrier to entry for a "winner take all" dynamic to work. The biggest barrier to entry seems to be the capital needed to train the models, but even free models are getting "good enough" for some uses and there's little friction to stop users from switching between models. Many frontends make this explicit by letting you pick the model you want to run inside the same environment.
If prices go up, I suspect a bunch of folks will jump to cheaper, less capable models instead of eating the added cost. The whole value proposition of AI in enterprise is around cost-cutting, so that mentality is likely to persist when choosing which model to pay for.
I imagine the calculus changes a little bit when you've invested hundreds of billions (trillions?) of dollars in a relatively short period of time. Priority number one is probably getting that money back. I think the fact that providers are RAPIDLY cutting back/jacking up prices points to this being the case.
Did I bring down the quality of GitHub? Shame on you for personal attacks and adding nothing that's even tangentially related to the topic (at least my comment was on topic)
You didn’t bring down GitHub. You brought down the thread.
There’s a difference between criticizing a company and just swapping names for insults. The former can be useful, the latter just turns the discussion into noise. If you’ve got a point about Copilot or the review feature, make it. Otherwise it’s hard to see what anyone is supposed to take away from “ShitHub” other than childish shit-posting.
I really don't understand your point. You're saying we're turning discussions into noise and continuing to spam comments that are not even related to topic. Hard to take this seriously other than just bullying because you get triggered by a word. I concede there's no point replying to you anymore although I tried to reason about your meaness.
[1] https://www.githubstatus.com/
[0] https://github.blog/news-insights/company-news/bringing-more...
I have always found it as a pretty nice to have feature if I am already using GitHub. It’s far from perfect or robust but I can get a lot of use out of it with low to no friction.
It is time to setup local models. It is cheaper, and you already have a computer. Why keep it idle and pay someone else for their CPU?
E.g. a well-designed deployment (infrastructure-as-code) repository doesn’t need a frontier model to be understood well-enough to create a new job / service using sibling jobs / services as templates.
And this already saves me dozens of minutes per week, although it’s not a 2x multiplier in my efficiency.
They never really get tight very long: the various states are way too busy flooding the world with endless money printing to kick the can of the public debt always further.
Covid financial crash? We went to new highs. 2022 tech flash crash (Meta and Netflix did -75% for example): we then went to new highs.
The only way for governments who ever spend way more than they bring in taxpayer dollars is to de-valuate the currency.
So "financial markets getting tighter": probably won't last.
So you'll probably never see government customers allow that and neither will a lot of commercial customers.
I don't see the risk. If your code is easily AI generated you don't have a moat anyways. A Chinese competitor probably won't have as easy of a time as a US one of you operate in the US
Further, at a lot of companies, the risk has to be acceptable to shareholders and auditors. Perceived risk is often a more powerful motivator than actual risk.
Also, how long do you think openai, Microsoft, Google, anthropic, etc could delay a lawsuit while you pay hundreds of thousands in legal retainers? 5 years? 10?
lmao tell that to the artists, authors and foss contributors whose work has been cloned into the llm oracle
From the perspective of someone currently living in the EU... I'd say thats pretty much a wash (or even slightly tilted in China's favour) for folks outside the US
But also - enterprise accounts already have budget assigned to github actions, and this allows them to start billing right away without having to actually get (or allow) businesses to evaluate the return of having copilot do code reviews.
So seems like it's a mix of immediate incentives and long term architecture. I don't like it, though. If I were an enterprise my first response would be to turn it off.
Hang on, I read this as copilot reviews with bill both actions minutes and AI credits. Did I miss something?
Though actually the more I think about it, I think this change actually does make more sense. In the case of the AI running on GitHub side, that does feel pretty equivalent to CI minutes. I would hope that the number of minutes they bill for is pretty minimal though, since the vast majority of that will be I/O waiting on the agent to return
Done that way it obfuscates cost of the code review and I think that's on purpose
That’s why.
Do they, though? I don't know a single person who uses GitHub who actually likes it. It's far more often something like "it's fine, but I miss (GitLab|Gerrit)" or "I stopped using it for personal stuff and moved to (Codeberg|GitLab)."
The brand recognition among non-technical folks is really the strongest selling point in my eyes. And that's irrelevant to ~95% of software development.
Stopped my recurring subscription at the end of last year when it started spinning up actions for review. Which as a side effect doubled the time (or so) to do a review. Whereas before that I would open a PR, wait at most a minute or two and the review was already done.
I’m blowing through my 1000 mins in days.
Thinking to either pool some free tiers or figure something out with spot instances.
Also is it just me or is CI/CD tooling still sort of rough all around.
Hetzner has cheap VPS that I host my CI on. It costs like $10/month.
Pick the cheapest region, since CI runners location doesn’t matter much.
But I think the issue is that my situation (solo dev, mono repo) is just not right for a dedicated instance.
With only 1-2 runners, the pipeline is slow (low parallelism) and resource constrained. And at least 50% of the time its idle (I'm not working/sleeping).
I guess what I'm really looking for is for some kind of aggressive autoscaling, and aggressive caching.
I tried a couple of things (GHA, Dagger + Hertzner, Buildkite)
And Im just not too sure theres going to be any out of the box solution since my priority is essentially to minimize cost and maximize efficiency. Not really a great customer for any providers.
Im tempted to just get agent to build something out quickly with cloudflare workers + spot instances.
I also have some other nice to have requirements:
- ts/code over config
- locally runnable and testable
- preferably no lock in
- repeatable/reproducible
https://docs.drone.io/server/provider/github/
Very easy to stand up, does just fine. Definitely doesn't have the "library" of prebuilt actions that GHA does, but for the most part... I consider that a plus.
Otherwise it's very similar in concept - define actions in a yaml file, run commands on an image, webhook integration with most repo providers.
I run it on some old hardware locally (k3s cluster on old machines) and it outperforms the 1000 minutes from GHA easily, and costs basically nothing but some maintenance and time.
I've been keeping my eyes open for something new in this space since Harness bought it, though - so if other folks have recommendations I'd be interested in alternatives.
Best decision we ever made
[1] Ed Zitron speculates the actual prices with token based billing for heavy users will be something like 10x the subscription price, but this seems high.
Although I would also point out that OpenAI recently tripled the amount of Codex inference you get per month for £200 (and to head off the suggestion, this is distinct from their current 2x promotion on £100/month plans)
Neither of those is how much it actually costs the company selling the service. And I have feeling they are running at loss here so the play is "get everything possible using LLMs then jack up the pricing"
Inference is cheap but training is quite expensive. Plus all the money they've invested and keep investing on hardware, data centers, etc. And evidently they also need to make a profit at some point.
Maybe from the perspective of traditional, turn-based chat. But when you start having developers command an army of agents that work around the clock, those cheap tokens start adding up fast...
I think the margins have to be a lot higher than that in order to give investors the return they're expecting, to continue the never-ending training treadmill, and to build more and more datacenters to accommodate people basically DDOS'ing the GPUs in order to run their workloads.
Yes, in theory what you said makes sense. But the tightrope these companies have to walk is that the per-token costs still have to be low enough that developers and companies don't just say "ehhh I guess we can still do all this work the old-fashioned way" but ALSO high enough to cover the massive expenses AND astronomical returns everyone's expecting.
If prices go up, I suspect a bunch of folks will jump to cheaper, less capable models instead of eating the added cost. The whole value proposition of AI in enterprise is around cost-cutting, so that mentality is likely to persist when choosing which model to pay for.
There’s a difference between criticizing a company and just swapping names for insults. The former can be useful, the latter just turns the discussion into noise. If you’ve got a point about Copilot or the review feature, make it. Otherwise it’s hard to see what anyone is supposed to take away from “ShitHub” other than childish shit-posting.