20 years on AWS and never not my job

(daemonology.net)

126 points | by cperciva 3 hours ago

12 comments

  • gobdovan 1 hour ago
    The author calls it a 'joke' that Heroes are just unpaid Amazon employees, but reality doesn't become a joke just because it's funny. The asymmetry here is staggering. I find myself holding back private research because I don't want to provide free R&D for a value-extraction machine that is already efficient enough.

    The author was at least dependency-driven in their contribution, but outside that kind of dependency, it's hard to justify contributing even 'in the open' when the relationship is this one-sided. Amazon in particular has done enormous damage to the economic assumptions that permissive open source once relied on. There's increasingly more projects adopting 'Business Source Licenses', precisely to prevent open work from becoming a free input into hyperscaler monetization.

    These devs know Amazon is grabby and, at some point, the only dominant outcome their community contribution is upstream of is unpaid labor for a trillion-dollar entity that also diverts support and community engagement away from the original projects by funneling users into managed versions of the same software.

    • queenkjuul 28 minutes ago
      I'm "lucky" to not be smart enough or important enough to think about this. Regardless, i wholeheartedly agree -- at this point, anything i personally could release publicly, will either be fully open source, or completely private. And I'm only choosing open source if I'm relatively sure it's not gonna make some asshole tons of money.
  • few 10 minutes ago
    > In April 2024 I confided in an Amazonian that I was "not really doing a good job of owning FreeBSD/EC2 right now" and asked if he could find some funding to support my work, on the theory that at a certain point time and dollars are fungible

    >I received sponsorship from Amazon via GitHub Sponsors for 10 hours per week for a year

    For whatever reason, I remember being shocked that you were only charging $300/hr [1] which was what a mere L6 google engineer would make salaried. I hope they are paying you more nowadays

    [1] https://news.ycombinator.com/item?id=30188512

  • bob1029 39 minutes ago
    I strongly disagree with the part about IAM roles for EC2

    > a useful improvement (especially given the urgency after the Capital One breach) but in my view just a mitigation of one particular exploit path rather than addressing the fundamental problem that credentials were being exposed via an interface which was entirely unsuitable for that purpose.

    What alternative interface does the author propose we use to securely exchange credentials? The only other approaches I can come up with involve allowing monkey hands to come into direct contact with secret materials. Outlook, slack and teams cannot possibly be more secure than IMDSv2. I think if you are manually passing around things like PFX files you've already lost the game.

    The entire point of the IAM roles is to make everything a matter of policy rather than procedure. The difference here is insane when you play through all of the edges. IAM policy management is significantly easier to lock down than the alternative paths. I can prove to an auditor in 5 minutes that it is mathematically impossible for a member of my team to even see the signing keys we use for certain vendors without triggering alerts to other administrators. I've got KMS signing keys that I cannot delete with my root account because I applied inappropriate policies at creation time. This stuff can be very powerful when used well. Azure has a similar idea that makes accessing things like mssql servers way less messy.

  • dchest 27 minutes ago
    Colin, if I remember correctly, you first ran Tarsnap servers on Ubuntu before you made FreeBSD work on EC2. At what point were you confident enough to switch to FreeBSD?
  • ysleepy 1 hour ago
    I remember many of these events as I was running FreeBSD a lot and subscribed to the mailing lists.

    Why on earth would you give this monstrosity of a company so much free labour?

    I get that volunteering is fun, but donating your time and competence to a hyper capitalist company is short sighted. I hope there was appropriate compensation, and I'm not including "early access".

  • latentframe 1 hour ago
    Interesting how this history is about the edge cases and the unlikely risks that turn into real incidents. the systems scale faster than what we think about their safety.
  • daemonologist 22 minutes ago
    Good domain name.
  • wahnfrieden 2 hours ago
    He gave them so much free labor
    • iceman28 1 hour ago
      It was a different time when software was seen as something that was built together and everyone was interested in learning the best from one another.
    • johng 2 hours ago
      This... they really owe him something, IMHO. Hell, discounted service so he can make a better margin on Tarsnap sounds good to me!
  • mlhpdx 2 hours ago
    I dug up my original AWS account confirmation email from 2006 a while (years) back. Now I need to go find it again to see if I was earlier.
  • villgax 1 hour ago
    That attested EC2 instance rollout after ~2 decades was a nice joke LOL
  • guardiangod 2 hours ago
    I just want to contrast this article on AWS to its Azure counterpart- https://news.ycombinator.com/item?id=47616242.

    2 companies have functionally similar products, but behaves completely different. One company makes technical decisions with security as the fundamental principal, while for the other company, security is not a consideration.

    • jiggawatts 1 hour ago
      That’s an unfair characterisation!

      Azure engineers absolutely considered security.

      They just chose other priorities: growth at any cost to catch up with AWS.

  • tryauuum 1 hour ago
    20 years of giving love to a soulless corporation