Model Training as Code

(aleph-alpha.com)

21 points | by peterBlue75 3 days ago

3 comments

  • delichon 2 hours ago
    Some good stuff here from Dwarkesh around mashing up training and inference:

    https://youtu.be/20p5-kQXF_Q?is=72ImTNxkOEKmOXQ9

    He predicts this kind of model factory will become central to organizational learning and operations. Updating and upgrading the model stack becomes the core staff function.

    • jaggederest 1 hour ago
      I think this is an interesting thing that will happen once the rate of change slows down a little bit - imagine a world where there's more or less a couple base models and everyone trains on top of them, and the bitter lesson is defunct just via sheer physics (maybe we have the best models we can physically run in reasonable energy density substrates, or something), then it becomes "your personal model" with your overlay, training, or feedback on top.
  • SpyCoder77 2 hours ago
    What is this "aleph" thing in names now? First aleph neuro, and now aleph alpha.
    • verelo 2 hours ago
      I'm glad you're asking because I've seen it too and don't get it either. I assumed initially it was alpha as a typo, then I Googled it and got even more confused.
      • boothby 1 hour ago
        First letter of the Hebrew alphabet, used by mathematicians to denote infinities.
        • verelo 1 hour ago
          That's what Google told me, but i still don't see how it links to this?
  • random3 2 hours ago
    > TL;DR: Model training has grown complex

    So they’ve built Savanah - a workflow engine because the existing zoo of hundreds of workflow engines didn’t cut it :)