• 0 Posts
  • 132 Comments
Joined 3 years ago
cake
Cake day: March 22nd, 2024

help-circle








  • I thought this would be nonsense at first, but the methodology is interesting. It seems plausible. And they tested quite a good spread of open weights, like MiMo and LongCat (which usually fly under the radar), all the way back to Yi 34B.

    They missed Russia’s LLM, but that’s okay; it’s very obscure.

    It’s also “lucky” they found this correlation, and so many recent weights happened to be initialized with PyTorch, on the GPU: very specific conditions.

    …Also, I think the FP4 precision is a dead giveaway too, as Hopper wouldn’t be able to train that effectively, right? I assumed they were using at least some smuggled B200s.


    And the export restrictions were stupid anyway.






  • You mean over the export restrictions? Those were always meant to protect Nvidia, but were both ineffective and ended up doing the opposite, by encouraging the development of Chinese chips (as Jensen tried to warn them of).

    So… honestly, that isn’t a big deal.

    What’s worse is the potential conflict of interest sleeping with an Nvidia executive.