WYGIWYG

  • 0 Posts
  • 34 Comments
Joined 2 years ago
cake
Cake day: September 24th, 2024

help-circle









  • You can host a small LLM model with a phone, or a whisper or a TTS, maybe some light diffusion.

    You cannot do tool use, reasonable coding, get non-hallucination answers on even moderate questions. That stuff starts to get viable around 24G and really needs closer to 200G to be able to do most of that they’re doing.

    the reason they bought up all the ram is to keep someone from buying 200G of vram and diying cards putting them out of business.









  • To an extent. Perf is often as much about architecture as it is about code. That guy who created Adobe alternatives- all those apps are dog slow; they have to do low-quality preview renders. That’s something Adobe probably spent a few lifetimes siphoning out every last watt of power from, and it’s the reason they didn’t want to move to Linux. The app is still serviceable, but it has to degrade active quality to work at a reasonable pace. To fix that in Rust is going to be an enormous ask. There needs to be hackery and jiggery-pokery, the kind of things that aren’t documented well in training.

    Maybe he can go back and have it try to figure out what Adobe is going or brainstorm ways to do it at full quality. Maybe someone will have to dive into the code, get familiar with it (hoping it’s not too confusing), and either fix the architecture, fuck with the security to let it use more base system calls, or maybe just tell another Claude session how to make it more performant.