bobburger

Llamafile is a great way to get use an LLM locally. Inference is incredibly fast on my ARM macbook and rtx 4060ti, its okay on my Intel laptop running Ubuntu.

Llamafile runs entirely on your machine. The largest one I can run locally is Mistral-7B and Wizardcoder 13B. They seem to be on par with chatgpt-3, but that’s okay for my purposes.

I use it for exactly the same thing.

I used to spend hours agonizing over documenting things because I couldn’t get the tone right, or in over explained, or some other stupid shit.

Now I give my llamafile the code, it gives me a reasonable set of documentation, I edit the documentation because the LLM isn’t perfect, and I’m done in 10 minutes.

A simpler answer might be llamafile if you’re using Mac or Linux.

If you’re on windows you’re limited to some smaller LLMs without some work. In my experience the smaller LLMs are still pretty good as chat bots so they might translate well.

🤨

I like to imagine this was thought up by some ambitious product manager who enthusiastically pitched this idea during their first week on the job.

Then they carefully and meticulously implemented their plan over 3 years, always promising the executives it would be a huge pay off. Then the product manager saw the writing on the wall that this project was gonna fail. Then they bailed while they could and got a better position at a different company.

The new product manager overseeing this project didn’t care about it at all. New PM said fuck it and shipped the exploit before it was ready so the team could focus their work on a new project that would make new PM look good.

The new project will be ready in just 6-12 months, and it is totally going to disrupt the industry!

Lol, yeah I’m going to organize an armed rebellion against a country that can shoot me in the chest with a fucking rocket covered in swords from the other side of the world while I’m chilling on my deck with my family.