> I asked the business owner a question about a business task. He sent me a ChatGPT screenshot with the answer. I replied that it had nothing to do with my question and everything there was wrong. A minute later he sent me another ChatGPT screenshot.
If this happens to me, it's a sign that they don't want to talk to me and I'm going to be let go.
> Nobody cracks open a programming book anymore
Not true for me. I still read the "Learning Rust in a month of lunches" although I ask AI to write Rust code all the times.
> It's not surprising that an LLM would produce different specs for the same work on different runs
This is what I don't understand: AI is a computer program with its own data. If we give the same input to that computer program every time, why does it produce different outputs every time? Or does the input include LLM data + our prompt + some random data that computer program picks from its Internet search?
LLMs have a temperature parameter. At zero temperature they are deterministic: they always choose the most likely next token at each step based on what came before and the model weights, and they will always generate the same output given the same input.
As you raise the temperature they will start (pseudo)randomly choosing tokens other than the single most likely token (though that one will still be the most likely to be chosen). It turns out this is almost always better than zero temperature, which has a tendency to get caught in repetitive loops. I imagine all the frontier labs have spent thousands (millions?) of CPU hours tuning the temperature parameters on their models for optimal performance.
> LLMs have a temperature parameter. At zero temperature they are deterministic: they always choose the most likely next token at each step based on what came before and the model weights, and they will always generate the same output given the same input.
"A value proportional to the reciprocal of β is sometimes referred to as the temperature: β = 1/kT, where k is typically 1 or the Boltzmann constant and T is the temperature. A higher temperature results in a more uniform output distribution (i.e. with higher entropy; it is "more random"), while a lower temperature results in a sharper output distribution, with one value dominating."
"Temperature" in the context of softmax does not change a "winning" token, it changes how much probable (in the sense of softmax distribution) winning token will be. If the winning token is "New York", it will be a winner with temperature close to 0 and with temperature of 1e9.
The actual selection of the random token is done separately by using inputs outside of the softmax distribution, for example, by using random number generator. I believe most of LLM configs have a seed for the random number generator.
More than that, generation of code in most programming languages is done with the more guardrails such as beam search guided by schema, syntax and semantics.
The thing that impresses me most is that the author knows everything (from the high level architecture to the small details) of "multi-Paxos consensus engine" (I have no idea what it is, but it must be very complicated) and can write everything out for AI to read (or did he/she use an app to convert speech to text)?
So, the complaint is that Linux (Fedora + KDE Plasma) is unstable after a week of usage and daily update? I'm not sure about Fedora as I'm using Arch (also with KDE Plasma on wayland). Everything is very stable for me. I `paru -Syu` whenever I remember or when VS Code shows at the lower corner that there's a new version.
Before reading that post, I would use sqlite/duckdb for everything related to data. After reading that post, I will use sqlite/duckdb + Rust for everything related to data :-)
Power companies will charge what they can, and to be fair most of their costs aren't generation, the guy who fixed that HV line a block over when the power went out during a winter storm? He doesn't work for free. And somebody paid for all those huge metal pylons or, if there aren't any where you live, the even more expensive underground cables.
But, the other practical effect is that if you use less fossil fuels you're making the climate worse more slowly. Now, given we'd like it to stop getting worse just making it worse more slowly isn't the whole answer but it does at least help.
If this happens to me, it's a sign that they don't want to talk to me and I'm going to be let go.
reply