On local models that cost power (post initial hardware cost), makes sense. My work is building this out and I think it's solid. But until we can use our own hardware and local models the long term cost is a big question mark.
With the right harness/agent/skill setup and requirements work, this (any specificity) can become part of the workflow from early stages.
Setting that up and making sure it works, at the early stages, is not what LLMs do best, today. This sort of work is possibly incredibly valuable in the next 12-36 months, depending on what LLMs can be designed to do out of the box.
An Agent that can deliver the correct legal review process for a patent and is correct often enough to validate the savings vs real humans, or at least the speed increase + manual review ala code reviews, is incredibly valuable.
Are we there yet? Maybe this is where current groups like FNCR are trying to go.
The Macbook Neo is about 500 nits but I'd bet the Asus laptop was more than $600 base.
Looks like the Framework laptops depend on model/screen, between 4-500 but with the new 13 pro hitting 700 nits. For a user replaceable screen, and backwards compatibility (I think), that's pretty solid.
I think only 32bit apps install there. Ideally there is a 64bit version that continues forward. This is mostly an issue for me with enterprise/etc software I support at work. A key system I just moved to Server 2022 (on prem and Azure VMs) is 32-bit and still uses 32bit ODBC. It's a great app for our need. Just, still 32bit...
The intent is only 32 bit apps get installed there, but there are a few problems in practice. Dunno why you're currently downvoted about it though.
The first is needing to know whether the app was 32 bit or not is sort of the main annoyance with that itself.
The second is not every app follows this rule correctly, for whatever reason.
The third is there isn't a clear path for mixed apps, e.g. Steam. On Windows, Steam is still a mix of 32 bit and 64 bit components, so there isn't really a "clean place for it to go. You could have one option of putting Steam in one or the other and then mixing 32/64 bit apps in that folder & you have the other option of duplicating things, moving the initial problem an extra directory level deeper. 3b is that Steam installs the games under its folder, and the games you can install can be 32 bit, 64 bit, or a mix - duplicating the problem yet again. Until the start of this year, Steam still supported 32 bit Windows and, therefore, you could also have 16 bit games be installed.
There were reasons to do the split in the early 2000s but holding on to each decision like this for decades after seems to cause more pain than it ends up avoiding.
Abstract/TLDR: LLMs are notoriously formulaic at writing, overusing certain tokens or phrases. I show that models trained with SFT fail to match the distribution of the training data by using Maximum Mean Discrepancy (MMD), Judge Model Quality (JMQ), and L2 Token Distribution.
That is like saying my new restaurant was a success, therefore powering it with a generator was better than connecting it to the grid.
The raw infra being local didn't enable any of that. Now if was building ASICs at TMSC that would a different thing because you'd then be using something different locally.
Idk if this turns into revenue or some financial metric but even if it does and it was a good outcome for author, it still says nothing of risk. What if he loses his timing opportunity / gets beat to market because he's unnecessarily futzing around with hardware? AI is rapidly advancing and he spent 2 years on this to save what was probably <2 months of faang income. There's multiple other angles I could dissect this from a risk perspective. I'm all for taking risks, but at least acknowledge them and preferably measure them as part of making big decisions like this to save a little bit of cash.
Let’s be clear, though, FAANG (as someone who has spent an awful lot of my life working at FAANG) was pays well but crushes your soul. There was a time, a very long time ago, where it didn’t, and there are the soulless soul crushers that love it there, but I would rather futz around with a mid range cars worth of hardware and be happy than spend a moment longer prostituting my soul for their money.
There was a time in this industry that it paid about as well as an accountant and people did it because they loved what they did. Then the money flooded in, a bunch of people switched majors from business to CS, washed out in industry, got their MBA, and became product managers and engineering managers and sucked all joy from it. God bless those that find that joy again.
> would rather futz around with a mid range cars worth of hardware and be happy than spend a moment longer prostituting my soul for their money.
So only 2 options in this profession are 1) sell your soul to 1 of 5 evil corporations that just so happen to also pay excessively well or 2) choose to be unemployed for years while spending a significant amount of money on hardware trying to turn a hobby into a business
Also by your reasoning, these GPUs are blood diamonds and the authors future product/business should warrant preemptive boycott by all the perfect people like you
They have a permit to destroy the environment and poison the water, which has more defense in the comments here than rejection of said destruction and poisoning.
reply