It’s actually really useful for web devs to have access to a local model. Whether or not browsers should bundle their own rather than using the system-provided model(s) is up for debate, however. For the time being, though, Google does have some of the better small ones.
Furthermore, users aren’t going to want to have to wait for an extra thing to download before their web apps can use AI.
That’s the thing… Without context of why, users probably wouldn’t want a 4 GB download. But they do want their web apps to work properly. When there’s a specific use case they’re interested in, they will want to have it, and they won’t want to wait.
You haven't even tried to provide a hypothetical example of what a web app should try to do using a local LLM, nor addressed the obvious questions about how that kind of thing should be standardized, what level of local LLM capability is reasonable for a web app to expect, or how permissions for that should be managed given that a local LLM is not just a tax on local storage capacity.
So why should anyone take it as a foregone conclusion that this is an instance where web devs should get what they want? In general, the browser should be acting in the best interests of the user and not automatically granting the wishes of every web site that wants to drain your battery.
One example I have that made me excited for this feature is the free recipe manager website I run.
Many of the paid-for competitors give users the ability to import unstructured recipe data these days from sites like instagram or at least text-only websites.
I can't afford to offer this as a feature since my website has no advertising and I just pay for it out of pocket, but it's an incredibly easy feature to add if you have the money to pay for tokens.
If I could use a local llm to do it though that runs in the person's own browser then I think it would definitely be valuable.
That said, I'm not sure the state of local llms provides a good enough experience yet (small models and slow) but that doesn't mean that in the future it might not be useful.
The propsosed apis do work for this purpose, albeit more slowly and lower quality
> not automatically granting the wishes of every web site that wants to drain your battery.
Pretty sure that ship sailed way back when Flash ruled the Internet, and it's still sailing more than ever today.
Browsers are just weird sandboxed VMs now. They have nothing to do with their original purpose. Don't be mad at me, I like shipping webapps that render documents server-side and use even JS incredibly sparingly. I'm just reporting what I see. The browser exists as a way to make developing completely proprietary apps with proprietary UIs for several platforms cheaper, and Chromium exists to help further that goal including, if necessary, being packaged up and shipped with those apps (Electron).
Google's marketing for their latest new browser feature nobody asked for shouldn't be taken at face value. Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
> Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
I'm sure that exists already, I've personally been waiting for some version of this to make it into browsers since like GPT 3.5. Every day on HN there is conversation about the tradeoffs of local vs. hosted models; the uses this API is intended for are perfectly within the capabilities of local models.
> And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
Most of this is answered under the tag "Intent to Experiment" and the associated link. It's not a mandate that they're forcing on the web today, it's a public experiment intended in part to solicit feedback for a potential spec: https://groups.google.com/a/chromium.org/g/blink-dev/c/6uBwi...
> Most of this is answered under the tag "Intent to Experiment" and the associated link. It's not a mandate that they're forcing on the web today, it's a public experiment intended in part to solicit feedback for a potential spec: https://groups.google.com/a/chromium.org/g/blink-dev/c/6uBwi...
That does pretty much settle the question: if it's still just an experiment, Google should be asking for consent and web developers should not be assuming that this is the future of browsers, and nobody should be acting like this is a foregone conclusion.
I for one run a small Scripture-study web app that makes use of this on Chrome browsers when available to provide summarizations of long commentary articles. I'm also looking to use it to power topical search.
I allow free open access to the content, as blocking it behind an account signup doesn't sit right ethically—it should be open and free for everyone.
The issue there is that there's no way to easily secure the API from being hammered by bad actors. (Due to the often-controversial response many have to Scripture, apps like these draw special kinds of negative attention.) You can set rate limits, but people can still abuse those, just to try to burn your money. I can get by for free (or relatively cheap, fully paid out-of-pocket as none of this is monetized) on Vercel/Netlify/etc, but inference is expensive, and a prime target for those trying to cause trouble.
If in the future the web exposes local "foundation" models that web apps can assume are present, that would open up great possibilities like these for indie devs like myself. Being able to offer useful and compelling features without worrying about abuse would be nice. That's my point.
If it's to allow the web apps I use to work with more privacy, or to enable smaller/indie players (that can't easily afford to burn a bunch of API tokens for every user) to offer some basic AI-based features in their web apps, then I'm all for that.
This is the whole appeal of Apple's Foundation models on iOS too.
You reminded me that I still find it interesting that no one ever copied meta-moderating. Even at reddit, we were all Slashdot users previously. We considered it, but never really did it. At the time our argument was that it was too complicated for most users.
Definitely sounds like clickbait, but this is a genuine problem I've been facing on my more complex projects. I was trying to work around it by making lots of .md files that contain context for various components of the app, as well as ones that "statefully" represent our steps along larger refactors, allowing things to persist across sessions. But that requires the hygiene/discipline of remembering to have the model update those.
Now… how much of our tiny context window does this eat up? And what if we're working on something very different from the previous tasks; might that irrelevant context risk confusing the model?
Also, how does this work if I work on multiple projects across different directories? Will it know to not get those mixed up?
>When FileVault is enabled, the data volume is locked and unavailable during and after booting, until an account has been authenticated using a password. The macOS version of OpenSSH stores all of its configuration files, both system-wide and per-account, in the data volume. Therefore, the usually configured authentication methods and shell access are not available during this time. However, when Remote Login is enabled, it is possible to perform password authentication using SSH even in this situation. This can be used to unlock the data volume remotely over the network. However, it does not immediately permit an SSH session. Instead, once the data volume has been unlocked using this method, macOS will disconnect SSH briefly while it completes mounting the data volume and starting the remaining services dependent on it. Thereafter, SSH (and other enabled services) are fully available.
Heh, reminds me of those boxes Sun used to make that only ran Java. (I don’t know how far down Java actually went; perhaps it was Solaris for the lower layers now that I think about it…)
With hypervisors and a Linux kernel doing the heavy lifting, the WASM on bare metal probably just looks a lot like a regular process. I would bet Sun did similar … minus the hypervisor.
I do miss the Solaris 10/OpenSolaris tech though. I don’t know anything that comes close to it today.
Technically, yes. I built+ported a majority of Debian packages onto Nexenta OS but that effort (and many parallel efforts) just vanished when Oracle purchased Sun. I don't miss SVR4 packages and I never grew fond of IPS. So many open-source folk warned of the dangers of CDDL and they were largely realized in fairly short time. Unsurprisingly, #opensolaris on irc also had a precipitous drop-off around the same time.
dtrace/zones/smf/zfs/iscsi/... and the integration between them all was top notch. One could create a zone, spin up a clone, do some computation, trash the filesystem and then just throw the clone away... in very short time. Also, that whole loop happened without interacting with zfs directly; I know that some of these things have been ported but the ports miss the integration.
eg: zfs on Linux is just a filesystem. zfs on Solaris was the base of a bunch of technology. smf tied much of it together.
eg: dtrace gave you access all the way down to individual read/write operations per disk in a raid-z and all the way up to the top of your application running inside of a zone. One tool with massive reach and very little overhead.
Not much compels me to go back to the ecosystem; I've been burned once already.
I think it was far less special that advertized, so it was probably a stripped Solaris that ran a JRE hoping noone would notice. Dog slow they were at least so from my viewpoint, there was nothing magic about those boxes at all.
Furthermore, users aren’t going to want to have to wait for an extra thing to download before their web apps can use AI.
That’s the thing… Without context of why, users probably wouldn’t want a 4 GB download. But they do want their web apps to work properly. When there’s a specific use case they’re interested in, they will want to have it, and they won’t want to wait.