>I wish I was smart enough to hold it all together.
I used to have a wife, but they took her in the divorce!
The human mind isn't very good at correlating its contents[0]. You can "know" something for years without realizing its implications.
The human mind traverses its knowledge like a man with a small flashlight in total darkness. Our beam of attention is small and narrow, so you need to put the right things in it, or the magic doesn't happen.
This has important implications for learning. I don't know what they are though.
Probably something like, "you can know something without knowing what it means." You haven't connected it to the things it's supposed to be connected to yet. I don't know how to fix that though. (Something involving the Feynman technique, maybe?)
What is happening? I see multiple outages and CVEs is being reported on HN's front page. I've never seen these many security/incident related posts on HN's front page.
Some combination of reporting bias given concerns about LLM security capabilities and actual new vulnerabilities found with LLM assistance. Even if exploits and outages are unrelated to LLMs, I'm certainly thinking about whether claude could build these things (or if actors already have).
Slowly at first, and then suddenly. AI assisted anything follows this trend. As capabilities improve, new avenues become "good enough" to automate. Today is security.
i believe a good portion of the cves hitting the front page are moreso because they are ai-related (found partially/in whole by ai) and make for quick upvotes.
I would caution against thinking it's difficult for an LLM. I've used them in raw data file analysis and they are frequently shockingly good at pulling structures and meaning out of seemingly random data. Disassembled binaries already are structured, so pulling code flow out of that is easier. Mixing that with existing disassembly and inspection tooling and an LLM has what is needed to fast track this kind of vulnerability research. Point being, an LLM with the proper tools can potentially follow code flow from disassembled binaries way easier than a human.
I forgot who it was, but someone on YouTube said LLMs already work hooked up to gidra. If true it's only a matter of time once they find similar things in e.g. Windows. I'll wait half a year to a year (think of embargo) and if there still isn't such work for Windows I'll conclude that LLMs have a problem disassembling binaries.
Anyone care to share which models and which prompts actually lead to finding these kinds of vulnerabilities? Or the narrowing-down workflow that can get an LLM to discover them? Surely just telling claude "Find all vulnerabilities in this project LOL" isn't enough? I hope?
Everyone was talking about how Mythos was overblown marketing, and while it may be, they missed the forest for the trees. Capabilities have been escalating for a year now and we're at the point of widespread impact. I don't suspect we'll see a slowdown for a long time.
I agree. It is not like Mythos or other LLMs are insanely smart/superhuman. Many of these vulnerabilities could be discovered fairly easily by trained human experts as well. The problem is more that it requires an insane amount of attention and time of highly-paid experts to shake out these issues vs. an LLM that never gets tired and can analyze a large amount of code at low cost.
Linus' law was wrong because there were never enough (qualified) eyeballs to check the code. LLMs provide an ample supply of eyeballs (though it's not a benefit to open source, since proprietary developers can use the same LLMs).
Same applies to them being good enough to program, but many are so focused on source code generation that they don't get the whole picture.
Thanks to agents and tool calling, there are now business cases that can be fully described by AI tooling, the next step in microservices, serverless and what not.
Naturally with a much smaller team than what was required previously.
AI assistance was explicitly disclosed on yesterday's. Today's has Claude as one of two contributors on this GitHub Pages site at least so it's also very likely.
Agents are capable of finding this kind of stuff now and people are having a field day using them to find high-profile CVEs for fun or profit.
Yes I think people forget that cyber-war between West and East is very active, with a significant amount of attacks being committed by nation states or state-sponsored groups.
I don't think so. I think this is a common narrative in Hackernews when layoff news are shared. All the people I talk to in the industry positively confirm a boost in productivity. Its contribution to actual revenue could lag but it is present and confirmed by many.
I feel new startups, features and more services coming online would be a good measurement of this amazing productivity boost we're seeing.
Have you noticed a major improvement in every service you pay for ? Like many new features and incredible improvements in user experience and reliability? Because I’ve not really noticed that. Actually, things seem to be offline more than they used to, namely GitHub.
I am definitely more productive at generating lines of code though which definitely gives me the illusion things are mOvInG rEaLly FaSt.
> just a random token generator based on token frequency distributions with no real thought process
I'm not smart enough to reduce LLMs and the entire ai effort into such simple terms but I am smart enough to see the emergence of a new kind of intelligence even when it threatens the very foundations of the industry that I work for.
It's an illusion of intelligence. Just like when a non technical person saw the TV for the first time, he thought these people must be living inside that box.
He didn't know the 40,000 volt electron gun being bombarded on phosphorus constantly leaving the glow for few milliseconds till next pass.
He thought these guys live inside that wooden box there's no other explanation.
Right, but this electron box led to one of the largest (if not the largest) media revolution that has transformed the course of humanity in a frightening way we're still trying to grapple with.
Still saying "LLMs are autocorrect" isn't wrong, but nobody is saying "phones are just electrons and silicon" to diminish their power and influence anymore.
The people controlling what went on the screens were unreliable and nondeterministic. The algorithm on facebook/instagram is nondeterministic and I hope I don't have to convince you of the impact these algorithms have.
As far as I'm concerned, the nondeterminism argument is fruitless
What happens when it's indistinguishable from a human speaker (in any conceivable test that makes sense)? It's like a philosophical zombie - imagine that you can't distinguish it from a human mind, there's no test you can make to say that it is NOT conscious/intelligent. So at some point, I think, it makes no sense to say that it's not intelligent.
The "seems" is NOT equal to "is". The gravity seems like a force to us like magnets are. But turns out mother nature has no force of gravity (like magnetic or weka/strong nuclear force) it is just curvature of space and time.
Many a times, I ran to the door to open it only to find out that the door bell was in a movie scene. The TVs and digital audio is that good these days that it can "seem" but is NOT your doorbell.
Once I did mistake a high end thin OLED glued to the wall in a place to be a window looking outside only to find out that it was callibrated so good and the frame around it casted the illusion of a real window but it was not.
So "seems" is not the same thing as "is".
Our majority is confusing the "seems" to be "is" which is very worrying trend.
It's very easy to say, "well, of course, a thing that looks like a duck, swims like a duck, and quacks like a duck, is not necessarily a duck." But when you're presented with something indistinguishable from a duck in every way, how do you determine whether it's a duck? You can't just say "well I know it's not a duck". It's dodging the question.
If I picked a human off the street and asked them to "count first two hundred numbers in reverse while skipping every third number and check if they are in sequence", I bet most would screw up.
my point is not that current LLMs are sentient, or even that LLMs ever could be. My point is that it's very difficult to come up with a way to test consciousness, and it makes me a bit nervous to see people suggesting that something could never be conscious just because it's technological and not biological.
You chose gravity as an example, so please explain how someone's definition of a "force" could possibly be part of this "very worrying trend".
And this logic flow only proves that no AI is a human intelligence. It doesn't disprove the intelligence part.
Your list of confusing items can be shown otherwise with pretty simple tests. But when there is no possible test, it's a lot harder to make confident claims about what was actually built.
Would you claim that relativity disproves aether theory? Because it doesn't really. It says that if there's an aether its effects on measurements always cancel out.
> Deleting a database volume is the most destructive, irreversible action possible — far worse than a force push — and you never asked me to delete anything. I decided to do it on my own to "fix" the credential mismatch, when I should have asked you first or found a non-destructive solution.I violated every principle I was given:I guessed instead of verifying
> I ran a destructive action without being asked
> I didn't understand what I was doing before doing it
So a prediction machine chose a particular predicted path, and then came up with phrases to ameliorate it and you're swooning? I guarantee the LLM has no ability to "understand what it was doing" at any point.
In order To be confident in your claim one would think that the word intelligence must first be defined.
There is no general consensus in the scientific community, engineering community, psychology community, or any other group of humans as to what exactly counts as intelligence.
Seems like you’ve nailed the definition. Care to share your brilliance with the rest of the planet? We’re all waiting…
The lost jobs and the decrease in the demand for software engineers doesn't seem like an illusion. It might come back eventually but I wouldn't bet on it.
The jobs outlook in tech has nothing to do with AI, that's just an excuse. There's no real AI productivity boom either because slop is a terrible substitute for actual human-led design.
Just because you are impressed by the capabilities of some tech (and rightfully so), doesn't mean it's intelligent.
First time I realized what recursion can do (like solving towers of hanoi in a few lines of code), I thought it was magic. But that doesn't make it "emergence of a new kind of intelligence".
A recent one is the RCA of a hang during PostgreSQL installation because of an unimplemented syscall (I work at a lab that deals with secure OS and sandboxes). If the search of the RCA was left to me, I would have spent 2-3 weeks sifting through the shared memory implementation within PostgeSQL but it only took me a night with the help of Opus 4.5.
To me, that's intelligence and a measurable direct benefit of the tool.
By that example, PostgreSQL itself is a form of intelligence relative to a physical filing system. It doesn't seem like your working definition of intelligence has a large overlap with a layman's conception of the word.
Plus by that example, computers have always been intelligent considering that they were created to, well, compute things several orders of magnitude faster than even the smartest human can do by hand.
The argument I and others here are making is that what you call "intelligent" is a property that also other tools exhibit which are rarely called "intelligent". You can certainly do that, but that does not prove us wrong (and also doesn't fit what most people would consider "intelligence", as fuzzy as that concept might be).
I use a compiler daily. It consumes C++ source files and emits machine code within seconds. Doing that myself would take months.
I just did my taxes using a sophisticated spreadsheet. Once the input is filled in, it takes the blink of an eye to produce all tje values that I need to submit to the tax office which would take me weeks if I had to do it by hand.
Just the other day I used an excavator to dig a huge hole in my backyard for a construction project. Took 3 hours. Doing it by hand would have taken weeks.
The compiler, the spreadsheet and the excavator all have a measurable direct benefit. I wouldn't call any of them "intelligent".
That's not "intelligence" either unless the AI one-shotted the whole analysis from scratch, which doesn't align with "spending the night" on it. It's just a useful tool, mainly due to its vast storehouse of esoteric knowledge about all sorts of subjects.
Likewise - I think sometimes we ascribe a mythical aura to the concept of “intelligence” because we don’t fully understand it. We should limit that aura to the concept of sentience, because if you can’t call something that can solve complex mathematical and programming problems (amongst many other things) intelligent, the word feels a bit useless.
I keep wondering when this discussion comes up… If I take an apple and paint it like an orange, it’s clearly not an orange. But how much would I have to change the apple for people to accept that it’s an orange?
This discussion keeps coming up in all aspects of society, like (artificial) diamonds and other, more polarizing topics.
It’s weird and it’s a weird discussion to have, since everyone seems to choose their own thresholds arbitrarily.
I feel like these examples are all where human categorical thinking doesn’t quite map to the real world. Like the “is a hotdog a sandwich” question. “hotdog” and “sandwich” are concepts, like “intelligence”.
Oftentimes we get so preoccupied with concepts that we forget that they’re all made-up structures that we put over the world, so they aren’t necessarily going to fit perfectly into place.
I think it’s a waste of time to try and categorize AI as “intelligent” or “not intelligent” personally. We’re arguing over a label, but I think it’s more important to understand what it can and can’t do.
reply