When I read the title of the article in my RSS feed my first instinct was to go straight to here with a snarky “How was it not actually AI that did this?” in my head…
I would love to be proven wrong, but if you ignore all the hype pieces and (from what I can tell) a TON of pseudo-scientific shit floating around, while it's possible to make these systems as accurate as possible, they would still be fundamentally unreliable since the systems aren't deterministic. Try asking the usual "is 9.11 greater than 9.9" question, on my last test with 3.7 Sonnet it first said "yes" and then eventually converged to "no". Quite hilarious at times.
Point being, LLMs don't really understand what they're saying; they're just predicting the next word. I'm not trying to downplay that, by the way - it's all very impressive. But it's not intelligent intelligent, and I think we should all be wary of comments like "oh we should just keep throwing more compute and data at it and consciousness/self awareness would eventually emerge". I think that's just fodder to raise/burn VC money (and the planet).
Conciousness/self awareness can emerge if they put more effort into accuracy and reasoning, ie the human touch. I agree with you on most companies thinking that AI can arrive to its own reasoning/conciousness. The hype is getting to their brains, lol.
I fucking hate LinkedIn so much it's unreal. When it comes to that platform, I lose all rationality altogether. Perhaps I need to go to therapy, but not before LinkedIn influencers are all assembled in the town square and publicly executed for their crimes upon humanity.
Over the years, I’ve applied to over a thousand jobs through LinkedIn, ‘earned’ dozens of skill badges; but all I’ve had to show for it was a compromised social security number. LinkedIn has proliferated the scammer.
Meanwhile, Indeed has connected me with real employers who’ve actually interviewed me. Most who started interviewing me had conducted multiple rounds, enough to get to the mature conclusion that it’s not a good fit. I’ve interviewed my way into two great fitting roles over the years via that platform.
> Firstly I would not go back to London unless I had the protection level of a government Minister. My life is not worth any amount of money and violence is out of control.
I feel like there's heavy observation bias here. Maybe you had a bad experience or two, but I've been living in London for the past 4 years and haven't had any such encounter(s) so far. You make it sound like London's some third world warzone; I personally felt that New York, SF, and LA were far more unsafe when I was living there with the amount of homeless people and fentanyl addicts walking around.
A bit OT - how do I work on developing the skill set necessary to find vulnerabilities like these? Should I take some particular courses, or some other “track” of sorts? At the moment, I have an undergraduate in Computer Sciences, and I’d say I’m a fairly OK programmer.
Check out LiveOverflow on YT. Maybe play some CTFs, but don't do that super seriously, just enough to get you hooked on binary exploitation. They're fun, especially if you find some teammates to cooperate with.
And then just, well, practice. A lot of practice. Mostly driven by curiosity about how things work - bugs will then just start to pop up and you are free to investigate whatever piques your interest. The more likely you are to just open up a debugger when a piece of software annoys you and try to binary patch it, the closer you are to being a security researcher :).
There's not much books/courses on this, low-level hacking is something that you kind of just learn as you go. But, for instance, if you never touched gdb/lldb, or never looked at assembly code, or never wrote C - you should investigate that first as base skills.
There is an excellent pre-packaged VM with levels of challenges that take you through the basics of exploitation to quite advanced levels called "Modern Binary Exploitation" [0]. I would highly recommend it.
You can also do the challenges using IDA/Ghidra instead of looking at the source for a proper challenge and I recommend doing this initially for each challenge.
I'd recommend CTF'ing a bit stronger than the other commenter. While there can be a distinct gap between the vulnerabilities in ctfs and real world applications, CTFs provide a great means of deliberate practice (work on a problem, potentially figure it out, and then read other peoples' write-ups after the competition ends).
I didn't meant to discourage from playing CTFs, I just became jaded by seeing the same kind of heap feng shui tasks over and over and over again :). You know, the note-management linked list task with a simple CLI menu. Not to mention the proliferation of 0/1day tasks, which are IMO just lazy.
Do play CTFs. Just pick the fun challenges. pwnable.kr used to have some good stuff if you want to level up.
I think we're on the same page. Once someone gets good enough at heap shenanigans, they likely have a good enough skill baseline to go after real targets. In terms of skill development though, I found ctf'ing gave me a decent sense of what may be exploitable, that it would be hard to get otherwise.
