Thank you for the reference. Its author defines a "rogue AI" as:
"an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere"
and explains that it would also need to be goal-directed in a way which would be at odds with human wellbeing.
Stipulating all that, what is still missing is an explanation of the mechanism by which an AI, rogue or otherwise, could do harm. How is it supposed to affect the world outside its computing substrate?
Absent humans making available the interfaces and resources to do so, it can't. The referenced article includes an example of a genocidal human doing exactly that, and using an AI as a force multiplier. That, as the trope goes, is a social problem, not a technical problem, and it needs a social solution, not a technical one.
Each of the other examples in the referenced article (military AI going rogue, wireheading, amoral corporate AIs manipulating humans) require AIs interfacing with the physical world outside their computing substrate or with the biosphere. Again, because these scenarios remain dependent on humans making available such interfaces, I fail to see how a hypothesized "rogue" AI could achieve any autonomy to do serious damage.
I see this panic about rogue AIs as well-intentioned but misguided, and perhaps exploited by folks who would like to control / diminish / force licensing of general purpose computing.
> How is it supposed to affect the world outside its computing substrate? Absent humans making available the interfaces and resources to do so, it can't.
True, but what more do you need than the ability to send web requests to arbitrary domains, and receive the responses?
Kinda like ChatGPT hired a person to just pass captchas, or Sydney reading news about it's own actions/interactions and therefore getting info it shouldn't have had, there's a lot of space for going out of the guardrails of not having a proper interface
Too many systems rely on people being unaware of exploits, but an AI would never forget something, or get bored or tired of trying, It doesn't need to be smarter than humans, just have enough persistence and attention to detail
> How is it supposed to affect the world outside its computing substrate? Absent humans making available the interfaces and resources to do so, it can't.
This would be more reassuring if hooking it up to a Python prompt wasn't virtually the first thing people did.
The article explains that too. But it's easy to think of millions of examples if you can hack everyone and persuade people to do stuff for you and you are smarter than them.
"an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere"
and explains that it would also need to be goal-directed in a way which would be at odds with human wellbeing.
Stipulating all that, what is still missing is an explanation of the mechanism by which an AI, rogue or otherwise, could do harm. How is it supposed to affect the world outside its computing substrate?
Absent humans making available the interfaces and resources to do so, it can't. The referenced article includes an example of a genocidal human doing exactly that, and using an AI as a force multiplier. That, as the trope goes, is a social problem, not a technical problem, and it needs a social solution, not a technical one.
Each of the other examples in the referenced article (military AI going rogue, wireheading, amoral corporate AIs manipulating humans) require AIs interfacing with the physical world outside their computing substrate or with the biosphere. Again, because these scenarios remain dependent on humans making available such interfaces, I fail to see how a hypothesized "rogue" AI could achieve any autonomy to do serious damage.
I see this panic about rogue AIs as well-intentioned but misguided, and perhaps exploited by folks who would like to control / diminish / force licensing of general purpose computing.