> What concerns me is that we start hooking these things up to systems that allow them to do actual harm.

This already happened long, long ago; notable example:

https://wikipedia.org/wiki/Dead_Hand