A way occurred to me to try to dodge the "undecidability of morality" theorem... but I think it runs into new different impossibility of morality theorems. Assume all the killer robots are constructed to have algorithms inside, which provably always halt in polynomial time. (You can easily design a programming language in which every program provably halts in polytime, e.g. since there is a cutoff timer initialized after reading input with some polynomial value.) In that case, the robots' behaviors are always decidable, so their impossibility proof fails. However... assuming robots flip coins inside their algorithms, the probability the program halts is #P-complete to compute or approximate. Hence if P < #P, it again is impossible for a polynomially-bounded entity to be moral. Further, it seems to me that if you construct an N-player game, even if each individual player is extremely simple, then the net expected outcome of the game, can be #P-complete to compute. Proof, the game is an N-input logic circuit and the inputs (players) are coin flips and the probability the output is '1' is #P-complete. Also true for "planar logic circuits." So even in a world containing N incredibly simple entities only, besides yourself, you still cannot make predictions about the world N timesteps ahead (hence cannot make moral choices) if you have less than #P computational power. -- Warren D. Smith http://RangeVoting.org <-- add your endorsement (by clicking "endorse" as 1st step)