Before he got famous playing gladiators and mathematicians Russell Crowe played an evil AI program named SID 6.7 (Sadistic, Intelligent, Dangerous) in the 1995 sci-fi movie Virtuosity. In the movie SID is a software agent evolved using advanced genetic algorithms to emulate serial killers. The police use him as part of a sophisticated virtual reality training simulation.
Until things go horribly and predictably awry and SID escapes from the simulation into the real world he can be regarded as simply another computer program - albeit a very advanced one. Once he's out of the computer and committing real-life atrocities he takes on a much more sinister, and arguably evil persona. Throughout the movie SID performs more and more dastardly acts until Denzel Washington eventually kills him at the end.
Despite the cheesiness of the overall movie the idea that a computer program could be evil struck me as an interesting concept. We have evil people, evil organizations, and even evil countries (axis of evil anyone?) but objects are generally not regarded as evil themselves. A gun, for example, can be used to further a malicious goal, but the gun itself is neither good nor bad. It's just a tool. Software, even when it's controlling a physical entity like an automotive assembly plant robot, is also just a tool.
But at what point does software become evil? When did SID cease to be a simple non-player character in a virtual reality game and become evil?
I think the quality "evil", at least in SID's situation, requires three things. The first is intelligence, the second is the motive to harm, and the third the ability to harm. Without all of these qualities SID couldn't have been evil.
For example, if SID had not been artificially intelligent and had simply been blindly executing preprogrammed instructions like a clock then he would have lacked the ability to understand what he was doing. Without that understanding, without the knowledge that what he was doing was wrong, SID would simply not be evil.
Also, if SID had not had the motive to harm others then he also would not have been evil, much as buggy guidance software that causes a fatal plane crash cannot be evil. The same thing goes for careless kids who set fire to a house while playing with matches. Dangerous? Yes. Evil? No. The intent must be there in order for something to take on a malicious quality.
And finally, SID needed the ability to harm others before he could be truly evil. Just as thinking about robbing a store is not wrong until one acts upon those thoughts, SID didn't become evil until he actually began to hurt real people. The non-player characters in the simulations didn't count.
Apparently, others have thought about this as well because today I read that researcher Selmer Bringsjord in New York has been trying to build a software agent that is "evil." Sci-Am has a two page article describing it at Are You Evil? That Which Is Truly Wicked.
To be truly evil, someone must have sought to do harm by planning to commit some morally wrong action with no prompting from others (whether this person successfully executes his or her plan is beside the point). The evil person must have tried to carry out this plan with the hope of "causing considerable harm to others," Bringsjord says. Finally, "and most importantly," he adds, if this evil person were willing to analyze his or her reasons for wanting to commit this morally wrong action, these reasons would either prove to be incoherent, or they would reveal that the evil person knew he or she was doing something wrong and regarded the harm caused as a good thing.
My definition of an evil software agent and Bringsjord's general definition are generally the same. The one difference is that I don't make a distinction between prompted evil and self-concocted evil. Who cares if you thought of the idea yourself or copied someone else? If you did it, understood what you were doing, and wanted to do it, then it's evil.
In any case, the research is a fascinating exploration of the boundaries between Philosophy and Artificial Intelligence. Reminds me a little of Kenneth Colby's "Parry" program.
