A serious problem in the Turing test — a test posited by the famed British mathematician and computer scientist Alan Turing which, if passed, would prove that machines could think — is exposed in a study conducted by Prof. Kevin Warwick and Dr. Huma Shah of Coventry University, UK.
Turing test: one judge-two hidden interlocutors. Image credit: Kevin Warwick & Huma Shah, Coventry University / Christopher Chapman, University of Reading.
The Turing test assesses a machine’s ability to exhibit intelligent behavior indistinguishable from that of a human.
“In his 1950 paper, Alan Turing wished to consider the question, ‘Can machines think?’ Rather than get bogged down by definitions of both of the words ‘machine’ and ‘think’ he replaced the question with one based on a much more practical scenario, namely his imitation game,” the authors said.
“Turing himself described the game in these terms: ‘The idea of the test is that a machine has to try and pretend to be a man, by answering questions put to it, and it will only pass if the pretence is reasonably convincing.”
In its standard form, the Turing test is described as an experiment that can be done in two different ways: (i) one-interrogator-one hidden interlocutor; (ii) one-interrogator-two hidden interlocutors.
In both cases, a machine must provide ‘satisfactory’ and ‘sustained’ answers to any questions put to it by the human interrogator.
“However, what about in the theoretical case when the machine takes the fifth amendment: No person shall be held to answer?”
“If a machine were to ‘take the fifth amendment’ – that is, exercise the right to remain silent throughout the test – it could, potentially, pass the test and thus be regarded as a thinking entity,” the scientists said.
The study looks at transcripts of a number of conversations from actual Turing tests in which the hidden machine remained silent.
In each case, the human judge was unable to say for certain whether they were interacting with a person or a machine.
Thus, a machine could potentially pass the Turing test simply by remaining silent.
The judge would be unable to determine whether the silent entity was a human choosing not to answer the questions, a smart machine that had decided not to reply, or a machine experiencing technical problems that prevented it from answering (as was actually the case in the transcripts studied).
“This begs the question, what exactly does it mean to pass the Turing test? Turing introduced his imitation game as a replacement for the question ‘Can machines think?’ and the end conclusion of this is that if an entity passes the test then we have to regard it as a thinking entity,” Prof. Warwick said.
“If an entity can pass the test by remaining silent, this cannot be seen as an indication it is a thinking entity, otherwise objects such as stones or rocks, which clearly do not think, could pass the test.”
“Therefore, we must conclude that ‘taking the fifth’ fleshes out a serious flaw in the Turing test.”
The study was published online in the Journal of Experimental and Theoretical Artificial Intelligence.
_____
Kevin Warwick & Huma Shah. Taking the fifth amendment in Turing’s imitation game. Journal of Experimental & Theoretical Artificial Intelligence, published online January 7, 2016; doi: 10.1080/0952813X.2015.1132273
