Although it's never mentioned anymore, the actual test that Turing proposed was that a man and a computer would each pretend to be a woman in a conversation with the judge. If the computer could fool the judges as well as the man could, that would be a mark of intelligence. The test was perhaps indicative of Turing's thoughts about sexual identity. Brent On 6/10/2014 7:37 AM, Andy Latto wrote:
Reading the transcripts, it seems to me that the people running this test didn't understand how it was supposed to work, as described by Turing.
As described by Turing:
The judge, in their half of the conversation, is doing their best to trip up the computer, saying things and asking questions which would be difficult for a computer to respond to, making it easier for them to distinguish the computer from the human.
The human talking to the judge is doing their best to help the judge distinguish the human from the computer, by saying things that the judge would find it unlikely that a computer would say.
As practiced, judging from the transcripts:
The judge makes conversation, following the lead of the person/computer they are talking to, making it as easy as possible for the computer to operate in its "comfort zone" where it can imitate a human well.
The human talking to the judge is relatively passive, participating in conversation, but making no attempt to say things a computer wouldn't say to make it easy for the judge to describe.
Even given this much easier task, the transcripts I saw seem obviously computer generated: despite the fact that the judges got it wrong a third of the time, I would happily take bets at 10-1 on my ability to distinguish a human from a chatbot. I don't know whether the judges were inept, or actually voted for what they knew was the computer when they thought it did an impressive job, even if they weren't fooled, but the claim that they were experts seems like nonsense. And how is this judged, anyway? Is there a formal certification program for Human-Computer-Distingushing Judges, or should "self-proclaimed" be inserted before the word "expert" wherever it occurs?
Watson didn't pass a turing test, but I find its performance truly impressive; the questions it read and understood were not written to be understood by a computer, and were in fact written in an unusual playful and punny style that Jeopardy uses, and the fact that Watson was able to understand these questions well enough to answer them is real progress in language understanding. The performance on this rigged "turing test" I find much less impressive.
Andy
On Mon, Jun 9, 2014 at 2:52 PM, Warren D Smith <warren.wds@gmail.com> wrote:
http://www.theguardian.com/technology/2014/jun/09/eugene-person-human-comput...
This story has some actual transcripts. Suppose you had to decide whether the judge was a "serious professional Turing contest judge" or an "idiot" just based on reading the transcript...
_______________________________________________ math-fun mailing list math-fun@mailman.xmission.com https://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun