FSTDT Forums
Community => Science and Technology => Topic started by: Askold on June 08, 2014, 11:18:47 pm
-
http://www.theverge.com/2014/6/8/5790936/computer-passes-turing-test-for-first-time-by-convincing-judges-it-is
Professor Kevin Warwick, a visiting professor at the university, noted in a release that "some will claim that the Test has already been passed." He added that "the words Turing Test have been applied to similar competitions around the world," but "this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted."
The program nearly passed the test back in 2012, when 29 percent of judges at another competition decided that it was a human
Cool.
...Jokes about computers taking over the wolrd are coming but apart from that I wonder if this is really such a huge milestone. I mean, apart from automated telemarketing and customer service does this particular trick really offer anything useful or is it just a sign of how much the computers have advanced generally.
Still, this did take much longer than I or the average scifi-writer would have assumed.
-
As my roommate put it, "It's interesting, but you're well on your way to nothing more than emulating a human."
-
The problem is that a parrot is better at passing the Turing test than an AI.
For an AI to truly be human-like, the AI has to be able to learn, not just parrot. They have to be able to understand context, which is a remarkably abstract concept that even human beings have a hard time consciously doing - and with a computer, there's no such thing as a subconscious. Yet. Furthermore, they can't just "know" a definition, they have to comprehend it in its entirety.
To be human-like, they also have to be able to naturally make mistakes, not just have mistakes forced on them. They also have to be able to react to those mistakes and understand the correction from verbal dialogue alone. They also have to be able to decide to reject the correction.
Or, in short, we basically need to create a human being from scratch with components that are not human components.
Ironically, parrots seem to be able go beyond parroting and into making grammatically correct senses, until they can eventually converse with humans (though the one parrot that really was famous for this died from stress so it's not something they necessarily SHOULD do).
-
The problem is that a parrot is better at passing the Turing test than an AI.
For an AI to truly be human-like, the AI has to be able to learn, not just parrot. They have to be able to understand context, which is a remarkably abstract concept that even human beings have a hard time consciously doing - and with a computer, there's no such thing as a subconscious. Yet. Furthermore, they can't just "know" a definition, they have to comprehend it in its entirety.
To be human-like, they also have to be able to naturally make mistakes, not just have mistakes forced on them. They also have to be able to react to those mistakes and understand the correction from verbal dialogue alone. They also have to be able to decide to reject the correction.
Or, in short, we basically need to create a human being from scratch with components that are not human components.
Ironically, parrots seem to be able go beyond parroting and into making grammatically correct senses, until they can eventually converse with humans (though the one parrot that really was famous for this died from stress so it's not something they necessarily SHOULD do).
Yeah, mimicry alone isn't enough.
-
Its no Mr. Data, but its certainly a milestone. Getting this far has been a hurdle for engineers and programmers alike; getting the hardware to where it can perform the necessary actions, which can be extremely complex when broken down into actual code, and actually sitting down and writing the millions upon millions of lines of code that this thing likely took to get it to just this level. But, this is something upon which we can build, something we can refine. Think about how long it took nature to get an intellect as complex as a parrot, and then look at how comparatively little time its taken us to get fairly close to emulating that creature.
As for a sub-conscious, you've never heard of asynchronous processing or threading, have you? That's basically what a sub-conscious is, another thread of our awareness program. It does its own thing while the conscious thread (or threads, as the case may be) is busy doing something else. Hell, they'd be able to do it better than us, because there's a far lesser risk of file corruption and pretty much no risk of unintentional (or unnecessary) cross-talk between threads. The human brain is gloriously inefficient and prone to sudden, massive amounts of data corruption. We don't need to emulate it all, just the good parts.
-
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...
I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.
But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/
(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)
-
Some jurisdictions have laws against emulating in public.
-
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...
I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.
But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/
(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)
Gonna have to agree with this, because I tried the bot and I certainly don't think it passes the test. Hell, I've had better conversations with Cleverbot. I took an entire course that forced us to consider the Turing test and various levels of AI, and nearly the entire class concluded that while passing the Turing test certainly doesn't mean a machine is equal to humans in almost every way, it's certainly a milestone worth noting.
-
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...
I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.
But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/
(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)
Gonna have to agree with this, because I tried the bot and I certainly don't think it passes the test. Hell, I've had better conversations with Cleverbot. I took an entire course that forced us to consider the Turing test and various levels of AI, and nearly the entire class concluded that while passing the Turing test certainly doesn't mean a machine is equal to humans in almost every way, it's certainly a milestone worth noting.
I think the problem lies in, as I noted when I heard about it, that the bar for 'passing' the test is miserably low. Fooling a third of folks? that's not a whole lot. Now half or better? That's much more intriguing.
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
And my mind immediately goes to "AM" in I Have No Mouth Yet I Must Scream.
-
I think the problem lies in, as I noted when I heard about it, that the bar for 'passing' the test is miserably low. Fooling a third of folks? that's not a whole lot. Now half or better? That's much more intriguing.
That might be a relevant factor, but I don't think it's the main problem. I would really like to know what instructions the judges were given, as well as how they were chosen. And I'd love to have the transcripts from the test.
A test that would actually convince me, regardless of whether it's 30% or 60% of the judges:
- Have the judges be people who are strongly incentivised to detect a chatbot. This could be as simple as giving them money for every bot correctly identified. Make them lose money for every human incorrectly identified, so they don't just say "bot" to everything. The main point is making sure they aren't some random person who doesn't really care either way, or who isn't trying too hard because they think it's just a cute game or whatever. I suspect something like that was the problem here.
- Have them talk to multiple candidates, half of which are bots and half human, chosen at random (make sure the human candidates don't try to pass themselves as lame chatbots, again with proper incentives).
- Don't give them a script or an allowed list of questions or anything. You'd think this would be obvious, but a number of articles on this event mentioned something like "There have been previous claims that the test was passed, but this version doesn't set the questions or topics ahead of time", implying that people have tried to run a Turing test with such restrictions.
- Make sure the people asking the questions are the same people judging the bots. Once again, you'd think that would be obvious, but last time I heard a chatbot had "passed the Turing test" the actual event consisted of one group of people talking to the chatbot while other people watched the conversation and then decided bot or not. Remarkably stupid, because the judge can't decide for themselves what line of questioning would convince them and be worth pursuing.
- The chatbot was supposed to be emulating a 13-year-old Ukrainian boy, presumably to cover up deficiencies in grammar, vocabulary, general knowledge, etc. I don't think this is invalid; a bot who can successfully emulate a young foreign speaker is still miles ahead of anything we have now. However, if you're gonna go that route, make sure the control group are real 13-year-old Ukrainians, so you're actually comparing something to the thing it's supposed to be emulating.
We know bots can simulate a conversation if you stay within a limited range; that's been happening since ELIZA (http://en.wikipedia.org/wiki/ELIZA). We know bots can fool people who aren't looking for it (I once did this over Omegle by copypasting to Cleverbot. Had a surprisingly long conversation until I finally got bored and told them what I was doing. The other person didn't take it well). What we want to know is whether you can build a bot that fools people who are actively looking for it and are free and willing to ask tricky questions or follow extended lines of argument. I'm not convinced this is what happened here.
-
"The words Turing Test have been applied to similar competitions around the world. However this event involved more simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday.”
...
The Turing Test 2014 was held in partnership with RoboLaw, an organization that examines the regulation of robotic technologies, and the judges included Red Dwarf actor Robert Llewellyn and Lord Sharkey.
Well, that is the closest that I have gotten to finding out how this test was done. For some reason my Google-Fu is failing me today. Seems like it was decently done, apart from the celebreties being included as judges.
EDIT: More info.
Simultaneous tests as specified by Alan Turing
Each judge was involved in five parallel tests - so 10 conversations
30 judges took part
In total 300 conversations
In each five minutes a judge was communicating with both a human and a machine
Each of the five machines took part in 30 tests
To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society
That sounds much better. So the computer fooled 10 people in total, not bad.
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.
Yup. Despite what Hollywood (read, people that think computers are fucking magic wonder boxes) likes to bullshit constantly with things like Skynet, a computer only does what its told. Its up to those who write code to tell it what to do and how to do it right. Hell, developing basic ethical subroutines would probably be easier than, say, giving it simple emotions. Some forms of ethics do require at least basic emotional reactions, or are enhanced by them, but the basic things like "don't murder people" require absolutely no emotion at all, just proper situational awareness.
-
I imagine a "make sure humanity is happy" might work with the caveats of "a killed person counts as extremely unhappy", "using drugs to force happiness doesn't count", and "happiness of one peoples shouldn't come at the cost of happiness of other peoples."
But then I'm talking in abstract.
-
I imagine a "make sure humanity is happy" might work with the caveats of "a killed person counts as extremely unhappy", "using drugs to force happiness doesn't count", and "happiness of one peoples shouldn't come at the cost of happiness of other peoples."
But then I'm talking in abstract.
The general thought is that you can't do this through patchwork "This is the ethical principle you have to follow, with exceptions a, b, c, d, e...", because if your ethical principle has so many exceptions, then it's not really a decent ethical principle and you'd best look for something better. I think the best idea right now for teaching ethics to an AI is saying "OK, look at humans, figure out what they think is right and wrong, now extrapolate that to a world where humans were smarter". Or something to that effect, but more precisely formulated.
Seems like it was decently done, apart from the celebreties being included as judges.
That's kind of a major point, though. Would a celebrity judge actually care about proper testing? Especially considering that a) celebrities tend to be people who enjoy publicity and b) "Turing test passed" is a headline, "Turing test failed" is not.
I'm not saying they deliberately passed a bot for publicity, mind you, just that they don't necessarily have any strong motivation to test properly.
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.
Yup. Despite what Hollywood (read, people that think computers are fucking magic wonder boxes) likes to bullshit constantly with things like Skynet, a computer only does what its told. Its up to those who write code to tell it what to do and how to do it right. Hell, developing basic ethical subroutines would probably be easier than, say, giving it simple emotions. Some forms of ethics do require at least basic emotional reactions, or are enhanced by them, but the basic things like "don't murder people" require absolutely no emotion at all, just proper situational awareness.
That's a very boring answer.
Also, again, what the hell are we going to do with it once we make it?
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.
Yup. Despite what Hollywood (read, people that think computers are fucking magic wonder boxes) likes to bullshit constantly with things like Skynet, a computer only does what its told. Its up to those who write code to tell it what to do and how to do it right. Hell, developing basic ethical subroutines would probably be easier than, say, giving it simple emotions. Some forms of ethics do require at least basic emotional reactions, or are enhanced by them, but the basic things like "don't murder people" require absolutely no emotion at all, just proper situational awareness.
That's a very boring answer.
Also, again, what the hell are we going to do with it once we make it?
What's it going to do once complete? Seriously, where the hell would it start? Would we just define an arbitrary starting point and let it continue, or would we let it decide what to do with its first moments of life?
-
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.
Yup. Despite what Hollywood (read, people that think computers are fucking magic wonder boxes) likes to bullshit constantly with things like Skynet, a computer only does what its told. Its up to those who write code to tell it what to do and how to do it right. Hell, developing basic ethical subroutines would probably be easier than, say, giving it simple emotions. Some forms of ethics do require at least basic emotional reactions, or are enhanced by them, but the basic things like "don't murder people" require absolutely no emotion at all, just proper situational awareness.
That's a very boring answer.
Also, again, what the hell are we going to do with it once we make it?
What's it going to do once complete? Seriously, where the hell would it start? Would we just define an arbitrary starting point and let it continue, or would we let it decide what to do with its first moments of life?
get thousands of supermodels running up and begging the scientists to give them babies.
I imagine we'd have to teach it the ins and outs of being alive and aware of your own existence.
-
get thousands of supermodels running up and begging the scientists to give them babies.
That reminds me of an old anecdote about George Bernard Shaw. Supposedly, a strange lady giving an address in Zürich wrote him a proposal, saying: “You have the greatest brain in the world, and I have the most beautiful body; so we ought to produce the most perfect child.” Shaw asked: “What if the child inherits my body and your brains?”
Don't know if it's true, but it's certainly funny.
-
And for fuck's sake keep it away from the internet! Otherwise our True AI will screw us all as soon as it discovers a comment section! At least until it's mature enough to ignore the bullshit.
-
It'll come with the maturity patch pre-installed. We only deal with that bullshit from human children because we have to, and some psychotics find it adorable.
-
I'm seeing more and more reports claiming that this article was just crap.
Some are just complaining about how the Turing test is just a thought experiment and not a real milestone or achievement. While other articles are claiming that this particular test wasn't done well enough to qualify as a pass.
And RAvynoushunter, I don't think that making a truly self aware computer is impossible. It might not be possible with the current technology we have but who knows how much computers can advanve in the next few centuries, if we don't kill ourselves accidentally on purpose before that happens.
(Seriously, computers are becoming more and more advanced and human sciences in general have advanced greatly in the last few centuries.)
-
Yup, its not a question of if we can do it, its a question of when we can achieve it. The more we understand about nature and the universe, the better we become at emulating it and improving upon it.