Author Topic: A computer has passed the Turing test  (Read 6311 times)

0 Members and 1 Guest are viewing this topic.

Offline Askold

  • Definitely not hiding a dark secret.
  • Global Moderator
  • The Beast
  • *****
  • Posts: 8358
  • Gender: Male
A computer has passed the Turing test
« on: June 08, 2014, 11:18:47 pm »
http://www.theverge.com/2014/6/8/5790936/computer-passes-turing-test-for-first-time-by-convincing-judges-it-is

Quote
Professor Kevin Warwick, a visiting professor at the university, noted in a release that "some will claim that the Test has already been passed." He added that "the words Turing Test have been applied to similar competitions around the world," but "this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted."

The program nearly passed the test back in 2012, when 29 percent of judges at another competition decided that it was a human

Cool.

...Jokes about computers taking over the wolrd are coming but apart from that I wonder if this is really such a huge milestone. I mean, apart from automated telemarketing and customer service does this particular trick really offer anything useful or is it just a sign of how much the computers have advanced generally.

Still, this did take much longer than I or the average scifi-writer would have assumed.
No matter what happens, no matter what my last words may end up being, I want everyone to claim that they were:
"If you strike me down, I shall become more powerful than you could possibly imagine."
Aww, you guys rock. :)  I feel the love... and the pitchforks and torches.  Tingly!

Offline Cerim Treascair

  • My Love Is Lunar
  • The Beast
  • *****
  • Posts: 3092
  • Gender: Male
  • Get me my arbalest... explosive bolts, please.
Re: A computer has passed the Turing test
« Reply #1 on: June 08, 2014, 11:43:22 pm »
As my roommate put it, "It's interesting, but you're well on your way to nothing more than emulating a human."
There is light and darkness in the world, to be sure.  However, there's no harm to be had in walking in the shade or shadows.

Formerly Priestling

"I don't give a fuck about race...I'm white, I'm American, but that shit don't matter.  I'm human."

Offline Witchyjoshy

  • SHITLORD THUNDERBASTARD!!
  • Kakarot
  • ******
  • Posts: 9044
  • Gender: Male
  • Thinks he's a bard
Re: A computer has passed the Turing test
« Reply #2 on: June 08, 2014, 11:50:53 pm »
The problem is that a parrot is better at passing the Turing test than an AI.

For an AI to truly be human-like, the AI has to be able to learn, not just parrot.  They have to be able to understand context, which is a remarkably abstract concept that even human beings have a hard time consciously doing - and with a computer, there's no such thing as a subconscious.  Yet.  Furthermore, they can't just "know" a definition, they have to comprehend it in its entirety.

To be human-like, they also have to be able to naturally make mistakes, not just have mistakes forced on them.  They also have to be able to react to those mistakes and understand the correction from verbal dialogue alone.  They also have to be able to decide to reject the correction.

Or, in short, we basically need to create a human being from scratch with components that are not human components.

Ironically, parrots seem to be able go beyond parroting and into making grammatically correct senses, until they can eventually converse with humans (though the one parrot that really was famous for this died from stress so it's not something they necessarily SHOULD do).
Mockery of ideas you don't comprehend or understand is the surest mark of unintelligence.

Even the worst union is better than the best Walmart.

Caladur's Active Character Sheet

Offline Ultimate Paragon

  • The Beast
  • *****
  • Posts: 8423
  • Gender: Male
  • Tougher than diamonds, stronger than steel
Re: A computer has passed the Turing test
« Reply #3 on: June 09, 2014, 07:33:56 am »
The problem is that a parrot is better at passing the Turing test than an AI.

For an AI to truly be human-like, the AI has to be able to learn, not just parrot.  They have to be able to understand context, which is a remarkably abstract concept that even human beings have a hard time consciously doing - and with a computer, there's no such thing as a subconscious.  Yet.  Furthermore, they can't just "know" a definition, they have to comprehend it in its entirety.

To be human-like, they also have to be able to naturally make mistakes, not just have mistakes forced on them.  They also have to be able to react to those mistakes and understand the correction from verbal dialogue alone.  They also have to be able to decide to reject the correction.

Or, in short, we basically need to create a human being from scratch with components that are not human components.

Ironically, parrots seem to be able go beyond parroting and into making grammatically correct senses, until they can eventually converse with humans (though the one parrot that really was famous for this died from stress so it's not something they necessarily SHOULD do).
Yeah, mimicry alone isn't enough.

Offline RavynousHunter

  • Master Thief
  • The Beast
  • *****
  • Posts: 8108
  • Gender: Male
  • A man of no consequence.
    • My Twitter
Re: A computer has passed the Turing test
« Reply #4 on: June 09, 2014, 08:16:13 am »
Its no Mr. Data, but its certainly a milestone.  Getting this far has been a hurdle for engineers and programmers alike; getting the hardware to where it can perform the necessary actions, which can be extremely complex when broken down into actual code, and actually sitting down and writing the millions upon millions of lines of code that this thing likely took to get it to just this level.  But, this is something upon which we can build, something we can refine.  Think about how long it took nature to get an intellect as complex as a parrot, and then look at how comparatively little time its taken us to get fairly close to emulating that creature.

As for a sub-conscious, you've never heard of asynchronous processing or threading, have you?  That's basically what a sub-conscious is, another thread of our awareness program.  It does its own thing while the conscious thread (or threads, as the case may be) is busy doing something else.  Hell, they'd be able to do it better than us, because there's a far lesser risk of file corruption and pretty much no risk of unintentional (or unnecessary) cross-talk between threads.  The human brain is gloriously inefficient and prone to sudden, massive amounts of data corruption.  We don't need to emulate it all, just the good parts.
Quote from: Bra'tac
Life for the sake of life means nothing.

Offline Sigmaleph

  • Ungodlike
  • Administrator
  • The Beast
  • *****
  • Posts: 3615
    • sigmaleph on tumblr
Re: A computer has passed the Turing test
« Reply #5 on: June 09, 2014, 02:09:39 pm »
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...

I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.

But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/

(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)
« Last Edit: June 09, 2014, 02:15:33 pm by Sigmaleph »
Σא

Offline Old Viking

  • The Beast
  • *****
  • Posts: 1454
  • Gender: Male
  • Occasionally peevish
Re: A computer has passed the Turing test
« Reply #6 on: June 09, 2014, 05:07:12 pm »
Some jurisdictions have laws against emulating in public.
I am an old man, and I've seen many problems, most of which never happened.

Offline Sleepy

  • Fuck Yes Sunshine In a Bag
  • The Beast
  • *****
  • Posts: 4598
  • Gender: Female
  • Danger zone
Re: A computer has passed the Turing test
« Reply #7 on: June 09, 2014, 05:48:03 pm »
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...

I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.

But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/

(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)

Gonna have to agree with this, because I tried the bot and I certainly don't think it passes the test. Hell, I've had better conversations with Cleverbot. I took an entire course that forced us to consider the Turing test and various levels of AI, and nearly the entire class concluded that while passing the Turing test certainly doesn't mean a machine is equal to humans in almost every way, it's certainly a milestone worth noting.
Guys, this is getting creepy. Can we talk about cannibalism instead?

If a clown eats salmon on Tuesday, how much does a triangle weigh on Jupiter? Ask Mr. Wiggins for 10% off of your next dry cleaning bill. -Hades

Offline Cerim Treascair

  • My Love Is Lunar
  • The Beast
  • *****
  • Posts: 3092
  • Gender: Male
  • Get me my arbalest... explosive bolts, please.
Re: A computer has passed the Turing test
« Reply #8 on: June 09, 2014, 10:18:43 pm »
Hmm. I've seen other chatbots which were claimed to pass the Turing test, but when I tried them they failed miserably. So I did the obvious thing, found the chatbot, and started talking...

I have to call bullshit. Either the version tested was much more advanced than the one you can find online, or the judges were idiots, or the test was set up in a stupid way, or something else is wrong. The bot I talked to wouldn't fool any reasonable person who was actively trying to find out if it was a program or not. Certainly not a third of the judges in a properly-conducted Turing test.

But don't take my word for it, feel free to talk to the bot yourself: http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/

(EDIT: most articles link to http://princetonai.com/bot/bot.jsp for the chatbot, but that page is down now presumably from all the other people trying to talk to it, which is why I considered the possibility the version I talked to was not the same as the one tested. If anyone finds a confirmed up-to-date mirror I would appreciate a link)

Gonna have to agree with this, because I tried the bot and I certainly don't think it passes the test. Hell, I've had better conversations with Cleverbot. I took an entire course that forced us to consider the Turing test and various levels of AI, and nearly the entire class concluded that while passing the Turing test certainly doesn't mean a machine is equal to humans in almost every way, it's certainly a milestone worth noting.

I think the problem lies in, as I noted when I heard about it, that the bar for 'passing' the test is miserably low.  Fooling a third of folks? that's not a whole lot.  Now half or better? That's much more intriguing.
There is light and darkness in the world, to be sure.  However, there's no harm to be had in walking in the shade or shadows.

Formerly Priestling

"I don't give a fuck about race...I'm white, I'm American, but that shit don't matter.  I'm human."

Offline I am lizard

  • The Beast
  • *****
  • Posts: 3545
    • https://www.instagram.com/p/9SIHifrULJ/
Re: A computer has passed the Turing test
« Reply #9 on: June 09, 2014, 10:33:53 pm »
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?

Art Vandelay

  • Guest
Re: A computer has passed the Turing test
« Reply #10 on: June 09, 2014, 10:38:12 pm »
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.

Offline SpaceProg

  • What you read is what you get.
  • The Beast
  • *****
  • Posts: 5507
  • Nocturnal
Re: A computer has passed the Turing test
« Reply #11 on: June 09, 2014, 11:05:53 pm »
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
And my mind immediately goes to "AM" in I Have No Mouth Yet I Must Scream.

Offline Sigmaleph

  • Ungodlike
  • Administrator
  • The Beast
  • *****
  • Posts: 3615
    • sigmaleph on tumblr
Re: A computer has passed the Turing test
« Reply #12 on: June 09, 2014, 11:48:47 pm »
I think the problem lies in, as I noted when I heard about it, that the bar for 'passing' the test is miserably low.  Fooling a third of folks? that's not a whole lot.  Now half or better? That's much more intriguing.

That might be a relevant factor, but I don't think it's the main problem. I would really like to know what instructions the judges were given, as well as how they were chosen. And I'd love to have the transcripts from the test.

A test that would actually convince me, regardless of whether it's 30% or 60% of the judges:

  • Have the judges be people who are strongly incentivised to detect a chatbot. This could be as simple as giving them money for every bot correctly identified. Make them lose money for every human incorrectly identified, so they don't just say "bot" to everything. The main point is making sure they aren't some random person who doesn't really care either way, or who isn't trying too hard because they think it's just a cute game or whatever. I suspect something like that was the problem here.
  • Have them talk to multiple candidates, half of which are bots and half human, chosen at random (make sure the human candidates don't try to pass themselves as lame chatbots, again with proper incentives).
  • Don't give them a script or an allowed list of questions or anything. You'd think this would be obvious, but a number of articles on this event mentioned something like "There have been previous claims that the test was passed, but this version doesn't set the questions or topics ahead of time", implying that people have tried to run a Turing test with such restrictions.
  • Make sure the people asking the questions are the same people judging the bots. Once again, you'd think that would be obvious, but last time I heard a chatbot had "passed the Turing test" the actual event consisted of one group of people talking to the chatbot while other people watched the conversation and then decided bot or not. Remarkably stupid, because the judge can't decide for themselves what line of questioning would convince them and be worth pursuing.
  • The chatbot was supposed to be emulating a 13-year-old Ukrainian boy, presumably to cover up deficiencies in grammar, vocabulary, general knowledge, etc. I don't think this is invalid; a bot who can successfully emulate a young foreign speaker is still miles ahead of anything we have now. However, if you're gonna go that route, make sure the control group are real 13-year-old Ukrainians, so you're actually comparing something to the thing it's supposed to be emulating.

We know bots can simulate a conversation if you stay within a limited range; that's been happening since ELIZA. We know bots can fool people who aren't looking for it (I once did this over Omegle by copypasting to Cleverbot. Had a surprisingly long conversation until I finally got bored and told them what I was doing. The other person didn't take it well). What we want to know is whether you can build a bot that fools people who are actively looking for it and are free and willing to ask tricky questions or follow extended lines of argument. I'm not convinced this is what happened here.
Σא

Offline Askold

  • Definitely not hiding a dark secret.
  • Global Moderator
  • The Beast
  • *****
  • Posts: 8358
  • Gender: Male
Re: A computer has passed the Turing test
« Reply #13 on: June 10, 2014, 12:51:29 am »
Quote
"The words Turing Test have been applied to similar competitions around the world. However this event involved more simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday.”

...

The Turing Test 2014 was held in partnership with RoboLaw, an organization that examines the regulation of robotic technologies, and the judges included Red Dwarf actor Robert Llewellyn and Lord Sharkey.

Well, that is the closest that I have gotten to finding out how this test was done. For some reason my Google-Fu is failing me today. Seems like it was decently done, apart from the celebreties being included as judges.

EDIT: More info.

Quote
    Simultaneous tests as specified by Alan Turing
    Each judge was involved in five parallel tests - so 10 conversations
    30 judges took part
    In total 300 conversations
    In each five minutes a judge was communicating with both a human and a machine
    Each of the five machines took part in 30 tests
    To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society

That sounds much better. So the computer fooled 10 people in total, not bad.
« Last Edit: June 10, 2014, 12:53:07 am by askold »
No matter what happens, no matter what my last words may end up being, I want everyone to claim that they were:
"If you strike me down, I shall become more powerful than you could possibly imagine."
Aww, you guys rock. :)  I feel the love... and the pitchforks and torches.  Tingly!

Offline RavynousHunter

  • Master Thief
  • The Beast
  • *****
  • Posts: 8108
  • Gender: Male
  • A man of no consequence.
    • My Twitter
Re: A computer has passed the Turing test
« Reply #14 on: June 10, 2014, 10:11:40 am »
I wonder what we'd do if we did creat a sapient computer?
I mean, how would you teach it things like ethics?
How would you make sure it didn't suffer pain from its existence?
It's just a matter of programming it to do those things, I would imagine.

Yup.  Despite what Hollywood (read, people that think computers are fucking magic wonder boxes) likes to bullshit constantly with things like Skynet, a computer only does what its told.  Its up to those who write code to tell it what to do and how to do it right.  Hell, developing basic ethical subroutines would probably be easier than, say, giving it simple emotions.  Some forms of ethics do require at least basic emotional reactions, or are enhanced by them, but the basic things like "don't murder people" require absolutely no emotion at all, just proper situational awareness.
Quote from: Bra'tac
Life for the sake of life means nothing.