Should a machine have to tell you if it’s a machine?
Share Now on:
This week Microsoft bought a company called Semantic Machines which works on something called “conversational AI.” That means computers that sound and respond like humans. Mostly it’s for digital assistants like Microsoft’s Cortana, Apple’s Siri, Amazon’s Alexa or Bixby on Samsung. Last month Google showed off its own smart assistant called Duplex, which can call a hair salon to make an appointment on your behalf, or a restaurant to make a reservation. But it’s clear from what Google showed that the people on the other end of these calls don’t know they’re talking to a computer. This has led some to ask what the rights of the human on the other end of line are. Marketplace Tech host Molly Wood spoke with Jacob Metcalf of the Data & Society Research Institute about the business case for making computers sound so real. The following is an edited transcript of their conversation.
Jacob Metcalf: You know, what’s odd about the product that Google showed at their IO conference is that it doesn’t put that signaling up front. And so, you kind of have to conclude that that means they put in effort, in fact a lot of effort, in order to hide the fact that it is a bot. Something is accomplished by the fact that it’s not obvious to the person on the other end that they’re talking to an automated machine. None of that conversation required the machine to insert “ahs” and “ums” as if it was a human. So then why are they trying to imitate a human? Is it because it looks cool? Or is it because the product’s more effective if it’s deceptive? And is that a tradeoff that we should accept as society?
Molly Wood: So, is it legal? I mean, is there a wiretapping concern too? This gets into the ethical question of whether these AIs need to identify themselves as not human, but also just the question of is it legal for them to collect data about their interactions with people who don’t know that they are part of that collection?
Metcalf: Yeah, well, that’s certainly an open question. I think it’s probably going to be litigated. There’s all kinds of ways that they could say that the algorithmic analysis is not the same thing as a wiretap. However, I really think it’s important that we not just focus on the legal question of wiretapping, but also on the question of them making use of other people’s time and energy in order to improve their product without those people being aware that they’re making that contribution.
Wood: I guess the devil’s advocate question then is how is this significantly different from any of the other automated systems that I might interact with, whether or not I think it’s a human. Is it different from any of the times that a computer calls me, whether for a robo-call or when I interact with an automated system for customer support? How is this different?
Metcalf: Well, if they identify themselves as a bot I think it’s not so different. However, I think it’s important to realize that there are ways in which if I call a restaurant and I make a reservation that interaction with that other person sort of sets social expectations, right? There’s a question here: Am I more likely to just skip out on the reservation and cost this business money if I used the assistant? Am I more likely to not feel compelled to act as a good citizen? So, I think it can be very helpful in some situations. It can be helpful for people who struggle to speak clearly on the phone or have anxiety about social interactions. But it’s critically important for us to get some early norms about how this kind of data is collected, how the bots identify themselves and for these companies to be clear that as they provide benefits to their users, it’s not coming at the cost of the other parties who might not even realize that they’re interacting with an automated system.
|How artificial intelligence can be applied to the real world|
|Will AI eventually let us enhance our own brains?|
|Bringing artificial intelligence to the factory floor|
We’re here to help you navigate this changed world and economy.
Our mission at Marketplace is to raise the economic intelligence of the country. It’s a tough task, but it’s never been more important.
In the past year, we’ve seen record unemployment, stimulus bills, and reddit users influencing the stock market. Marketplace helps you understand it all, will fact-based, approachable, and unbiased reporting.
Generous support from listeners and readers is what powers our nonprofit news—and your donation today will help provide this essential service. For just $5/month, you can sustain independent journalism that keeps you and thousands of others informed.
Donate now to get almost any thank-you gift.