What have you always wondered about the economy? Tell us

Smile! We’re analyzing how you’re feeling using videos

Kai Ryssdal and Mukta Mohan Apr 16, 2015
HTML EMBED:
COPY

Smile! We’re analyzing how you’re feeling using videos

Kai Ryssdal and Mukta Mohan Apr 16, 2015
HTML EMBED:
COPY

The advertising world spends a lot of money trying to make you feel something. They want to make you laugh, cry, feel hungry, even. Anything to sell their products and build brand loyalty. However, tracking whether or not their commercials are working is hard to do. A company called Affectiva says it can help by using video footage to collect emotional data.

Co-founder and Chief Science Officer Rana El Kaliouby says, “Emotions influence every aspect of our life; how we connect and communicate with each other and also how we make decisions. We feel that our emotions are missing from the digital world and our digital experiences.”

Affectiva is trying to change that.

Its flagship program, Affdex, asks viewers to consent to be monitored, and then users’ facial expressions are captured while they watch advertisements. Affectiva has collected 2.7 million videos in 75 different countries. That data is aggregated to analyze how emotionally engaged users are with a service or ad. The program is able to detect smiles, confusion, brow furrow, even how wrinkles may move around an eye. Affdex is now able to understand smiles better than most people can.  

The program is being used primarily for advertising companies, but they are also working on real-time video communication with multiple people in a conversation. It may be used to track engagement in online courses, webinars, business meetings, even job interview videos.

Kaliouby recognizes that there are some applications that blur the line, but Affectiva has stayed away from anything without a clear opt-in. “We’re more focused on the ones where we feel we can bridge a communication gap,” says Kaliouby. 

Kaliouby’s background is in computer science. She says, “I spent many, many hours with my computer and it really bugged me that it was very oblivious to my emotional state. And that kind of inspired and motivated me to build an emotionally intelligent computer.”

Watch Kai test out Affdex below:

Marketplace is on a mission.

We believe Main Street matters as much as Wall Street, economic news is made relevant and real through human stories, and a touch of humor helps enliven topics you might typically find…well, dull.

Through the signature style that only Marketplace can deliver, we’re on a mission to raise the economic intelligence of the country—but we don’t do it alone. We count on listeners and readers like you to keep this public service free and accessible to all. Will you become a partner in our mission today?

Your donation is critical to the future of public service journalism. Support our work today – for as little as $5 – and help us keep making people smarter.