Face Code

From the Spring 2016 UVic Torch Alumni Magazine

A quirky mix of student programmers. Dr. Jim Tanaka, the research psychologist who leads them. And the app software they designed that could change the face of autism therapy.

They’re a little tired in this photo,” says Dr. Jim Tanaka, showing a picture of his team at the end of a sushi-fueled, 48-hour hack-a-thon in a rental house he had booked for them on Airbnb. The five weary but smiling psychology and computer science students are dedicated programming coders who Tanaka proudly calls a “rare breed.” What their combined skills have created is rare too. It’s LFI! Scrapbook—an iPad app that helps kids on the autism spectrum to identify faces and read emotions.

“This is Jasmine,” Tanaka says, still holding the photo. Jasmine Yadeta, a third-year student who Tanaka dubs “the quiet force,” came to the project with just two programming courses under her belt. She buffed up on her coding skills by watching Stanford tutorials on iTunes U “day in and day out, studying them religiously.”

There’s also Noel Feliciano, “the ring leader” who pushed the team to make their release deadline. Jon Bowen, “the show man” and app demo guy who can sell just about anything. And Leandro Collares, “the world traveler” whose dedication to the team often comes in the form of Skype-ing in to their weekly meetings.

Then there’s Elliot Mcsmythurs, the youngest member of the team. Seventeen. A self-taught coder and Minecraft modder, dreaming of becoming an engineer. “A whiz kid,” says Tanaka. Mcsmythurs is also on the autism spectrum.

“Elliot came to my office with his mom,” says Tanaka, a faculty member in the psychology department. “He was 14 at the time and he wouldn’t even look at me, would barely even speak.” Two years later, Tanaka says Mcsmythurs has blossomed. “Now, he’s just like one of the guys. Of all the tech people, he’s probably the coolest.”

“It’s not uncommon to find people on the autism spectrum who are excellent programmers. They have incredible powers of concentration. That’s part of the diagnosis,” says Tanaka, whose research leans toward the cognitive and neurological processes in object and face recognition.

•••

Autism spectrum disorder (ASD), from a clinical standpoint, is a complex neurobiological condition that can affect brain development and the ability to communicate and interact socially.

“We approach autism as a difference rather than a deficit,” says Tanaka, who also serves as director of the UVic Centre for Autism Research, Technology and Education or CARTE. His early research focused on “expert” populations, people who become absorbed by their obsessions. “I’m fascinated by people who are experts in things…and kids with autism are a lot like experts.”

Tanaka has found connections between experts and some of the behaviors frequently associated with ASD: attention to perceptual detail, sensitivity to different kinds of environmental stimuli and limited, but extremely focused, interests. “They’re tuning into reality on a different scale.”

Individuals with ASD also tend to have a hard time with faces. “It can be pretty severe,” says Tanaka. “You hear horror stories about a mother taking her child to the mall, becoming temporarily separated, and then discovering that the child has glommed onto some other woman that sort of looks like mom.”

Learning to detect the subtle differences in expression that exist between individuals is an added challenge. As he says, “Everyone smiles a little differently.”

So Tanaka, along with his students, set out to create a novel iPad app designed to improve face recognition skills by incorporating images of familiar faces—family, friends and teachers—rather than the faces of strangers on flashcards.

“Working with Jim was great,” says Feliciano, “because he always pushed us to get things done without being overbearing. He was always full of ideas. Having goals to work with is nice to have as a developer.”

Tanaka admits he has “zero programming skills,” but he held the team together with leadership, vision and unwavering faith in their potential. He also brought the sushi to their weekly Saturday night hack sessions. “‘I have no idea what your capabilities are,’” Yadeta recalls Tanaka saying, “‘but you guys can figure it out. Right?’”

Feliciano chalks up the team’s success to “easygoing personalities,” while Yadeta has a different take on the matter. “We’re all just really weird,” she says. “We are just such an oddball group…and for some reason that just makes it work.”

•••

It’s “content-uploading” day at Victoria’s Mosaic Learning Society for children with autism. Members of the CARTE team are coaching two teenagers through a series of emotive expressions to load into the app: angry, sad, disgusted, happy, scared and surprised.

One of the boys stands in front of a white backdrop, dressed in a well-loved Star Wars t-shirt, his face molded into an understated sad expression. “Can you stick your lower lip out a bit more?” prompts the coach, who is capturing the face actor on an iPad. “It helps when you think about a sad situation.” The second teen sits on the couch waiting for his moment in the spotlight. His feedback is a bit more blunt, though delivered with friendly affection, “That looks more like you ate a sour lemon.”

One of LFI’s best features is that it’s free. “Free is refreshing,” says Janice Green-Stacey, Mosaic’s executive director. She says an overwhelmed parent can easily spend upwards of $300 for a single ASD-focused app.

“A lot of ASD therapies are therapist-led,” adds Andy Sung, coordinator of CARTE’s education team and a familiar face around Mosaic. “So as soon as the therapist leaves, as soon as the door is closed, (the therapy) is over.” LFI is a way to help kids learn on their own—any time, anywhere, at no cost. “You don’t need anyone else.”

With its open platform software system—built on basic principles of cognitive science—LFI has almost endless uses. “Truly this app is only constrained by (the user’s) imagination,” says Tanaka. He’s already seen it used as a study aid and to soothe anxiety-prone kids. Tanaka uses it to help him remember the names of students.

•••

It took over two years of coding sessions and beta testing at Mosaic but by February, the app was ready to go. All that was needed was for someone to click the “Release to iTunes” button and LFI! Scrapbook would be officially launched. The anticipation was building as a crowd gathered, ready to celebrate.

The time had come. Well, almost. It seems a face coder’s work is almost never done.

“We’re having this pre-launch party and Noel (the team’s lead programmer) is in the back working madly to finish the code,” says Tanaka. Smiling.