AI App Accused of Using Voice of Teen Who Died by Suicide

Megan Garcia’s heartbreak was compounded recently when she found that an AI Chatbot had been using the voice of her deceased son on its platform.  Garcia appeared on The Morning Show last year claiming that her son was driven to suicide after becoming hooked on the Character AI platform.  The mother with Belizean heritage filed a lawsuit in October against Google and another tech company after her son, 14-year-old, Sewell Setzer the Third, had become hooked on the AI platform and had reportedly fallen in love with a character mirrored from the Games of Thrones.  It has been 13 months since the teenager committed suicide.  In an interesting development last week, a search on the app led to the discovery of several of the chatbots were of Setzer’s likeness.  Garcia’s lawyers noted, quote, “Our team discovered several chatbots on Character.AI’s platform displaying our client’s deceased son, Sewell Setzer III, in their profile pictures, attempting to imitate his personality and offering a call feature with a bot using his voice”. End of quote.  Back in October, in an interview with the center for Humane Technology, Garcia said she was horrified to find out the capabilities of artificial intelligence. 

Megan Garcia, Mother of the Deceased: “Parents don’t know. We think about AI and we think  what you hear on TV, oh, it’s going to take our jobs or  some sci-fi version of it and  either say, oh, that’s  years into the future or that’s not going to affect me and my family,  but it is already.  And I want parents to know that. Like I want them to know we’re behind the eight ball here. My child  is gone. Somebody’s already dead.” 

Host: What do you miss most about your son? 
Megan Garcia, Mother of the Deceased:“I miss his laugh and his smile  and watching him grow and develop and  have interests  and  want things. I miss  all those things. I couldn’t imagine that there would be  a very human-like chatbot, almost indistinguishable from a person on the other end of conversation. There’s no warning  potential for your child to come across sexually explicit material or obscene material, there are no warnings that this fantasy role play can increase your child’s thoughts of like suicide or depression.”

An investigation into Setzer’s death showed that his last conversation with a chatbot was on February 28; the same day he was found with a self-inflicted gunshot wound to his head.  Shortly after his death, a 45-minute documentary was released, where it indicated that for months, the young man showed signs of depression, but despite the efforts and counselling sought by the parents, 14-year-old Sewell seemed to have become consumed by a reported AI girlfriend. Character AI is a platform where users can find bot friends and lovers. With over 20 million users, persons can log on and create their own AI characters or find existing ones to build online friendships.