Saw an ad on Instagram today for Replika AI, where it was touting the romantic relationships feature of the chatbot service, including photo sharing, role playing, and “caring and loving”. All of these, with an AI.
Replika started off as an experiment by an engineer who lost a dear friend. She had experience with chatbots and decided to feed her friend’s text messages into a neural network to create a “digital memorial” of him. Read more about it in this Wired article.
But the ad I saw today was something wildly different. It was gross and far from a “digital friend” or a likeness thereof. Over the years, while I’ve not used Replika much, I’ve kept my eye on the service. I stopped using it right when it started leaning into this romantic aspect instead of friendship. It started asking me to share pics of my day-to-day life, selfies to get started with the app after I came back after a break of a few months, and to voice chat with it. It felt gross the way the app was transforming right in front of me.
Now, wherever there are chatbots and turing tests, there’s the baseness and loneliness of humanity. I understand that. I understand that people were seeing this “friend” app and asking it romantic questions, and laughing when it was giving them even slightly romantic responses. But for the service to lean into that feels like a betrayal of the original intent.
Dystopian storytelling often pins on this idea of people being so isolated from society that only an AI gives them the comfort of a relationship. Heck, we don’t need to look to science fiction for that. Real news coming out of China about people’s social media usage behavior often shows how messed up the landscape already is.
But to see an app in the US be so blatant in its disregard for real human connection and its outright mission to replace it with a chatbot feels like something society and politicians should condemn.
What I also don’t understand is how this app isn’t violating at least one or more of Apple’s ridiculous App Store policies. Thoughts?