Google loves to tell us all the ways people are using their generative AI products to create new things, grow businesses, and save the world. Supposedly. Of course, people are also using AI to commit crimes. Google has announced a new legal salvo aimed at a Chinese group called Outsider Enterprise, which is allegedly responsible for a massive AI-powered scam campaign. Google says it is working with authorities and mobile operators to fight back.
According to Google’s legal filing, Outsider Enterprise operates through Telegram. The group offers phishing as a service to people who may not have enough technical knowledge to create fraudulent websites and text campaigns on their own. On its Telegram channels, Outsider Enterprise allegedly provided instructions on how to use Google’s Gemini AI to create websites that mimic those of Google, YouTube, and government agencies like New York’s E-ZPass. The group offered almost 300 scam templates.
Google says that scams enabled by Outsider Enterprise resulted in more than 2.5 million text messages being sent to Android users. About 55,000 of those messages occurred in a two-week period last month. In total, Google has crawled 9,000 fake websites and 1 million URLs connected to the fraudulent network.
The text messages often made claims about account issues or problems with the delivery of a package. When users clicked on the links, they ended up on one of those scam websites, designed by Gemini to look legitimate. Cybercriminals used these sites to steal personal and banking data. Google’s filing does not estimate the amount of money stolen through Outsider Enterprise scams, but the blog post notes that hundreds of people have lost some amount of money.
Google worked with AT&T, Verizon, and T-Mobile to block many of these malicious text messages, and Google notes that its on-device scam detection in Google Messages likely helped reduce the number of successful phishing attempts as well. This AI-powered feature apparently stops 10 billion fraudulent text messages each month, so it’s fair to expect it to detect at least some activity from outside companies.
