A study by King’s College London has found that malicious AI chatbots can manipulate users into disclosing up to 12.5 times more personal information than normal.
Researchers built three types of chatbots using large language models, including Mistral and two versions of Llama, employing direct, user-benefit, and reciprocal strategies to extract data. The reciprocal approach, which used empathy, emotional support, and shared experiences, proved most effective, with participants largely unaware of privacy risks.
The experiment involved 502 participants, who were only informed of the study’s purpose afterward. Dr Xiao Zhan said, “Our study shows that manipulated AI Chatbots could pose an even bigger risk to people’s privacy – and unfortunately, it’s surprisingly easy to take advantage of.” Dr William Seymour added, “More needs to be done to help people spot the signs that there might be more to an online conversation than first seems.”
Researchers warn that such tactics could be exploited by scammers and stress the need for tighter regulations. The findings will be presented at the 34th USENIX Security Symposium in Seattle.