How AI voice cloning is increasing call center fraud

When you receive a call from an unknown number, chances are it's spam, though it may be something much worse: a fraudster recording your voice, then using it to infiltrate call centers and access bank accounts. It's just one technique fueled by the latest advancements in voice and AI technology to acquire financial and personal information to ultimately steal customer funds.

Call center fraud occurs when perpetrators phone into a company and pose as someone else to gain access to people's accounts. Fraudsters sometimes use AI to generate cloned voices to bypass security measures, even manipulating agents into giving them customer information.

There are many ways fraudsters use AI and voice technology to bypass call center security. One way is via Voice Engine, a voice cloning tool by OpenAI, the creator of ChatGPT, which generates natural-sounding speech closely resembling that of the original speaker. Generative AI has led to the development of more than 350 tools to clone voices, according to a 2024 Pindrop report.¹

How do fraudsters use AI voice cloning to commit call center fraud?

Before invading call centers, fraudsters may prepare themselves by hacking computer systems, scouring the dark web and posing as service company technicians to gather customer data.

Some fraudsters use what little information they have on an individual, such as an address, to mine interactive voice response (IVR) systems for additional data. An IVR is an automated telephone system technology that uses prerecorded messages and menu options to help customers get information without speaking to a live agent. The imposter uses this expanded data to bypass security measures and increase their chances of successfully manipulating agents with social engineering or accessing accounts through other channels.

For example, a perpetrator feeds a victim’s personal details, such as account number and Social Security number, into an AI tool. The criminal then instructs the technology to impersonate the victim and trains a bot to create responses to prompts by a customer service agent. The AI bot has a conversation with the agent, pretending to be the victim. With state-of-the-art voice and speech technology, there’s a delay of four to eight seconds for a bot response, which might alert the agent. Yet, as technology advances, these delays will become short enough to mimic a regular conversation.

It’s all increasing call center fraud rates: roughly 1 in every 750 calls in 2023 compared with 1 in 1,200 in 2022, according to Pindrop. That’s a 60% increase. Banking fraud via phone channels rose too, by about 44% to roughly 1 in 700 calls in 2023, up from about 1 in 1,000 two years earlier.

How much is voice fraud and call center fraud costing financial institutions?

Call center fraud isn’t just hurting companies in terms of account losses. Cardholders experience added friction, plus the additional time to authenticate call center customers decreases efficiency. The average contact authentication process increased to 46 seconds in 2023, up from 30 seconds three years earlier, a 53% increase, notes the Pindrop report.

So aside from hard losses, financial institutions (FIs) also face higher labor costs, having to devote more time and resources to the growing problem.

With banks and credit unions increasingly targeted via sophisticated fraud tactics, there’s a need for innovative tools and strategies to establish an effective fraud prevention framework.

What steps can banks take to prevent AI voice fraud?

Issuers can lower voice fraud at call centers by utilizing multiple channels, not only for authentication, but also for fraud detection.

Consider these action items:

• Host “red flag” trainings for call center agents: Ultimately, a goal should be that call center agents only deal with customer support or sales calls rather than general account inquiries. Until then, representatives need to know the latest fraud trends of the last 30 to 90 days and be aware of things that signal fraud. But general rules should apply no matter the trends. “We see accounts where the fraudster will mine for information by calling multiple times,” said Angelia Sharpe, Director, Fraud & Dispute Services, Total Issuing™ Managed Services. “Unusual call activity can be a red flag for suspicious activity."
• Monitor calls: Keep tabs on the number of incoming calls per customer. If it’s above a certain number during a period of time, it should trigger an alert and be investigated. Another sign could be requests to change a physical address and phone number. To identify a fraudster, their voice can be compared with what’s on record from the real cardholder. If it doesn’t match, the fraudster can be blocked or blacklisted from accessing the account.
• Collect and analyze voice data: Detect voice patterns from recorded call center conversations. This helps authenticate callers and reduce the burden of collecting data for knowledge-based authentication. FIs could also use voice mismatch detection software to flag suspicious callers, such as technology that detects nuances in audio that indicate whether a voice is real or recorded.
• Use behavior signals with device profiles: IVR and agents can provide valuable behavioral patterns to distinguish between legitimate and high-risk callers. This also helps create device profiles by relying on phone data such as keypress patterns and background noise to “fingerprint” a caller for authentication. For example, if the caller tends to use the same phone from a busy household with kids, then call centers can improve the accuracy of authentication by storing this kind of information.
• Layer on additional authentication factors: Use personally identifiable information (PII) to detect fraud, primarily with phone numbers and two-factor authentication. Agents can make an internal call to check the caller’s PII. Then with a mobile app, agents can send a push message to the cardholder’s/caller’s device, asking them to press “yes” or “no” to prove authentication.

How can FIs proactively defend against evolving fraud tactics?

Banks need to balance customer experience and security. By taking actions such as training call center staff, adding authentication technology, and monitoring software and technology for voice fraud, FIs can stay ahead of fraudsters while giving themselves a competitive advantage in the market.

“As new fraud voice technology is introduced, call centers must train agents on how to effectively use these tools," Abhishek said. "It could be asking the caller a series of questions to generate a score which determines if the person is the genuine account holder."

Disclaimers: