Back to blog

Blog

Phishing, Smishing and Vishing: What to Measure

For CISOs, IT, and compliance, these are not the same metric. Channels, signals, reporting, and controls differ, and training shows mixed results.

Phishing, Smishing and Vishing: What to Measure

Phishing, smishing, and vishing share the same impersonation logic, but they should not be managed as a single awareness category. For CISOs, IT, and compliance, the operational issue is different: the channel changes, the warning signs change, the expected response changes, and so does what is worth measuring. The evidence on training also cuts both ways, so a single aggregated click rate, or an isolated annual course, is not enough to govern risk.

What changes when they are compared side by side

When the sources are viewed together, one thing becomes clear that none of them shows on its own: the issue is not only spotting a scam, but responding differently depending on the channel. It also shows that compliance and operations converge on the same requirement, measure and improve continuously, not just train.

Topic Channel How it is carried out Signal or vector Recommended response Source
Phishing email or other messages uses convincing emails or other messages to induce the target to open harmful links or download malicious software, and may seek sensitive data from a victim by impersonation, according to NIST spelling or grammar mistakes, generic addresses, artificial urgency, and requests for personal information, according to NIST do not click or enter information, verify the request using known or public contact information, and report suspicious messages, according to NIST and the UK NCSC NIST, UK NCSC
Smishing text messages or SMS sends a message to a phone with a selectable link or a callback number, and may induce the download of malware onto the device, according to NIST links in SMS, callback numbers, and direct messages that seek personal data, according to NIST, CISA, and GOV.UK verify out of band using known or public contact information and forward suspicious messages to 7726, according to the FTC and GOV.UK NIST, CISA, FTC, GOV.UK
Vishing voice calls the attacker contacts the victim by voice call or leaves messages while posing as reputable companies to induce disclosure of personal information, according to NIST and CISA calls or voice messages that impersonate a person or company to obtain sensitive data, according to NIST and GOV.UK pause and verify the request through a trusted source or known or public contact information, not the information provided in the call, according to NIST NIST, CISA, GOV.UK
ISO/IEC 27001:2022 not specified requirements standard to establish, implement, maintain, and continually improve an information security management system, according to ISO not specified awareness for people under the organization's control, monitoring, measurement, analysis, and evaluation of the system, and use of secure authentication techniques, according to ISO and ISO/IEC JTC 1/SC 27 ISO, ISO Help Centre, ISO/IEC JTC 1/SC 27
NIST learning program phishing, smishing, and vishing hands-on social engineering exercises that include phishing, smishing, and vishing simulations, plus games, quizzes, tabletop exercises, and virtual labs, according to NIST not specified combine simulations and awareness with phishing-resistant authenticators, email protection controls, data loss prevention, and network security capabilities, according to NIST NIST
GOV.UK reporting suspicious email and suspicious text messages separate reporting channels, suspicious emails to report@phishing.gov.uk and suspicious text messages to 7726, according to GOV.UK not specified report by separate channel, email to report@phishing.gov.uk and SMS to 7726, according to GOV.UK GOV.UK

Why a single click metric is no longer enough

Whalemate's reading is straightforward: if risk enters through different channels, measuring everything with a single email click rate leaves blind spots. NIST describes an organizational phishing simulation as sending an email to measure whether the user detects the attempt or clicks, according to its learning guidance, but APWG reported that in the fourth quarter of 2025 the use of SMS for phishing increased, and that in the third quarter of 2025 SMS-based fraud rose by nearly 35%, while vishing and smishing volumes continued to increase.

The tension is operational, not semantic. If the main metric is still tied to email, the dashboard does not reflect the growth of channels that do not behave the same way. The FTC adds another relevant signal: reported losses from text-message scams reached $470 million in 2024, more than five times the 2020 figure, according to the FTC.

What is the real difference between phishing, smishing, and vishing?

NIST, CISA, GOV.UK, and Investor.gov agree that the main difference is the channel. According to Investor.gov, phishing generally uses email, smishing uses text messages or other direct messages, and vishing uses phone calls. NIST also says phishing can happen across multiple channels or move across channels, with the same goal of exploiting human behavior to obtain sensitive information.

Whalemate's reading is that classifying by channel is not a teaching detail. It defines which signals are expected, what operational friction makes sense, and what kind of reporting the organization should make easy.

Which signals change by channel?

For email phishing, NIST lists concrete signs: spelling or grammar errors, generic addresses, artificial urgency, and requests for personal information. In smishing, according to NIST and CISA, the main indicators are SMS links, callback numbers, and messages that try to move the conversation to another channel. In vishing, according to NIST, the vector is the call or voice message that impersonates a trusted company or person.

Whalemate's reading is that these modes should not be trained with the same warning list. False urgency can exist in all three, but the detection path and the safe decision are different in an inbox, on a phone, or during a call.

What operational response changes between email, SMS, and voice?

The sources converge on one central idea: verify outside the suspicious channel. NIST recommends verifying the request using known or public contact information, not the information included in the message, and confirming changes to accounts, banking details, urgent payments, or gift card requests through a trusted source. The UK NCSC recommends not clicking or entering information when a suspicious message is received and reporting it.

The difference shows up in execution. GOV.UK offers separate reporting channels, suspicious emails to report@phishing.gov.uk and suspicious text messages to 7726, while the FTC also recommends forwarding suspicious texts to 7726. Whalemate reads this as a useful pattern for LATAM: if the channel changes, the reporting and triage mechanism should change as well.

Does training work or not?

This is one of the most uncomfortable tensions in the material. KnowBe4 reported a base of 12.5 million users, 35,681 organizations, and more than 32.1 million simulated tests, and said the share of users who clicked fell to about 5% within one year of training and frequent testing. It also said that weekly or more frequent tests can push the click rate below 2%, according to its own analysis.

But a field study of 12,511 employees reported an overall click rate of 10.4% and found no statistically significant main effects from training on click rate, p=0.450, or on reporting rate, p=0.417, according to the academic study hosted on arXiv. In that same work, the control group had an average click rate of 9.8% compared with 10.5% among trained employees.

Whalemate's reading is that the answer is not binary. The evidence does not justify presenting training, by itself, as a sufficient control or as a universal failure. It does justify demanding segmentation, context, and measurement beyond the click.

What matters more, training or lure difficulty?

The academic study of 12,511 employees adds a key tension against simplistic readings of the results. According to that work, the NIST Phish Scale predicted behavior better than training, with a 7.0% click rate for easy lures and a 15.0% click rate for difficult lures. That difference suggests that a major part of the outcome depends on how convincing the simulated attack is.

Whalemate's reading is that comparing programs only by click rate can lead to mistakes if lure difficulty is not controlled. Two campaigns can show different rates without that reflecting a real improvement in human risk.

Why do vendors and academic studies show such different results?

Proofpoint reported, among customers who adopted its Security Awareness solution, an average 31% reduction in clicks on malicious links at three months and 40% at six months. At the same time, a synthesis hosted by NIST says that large-scale organizational experiments suggest training programs may not significantly improve real phishing email detection, and that people were 2% less likely to fall for phishing when they received some form of training.

Whalemate's reading is that the sources are measuring related but not identical things. The population, frequency, experimental design, type of intervention, and possibly the quality of the lure all change. For a security team, that makes it invalid to use a single benchmark as proof of universal effectiveness.

What does ISO/IEC 27001:2022 require, and what does it not allow you to claim?

ISO/IEC 27001:2022 presents, according to ISO, a requirements standard to establish, implement, maintain, and continually improve an information security management system. ISO's guidance describes implementation as an ongoing process that includes risk assessment, staff training, compliance monitoring, audit, and continuous improvement. In addition, requirement 7.3 of ISO calls for awareness among people under the organization's control, and requirement 9.1 calls for monitoring, measurement, analysis, and evaluation of the system, according to ISO's reference text.

Whalemate's reading is that compliance should not present an annual course as sufficient evidence of control over phishing, smishing, and vishing. ISO asks for a system, not an event. It also asks for measurement and improvement, so a single, undifferentiated metric falls short of the standard's own logic.

What role do technical controls and authentication play?

NIST recommends combining phishing-resistant authenticators with a comprehensive program that includes awareness and training, email protection controls, data loss prevention, and network security capabilities. Meanwhile, ISO/IEC 27001:2022 Annex A 8.5 is identified as "Secure authentication," according to ISO/IEC JTC 1/SC 27.

Whalemate's reading is that the debate over training, yes or no, is framed the wrong way. The strongest sources do not propose replacing technical controls with awareness, but combining them with protections that resist the channel, especially when voice and messaging move part of the risk outside the corporate email environment.

What should be measured next to make better decisions?

The material supports a concrete decision: separate by channel the volume of attempts, interaction rate, reporting rate, time to report, risky actions, and recurrence by person, role, and business unit. That segmentation is consistent with the growth of SMS and vishing reported by APWG, with GOV.UK and FTC's separate reporting channels, and with ISO's requirement for continuous monitoring and measurement.

It is also worth reading the internal differences inside the organization. A study at enterprise scale published in Computers & Security observed variation in click rates across business units and found that the timing of training before the simulated email did not significantly influence click rates. That reinforces the need to prioritize by risk and context, not by overall average.

What does this mean for CISOs, IT, and compliance in LATAM?

For CISOs, it means stopping the use of one aggregated click rate as the master metric and moving to a dashboard by channel, role, and business unit. For IT, it means enabling separate reporting, out-of-band verification for sensitive requests by voice and messaging, and combining those measures with phishing-resistant authenticators and complementary technical controls, according to NIST. For compliance, it means demonstrating a continuous system of awareness, monitoring, measurement, analysis, evaluation, and improvement, as required by ISO/IEC 27001:2022, not an isolated course.

In that logic, Human Risk Management should not be reduced to awareness either. According to Whalemate's material, HRM models context, simulates, trains, measures, intervenes, and reports, and its risk score consolidates signals from simulations, courses, reports, and behavior to prioritize interventions, not to punish. The actionable decision is to use segmented results by channel to intervene where the real risk changes, not where an average metric looks cleaner.

What to read next?

See the full topic guide

Questions

Frequently asked questions

Are phishing, smishing, and vishing the same thing?

No. They share the impersonation of a trusted entity to obtain sensitive information or induce another action, but the channel changes. According to NIST, CISA, and Investor.gov, phishing is usually associated with email or other messages, smishing with SMS or direct messaging, and vishing with voice calls. That difference changes signals, controls, and metrics.

What minimum metrics should a company separate by channel?

According to the material's finding, it is advisable to separate at least volume of attempts, interaction rate, reporting rate, time to report, risky actions, and recurrence by person, role, and business unit. That segmentation is consistent with ISO/IEC 27001:2022, which requires monitoring, measurement, analysis, and evaluation of the system, according to ISO.

Is an annual course enough for compliance?

No as sufficient evidence of control. According to ISO, ISO/IEC 27001:2022 requires establishing, implementing, maintaining, and continually improving a management system, and its implementation includes risk assessment, training, monitoring, audit, and continuous improvement. Requirement 7.3 calls for awareness and 9.1 calls for monitoring and measurement of the system.

Does training really reduce phishing risk?

The evidence in the material is mixed. KnowBe4 and Proofpoint report click reductions in their customer sets, while recent academic studies and a synthesis hosted by NIST do not find significant improvements or show modest effects. The operational reading is not to rely on training alone and to measure more than clicks.

What control is advisable to prioritize when risk grows through SMS and voice?

According to NIST, it is advisable to combine awareness with phishing-resistant authenticators and other technical controls. When risk comes through SMS or voice, the material also supports prioritizing out-of-band verification using known or public contact information, and reporting mechanisms differentiated by channel, as shown by GOV.UK and the FTC.

Would you like to go deeper on this topic?

Open this article directly in Claude or ChatGPT — ask questions, get a summary, or explore related ideas.

Open with ClaudeOpen with ChatGPT

Related articles