By 
For CISOs, IT, and compliance teams in LATAM, a security awareness training platform should be evaluated as a system for measurement, segmentation, and evidence, not as a course catalog. The material's frameworks consistently call for ongoing, updated training, and several add exercises, effectiveness testing, and verifiable records. The tension comes when that regulatory demand meets empirical evidence: training alone does not show a consistent reduction in phishing, so the useful decision is who to intervene on, with what intensity, and how to prove whether the program works.
What the regulatory requirements show together
Placed side by side, the sources do not simply describe a duty to "train." They describe a system that combines frequency, exercises, measurement, and evidence, with different levels of detail depending on the framework.
| Standard | What it requires | Frequency | Exercises | What to measure | Evidence | Source |
|---|---|---|---|---|---|---|
| NIST SP 800-53 Rev. 5 AT-2 | Security and privacy literacy training for system users, including managers, senior executives, and contractors; update content and incorporate lessons learned from internal or external incidents or breaches | Initial training for new users and then at the frequency defined by the organization; also when system changes or organization-defined events require it | Includes AT-2(1) "Practical Exercises" and AT-2(3) "Social Engineering and Mining" | not specified | not specified | according to NIST SP 800-53 Rev. 5 |
| PCI DSS v4.0 Req. 12.6 | A formal awareness program so all personnel understand the information security policy and procedures and their role in protecting account data; training must include phishing and related attacks and social engineering | Security awareness education is an ongoing activity | not specified | not specified | not specified | according to PCI DSS v4.0 |
| ISO/IEC 27001:2022 Annex A 6.3 + clause 7.2 | Personnel and relevant interested parties must receive appropriate awareness, education, and training and regular updates; the needed competence must also be determined, competence ensured, measures taken to acquire it, and the effectiveness of those measures evaluated | Regular updates | not specified | Evaluate the effectiveness of the measures taken to acquire competence | Retain documented information as evidence of competence | according to ISO/IEC 27001:2022 and the interpretation cited by Sentrix |
| CISA phishing guidance + CISA FISMA metrics | Implement user training on social engineering and phishing; regularly educate users to identify suspicious emails and links, avoid interacting with them, and report them; combine awareness and training, simulated attacks, and results analysis to adjust training and reduce risk | Employees should review anti-phishing training material annually; monitoring and analysis of quantitative and qualitative metrics is also required | Phishing vulnerability assessment using simulated emails to defined groups; includes evidence of exercises aimed at senior executives, privileged users, or cybersecurity staff | Percentage of users who completed annual training; percentage of new users who completed training before access; number of users who participated in exercises, number who passed, and number who identified and reported phishing | Data supporting the metrics must be collected accurately, consistently, and reproducibly | according to CISA |
| ENISA | Programs should include capability building, periodic assessments of trends and challenges, measurement of cybersecurity behavior, and campaign planning; it recommends identifying objectives, segmenting employees into target groups, selecting channels, implementing, and evaluating | Regular phishing simulations and periodic assessments | Recommends a baseline assessment and regular phishing simulations to identify staff who need additional training | Measurement of cybersecurity behavior; baseline assessment to determine where risks are | not specified | according to ENISA |
| Central Bank of Brazil CMN Resolution 4,893/2021 | Mechanisms to spread a cybersecurity culture, including implementing training programs and periodic employee assessments; the policy must reflect senior management's commitment to continuous improvement of cybersecurity-related procedures | Periodic employee assessment | not specified | Periodic employee assessment | not specified | according to CMN Resolution 4,893/2021 of the Central Bank of Brazil |
Is the platform bought to deliver courses or to measure behavior?
Whalemate's reading is that the material's frameworks look more like a management system than a content library. According to NIST SP 800-53 Rev. 5, control AT-2 requires initial training, follow-on training at a frequency defined by the organization, content updates, and incorporation of lessons learned from incidents. According to PCI DSS v4.0, Requirement 12.6 says awareness education is an ongoing activity, not a one-off course.
That changes the buying criterion. If the problem were only distributing content, a basic LMS would be enough. But according to ENISA, the program should start with a baseline, segment employees into target groups, run regular simulations, and measure behavior, so the useful platform is the one that turns observed behavior into intervention decisions.
Does annual training reduce phishing?
CISA places anti-phishing training at the minimum floor. According to CISA guidance, employees should review training material annually, and the organization should combine awareness, simulated attacks, and results analysis to adjust training and reduce risk.
The empirical evidence in the material complicates any linear promise. According to the large-scale study published on arXiv, the control group had a click rate of 9.8% versus 10.5% among trained participants, with no statistically significant difference, p=0.450. The same study also found no statistically significant difference in reporting, with p=0.417 and effect sizes below 0.01. Whalemate's reading is that annual training may be necessary for compliance and for establishing a baseline, but it is not enough as proof of behavioral effectiveness.
Which matters more, the click or the report?
Organizational practice still focuses heavily on the click. According to a NIST study, 72% of surveyed organizations used phishing simulation click rates to measure effectiveness and 62% used simulated phishing reports. According to another NIST study, training completion was used by 84% of surveyed organizations and click rate by 72%.
The material's recommendations are broader. According to the NCSC, organizations should not measure only how many people clicked but also how many reported. According to CISA, one of the behaviors that should be taught is reporting suspicious emails, links, and attachments. Whalemate's reading is that a platform that only captures clicks leaves out a central signal of operational resilience, because early reporting speeds response and containment.
Can a click rate be compared without context?
According to NIST, no. The NIST Phish Scale exists precisely to give context to click rates based on the human difficulty of detecting the lure. NIST says a rate should not be compared without considering that difficulty.
The same point appears in the empirical evidence. According to the arXiv study, the click rate rose from 7.0% for easy lures to 15.0% for difficult lures. Whalemate's reading is straightforward: a platform that shows a raw rate without classifying campaigns, lures, and populations can lead to wrong decisions because it mixes attack difficulty with user performance.
Do studies show sustained improvement or null results?
They show both, and that tension matters. According to the longitudinal study from Ruhr University Bochum, click rates fell from 19% to 10% as training progressed, with 429,418 emails sent and 59,689 clicks, equivalent to 13.90%, and in a second phase 27,501 links clicked, equivalent to 10.55%. The same study reported that 61 of 77 clients improved their click rate after using simulated emails.
But according to the scoping review in Computers & Security, which identified 42 studies, short-term impact is documented and the evidence for sustained behavioral change is limited. According to arXiv, a recent large-scale study also found no significant main effects on either clicks or reports. Whalemate's reading is that organizations should not buy the promise of guaranteed reduction, but rather the ability to measure locally whether the intervention works in their own population and campaigns.
What should you do with optimistic vendor benchmarks?
Treat them as commercial references, not independent verdicts. According to KnowBe4, its analysis covered 67.7 million phishing simulations, 14.5 million users, and 62.4 thousand organizations. The vendor reported a global average simulated phishing rate of 33.1% before training, a 40% reduction after three months, and a cumulative 86% reduction after 12 months. In another report, KnowBe4 reported a drop from 34.3% to 4.6% for its training category.
The material itself requires those figures to be tested against other evidence. According to SC Magazine UK, in a study of 10 campaigns over eight months involving more than 19,500 employees at a U.S. healthcare organization, click rates did not improve over time and traditional in-person or static training had little measurable effect. Whalemate's reading is that benchmarks help size scenarios, but the purchase decision should rest on methodological traceability, segmentation, and verifiable results inside the organization itself.
