PCI, ISO and NIST: awareness is not annual training

Cybersecurity awareness training, viewed through ISO/IEC 27001:2022, ISO/IEC 27002:2022, NIST, PCI DSS v4.0, CISA, NCSC and ENISA, cannot be managed like an annual course with a completion rate. Comparing the frameworks with operational evidence points to a different model: a continuous program tailored to role, delivered through multiple channels, and measured through behavior and reporting as well. At the same time, several sources question whether click rate should be the main or only KPI for judging risk or effectiveness.

What changes when the frameworks are compared side by side

None of the source material brings together in one view what each framework requires or recommends on frequency, role tailoring, channels, content and evidence. Placed side by side, the issue is not choosing between a course, a simulation or a campaign, but designing a continuous, auditable control with behavior-based metrics.

Why annual training is no longer enough

PCI DSS v4.0 turns the requirement into concrete obligations. According to PCI DSS v4.0, the program must be reviewed at least once every 12 months, training must be given at hire and at least once every 12 months, and acknowledgment that the person read and understood the security policies and procedures must also be obtained at least once every 12 months.

But the PCI Security Standards Council guidance says something stricter in practice: awareness should be a continuous program, not a yearly standalone activity, to maintain awareness on a daily basis. Whalemate’s reading is that the annual minimum may satisfy documentary compliance, but it does not prove sustained management of human risk.

What ISO adds that a course does not solve

ISO/IEC 27001:2022, in clause 7.3 Awareness, requires people to know the information security policy, their contribution to the system’s effectiveness, and the consequences of not meeting the requirements, according to the public text of the standard. That shifts the focus from attending training to proving the organization achieved understanding relevant to the management system.

ISO/IEC 27002:2022 control 6.3 adds that staff and relevant interested parties should receive appropriate information, education and training, with periodic updates according to their role, according to the public copy of the control. It also lists physical and virtual channels, from campaigns and posters to newsletters, briefings, e-learning and emails. Whalemate reads this as ISO pushing a segmented, multimodal program, not a single uniform module for the entire organization.

Are awareness and training the same?

NIST says no. According to the NIST glossary, awareness aims to focus attention on security and enable people to recognize issues and respond, while training is more formal and seeks to build knowledge and skills for job performance.

The difference matters for budget, design and evidence. According to NIST SP 800-50 Rev. 1, awareness and training is a foundational program for all personnel, and training can include instructor-led or online courses, exercises or other methods. Whalemate’s reading is that mixing both concepts into one course often produces programs that neither change behavior nor build role-specific capabilities.

What is being measured today, and what does NIST criticize?

In a survey of federal organizations cited by NIST, training completion was the most common effectiveness measure, at 84%, and click rate in phishing simulations was second, at 72%. That shows how many organizations still evaluate the program with two easy-to-audit metrics.

But NIST warns that phishing programs often use click rate as the only metric, and says exercises should consider the human element and context to assess effectiveness and progress. According to the NIST Phish Scale, reporting rates, repeat click rates and protectors should also be considered. Whalemate’s reading is that a dashboard based only on completion rate and click rate simplifies audit work, but weakens risk decisions.

Does click rate work as the main KPI?

It does, but used badly it can confuse more than it clarifies. Proofpoint says click rate in phishing tests alone should not represent the impact or success of an awareness program, even though it reported an average failure rate of 4.93% across all organizations and campaigns analyzed.

NCSC goes in the same direction. According to NCSC, in addition to measuring how many people clicked, organizations should measure how many reported, and measuring only the absence of reports can encourage employees to hide problems. Whalemate’s reading is that click rate can be a useful signal, but not a governing KPI or a basis for discipline.

What do simulations show when put in context?

UNICC offers a key data point for avoiding apples-to-oranges comparisons. According to UNICC, using NIST difficulty in its report, click rate was 5.74% for low-difficulty messages, 12.10% for medium difficulty and 36.15% for high difficulty.

That contrast makes it risky to use benchmarks without normalizing difficulty or population. Comparing those numbers with the 4.93% average failure rate reported by Proofpoint cannot be read as if both figures measured exactly the same thing, because Proofpoint itself says the number depends on the campaigns and the study population. Whalemate’s reading is that a benchmark without difficulty context can reward easy campaigns and punish better-designed ones.

Are simulations overrepresented?

CISA uses them as an operational tool. According to CISA guidance on counter-phishing, a six-week Phishing Campaign Assessment measures workforce susceptibility to phishing lures and its results can guide anti-phishing training.

NCSC puts a limit on that centrality. According to NCSC, phishing simulations are often overrepresented in defenses, detecting all phishing is difficult, and spear phishing is even harder to identify. Whalemate’s reading is that simulations are useful for spotting exposure and segmenting interventions, but lose value when they replace the entire program.

Is the evidence of improvement solid or disputed?

There is evidence of improvement, but it is neither linear nor sufficient on its own. According to KnowBe4, based on vendor data covering 67,718,305 simulations, 14,508,441 users and 62,460 organizations, the global average Phish-prone Percentage fell from 32.9% before training to 4.8% after 12 months of sustained training. The source matters because it is not an independent study.

On the other side, the arXiv study titled "Anti-Phishing Training Does Not Work" questions whether anti-phishing training necessarily produces the expected result and says an isolated click rate is not enough to infer learning or reduced risk. Whalemate’s reading is that the debate is no longer about whether to train or not, but what kind of training, for whom, with what measurement and against which expected behavior.

Which metrics hold up better in an audit and a risk decision?

ENISA recommends defining metrics before designing awareness activities so their effectiveness and reach can be assessed in the short and long term. It also proposes measuring the percentage decrease in incidents, the number of incidents reported, the number of positive test results and the increased knowledge level, according to its practical material.

NIST SP 800-50 Rev. 1 proposes measuring behavior change per participant, for example the percentage of participants who changed a work behavior such as reporting a suspicious email. PCI SSC recommends recording who takes the training, whether it is completed successfully, comprehension through assessments and feedback, and reports of malicious emails. Whalemate’s reading is that the minimum useful set combines compliance evidence, comprehension and observable behavior.

What minimum content should the program cover?

Content depends on the applicable framework, but there is a common core. According to ISO/IEC 27001:2022, people must know the policy, their contribution to the system and the consequences of noncompliance. According to ISO/IEC 27002:2022 control 6.3, awareness should cover management commitment, applicable obligations, personal responsibility, basic security procedures and points of contact for advice.

If the organization is under PCI DSS v4.0, the floor is more specific. According to requirement 12.6.3.1, training must include threats and vulnerabilities that could affect CDE security, including phishing and related attacks, and social engineering. CISA also recommends covering phishing, business email compromise, operational security and passwords. Whalemate’s reading is that the content should come from the risk map and the applicable framework, not from a generic vendor catalog.

What does this mean for LATAM, compliance and data protection?

In Brazil, LGPD, Law 13.709, requires controllers and others involved in any phase of processing to guarantee the information security provided for in the law regarding personal data, even after processing ends, according to Brazil’s ANPD. In addition, ANPD published information security guidance for small processing agents that includes, according to DLA Piper, suggestions such as awareness and training programs.

For Chile, one Fensivo source describes for providers of essential services and operators of vital importance an obligation for continuous training and awareness, and adds that the lack of records of training actions and programs counts as noncompliance. Because the material itself says so, that should be read as that source’s regulatory interpretation, not generalized to all of LATAM. Whalemate’s reading is that, even outside PCI, the regional expectation points to documented, continuous and traceable programs.

Operational consequences for CISOs, IT and compliance teams in LATAM

If the program had to be redesigned tomorrow, the useful decision is not to chase only who completed the course. The combination of NIST, ISO/IEC 27001:2022, ISO/IEC 27002:2022, PCI DSS v4.0, CISA, NCSC and ENISA suggests treating cybersecurity awareness training as a continuous, auditable control, segmented by function, with onboarding, periodic review, multiple channels and specific content on phishing and social engineering when relevant.

The dashboard changes too. According to ENISA, NIST, NCSC and PCI SSC, it makes sense to combine completion and policy acknowledgment with behavior and risk metrics, such as phishing reports, behavior change, incidents reported, before-and-after results and the difficulty context of simulations. For IT, CISO and compliance teams in LATAM, a Human Risk Management approach lets them use those signals to prioritize interventions and show effectiveness without reducing the program to clicks or attendance.

What to read next?

• Awareness on the platform
• LATAM Awareness Platforms: What to Evaluate
• NIST and CISA on social engineering