Cyberattack simulations should not be measured by a single click rate. For a CISO, IT, or compliance team, the useful question is what capability the exercise tested, which variable changed the outcome, and what program adjustment it makes possible. ISO/IEC 27001:2022 requires awareness, education, training, and periodic updates based on job role, while CISA and NIST push toward exercises, metrics, and more operational corrections.
Before looking at each source separately, the combined picture shows something none of them provides on its own: the standard defines the program, CISA defines the exercise, and NIST plus the studies point to the metrics and variables that actually move the result.
Standard, source, or study
What it asks for
What it measures
What changes it
Decision it enables
Source
ISO/IEC 27001:2022 clause 6.3
awareness, education, training, and periodic updates appropriate to job function
not specified
not specified
define an awareness and training program tailored to the role
ISO/IEC 27001:2022
ISO/IEC 27001:2022 control A.6.3
awareness, education, training, and periodic updates appropriate to job function
not specified
not specified
define an awareness and training program tailored to the role
ISO/IEC 27001:2022
CISA, cyber exercise and simulated cyberattack
a planned event or authorized simulated attack
prevention, detection, mitigation, response or recovery, and exploitable weaknesses
not specified
test controls and operational capability before a real exploitation
CISA NICCS
CISA, Cybersecurity Performance Goals
announced and unannounced exercises, with penetration testing, bug bounties, incident simulations, or tabletop exercises
not specified
announced or unannounced design
choose the exercise type to get stronger operational evidence
CISA
CISA, mock phishing emails
simulated emails sent to defined groups to assess phishing vulnerability
opens, clicks, and potential impact level
not specified
measure real phishing susceptibility
CISA
NIST, study on awareness programs
not specified
training completion rate, clicks in simulations, audits, simulated and real phishing reports
not specified
select metrics to evaluate program effectiveness
NIST
SAGE, hospital study
authorized simulations to assess email phishing recognition
staff ability to recognize email attacks
email personalization
segment and personalize lures to measure real susceptibility more accurately
SAGE Journals
Phin Security, longitudinal study summary
not specified
phishing resilience, clicks, and reports
difficulty of the simulated phishing
adjust difficulty because it changes the outcome significantly
Phin Security
JISTIC, published simulation
not specified
17,062 delivered emails, 30.35% opens, 1.09% clicks, and 0.43% data submission
not specified
use an operational baseline to compare campaigns
JISTIC
What does ISO/IEC 27001:2022 actually require, and what does it not define?
ISO/IEC 27001:2022, in clause 6.3 and control A.6.3, requires awareness, education, training, and periodic updates appropriate to the job function, according to the provided copy of the standard and control summary. It does not set a simulation KPI or require a single metric. Whalemate's reading is that the standard defines the duty to run a program, not the control panel.
What does CISA mean by cyber exercise and simulated cyberattack?
CISA, through NICCS, defines a cyber exercise as a planned event to simulate a cyber disruption and develop or test prevention, detection, mitigation, response, or recovery. CISA also defines a simulated cyberattack as an authorized simulated attack to assess defenses and identify exploitable weaknesses. Whalemate's reading is that the value of the exercise comes from the capability it tests, not the theatrics of the lure.
Why does CISA insist on announced and unannounced exercises?
In its Cybersecurity Performance Goals, CISA says exercises can include penetration testing, bug bounties, incident simulations, or tabletop exercises, and should include both announced and unannounced tests. That changes the quality of the evidence because it keeps the team from responding only because it expects the calendar event. Whalemate's reading is that the announcement or the surprise is a design variable, not an administrative detail.
What metrics does NIST use to evaluate awareness?
In its study on awareness programs, NIST reports that the most common metrics were training completion rate, clicks in phishing simulations, and audits or program assessments. The same study also mentions simulated and real phishing reports as evaluation inputs. Whalemate's reading is that useful measurement combines formal compliance with observable behavior.
What happens when lure difficulty or personalization changes?
Phin Security, summarizing an academic study, says the difficulty of the simulated phishing had a substantial effect, with clicks rising from 7% with easy lures to 15% with difficult lures. In the SAGE hospital study, personalization significantly increased the likelihood of interaction, with 64% not opening the generic phishing email versus 38% not opening the personalized one. Whalemate's reading is that the result depends heavily on how the stimulus is designed.
Is conventional training enough to reduce risk?
Phin Security summarizes a longitudinal study of more than 14,000 employees over 15 months and says conventional integrated training did not make employees more resilient to phishing and may have had unintended negative effects. The same summary adds that sustained simulations and targeted training cut successful compromise rates by about half within six months. Whalemate's reading is that the program has to close the loop between simulation, analysis, intervention, and training adjustment.
What does this mean for a CISO, IT, or compliance team in LATAM?
The useful decision for tomorrow is to stop treating the simulation as an awareness formality and turn it into a capability test. That means defining whether the program will measure opens, clicks, data submission, reporting, audits, and performance in announced and unannounced exercises, depending on the goal. It also means deciding whether the problem lies in lure design, detection and reporting capability, or the training that followed.
For IT and security, that changes operations because it requires tracking campaigns, segmentation, results, and corrective actions. For compliance, it changes the evidence because it is no longer enough to show that training took place; the program must produce periodic updates and observable corrections. For a CISO, the practical decision is whether the program is governed by a single click rate or by a set of signals that make it possible to adjust controls and training.
Is a cyber attack simulation the same as a course?
No. CISA treats it as a planned event or an authorized simulated attack to test capabilities, while ISO/IEC 27001:2022 refers to awareness, education, training, and periodic updates. A course informs; a simulation observes behavior and capability under controlled conditions. In practice, one does not replace the other.
Which metric is best to look at first?
It depends on the objective, but NIST shows that organizations usually look at training completion, clicks in simulated phishing, audits, and reports. If the goal is to detect real improvement, it is advisable to add reports and not rely only on clicks. If the goal is to test operational resilience, you also have to look at opens, data submission, and team response.
Why is lowering the click rate not enough?
Because the cited evidence shows that lure difficulty and personalization greatly change the outcome. A low click rate may come from a lure that was too obvious, not from real improvement. And a high click rate may depend on the chosen segmentation. That is why the right decision is made using several signals, not a single KPI.
What should an organization do after a simulation?
CISA suggests using the analysis of results to modify training and mitigate phishing risk. That means reviewing what happened, identifying whether the failure was in the lure, detection, or reporting, and adjusting the program. If there is no subsequent change, the simulation remains as a record, but not as a control improvement.
What does this add to compliance?
It provides evidence that the program is not just documentary. ISO/IEC 27001:2022 requires updating and suitability to the role, and well-designed simulations make it possible to show that this obligation translates into measurement and adjustments. For compliance, the value lies in proving that a cycle of improvement exists, not just an isolated campaign.
Would you like to go deeper on this topic?
Open this article directly in Claude or ChatGPT —
ask questions, get a summary, or explore related ideas.
By 
