--- title: "CISA and NIST on cyberattack simulations | Whalemate" description: "CISA, NIST, and ISO/IEC 27001 show what to measure in simulations: clicks, reports, opens, difficulty" url: https://www.whalemate.com/en/resources/blog/cisa-nist-cyberattack-simulations/ lang: en updated: 2026-11-10 alternates: es: https://www.whalemate.com/recursos/blog/cisa-nist-simulaciones-ciberataque/ pt-BR: https://www.whalemate.com/br/recursos/blog/cisa-nist-medem-simulacoes-ciberataque/ index: https://www.whalemate.com/llms.txt --- [Back to blog](https://www.whalemate.com/en/resources/blog/) Blog # CISA and NIST on cyberattack simulations CISA, NIST, and ISO/IEC 27001 show what to measure in simulations: clicks, reports, opens, difficulty Nov 10, 2026 By [Federico Hombre](https://www.whalemate.com/en/resources/blog/author/federico-hombre/) 5 min read [Awareness](https://www.whalemate.com/en/resources/blog/topic/awareness/) ![CISA and NIST on cyberattack simulations](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/cisa-nist-cyberattack-simulations.webp) In this article 1. [What does ISO/IEC 27001:2022 actually require, and what does it not define?](#s1) 2. [What does CISA mean by cyber exercise and simulated cyberattack?](#s2) 3. [Why does CISA insist on announced and unannounced exercises?](#s3) 4. [What metrics does NIST use to evaluate awareness?](#s4) 5. [What happens when lure difficulty or personalization changes?](#s5) 6. [Is conventional training enough to reduce risk?](#s6) 7. [What does this mean for a CISO, IT, or compliance team in LATAM?](#s7) 8. [What to read next?](#s8) Cyberattack simulations should not be measured by a single [click rate](https://www.whalemate.com/en/resources/blog/pci-dss-phishing-vs-spam/). For a CISO, IT, or compliance team, the useful question is what capability the exercise tested, which variable changed the outcome, and what program adjustment it makes possible. ISO/IEC 27001:2022 requires awareness, education, training, and periodic updates based on job role, while CISA and NIST push toward exercises, metrics, and more operational corrections. Before looking at each source separately, the combined picture shows something none of them provides on its own: the standard defines the program, CISA defines the exercise, and NIST plus the studies point to the metrics and variables that actually move the result. | Standard, source, or study | What it asks for | What it measures | What changes it | Decision it enables | Source | | --- | --- | --- | --- | --- | --- | | ISO/IEC 27001:2022 clause 6.3 | awareness, education, training, and periodic updates appropriate to job function | not specified | not specified | define an awareness and training program tailored to the role | ISO/IEC 27001:2022 | | ISO/IEC 27001:2022 control A.6.3 | awareness, education, training, and periodic updates appropriate to job function | not specified | not specified | define an awareness and training program tailored to the role | ISO/IEC 27001:2022 | | CISA, cyber exercise and simulated cyberattack | a planned event or authorized simulated attack | prevention, detection, mitigation, response or recovery, and exploitable weaknesses | not specified | test controls and operational capability before a real exploitation | CISA NICCS | | CISA, Cybersecurity Performance Goals | announced and unannounced exercises, with penetration testing, bug bounties, incident simulations, or tabletop exercises | not specified | announced or unannounced design | choose the exercise type to get stronger operational evidence | CISA | | CISA, mock phishing emails | simulated emails sent to defined groups to assess phishing vulnerability | opens, clicks, and potential impact level | not specified | measure real phishing susceptibility | CISA | | NIST, study on awareness programs | not specified | training completion rate, clicks in simulations, audits, simulated and real phishing reports | not specified | select metrics to evaluate program effectiveness | NIST | | SAGE, hospital study | authorized simulations to assess email phishing recognition | staff ability to recognize email attacks | email personalization | segment and personalize lures to measure real susceptibility more accurately | SAGE Journals | | Phin Security, longitudinal study summary | not specified | phishing resilience, clicks, and reports | difficulty of the simulated phishing | adjust difficulty because it changes the outcome significantly | Phin Security | | JISTIC, published simulation | not specified | 17,062 delivered emails, 30.35% opens, 1.09% clicks, and 0.43% data submission | not specified | use an operational baseline to compare campaigns | JISTIC | ## What does ISO/IEC 27001:2022 actually require, and what does it not define? ISO/IEC 27001:2022, in clause 6.3 and control A.6.3, requires awareness, education, training, and periodic updates appropriate to the [job function](https://www.whalemate.com/en/resources/blog/iso-27001-2022-requires-continuous/), according to the provided copy of the standard and control summary. It does not set a simulation KPI or require a single metric. Whalemate's reading is that the standard defines the duty to run a program, not the control panel. ## What does CISA mean by cyber exercise and simulated cyberattack? CISA, through NICCS, defines a cyber exercise as [a planned event](https://niccs.cisa.gov/resources/glossary) to simulate a cyber disruption and develop or test prevention, detection, mitigation, response, or recovery. CISA also defines a simulated cyberattack as an authorized simulated attack to assess defenses and identify exploitable weaknesses. Whalemate's reading is that the value of the exercise comes from the capability it tests, not the theatrics of the lure. ## Why does CISA insist on announced and unannounced exercises? In its Cybersecurity Performance Goals, CISA says exercises can include penetration testing, bug bounties, incident simulations, or tabletop exercises, and should include both announced and unannounced tests. That changes the quality of the evidence because it keeps the team from responding only because it expects the calendar event. Whalemate's reading is that the announcement or the surprise is a design variable, not an administrative detail. ## What metrics does NIST use to evaluate awareness? In its study on awareness programs, NIST reports that the most common metrics were [training completion rate](https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934952), clicks in phishing simulations, and audits or program assessments. The same study also mentions simulated and real phishing reports as evaluation inputs. Whalemate's reading is that useful measurement combines formal compliance with observable behavior. Whalemate Want to see Whalemate in your organization? Book a demo. We will walk through email and QRishing simulations for your team, and what CISO, IT or compliance can decide from there. [Book a demo](https://www.whalemate.com/en/demo/) ## What happens when lure difficulty or personalization changes? Phin Security, summarizing an academic study, says the difficulty of the simulated phishing had a substantial effect, with clicks rising from 7% with easy lures to 15% with difficult lures. In the SAGE hospital study, personalization significantly increased the likelihood of interaction, with 64% not opening the generic phishing email versus 38% not opening the personalized one. Whalemate's reading is that the result depends heavily on how the stimulus is designed. ## Is conventional training enough to reduce risk? Phin Security summarizes a longitudinal study of more than 14,000 employees over 15 months and says conventional integrated training did not make employees more resilient to phishing and may have had unintended negative effects. The same summary adds that sustained simulations and targeted training cut successful compromise rates by about half within six months. Whalemate's reading is that the program has to close the loop between simulation, analysis, intervention, and training adjustment. ## What does this mean for a CISO, IT, or compliance team in LATAM? The useful decision for tomorrow is to stop treating the simulation as an awareness formality and turn it into a capability test. That means defining whether the program will measure opens, clicks, data submission, reporting, audits, and performance in [announced and unannounced exercises](https://www.whalemate.com/en/resources/blog/nist-cisa-context-based-phishing/), depending on the goal. It also means deciding whether the problem lies in lure design, detection and reporting capability, or the training that followed. For IT and security, that changes operations because it requires tracking campaigns, segmentation, results, and corrective actions. For compliance, it changes the evidence because it is no longer enough to show that training took place; the program must produce periodic updates and observable corrections. For a CISO, the practical decision is whether the program is governed by a single click rate or by a set of signals that make it possible to adjust controls and training. ## What to read next? - [Awareness on the platform](https://www.whalemate.com/en/platform/advanced-simulations/) - [LATAM Awareness Platforms: What to Evaluate](https://www.whalemate.com/en/resources/blog/latam-awareness-platforms-what-evaluate/) - [NIST and CISA on social engineering](https://www.whalemate.com/en/resources/blog/nist-cisa-social-engineering/) [See the full topic guide](https://www.whalemate.com/en/platform/advanced-simulations/) Questions ## Frequently asked questions **Is a cyber attack simulation the same as a course?** No. CISA treats it as a planned event or an authorized simulated attack to test capabilities, while ISO/IEC 27001:2022 refers to awareness, education, training, and periodic updates. A course informs; a simulation observes behavior and capability under controlled conditions. In practice, one does not replace the other. **Which metric is best to look at first?** It depends on the objective, but NIST shows that organizations usually look at training completion, clicks in simulated phishing, audits, and reports. If the goal is to detect real improvement, it is advisable to add reports and not rely only on clicks. If the goal is to test operational resilience, you also have to look at opens, data submission, and team response. **Why is lowering the click rate not enough?** Because the cited evidence shows that lure difficulty and personalization greatly change the outcome. A low click rate may come from a lure that was too obvious, not from real improvement. And a high click rate may depend on the chosen segmentation. That is why the right decision is made using several signals, not a single KPI. **What should an organization do after a simulation?** CISA suggests using the analysis of results to modify training and mitigate phishing risk. That means reviewing what happened, identifying whether the failure was in the lure, detection, or reporting, and adjusting the program. If there is no subsequent change, the simulation remains as a record, but not as a control improvement. **What does this add to compliance?** It provides evidence that the program is not just documentary. ISO/IEC 27001:2022 requires updating and suitability to the role, and well-designed simulations make it possible to show that this obligation translates into measurement and adjustments. For compliance, the value lies in proving that a cycle of improvement exists, not just an isolated campaign. Whalemate Is this risk in your program? If this article describes something you already faced — or that the board is asking you to explain — book a demo. [Book a demo](https://www.whalemate.com/en/demo/) ## Would you like to go deeper on this topic? Open this article directly in Claude or ChatGPT — ask questions, get a summary, or explore related ideas. - [Open with Claude](https://claude.ai/new?q=I%27m%20reading%20a%20Whalemate%20article%20titled%20%E2%80%9CCISA%20and%20NIST%20on%20cyberattack%20simulations%E2%80%9D.%20Explain%20the%20key%20ideas%20and%20what%20an%20IT%20team%20or%20CISO%20should%20do%20next.%20Reply%20in%20English%2C%20clearly%20and%20with%20actionable%20steps.) - [Open with ChatGPT](https://chatgpt.com/?q=I%27m%20reading%20a%20Whalemate%20article%20titled%20%E2%80%9CCISA%20and%20NIST%20on%20cyberattack%20simulations%E2%80%9D.%20Explain%20the%20key%20ideas%20and%20what%20an%20IT%20team%20or%20CISO%20should%20do%20next.%20Reply%20in%20English%2C%20clearly%20and%20with%20actionable%20steps.) In this article 1. [What does ISO/IEC 27001:2022 actually require, and what does it not define?](#s1) 2. [What does CISA mean by cyber exercise and simulated cyberattack?](#s2) 3. [Why does CISA insist on announced and unannounced exercises?](#s3) 4. [What metrics does NIST use to evaluate awareness?](#s4) 5. [What happens when lure difficulty or personalization changes?](#s5) 6. [Is conventional training enough to reduce risk?](#s6) 7. [What does this mean for a CISO, IT, or compliance team in LATAM?](#s7) 8. [What to read next?](#s8) Whalemate Is this risk in your program? [Book a demo](https://www.whalemate.com/en/demo/) ## Related articles [!\[Phishing types need separate metrics\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/phishing-types-need-separate-metrics.webp)](https://www.whalemate.com/en/resources/blog/phishing-types-need-separate-metrics/) Awareness Nov 9, 2026 By Federico Hombre ### Phishing types need separate metrics CISO, IT, and compliance teams should manage phishing by channel, target, control, and metric, not as a list of labels. Read more [!\[NIST, ISO and CISA Raise the Bar on Phishing\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/nist-iso-cisa-raise-bar.webp)](https://www.whalemate.com/en/resources/blog/nist-iso-cisa-raise-bar/) Awareness Nov 6, 2026 By Federico Hombre ### NIST, ISO and CISA Raise the Bar on Phishing NIST, ISO, CISA and Brazil show that click rate and an annual campaign are not enough to defend against phishing risk. Read more [!\[NIST on What to Measure in Security Awareness\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/nist-what-measure-security-awareness.webp)](https://www.whalemate.com/en/resources/blog/nist-what-measure-security-awareness/) Awareness Nov 4, 2026 By Federico Hombre ### NIST on What to Measure in Security Awareness NIST says clicks alone are not enough. A useful dashboard also tracks reports, opens, data entry, response time, and message difficulty. Read more [See all articles](https://www.whalemate.com/en/resources/blog/)