--- title: "NIST on What to Measure in Security Awareness | Whalemate" description: "NIST says clicks alone are not enough. A useful dashboard also tracks reports, opens, data entry, response time, and message difficulty." url: https://www.whalemate.com/en/resources/blog/nist-what-measure-security-awareness/ lang: en updated: 2026-11-04 alternates: es: https://www.whalemate.com/recursos/blog/nist-medir-awareness/ pt-BR: https://www.whalemate.com/br/recursos/blog/nist-medir-awareness/ index: https://www.whalemate.com/llms.txt --- [Back to blog](https://www.whalemate.com/en/resources/blog/) Blog # NIST on What to Measure in Security Awareness NIST says clicks alone are not enough. A useful dashboard also tracks reports, opens, data entry, response time, and message difficulty. Nov 4, 2026 By [Federico Hombre](https://www.whalemate.com/en/resources/blog/author/federico-hombre/) 5 min read [Awareness](https://www.whalemate.com/en/resources/blog/topic/awareness/) ![NIST on What to Measure in Security Awareness](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/nist-what-measure-security-awareness.webp) In this article 1. [Why does NIST not leave it at clicks alone?](#s1) 2. [What does the NIST Phish Scale add to the dashboard?](#s2) 3. [What metrics appear beyond clicks?](#s3) 4. [What happens when you compare by business unit?](#s4) 5. [What do successive waves teach us?](#s5) 6. [What tension exists between NIST and JISTIC?](#s6) 7. [What tension exists between NIST and the enterprise study?](#s7) 8. [What about vendor figures and preprints?](#s8) 9. [What operational takeaway does this leave for tomorrow?](#s9) 10. [What to read next?](#s10) "Awareness indicators" cannot be read as a single click rate. The material argues that a useful dashboard for CISOs, IT, and compliance combines opens, clicks, reports, data entry, and [response time](https://ejournal.unikama.ac.id/index.php/JISTIC/article/download/14066/5316), and that those numbers change meaning based on message difficulty and the business unit context. The practical choice is not to measure more just to measure more, but to avoid false conclusions when a wave improves or worsens for reasons that have nothing to do with clicks. Before getting into the tensions, it helps to organize what each source brings to the table. The comparison shows that NIST pushes context and reporting, JISTIC broadens the set of measurable indicators, and the business study adds segmentation by business unit and wave-by-wave reading. Looking at them together makes the gap clear between an isolated metric and an operational reading. | Standard or source | Indicator | What it measures | What the source warns about | Source | | --- | --- | --- | --- | --- | | NIST Phish Scale | human detection difficulty | how hard it is to detect a phishing email | it contextualizes click-rate variability and is not limited to a single click metric | NIST | | NIST awareness program | click rate, report rate | how many people click or report phishing | click rates alone do not provide a complete view of employee behavior | NIST | | JISTIC article | open rate, click rate, data submission rate, response time | measurable indicators for evaluating phishing simulations | it defines open rate, click rate, and the other indicators as exercise metrics | JISTIC | | Enterprise phishing study | click rate, report rate | changes in response to simulations across business units and over three waves | the timing of training before simulated emails did not significantly affect click rates | ScienceDirect | | TitanHQ | PVP and SPR | employee susceptibility and mobile phishing | its metrics are presented as vendor definitions for analyzing simulations and training | TitanHQ | | arXiv | phishing susceptibility | reduced susceptibility after sustained simulations and immediate feedback | the result should not be treated as a universal benchmark | arXiv | ## Why does NIST not leave it at clicks alone? NIST says click rates alone do not provide [a complete view](https://www.nist.gov/blogs/cybersecurity-insights/cybersecurity-awareness-month-2023-blog-series-recognizing-and) of employee behavior, and that the Phish Scale is meant to contextualize [email difficulty](https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=956439). That changes the compliance discussion, because the same percentage can mean different things if the scenario was easy or hard to spot. Whalemate treats that point as the baseline: the click is interpreted, not worshiped. ## What does the NIST Phish Scale add to the dashboard? According to NIST, the NIST Phish Scale includes signals from the email and the alignment of the premise with the recipient's context. It also takes into account the job relevance of the scenario, alignment with other situations or events, the level of concern about the consequences of not clicking, and prior exposure to training or warnings. For a CISO, that means performance data cannot be separated from exercise design. ## What metrics appear beyond clicks? NIST mentions click rate and [report rate](https://www.whalemate.com/en/resources/blog/nist-cis-measuring-social-engineering/), but the academic article in JISTIC expands the dashboard with open rate, data submission rate, and response time. JISTIC also defines open rate as the percentage of users who open the simulated email out of the total correctly delivered, and click rate as the percentage that clicks the link. Whalemate's reading is that mature operations do not stop at a single user response. ## What happens when you compare by business unit? The enterprise-scale study found variations in click rates across different business units, and also reported that the timing of training before the simulated emails did not significantly affect that rate. That combination breaks a common practice, which is attributing improvement or decline to a single cause. The useful reading is different: response changes by group, and that requires segmentation before deciding on a broad action. Whalemate Want to see Whalemate in your organization? Book a demo. We will walk through email and QRishing simulations for your team, and what CISO, IT or compliance can decide from there. [Book a demo](https://www.whalemate.com/en/demo/) ## What do successive waves teach us? ScienceDirect reported that click and report rates improved over three waves of simulated phishing. That matters because it shows behavior is not static, and a single snapshot can be misleading. For Whalemate, the operational question is not whether there was improvement, but in which wave, in which unit, and with what kind of message it happened. ## What tension exists between NIST and JISTIC? NIST presents click rate as a common metric that should be read together with reporting and [scenario context](https://www.whalemate.com/en/resources/blog/nist-cisa-context-based-phishing/). JISTIC, by contrast, adds opens, data submission, and response time as measurable indicators, broadening the dashboard beyond clicks. Whalemate's reading is that there is no contradiction, only different layers: NIST explains how to interpret, while JISTIC helps expand what to observe. ## What tension exists between NIST and the enterprise study? NIST insists that message difficulty and scenario context are key to understanding click variability. The enterprise study adds that this variation also appears across business units and that training timing did not significantly move click rates. For a CISO or compliance team, that means the common mistake is not measuring too little, but assigning causality in bulk when the phenomenon is segmented. ## What about vendor figures and preprints? TitanHQ says susceptibility to phishing dropped by 92% after training with SafeTitan, and arXiv reports a 52% reduction over a six to eight month period with simulations and immediate feedback. Both results are useful as specific evidence, but they should not be read as universal benchmarks. Whalemate treats them as attributed results, not as a general rule for every organization. ## What operational takeaway does this leave for tomorrow? For CISOs, IT, or compliance teams in LATAM, the [decision should not be](https://www.whalemate.com/en/resources/blog/human-risk-management-what-measures/) "did the click rate go up or down," but where to intervene. If reporting improves but data submission remains high, training is not stopping practical damage. If the variation is concentrated in one unit, or if the exercise was highly difficult according to the Phish Scale, it makes more sense to prioritize targeted reinforcement, segmentation, and wave-by-wave analysis than a [generic program](https://www.whalemate.com/en/resources/blog/security-awareness-training-measure-clicks/). ## What to read next? - [Awareness on the platform](https://www.whalemate.com/en/platform/advanced-simulations/) - [LATAM Awareness Platforms: What to Evaluate](https://www.whalemate.com/en/resources/blog/latam-awareness-platforms-what-evaluate/) - [NIST and CISA on social engineering](https://www.whalemate.com/en/resources/blog/nist-cisa-social-engineering/) [See the full topic guide](https://www.whalemate.com/en/platform/advanced-simulations/) Questions ## Frequently asked questions **What is the minimum an awareness program should measure?** According to the material, the reasonable floor is not just the click. A useful dashboard adds report rate, open rate, data submission rate, and response time, in addition to click rate. NIST adds that message context also matters, because the same reaction does not mean the same thing if the scenario was easy or difficult to detect. **Why can click rate alone be misleading?** Because NIST warns that it does not provide a complete view of employee behavior. A high click rate may reflect a very convincing message, not necessarily a global failure of the program. In addition, the enterprise study showed differences across business units, so the average may hide very different behaviors within the same organization. **What is the NIST Phish Scale used for in practice?** It is used to interpret whether an email was easier or harder to detect, using signals from the message and the audience context. That helps read click variation more accurately and avoids jumping to conclusions. For a security team, it also structures the conversation with the business, because performance is compared against the difficulty of the exercise, not against an isolated number. **How is this used to decide an intervention?** First, segment by business unit and review the waves separately, according to the enterprise study. Then, cross click rate with report rate, open rate, data submission rate, and response time, and interpret it with the Phish Scale when applicable. Based on that reading, the action may be targeted training, reinforcement by group, or an adjustment to the type of simulation. Whalemate Is this risk in your program? If this article describes something you already faced — or that the board is asking you to explain — book a demo. [Book a demo](https://www.whalemate.com/en/demo/) ## Would you like to go deeper on this topic? Open this article directly in Claude or ChatGPT — ask questions, get a summary, or explore related ideas. - [Open with Claude](https://claude.ai/new?q=I%27m%20reading%20a%20Whalemate%20article%20titled%20%E2%80%9CNIST%20on%20What%20to%20Measure%20in%20Security%20Awareness%E2%80%9D.%20Explain%20the%20key%20ideas%20and%20what%20an%20IT%20team%20or%20CISO%20should%20do%20next.%20Reply%20in%20English%2C%20clearly%20and%20with%20actionable%20steps.) - [Open with ChatGPT](https://chatgpt.com/?q=I%27m%20reading%20a%20Whalemate%20article%20titled%20%E2%80%9CNIST%20on%20What%20to%20Measure%20in%20Security%20Awareness%E2%80%9D.%20Explain%20the%20key%20ideas%20and%20what%20an%20IT%20team%20or%20CISO%20should%20do%20next.%20Reply%20in%20English%2C%20clearly%20and%20with%20actionable%20steps.) In this article 1. [Why does NIST not leave it at clicks alone?](#s1) 2. [What does the NIST Phish Scale add to the dashboard?](#s2) 3. [What metrics appear beyond clicks?](#s3) 4. [What happens when you compare by business unit?](#s4) 5. [What do successive waves teach us?](#s5) 6. [What tension exists between NIST and JISTIC?](#s6) 7. [What tension exists between NIST and the enterprise study?](#s7) 8. [What about vendor figures and preprints?](#s8) 9. [What operational takeaway does this leave for tomorrow?](#s9) 10. [What to read next?](#s10) Whalemate Is this risk in your program? [Book a demo](https://www.whalemate.com/en/demo/) ## Related articles [!\[CISA and NIST on cyberattack simulations\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/cisa-nist-cyberattack-simulations.webp)](https://www.whalemate.com/en/resources/blog/cisa-nist-cyberattack-simulations/) Awareness Nov 10, 2026 By Federico Hombre ### CISA and NIST on cyberattack simulations CISA, NIST, and ISO/IEC 27001 show what to measure in simulations: clicks, reports, opens, difficulty Read more [!\[Phishing types need separate metrics\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/phishing-types-need-separate-metrics.webp)](https://www.whalemate.com/en/resources/blog/phishing-types-need-separate-metrics/) Awareness Nov 9, 2026 By Federico Hombre ### Phishing types need separate metrics CISO, IT, and compliance teams should manage phishing by channel, target, control, and metric, not as a list of labels. Read more [!\[NIST, ISO and CISA Raise the Bar on Phishing\](https://lvaxvhnlaeradaprhyqy.supabase.co/storage/v1/object/public/editorial/blog-covers/nist-iso-cisa-raise-bar.webp)](https://www.whalemate.com/en/resources/blog/nist-iso-cisa-raise-bar/) Awareness Nov 6, 2026 By Federico Hombre ### NIST, ISO and CISA Raise the Bar on Phishing NIST, ISO, CISA and Brazil show that click rate and an annual campaign are not enough to defend against phishing risk. Read more [See all articles](https://www.whalemate.com/en/resources/blog/)