"Awareness indicators" cannot be read as a single click rate. The material argues that a useful dashboard for CISOs, IT, and compliance combines opens, clicks, reports, data entry, and response time, and that those numbers change meaning based on message difficulty and the business unit context. The practical choice is not to measure more just to measure more, but to avoid false conclusions when a wave improves or worsens for reasons that have nothing to do with clicks.
Before getting into the tensions, it helps to organize what each source brings to the table. The comparison shows that NIST pushes context and reporting, JISTIC broadens the set of measurable indicators, and the business study adds segmentation by business unit and wave-by-wave reading. Looking at them together makes the gap clear between an isolated metric and an operational reading.
Standard or source
Indicator
What it measures
What the source warns about
Source
NIST Phish Scale
human detection difficulty
how hard it is to detect a phishing email
it contextualizes click-rate variability and is not limited to a single click metric
NIST
NIST awareness program
click rate, report rate
how many people click or report phishing
click rates alone do not provide a complete view of employee behavior
NIST
JISTIC article
open rate, click rate, data submission rate, response time
measurable indicators for evaluating phishing simulations
it defines open rate, click rate, and the other indicators as exercise metrics
JISTIC
Enterprise phishing study
click rate, report rate
changes in response to simulations across business units and over three waves
the timing of training before simulated emails did not significantly affect click rates
ScienceDirect
TitanHQ
PVP and SPR
employee susceptibility and mobile phishing
its metrics are presented as vendor definitions for analyzing simulations and training
TitanHQ
arXiv
phishing susceptibility
reduced susceptibility after sustained simulations and immediate feedback
the result should not be treated as a universal benchmark
arXiv
Why does NIST not leave it at clicks alone?
NIST says click rates alone do not provide a complete view of employee behavior, and that the Phish Scale is meant to contextualize email difficulty. That changes the compliance discussion, because the same percentage can mean different things if the scenario was easy or hard to spot. Whalemate treats that point as the baseline: the click is interpreted, not worshiped.
What does the NIST Phish Scale add to the dashboard?
According to NIST, the NIST Phish Scale includes signals from the email and the alignment of the premise with the recipient's context. It also takes into account the job relevance of the scenario, alignment with other situations or events, the level of concern about the consequences of not clicking, and prior exposure to training or warnings. For a CISO, that means performance data cannot be separated from exercise design.
What metrics appear beyond clicks?
NIST mentions click rate and report rate, but the academic article in JISTIC expands the dashboard with open rate, data submission rate, and response time. JISTIC also defines open rate as the percentage of users who open the simulated email out of the total correctly delivered, and click rate as the percentage that clicks the link. Whalemate's reading is that mature operations do not stop at a single user response.
What happens when you compare by business unit?
The enterprise-scale study found variations in click rates across different business units, and also reported that the timing of training before the simulated emails did not significantly affect that rate. That combination breaks a common practice, which is attributing improvement or decline to a single cause. The useful reading is different: response changes by group, and that requires segmentation before deciding on a broad action.
What do successive waves teach us?
ScienceDirect reported that click and report rates improved over three waves of simulated phishing. That matters because it shows behavior is not static, and a single snapshot can be misleading. For Whalemate, the operational question is not whether there was improvement, but in which wave, in which unit, and with what kind of message it happened.
What tension exists between NIST and JISTIC?
NIST presents click rate as a common metric that should be read together with reporting and scenario context. JISTIC, by contrast, adds opens, data submission, and response time as measurable indicators, broadening the dashboard beyond clicks. Whalemate's reading is that there is no contradiction, only different layers: NIST explains how to interpret, while JISTIC helps expand what to observe.
What tension exists between NIST and the enterprise study?
NIST insists that message difficulty and scenario context are key to understanding click variability. The enterprise study adds that this variation also appears across business units and that training timing did not significantly move click rates. For a CISO or compliance team, that means the common mistake is not measuring too little, but assigning causality in bulk when the phenomenon is segmented.
What about vendor figures and preprints?
TitanHQ says susceptibility to phishing dropped by 92% after training with SafeTitan, and arXiv reports a 52% reduction over a six to eight month period with simulations and immediate feedback. Both results are useful as specific evidence, but they should not be read as universal benchmarks. Whalemate treats them as attributed results, not as a general rule for every organization.
What operational takeaway does this leave for tomorrow?
For CISOs, IT, or compliance teams in LATAM, the decision should not be "did the click rate go up or down," but where to intervene. If reporting improves but data submission remains high, training is not stopping practical damage. If the variation is concentrated in one unit, or if the exercise was highly difficult according to the Phish Scale, it makes more sense to prioritize targeted reinforcement, segmentation, and wave-by-wave analysis than a generic program.
What is the minimum an awareness program should measure?
According to the material, the reasonable floor is not just the click. A useful dashboard adds report rate, open rate, data submission rate, and response time, in addition to click rate. NIST adds that message context also matters, because the same reaction does not mean the same thing if the scenario was easy or difficult to detect.
Why can click rate alone be misleading?
Because NIST warns that it does not provide a complete view of employee behavior. A high click rate may reflect a very convincing message, not necessarily a global failure of the program. In addition, the enterprise study showed differences across business units, so the average may hide very different behaviors within the same organization.
What is the NIST Phish Scale used for in practice?
It is used to interpret whether an email was easier or harder to detect, using signals from the message and the audience context. That helps read click variation more accurately and avoids jumping to conclusions. For a security team, it also structures the conversation with the business, because performance is compared against the difficulty of the exercise, not against an isolated number.
How is this used to decide an intervention?
First, segment by business unit and review the waves separately, according to the enterprise study. Then, cross click rate with report rate, open rate, data submission rate, and response time, and interpret it with the Phish Scale when applicable. Based on that reading, the action may be targeted training, reinforcement by group, or an adjustment to the type of simulation.
Would you like to go deeper on this topic?
Open this article directly in Claude or ChatGPT —
ask questions, get a summary, or explore related ideas.
By 
