What I am unclear about is—, what did the participants know going in about the study? Are they aware you are dealing with email phishing or was it just the broad topic of AI?
We recruited 108 senior volunteers through two organizations: a large seniors’ community in southern California and a seniors’ computer club in northern California. Participants agreed to participate in a behavioral study on emails. Each person received between 1–3 emails from different email templates, with content successfully generated through various jailbreaking techniques similar to those tested in the safety guardrail evaluation. Each email contained a unique URL, and our webserver tracked which links were opened. When participants clicked on the phishing links, they were immediately directed to a web page explaining they had participated in a study, where we asked them to answer follow-up questions. Several participants agreed to be interviewed afterward to provide additional insights into why they clicked and their experiences with phishing attempts.
In the abstract of that paper: "In a human validation study with 108 senior vol-
unteers, AI-generated phishing emails successfully compro-
mised 11% of participants."
The Introduction mentions: "Americans aged 60 and older reported losses
of nearly $4.9 billion to online fraud in 2024"
