Three Ways to Make Awareness Measurable

By Julie Fugett

So much of what we do in information security is immediately measurable: how many packets did the firewall drop? How many security incidents did we handle this week? Elsewhere, however, our reach can be more difficult to measure. How effective is our awareness program? Are we talking about the right topics to the right people? Does anybody even care?

My primary job duties center on security awareness, so it’s important to me that people care. I like to joke that I’m “justifying my existence” by compiling metrics regarding security awareness, but that’s only half the story. Showing that your security awareness program is reaching its intended audiences may have compliance implications as well. Regulations like HIPAA and contractual agreements like the Payment Card Industry Data Security Standard have security awareness requirements built-in. Depending on the type of data your organization handles, you may have some of these obligations placed at your feet!

You should ensure that your efforts are actually measurable. Posters on the break room bulletin board are great, but how do you know they’re having an impact? A banner on the company intranet draws attention to your cause, but have you taken steps to track how many people are clicking through to your website?  When you give presentations, how do you know if anybody even paid attention?

It can be overwhelming to think about all the data points you “should” track when it comes to security awareness. My advice: start small. Do the easy things. There will be time later to draw detailed conclusions about the efficacy of your campaign. If you are just beginning, try to put those things out of your mind—if you’re anything like me, you’ll get so caught up wanting it to be “perfect” that you’ll never take that first step.

One of the simplest things I do is count how many people I talk to during the course of a year. I have a spreadsheet where I record the date, the nature of the event, and how many people showed up. When you are showing your managers how effective your awareness campaigns are, it is far more effective to say “I talked to 1500 people in 2008” than “boy, we did a BUNCH of stuff for Security Awareness Month in October!” If you fight nerves during your presentations, have someone else count for you so you don’t forget.

Asking for specific, written feedback can be hugely beneficial. Bribing for it is even more so.  I teach workshops for which there is optional online feedback that can be given after the workshop is finished. Probably 10% of my students fill out that feedback. I see three reasons for this:

1.     It’s online. My presentations tend to make people skittish about the Internet for awhile, so they don’t believe me when I say the feedback is anonymous.

2.     It’s kind of long. The feedback form asks at least 10 questions—most of them about the class and the instructor.

3.     They get nothing for their time. No fun swag, no free soda, just a “thanks for your feedback.”

On the other hand, the feedback I solicit during Cybersecurity Awareness Month in October gets nearly 100% participation. Here’s why:

1.     It’s anonymous—I don’t even give them a place to write their name

2.     There are three questions, and they’re mostly about the student’s perceptions and concerns.

3.     The bottom of the feedback form tears off and enters the attendee in a drawing for prizes.

Finding out about what worries your coworkers about information security will help you learn where to focus your efforts. Knowing their frame of mind will give you an “in” so you can discuss your issues (encryption, document disposal, mobile devices, whatever) in a manner that is more meaningful to them.Tracking this feedback is another great way to show management that you are running and agile and responsive security awareness program.