Striking quantitative gold

Alternate Title: Metrics?   We don’t need no stinking metrics…

In the last post I talked about where you can find information to support management in risk decisions.   This time we’ll look more closely at when quantitative input is most valuable to a small organization and where we might find it in your organization.

For my purposes, I’ll use a specific meaning of quantitative data:   b. Of, relating to, or susceptible of measurement.   While the other meanings in the dictionary are valid and useful, given the difficulty of obtaining data in a small company, this one is most practical.    This definition is also useful because it identifies the greatest challenge for you in a small organization while also providing a path to discovering the difference between gold (persuasive information) and fool’s gold (nice data but ultimately wasted dollars if you did anything more to gather it than just ask a question of someone you knew had the answer).

Let’s revisit the locations where we could find useful data and I’ll highlight a few examples of ways to gather quantitative data to help persuade management for change.  In an emergency, it could also dissuade them from reducing your budget by demonstrating the value of your work.   Please don’t forget these are only examples and not a best practice or finite list that you can use to solve all your problems.   Note:  If these don’t make a lot of sense, I’d encourage you to pick up “How to Measure Anything:  Finding the Value of Intangibles” by Douglas W. Hubbard for some great tips and tools.

• Internal Experience - As soon as you use the word experience, people naturally think the info will be subjective, but what if you put a lot of people’s experiences together?  Does it invalidate all their qualitative opinions if one element of a set of data is qualitative?  No, 80% satisfaction from 10 managers = 8 managers on your side.   That’s very meaningful to your boss and the CEO so it's useful and easy to measure.

• Event Records and Logs - Logs are great sources of a lot of information.  Others have devoted whole blogs, whitepapers and even books on the subject.   But what if you don’t have a fancy SIEM with some canned reports to help you spot trends?   How about using Excel or grep to parse the logs for key events such as AV update failures/successes, AV events of a certain type?   That’ll give you useful reports but is it actionable?  Not without additional processing.   What if you counted the number of updates and subtracted the number of systems to show trends of update failures?   What if you wait a day and then took the most recent update for each system and subtracted the time-stamp from now to see how long on average it typically takes for signatures to propagate.   Would this be a useful factor to estimate exposure to the next AV attack?  (I can’t say great or the SIRA list will crucify me but it can be a “great” factor in the right context.)

• Financial Records - Financial records can be hard to obtain however, they are already in the hands of the people you most want to persuade after your manager.  Getting value and support for your plan may be as simple as getting them to ask the question on their own data.

o How many laptops did we replace last year because they were lost?
o How many vendors were penalized for a failure to deliver on a Service Level Agreement (SLA)?
o How many terminated employees failed to return equipment?

Of course, you want to be ready to ask the next question or propose a solution when they come back with their answer.   Remember, in a small, busy organization, they won’t bring it back to you unless the answers is bad enough to worry them.  After all, you’re the person they least want to see in the hallway.

• Capacity Reports - These are a little tougher to probe for quick value as it can be tricky to cast your question in a way that is NOT putting the person maintaining the system or process on edge.   Best to start with telling them what you think.   “I’m worried that we may not be testing changes well and think I know a way for me to find out whether I’m wrong.   Do your outage reports show many system or component outages within 24 hours after a change?  Do you recall any times when expected capacity improvements didn't materialize following the changes and they actually got worse?”   Since the capacity reports are most useful to management for keeping ahead of growth, using them to identify the value of risk mitigation takes a bit more knowledge of the business and correlation with various goals to locate the gold.

• Customer History - This can be a gold mine of supporting info.  Customer satisfaction is an established measure of product and company quality and it’s easy to tie to revenue values for an argument in favor of preventing loss.    Also, Sales and Marketing as well as customer support often already gather these metrics.   Some questions to ask are:  “What are your…

o top 10 complaints,
o most common compliance questions and
o most common reasons that customers choose to leave?

You need to also recognize that people may not have this information so there are degrees of quality of the data to consider.   The following are factors that impact the quality of the answers you received in order of increasing ability to help you reach your goal.

• Memory - This is the weakest form of data but having it has benefits.   Consider whether the source is the target audience or is a trusted adviser to your target audience.   How helpful would it be then?

• Extrapolation (without real data and statistical method its more “guess-trapolation” but ok…) -   Having “experts” is nice as they can bring valuable facts to the table but what if you’re trying to get by with just yourself and a fairly large chasm between your story and management’s capability to understand the challenge?   How about if you ask them to extend what they know into the future based on their past experience?   It’s not as much FUD if they envision the possibilities on their own using their own nightmares.

• Research – This is the best source of data but the hardest to get, particularly in a small organization.   Since the data comes from a significant effort to gather, compile and analyze, everyone at the table is more likely to recognize the value more quickly as long as it is presented well.   Research comes from three sources:

o External – Someone else has done the work with a similar environment or environmental conditions.
o Internal – Someone inside saw the need and already gathered the data.
o Start tracking – It’s never too late to start collecting data so grab a shovel and dig in.   Just remember not to become guilty of the sin of Gluttony.

Hopefully you now have some ideas for where to look for more data to support your negotiations.  Good luck and happy digging prospector!