Rationale

When one thinks of creating an index, the obvious indices that come to mind are those focused on financial markets. Those can be straightforward – in the S&P 500 Index, the prices of the constituents are known as are the weights in the form of the free float of the stock. The S&P 500 Index is thus completely transparent, independent, and objective, with neither of the two variables controlled by those creating it. The Lehman/Barclays Global Aggregate and its sub-indices can be clearly defined in terms of what goes into constructing them, and both prices and issued quantities are known as objective numbers.

Yet not all financial indices enjoy the benefit of such clarity on both prices and weights. The Dow Jones Industrial Average (DJIA) is a price based index, but of 30 subjectively selected stocks; it is, of course, widely used, including as the basis of financial derivative contracts and Exchange Traded Funds (ETFs). The key feature in any index is consistency, which is what makes an index useful. Subjectivity in determining an index does not erode credibility so long as transparency and consistency are maintained.

For any financial index to be maximally useful, it should be tradable in some way. A group of people should be able to associate the value of the index with the financial well being of their portfolio, their business, or something that has economic relevance. If trade-ability is not possible, then the index becomes an object of mere entertainment, cocktail conversations, and not much more. It becomes like the daily number for humidity, or the Big Mac index, or something like the “smoothed-sunspot-number” (SSN) which while published daily is of interest only to an esoteric community.

For an index to be “really useful,” it needs to be able to do the following two things at the very least:

1. Risk managers should be able to use the index to hedge their risks, and
2. Investors should be able to take on exposure to the risks that the index incorporates, and be rewarded for taking on the risk by those who do not desire such risk.

The above are, needless to say, in addition to the basic requirements of objectivity, consistency, rule-based computation, and transparency which we consider a given.

Information security risks and financial risks differ in a number of fundamental ways that make achieving the above “economic objectives” difficult. While obvious, there is a need to explicitly recognize these differences. The lack of data on information security is a known matter, but it does not represent a fundamental or conceptually insurmountable issue. The past decade has shown indices being very usefully put together for risks that were for very long times not considered computable.

Operational risks, which include technology risks, are fundamentally different from market and credit risks. Here are some key differences:

• Risk premiums: Investors get paid positive risk premiums for taking market and credit risks on. For technology risks, there is no positive return, only possibly an avoidance of some downside. When it comes to information security risks, we are on the ‘left’ side of zero on the number line and that is not a realm intuitive to humanity. A fund manager can say, “I earned 6% returns when the index earned 9%,” and people get that. By contrast, it is difficult for an IT risk manager to explain that he/she spent $2m on information security and has no idea of the measure of the benefit. This problem exists with buyers of insurance all the time.
• Relative importance: In the financial services sector, it is market and credit risks that dominate. These are the risks that can make an institution go out of business. Operational risk or systems risk events may sting, but are rather unlikely to make one keel.
• Availability of hedges: Most market and credit risks can be offset by acquiring positions in other securities. There are no easy hedges for cybersecurity risks other than implementing internal controls (though some insurance protection can now be bought). These are largely process redesign exercises: segregation of duties, re-performance, analytical checks, prevention controls embedded in systems etc. These exercises are more of an art than a science.
• Measurement: Market & credit risks can be measured and reported using Value at Risk (VaR), exposures vs. limits, and other quantitative tools. But technology risk is difficult to measure. Most risk managers find it difficult to get beyond subjective red, amber and green measures and their equivalents.
• Fungibility: In the financial markets, assets are identical and carry the same risk that can be hedged. On the other hand, if a company wishes to hedge against their Juniper routers being compromised, it is a difficult thing to do because on average only some of the total global population of Junipers will be compromised, and not all. The difference is that “My Euros went down but yours didn’t” is never a possibility in the financial markets while in the IT risk arena, “My Junipers went down and caused me a loss while everyone else was okay” is the norm.

There are three different agents that are responsible for cybersecurity risks: the economic agent who does it for the money; the activist/hacktivist/hacker, who does it for recreation or ideological reasons; and the nation-state, i.e., the sovereign, where the driver is geo-political strategy. Except for the first agent (who can be expected to act rationally), the motives, incentives and timing of the other two may be difficult to predict.

Ignoring the legal and moral issues relating to the underground markets of botnets and other things, consider it simply a market and think about the factors that drive the behavior of different economic agents in this market, with a view to comparing and contrasting some of this to what is known for financial markets.

 

Sellers – whether they are sellers of credit card numbers (CCNs), zero-day-exploits, or botnets – have a business with a high operational leverage, which means a business with high fixed costs of production and low marginal costs. It takes a certain investment to acquire a “book” of CCNs or botnets, but once it is there, it can be sold multiple times without a great deal of marginal costs for each subsequent sale. High fixed cost businesses, like the airline industry, tend to be rather unforgiving and are characterized by cut-throat competition that tends to drive selling prices close to marginal cost, leading to periodic purges, then consolidations followed by periods of high profitability – but only for so long.

The sellers therefore face a great deal of competition driving down prices all the time till the next big thing comes along (such as a new exploit), though that becomes useless as soon as that knowledge becomes pedestrian, its utility reduced as firms react.

 

 

The buyer in these markets is effectively paying for a “factor input” to a larger process, which allows something else to be built which is of greater value than the inputs. What the seller is selling needs to be combined with other inputs – managerial, logistical, capital – to create something of value. A spam list is useful only if you have the infrastructure to sell Viagra. A list of credit cards is useful only if you can use it to order expensive electronics online and have the means to physically receive and resell the stuff you buy (& safely pocket the cash). Similarly, law enforcement imposes a cost on the buyer and is a part of the buyer’s cost structure.

Though the risk from these agents is real and significant, it is difficult to predict. Current events such as the reaction from Wikileaks supporters or sudden changes in the relationship between nation states usually alerts security professionals to expect something, and make a determination if they could be likely targets.

The twin challenges of building an index and creating a risk marketplace

The task of creating an information security index suitable for hedging such security risks faces two key challenges. The first is how the index can be calculated in a credible way, and, second, the specification for the securities that could be based on such an index and the design of the markets where these could be traded. This project deals with the first problem. The second challenge may be addressed as a subsequent project upon the success of the first one.

Some thoughts on how we can go about addressing each of these challenges in a practically implementable way appear below.

The first challenge – the construction of the security index – is not trivial and poses rather difficult issues given the lack of credible data upon which to base it. Prices for the illegal underground markets are difficult to get, whether directly or otherwise. Third party sources may neither be dependable over a period of time, nor be credible. Primary collection of data by directly accessing chat rooms or contacts with people who are engaged in illegal activities will not only be time consuming and unreliable, it is also probably legally risky. Companies that do collect & publish data are quickly tire of the activity. One simply cannot rely on such sources of data; the externally-based data route for calculating a cybersecurity index leads (and has repeatedly led) to a dead-end.

It is therefore proposed that the index be constructed using survey-based methodology. Survey based indices are common in the world of economics. Some of the most successful and reliable indicators of economic activity are survey based.

The foremost is the Institute of Supply Management’s (ISM) Purchasing Manager Index (the PMI), which is published monthly and used with all seriousness by professional investors in guiding their decision making. The PMI is based upon a survey of about 300 purchasing managers, and different parts of the survey contribute to different “components” of the index. Each of the components is equally weighted to arrive at the overall PMI. The questions asked require non-numerical answers. For example, a question might be “Has the economic activity in question increased, decreased or remained the same?” The responses are converted to numbers using a method used for what are called “diffusion indices”. To quote an example verbatim from the ISM’s website, if the response is 20 percent “Better,” 70 percent “Same,” and 10 percent “Worse,” the Diffusion Index would be 55 percent (20% + [0.50 x 70%]). A reading of 50 percent indicates “no change” from the previous month, a reading of less than 50 indicates economic contraction, and a reading of more than 50 indicates expansion.

The PMI is one of the most influential economic indicators, and the fact that it is survey based does not diminish its credibility in any way. JP Morgan and Markit have collaborated with the ISM to produce what they call the “JPMorgan Global Services PMI”, which is a branded index backed by very respected market participants

A variety of economic indicators published by the Conference Board are similarly based on a survey based methodology – an often quoted index being the Index of Consumer Confidence (CCI) which is based upon a survey of 5000 US households. They also publish a whole family of indices on the business cycle, labor markets etc.

The University of Michigan also publishes a similar “University of Michigan Consumer Sentiment Index” which is based upon telephonic surveys of 500 respondents. Each of the above indicators have been around for many decades each, and are considered credible and concrete by business and government alike.

An index based on prices, botnet data, etc., is limited to certain kinds of risks. It will not cover newer risks that may arise, many of which may not have any price or attack data ever available – for example the risk of organized attacks on infrastructure by a sovereign nation. Or the risk of unsafe application development practices in the emerging world. A survey based approach can easily address such risks and any new risks that may arise.

An index produced without collaboration with industry professionals/CISOs may intrigue their curiosity, but may never get adopted. By involving 100 up to 300 CISOs or security practioners in a survey based process, we gain better acceptance of the index and adoption by their organizations as their participation means they are “invested” in the index.

By doing the survey monthly, and releasing the index on a defined date each month, we get rid of the need to maintain real-time data feeds and/or daily updates, thereby significantly reducing management overhead and the cost of index maintenance.

The survey can be constructed to provide sub-indices, as well as an overall index relating to information security.

The infrastructure to record surveys, send out reminders, provide automated status reports on respondents lends itself to a web based tool; which we can build and sustain for a modest budget even out of limited resources.

The methodology will be clear and transparent, the math to convert survey data to an index is known, and given a reasonably large number of respondents, we can be fairly assured that any outlier responses will be diminished in importance.

We know (or can know, at least on a confidence interval basis) that a certain percentage of Junipers (picking on Juniper as an example) will be compromised causing some expected dollar loss. For the individual Juniper owner, the realized probability of the loss occurring will be binary – 0 or 1 – and not, for example, 0.03 ever. For the single company, the expected value is far less relevant than the worst case loss. Therefore what the Juniper owning company really wants is insurance, not unlike the house owner wanting to protect against fire. The only party truly exposed to the full distribution of the probability of house fires is the insurance company.

Sub-indices:

The sub-indices could be each of the n components of the general index. For example, the “index of the threat of sovereign attacks” might be a separate sub-index.

Each of these sub-indices would have some non-zero correlation with the general index, and there is likely a Capital Asset Pricing Model (CAPM) style relationship where each sub-index has a beta representing its correlation with the main general index and its degree of volatility compared to the volatility of the general index, i.e., 
ΔSub-Index = α + β*ΔGeneral-Market-Index

Financial instruments that allow people to take real dollar positions are legally complex and require extensive legal considerations and regulatory approvals, for example, from the Commodity Futures Trading Commission (CFTC) in the US. At this time, no such markets or instruments are planned.

The above is a simplistic explanation of the initial thoughts on constructing an Index of Cyber Security. We will stop here just to be sure that we take one step at a time, the first of which is launching the index. For the proposed calculation methodology, please refer to the ‘calculation’ tab.