Ditching the ALE

At this point in my career, I deliver a lot of certification prep content, through teaching and writing. And I see certain things that were included at the outset of the industry as guidelines and suggestions that just aren't applicable anymore (or at least, not applicable in the same way as when they were proposed). My primary customer is ISC2, for the CISSP and CCSP certs, but I've taught ISACA and CompTIA certification prep courses in the past, and many of them suffer from the same problems. While I can't say for certainty exactly why all the major INFOSEC certifications suffer from the same blind spots, I can guess: most of the test writers have the same training in the same fundamental concepts, get the same certifications (from multiple vendors), and have received that content from their predecessors, and will pass it to the next generation in kind.

This leads to the possibility of stagnancy in content and approach. Which isn't terrible, for certain fundamental security concepts (say, defense-in-depth/layered approach/multiple redundant controls, or the use of two-person integrity), but there are other notions/ideas that are simply treated as sacrosanct in perpetuity, instead of being re-examined for validity, assessed as nonsense, and thrown onto the trash pile of history.

Today, I want to talk about one of the latter: the ALE formula.

If you don't what it is, consider yourself lucky. Then consider yourself unlucky, because if you're going to go get an INFOSEC cert, I can tell you for damn sure that it's going to be one of the things you're going to have to learn and memorize whether you like it or not.

Simply put, it's an approach to estimating the cost of a given type of negative impact as the result of security risk being realized. We teach INFOSEC practitioners that this value determination can be used to weigh the possible costs of controls to address a particular risk, and figure out whether or not to spend the money protecting against it.

Which is a good idea: spending too much on addressing a particular threat is just as bad as not spending enough...and, arguably, sometimes worse, because spending too much leaves you with a false sense of security and a lack of money, where not spending enough just means you have some of that risk left.

But the ALE formula is not really the best tool to accomplish this in our realm of INFOSEC, for many, many reasons. And we should stop requiring its use, and teaching it to newbies.

Why? Well, for starters, let's talk about the potential cost of a single type of incident, known in the formula as the SLE.

It's worth noting that the ALE formula works great in the physical security universe, where tangible assets can be mapped to specific losses. If I'm trying to secure a retail space selling goods that are of a particular size, shape, weight, and cost, I know some discrete, objective information about those assets. I know how many can be stolen at one time, by a single person picking them up and walking off with them. I know the amount (number and dollar value) of my inventory, based on another limiting factor: the footprint of my retail space and storage area. I know the various access points to get at my inventory: the doors/windows/loading areas. All these things can be defined and somewhat limited.

With electronic data as assets, all this numeric determination goes out the window (I mean, not the literal window, like tangible assets, but a metaphorical window, because the determination is impossible). I can't really know how many "data"s a person can steal at any given moment, because the size of files or objects or characters don't really have any meaning in the physical universe-- a flashstick that weighs less than an ounce can carry one file or a thousand files, and any given file can contain one character, or a million characters, and all of this fits inside one person's pocket, anyway (and that person doesn't need any exceptional muscles to carry even the heaviest flashstick).

So trying to determine the monetary impact of a single security event involving data is impossible, unlike the impact of a single security event involving physical assets. If someone steals one spoon in a retail environment, we know the cost of that spoon (and we actually know several costs: the wholesale cost we paid to get the spoon, the retail cost of what we would have realized in revenue if we sold that spoon, and the logistical cost of getting that spoon to the retail location)...but if someone steals a file, the value of the information in that file can vary wildly. A file might contain a photo of the user’s pet kitten (which is of value only to the user, and then only arguably at that, if the user has a copy of the photo), or it can contain the privacy data of the target organization’s entire customer base, and the relevant monetary impact can stretch into the range of millions of dollars, as the result of statutory damages assessed against the organization, or the loss of market share, or direct fraud on the part of the perpetrator using that information, and so on.

Sure, insurance companies in recent years have created various approaches to assigning value to data, but these are all just gibberish. Take, for instance, the idea of “average file cost”-- even if we were to determine the midpoint of value between the kitten photo and the customer list, that medium value would be meaningless when we suffered an actual loss: if we lost the kitten photo, and the insurance claim paid the amount of “average cost,” we’d be receiving far more in cash payout than the thing was worth, and if we lost the customer list the “average cost” claim payout would be far less than the damage we’d suffered. And what’s the size/value of an “average” file, anyway? How many files are there in a given business environment? The concept is absolutely pointless.

When the SLE is just a fictional construct, the entire ALE formula is ridiculous. We could use just this argument to eliminate the wretched thing from our industry. But there are even more reasons why ALE is stupid in the INFOSEC world-- and I’ll get to those in subsequent articles.

 

 

CISSP CAT Format Feedback, Part 2

A second former student has reached out with some feedback...he passed, as well! Smart class, that.

Here's what he had to say (and he says he's glad to answer questions about the experience, too, and will be checking the blog Comments, so feel free to chime in):

"Since you probably haven't gotten much feedback about the CAT yet, I thought I would provide you with my preparation strategy and exam experience.

 

Here were the study materials I used and their usefulness (in no particular order):

 

Classroom notes - 10/10 - This is where I began my studying and it helped me tailor my studying to topics I was unfamiliar with.

The Official CBK CISSP text - 1/10 - I used it during class for subjects I had absolutely no familiarity with, but in general, there is too much information to internalize and lot of rabbit holes that the exam will simply not ask about. Not to mention it's unbearably dry.

Eric Conrad's 11th Hour CISSP Study Guide - 9/10 - Effectively a condensed version of the most important CISSP topics. There were a few areas that may require additional reading (i.e. RMFs) but in general, this is an excellent text reference.

Kelly Handerhan's Cybrary video series - 10/10 - This was by far the most useful resource I used. If I had the time, I would have watched the full series twice, taking copious notes. She also offers an excellent bit of advice about approaching the exam with a managerial mindset, rather than a troubleshooting or technical one.

Phil Martin's Simple CISSP - 10/10 - I found this book on Audible and listened to it during my commutes. The author narrates in a very slow, deliberate, and clear Texan drawl, clearly explaining even some of the most difficult subjects.

Sybex Test Questions - 5/10 - Compared to the actual exam, the practice questions in the Sybex bank are so-so. Many of them ask about details the exam couldn't care less about; many more of them are simply too easy and direct. (For example, the exam will never phrase a question such as "blah blah blah describes which security control/process"). There aren't enough "which of these is the BEST/MOST accurate," which is the entirety of the exam.

Transcender Test Questions - 7/10 - This bank contains many more of the BEST/MOST accurate style questions, but still not enough to truly simulate the exam.  Fun fact: if you purchase the bank from Transcender, six months of access is $160, if you buy it through Cybrary (via the Kelly Handerhan videos, which are free), access is only $40. That's a useful bit of knowledge for the financially-minded."

 

Great stuff to know, and really glad he offered to share.

New Year, New CISSP Exam

Just in time for 2018, the CISSP exam from ISC2 has converted from standard multiple-choice format to a Computerized Adaptive Testing model for exams delivered in English (foreign-language versions of the test currently remain in the traditional format). This means that instead of the grueling 6-hour, 250-question test, CISSP candidates now face only 100 to 150 questions, in a maximum of three hours.

Depending on your success with multiple-choice tests, and your personal technique, the new experience could be either a massive boon or a ridiculous hurdle to get the certification.

I got my CISSP back when the test was in the traditional format...and done with pencil and paper. I have no clue how I'd do on the current version.

I have, however, received feedback from the first of my students to take the new version of the test: they passed! Their exam was also only 100 questions long (meaning the student demonstrated sufficient command of the material so that the testing engine didn't have to throw more questions at the student), and it took the student an hour to complete. Perhaps most interesting, this particular student is not an IT practitioner, but is familiar with the industry in other roles. Main impression? The student repeated what I always try to stress to anyone taking one of the certification tests: READ. THE. FULL. QUESTION. Make sure you read it completely, and understand what's being asked, and that you read all of the possible responses.

The exam is still being administered by PearsonVUE, and you can download the outline from ISC2's website.

Have you taken the exam in the new format? Please add some feedback about your experience in the Comments!