Measuring security the Improvement Kata way

We have been using improvement katas to improve our processes. We wrote an in-depth blog about this, but the gist is that you improve a process by making small, focused changes and measuring if you got closer to your goal. Measuring is essential in this process; if you don’t measure, you don’t know if there is something to improve or if your changes were effective. Luckily, finding what to measure is relatively straightforward… usually.

Why security?

Let’s shift our focus to security. Security is important. We all know that. We don’t want your name and address to be shared on the internet, and we don’t want you to lose your exam results. More secure is better. But more security is not necessarily more secure. If we put an alarm on anything that can go wrong and monitor every movement on our systems, we would drown in distracting notifications. We would have no time to work on anything else. So we need to choose what to do. We need to measure security if we want to improve our security processes. But how do you measure security?

Ways to measure security

Security is not very straightforward to measure, especially within a short timeframe. For some processes, you could measure the time it takes for a user to do a certain task on a webpage, or you can tally the number of questions about a specific subject. You can then make your changes and measure again to see how the numbers changed. But for security processes, there is simply not enough flow through the process. For example, we cannot get meaningful measurements from the number of data breaches. We cannot improve from 0, and not having any data breaches does not mean there is no possibility for data breaches, which is one of the things we are interested in. So we need to think of other ways.

Counting security issues

Let’s face it, there is no digital system without any security issues. Especially a system that is always changing. When these issues are found, they should be reported. These are any reports of security problems in a system, from available data where it should not be to someone who lost their key to the office. One way you can think of measuring is by counting these security issues.

This metric can be very useful but can also be very deceptive. Having fewer issues does not necessarily mean fewer problems, only that you can find fewer. It’s like treasure hunting; if you don’t dig in the sand, you won’t find anything interesting, but you cannot say there is no treasure. In a way, increasing the number of security reports means that we get a better overview of our security state. This can be an important goal, but it is not the way to measure security.

Compliance with laws and regulations

Instead of through measuring, security is often valued based on specific standards. In the EU, this is usually the GDPR, the strictest security law in the world. It describes rules for how to deal with different types of data. This provides a base level of security and accountability, specifically regarding the misuse and abuse of data.

Of course this is important. If you have GDPR issues, they need to be solved. Not only for legal reasons but because we care about the security of our clients. We are GDPR compliant and will stay compliant. Then again, it is a baseline after which there is still a lot of flexibility and opportunity for security problems.

Risk assessments

A while back, we started looking into the ISO 27001 standard. The standard describes a way of working with an Information Security Management System (ISMS) to oversee and treat information security. So we started working on this ISMS because it would provide a better framework for us to see how we are doing regarding security.

As part of the ISMS, we started doing security risk assessments. In a risk assessment meeting, we discuss a long list of possible security risks with a group of employees from different departments; A software engineer, the CTO, and the privacy officer. For each risk, we give a rating for their likelihood to occur and their impact if they would occur. This gives us a list of different possible problems that need to be prevented or mitigated, sorted on their exposure, a combination of likelihood and impact.

This may look a lot like counting security issues, but it has some interesting differences:

1. They have an exposure rating, helping us with choosing what to tackle for the largest effect;
2. The next risk assessment has the same list of risks, meaning we can compare the progress of the risks' exposure instead of the number of them.

This makes measurements from the risk assessments a lot easier. We now have a structured way of looking at the security of Easy LMS. We fill the risk assessment with the knowledge we get from security issue reports and our day-to-day observations, then we improve one step at a time.

Certification

There is a problem with measuring from the risk assessment: it may not be entirely objective. We can only report what we see and rate them based on our own experiences and thoughts about the subject. This basically creates a bubble with what we know and what we are working on, but it is not showing us what is outside the bubble.

Therefore, we decided to continue with the ISO 27001 standard as a whole, which also describes a process for certification. An external party will rate us on our implementation of security measures and the ISMS; basically a check to see if others agree that we are where we think we are.

So we are working hard on getting ourselves ready for this certification process, and we hope to call ourselves ISO 27001 certified somewhere in 2023!

How we implement security measures that stick

Now we have a process to improve our security. We have picked the things we need to improve and started implementing them in our company. However, we aren’t fully there yet. We also need to make sure that the implementation will be successful.

The thing is, everyone knows security is important, but people don’t actually like working on it. It can quickly become like taxes; it should be done, but it is an annoying chore. How do we make sure that this does not become a problem?

Removing obstacles

The biggest reason why security measures may not work is that people work around them. This problem lies with the measure, not with the people. It means that creating workarounds is simply more convenient than actually following the rules and processes. The process has too many obstacles to overcome, and it is in our nature to take the path of least resistance. This can have disastrous effects.

An extreme example would be the Bhopal disaster in 1984, where several security measures were neglected. A city was flooded with toxic gas, killing thousands and injuring more than half a million people. Of course, security incidents at Easy LMS would not have these extreme effects, but it does show how easily security measures can be circumvented, despite the possible consequences.

It is best to make following security procedures as simple as possible. More specifically, try to make it easier than circumventing the measures. There can be some additional steps that need to be taken for security purposes, but not too many. There should not be additional waiting time. Security procedures should not stop your employees from doing their work; they should guide them through how to do it more securely.

Security procedures should not stop your employees from doing their work; they should guide them through how to do it more securely.

Let’s take the example of reporting a security issue. Say this needs to be done through a custom filing system where many specifics need to be searched for and filled in, most of which won’t be used anyway, costing the employee 15 minutes of their time. It may be used in the beginning, but it is not sustainable. It is easier to do it later and forget about it. Now take another system where an employee can send a simple message on Slack with a short description of the issue, and it will be dealt with. There are still some steps for the employee, but it is very easy and low-effort. Maybe even more important, the employee is no longer responsible for resolving it. They send the message and can continue with their day, which is a lot easier than dealing with the inevitable consequences of a security incident. So instead of forcing people to work inefficiently and then blaming them for making mistakes, you look at what is inefficient about it, and then you improve it, so it becomes easier.

Of course there are steps to take to be more secure, but obstacles on the road for your employees make their work unnecessarily harder. A lot of these obstacles can be taken away by automation or simple delegation of tasks.

Safety for people

We have talked before about safety for people in the context of making mistakes. We can also apply this to security. People need to be able to talk about security, mention security problems, and correct each other’s behavior without any negative consequences. If employees feel like they cannot talk about security or they don’t know where to do so, security measures will form an obstacle for them. Employees all have their own expertise and experience in their domain, so they know what they are working with.

People need to be able to talk about security, mention security problems, and correct each other's behavior without any negative consequences.

They can help by suggesting measures for specific security issues, they can give feedback on already implemented measures, and they can notify you of obstacles on their way. So when we heard that someone had lost their keys or that the paperwork around setting up a client interview was too much work, we weren’t angry or dismissive. We were happy that they came to us with this information. If you do not treat this feedback as valuable or if you do not do anything with the feedback, you lose your employees’ interest in working securely.

Keep improving 

We already talked about improving our information security, but it is just as important to keep improving the security processes for our employees. If you don’t actively listen to your employees and change what doesn’t work, the security measure will eventually stop working. The employees, like the company itself, will keep changing. Easy LMS will keep changing, and so will our security procedures, not just to make them better but to make them easier.