Mark Curphey - SecurityBuddha.com

I am working in the OWASP Web Certification Project and planning to make some serious progress this week. One of things I have done is to step back and think about what makes a good evaluation criteria. Here are some notes.

- Risk Based Security

- Assurance

- Unambiguous

- Repeatable

- Flexible

Risk Based Security

Risk based information security may not always be a trendy term, yet established thinking rarely goes out of fashion. Different types of web sites face different levels of risk. Without descending into the heady depth of discussions around what risk really is, in general most people will agree that it is made up of the threats (from threat agents), vulnerabilities and the potential impact to the environment (usually the business). It naturally follows that different classes of web sites have different levels of inherent risk.

Some web sites such as financial services may have more threat agents with better capabilities and who are more motivated to attack them than perhaps a local gardening club site.

In general the classes of vulnerabilities in different types of web sites are similar but are of course dictated by the functionality in the system. For instance some types of systems using certain types of technology such as AJAX may have to consider certain categories of vulnerabilities where a web site that doesn’t use a technology such as web services may not have a consider another specific category.

The potential impact is usually viewed as the direct impact to the business but it is becoming increasingly common place that we are seeing impact viewed as the impact to an entire industry. This seems to have been the case with the payment card industry. If one web site is hacked and the company credit card database exposed, confidence in the whole industry is affected.

Any web security criteria should take into account the risk of the target of evaluation.

Assurance

To have assurance about something implies that you have confidence in it. Having confidence that a statement or set of claims about the security of a web site is accurate is clearly very desirable yet assurance has implications beyond just accuracy. For instance a higher degree of assurance may mean that you can place more trust in a set of claims knowing that they are more likely to be repeatable and consistent.

Different assessment techniques can provide different levels of assurance (confidence) on claims about the security of a web site. Specifically some tools are better than others at determining specific classes of vulnerabilities and some techniques are better at finding certain types of problems than others.

It also follows that a company who embeds security in their entire software development lifecycle (SDLC) will be more likely to consistently produce the assessed results and therefore a 3^rd party can have a higher level of assurance on any claims.

 We must accept that in general assurance comes at a cost. It is more expensive to produce facts on which higher assurance claims can be made. For example it is more expensive to test something in a way in which you can make a higher assurance claim. Using technical testing as the example, a higher degree of manual effort (higher cost) is generally required over lower assurance automated approaches (lower cost).

If we define the level of risk inherent in specific classes of web site, we are then able to pragmatically apply the principles of assurance. For instance it makes perfect sense that for a high transaction value financial services application we would want to have a high level of assurance in the security claims whereas a very low transaction value site would not.

Any web security standard should offer assurance levels.

Auditable

Any criteria for security must be Auditable; that is to say it must be something that can be inspected and validated. If something cannot be inspected and validated then it should be defined as a set of principles and not as a set of criteria.

Unambiguous

Some security standards make statements that are ambiguous. One example is the PCI DSS that says “only necessary ports should be open”. The default effect of this ambiguous statement is for all sites to legitimately claim that all open ports are necessary and everyone passes. “The Remote Desktop Protocol is necessary to remotely manage the web site host” for example. In general ambiguity usually occurs when standards writers attempt to write “catch all” statements without precision.

A better approach to avoid ambiguity is to write concise statements and provide a flexible process to request exceptions. For example a better approach to the PCI DSS open ports issue above would be to say “Only ports 80 and 443 allowing inbound and outbound HTTP and HTTPS traffic should be enabled”. All other ports requests could then be documented by an accredited auditor and submitted to an approval process along with a set of documented guidelines for approval or rejection (see flexible below).

Repeatable

No two people are the same. While intelligent interpretation is one of the beauties of the human race it is important that evaluation claims are repeatable. Two individuals or two teams should arrive at the same conclusion and be able to justify their results. In a commercially driven world where the lowest cost or the fastest results often wins business, we must ensure that specific security claims are not compromised by commercial pressure or that we understand the trade-offs. The notion of assurance levels of course allows these influencing factors to be taken into account when producing security claims.

Flexible

Some security criteria fail by trying to dictate a general control that may work in the majority of situations but doesn’t take into account other perfectly legitimate approaches. This forces companies to adopt specific approaches that may not be best for their business. A better approach is to provide standard criteria and a well defined process to request exceptions. If the mantra of a security criteria is to be able to design, develop secure web sites then flexibility is essential.