Why the PCI Standard needs as serious re-think
For as while I have had serious concerns about the PCI Security Standard. I have raised them with the folks who manage it (or did manage it at the time) and got nothing but short shrift. In fact it was worse than that; instead of listening to the concerns I had a bunch of total nonsense thrown back at me claiming I was only raising the issues to further our own business. At the time I was unable to really comment, now I can. Total hogwash!
Here are some of my issues with it.
1. Scanning vendors – This is a total cop-out. According to the PCI you can check for compliance using a scanning vendor. The catch of course is that by definition a scanning vendor is a company who can use people or tools to check. Under most circumstances this high ground would have been admirable but when the PCI management decided to recruit an army of low skilled “joe blows” from the street to become scanning vendors they created a market based solely on price. I know of people that do PCI assessments that had never worked in the industry before. With people doing PCI assessments for 150 bucks per quarter, how much time, diligence and skill do you really think they are putting into checking a web site you may put your credit cards into? Not much! By defining scanning vendors in this way PCI have created a cop-out clause. They have pushed the vendors to assume the risk and created a market where the reward doesn’t allow anyone responsible to do really do so. They can say “we told you how to do it, look here it is in writing” if anything goes wrong (AKA Card Systems).
2. Its badly written – Standards have to be implementable. That means you must minimize ambiguity. As an example that I know a lot about lets take the OWASP Top Ten. This was never designed to be referenced by standards. The OWASP Top Ten is a high level marketing document to raise awareness of the broad issues. Saying you must implement the OWASP Top Ten is sheer nonsense.
NB: One of the original PCI documents referenced the OWASP Secure Coding Guidelines which has never existed BTW. I was always mildly amused at how many people could have been certified to check against something that was never created and amazed that no one asked the question about which document it actually was. I guess reading the standard you are certifying against is only optional!
3. Application security - I offered to gather a set of industry experts like Gary McGraw and John Viega around a table and define a good set of criteria when I explained the OWASP Top Ten was not fit for purpose. My offer fell on deaf ears. Instead the powers that be decided to modify the standard and specify that automated scanning for XSS and SQL Injection was required. I guess they just don’t understand that you can’t automatically scan for these things with any degree of confidence. And I mean maybe 1 in 10 issues found this way if you are lucky! Ever found a stored XSS via a web app scanner? Most web app scanners only look for basic SQL injection issues (4 or 5 signatures) to avoid their sessions being killed. This just leads to a HUGE false sense of security.
4. Application firewalls or a code review- I can only assume someone is on the board of a VC firm with money into the over-burgeoned and over hyped web app firewall market. Useful tools, possibly (although I would never buy one), but telling people they have to have one or have a code review is plain old wrong. Its just not right. Can my wife do the code review (she’s an accountant)? I guess so. What should she look for? Oh its not defined anywhere.
In truth I could go on and on about the PCI standard for a while and really pick it apart. I think it had good intentions and has become one of the most powerful stimuli for a long time. But its so far from being close to a standard that even needs refining. It needs to be ripped apart and started again. The program, the documents and the certification process. And this time get some good information security people, some standards folks and some sharp business people in to help!
January 30, 2007 at 11:32 am
[...] under the bus and see if the assessor who may well (we don’t know yet) have followed the loosely worded book to the letter is [...]
February 8, 2007 at 1:03 pm
If low skilled “joe blows” can pass why couldn’t foundstone?
February 8, 2007 at 1:08 pm
Well John, I think you know they are a qualified assesor. At times like this I wish legal obligations would allow me to spill the full beans. Its a juicy story as well you know!
February 8, 2007 at 6:32 pm
Think we are missing the point maybe. I’ve worked for 3 ‘Approved’ Scanning Vendors. The first pair practised the same thinking toward PCI compliance;
1> Let’s hire someone who can get us through this. (Me on two occasions)
2> Let’s use cheap labor for the ‘real’ work once we are certified.
Maybe it doesn’t happen in Europe but I reckon it’s pretty standard here.
The compliance test in itself is not so bad. I believe that not many Vendors make it through the first shot. Maybe even Foundstone didn’t manage. (eh hemm)
Whether or not the PCI Compliance system is good, bad or burnt down it’s not going to change this. The real problem IMHO is that oce certified the Vendor is not policed at all.
There’s another thing that bothers me. Several approved Scanning Vendors are just scanning tool vendors. They need the certidfication in order to market their tools. In no special order – Foundstone, WebInspect and Qualys spring to mind. They provide the tools that dilute the quality surely?
So maybe the PCI Council, for all it’s sins aren’t all to blame, or are you plugging for a Consultency job Mark? :-p
Perhaps they need a guy like you. I for one would enjoy sanction that Vendors are consulted and involved in the next revision of the Standards document.
L.J.
February 8, 2007 at 7:08 pm
No consultancy for me, have my hands full building a software company but I do passionately want to fix the issues I see with PCI. I’ll happily donate my time for free and promise I will never become a scanning vendor. Can’t get straighter or fairer than that!
I think the scenario you describe about getting certified is actually quite common. I actually know of several firms that took an interesting approach. I am obviuously not able to talk about Foundstone but these are other similar sized firms. Its a small industry, we all talk! As you are not told what you have to pass or what you have failed on, some firms tested the water. They did the bare minimum and then cracked it up a notch until they passed. The reason they did this (and I know of at least 3) was that this is a price sensitive market and unless you can do it with the minimal amount of effort there is no real way to make money at it. I suspect (they will admit it off the record) those firms are subtly turning away PCI work as they don’t want to assume the risk. They got certified to keep others in their company happy. This is why I think the business model thats been created is broken.
A similar story is true with Qualys and other tools verndors as you point out. This was my first point. People sell tools as PCI scanning tools, they dont sell tools as scanning vendors. Its all semantics but very important ones.
And the continuous policing, yeah. Big problem.
I actually heard a really funny story from someone over beers in Malaysia a while back. One person was doing the web app testing part. The app had a hole the PCI folks didnt know about and they were able to SQL inject it and get everyone elses results. They could reset others scores as well. I don’t know if its true but if it is its scary. Apparently most people had failed at that point (unless someone else had already reset their scores down words). I do know the web app scanners were tested against the PCI test bench by some people and found virtually nothing. Again strange why anyone would then go and mandate (implicitly) automated scanning for SQL injection and XSS.
Oh well, blue skies!
February 8, 2007 at 10:16 pm
I don’t really afford much credibility to your article. Who are you to criticise PCI?
February 9, 2007 at 6:06 am
The guy in Malaysia had few too many beers I reckon. He’s saying PCI kept the other Vendors scores on the flawed web app box? Yeah right…
From memory that box was a simple VMware machine with a web server & a ‘dummy’ website selling widgets or something like that with SQL injection and XSS vulnerabilities.
A bunch of Bud bottles colors the imagination somehow I reckon.
L.J.
March 23, 2007 at 6:40 pm
[...] What I have heard happens in the real world is the following; vendors get certified. Some hire 3rd parties to get them through it. This is wrong for obvious reasons. The folks that do this are essentially taking on the risk [...]
November 30, 2007 at 3:50 pm
[...] [...]