higB, I plan to cover the ideas behind why we built CodeScout and SiteScope at Foundstone, tools which I think are really cool (in concept if not in practice).

Dre, brilliant comments as always. I will take a look at the Atlassian stuff. I plan to talk about some of the things you mentioned in the next set of posts especially walkthroughs etc in the types of testing post. It maybe Sunday now as I have an 18 hour flight to busy myself with! Again I so appreciate your comments, always insightful!

Yes, and we need *two* accounts/credentials, not just one. And, no, I’m not using my SSN and my daughter’s. Although I have thought about using my ex-wife’s.

Imagine trying to test business logic flaws with only one account / set of credentials. It doesn’t work that way.

You’re right, it’s probably a good idea to put all this stuff up front - in the SOW/CSA as a requirement. I guess this largely depends on how advanced your client is / how mature their organization is.

What’s frustrating for your vendors is we ask key information about what the application does and we can never get a straight answer. Most of the time the POC we deal with who is commissioning the the review has no concept of what the application does.

“I told them what I wanted and didn’t ask them to tell me what I need. ”

Can I get your contact information? It would be refreshing to have a customer who really does know what they need (and is right.)

I have a lot of customers who think they know what they need. It’s not easy to tactfully convince customers that what they are asking for makes no sense. In 2007 I still convince customers that an application review is more then running webinspect. In 2007 customers still don’t understand that testers need credentials (that work) for the applications they are testing.

If I had a nickel for every customer who truly knows what they need, gives out accurate scoping information, and supplies working credentials on time, id be very poor.

But, since I love this stuff as much as you do, I’ll give you some feedback and what already looks to be a great piece of work in progress. Here goes:

Fixed Price or Time and Material?

The key to fixed price and time and materials work for both parties is in defining an accurate statement of work. A good statement of work will describe in unambiguous language exactly what is expected of both parties. It is the agreement upfront of what will (and what won’t) be done. If you don’t have an accurate and detailed statement of work then in general expect trouble.

I think it is key to differentiate between your SOW (statement of work) and a CSA (consulting service agreement). The SOW should be an example of what the deliverable(s) will look like. This makes it easy for people to know what they are getting as a result in the end. At first glance, a SOW will look like it already has all the information a company needs to know - so they don’t need to hire you. But after looking at the level of customization and detail that can be provided - the customer should realize the value that you would add if you do a code review. How win-win is that?

The CSA is all the negotiable terms: a) how long/hours, b) pricing, c) contractual terms. Put these in separate sections (or even separate pages) to allow ease of changes.

Curphey, you also forgot the most important part of “the Business” - invoicing. Shame on you!

Personally, I prefer FreshBooks because this is simply an amazing web tool. Maybe I can do a code review for them and get my own rate plan! With this, I’m trying to say two things: nothing beats partnerships and relationship building (quid pro quo), and almost nothing beats getting an easy-to-pay / accurate invoice. Actually, no, nothing beats a paycheck for a job well done!

Tools or People?

It’s well worth the effort of understanding the parts of the scope where automated tools be used and which parts will require humans.

Don’t forget the humans on the buyer side as well. One of the best ways to do a code review is to start with a walkthrough. Identifying stakeholders and putting them into the SOW/CSA is extremely important. Managers that sign-off the CSA and pay you are great, but what about the developers and testers?

It should be a basic requirement for a “code review” SOW/CSA to identify at least one person who will perform a walkthrough - and that the code review and work is dependent somewhat on the success and involvement of this person (and the process behind it, which doesn’t have to strictly be IEEE Std. 1028-1997 - but at least the rules are written down somewhere that is easily referenced). Let’s not forget CYA…

Identifying other stakeholders / key players early on is also recommended.

Bling Bling and Bang Bang?

I strongly encourage both sides to think about better ways of reporting such as using defect tracking systems. Testers will be happier and companies can get around 15% more work for the same cost!

Have you seen Atlassian and Cenqua integration? I’m not talking about the merger/acquisition, I’m talking about JIRA + FishEye, or Crucible. Amazing sets of tools! I think I sent you a link on continuous testing from the CI book. Continuous integration with a build server doing continuous-prevention development testing (based off of defect-tracking) is application security assurance Six Sigma!

As someone doing only audit/pentest/va and since the last few years more and more webapp related work, i always have difficulty scoping the work to be done. hopefully the next articles will shed some useful light on this issue