Open Source Scanning Tool Eval

One of the risk areas for ISVs and others who make software for use outside of their company is the inclusion of open source packages in their software and the license obligations associated with those packages.  Most of our R&D groups have a pretty good handle on what open source packages they are using, however, our legal department and R&D management felt it would behoove us to have a scanning tool in order to verify everything included in our code.  So, with that in mind, the governance board kicked the tool evaluation over to my group and asked us to get an open source scanning tool.

Players


We found only a couple of players in the open source scanning space, BlackDuck, Palamida and OpenLogic.  There are a couple of other supporting groups like Veracode (they are mostly for security), Protecode (the engine used by OpenLogic), and  FOSSology (a community that develops a package to analyze code for open source software), but our evaluation stuck pretty much to BlackDuck, Palamida and Open Logic.


Overall, BlackDuck is the front runner in the field.  Gartner, Forrester and the like all recognize BlackDuck as having the largest structure, customer base and offerings of the three.  Palamida labels themselves as "application security for open source software", but their sales person dismisses the security talk and concentrate their message about their scanning capabilities.  OpenLogic is pretty small, but they have a decent scanning tool and a hosted offering to upload a fingerprint to analyze and then review the results.


Requirements


The requirements we looked at were:

1. Ability to review source code and binaries
2. Understand license obligations
3.  Provide software inventory (bill of materials)
4. Comprehensive library
5. Multi-user/Multi-role system
6. Common IDE interface
7. Report generation
8. Installation ease 
9. Ease of use
10. File comparison feature
11. Performance
12. Automation
13. Cryptography

Stakeholders

1. The following groups participated in the requirement gathering process and product evaluations:
1. Legal
2. Open Source Policy Subcommittee
3. Open Source Risk Assessment Subcommittee
4. IT Risk Management/GA Audit
5. Business Unit R&D Leadership

Outcomes


We did a POC with BlackDuck, Palamida and OpenLogic.  We used the same code package and gave each vendor a week to do the scan, reconcile the findings and do a presentation of their findings.  Both BlackDuck and Palamida came onsite to do their scans, while OpenLogic did theirs remotely.  Palamida brought their own box while BlackDuck had us install their software on a virtual machine in our development center.

All the scans came back with pretty much the same results and luckily our code didn't have any major violations.  We compared the found list and there were small variations in the number of exact hits, but between the exact matches and partials, each tool did a similar job of finding the open source components in our code.

All three vendors warned us about the large effort to analyze the scan after it was done and based on their results and presentation, they do have a point.  All three had delta scan ability, so the initial scan would have the majority of the work, but once a baseline was set, then later scans would be easier.

This may have been just our thinking, but it appeared to us BlackDuck and Palamida were trying to bundle their scanning software with their analysis service and when we tried to divorce the software from the service, the price of the software rose.  Also, the environment overhead for BlackDuck and Palamida was not overly large, but because OpenLogic was a hosted solution, the two were large in comparison.

Each of the tools had a customizable workflow engine, with BlackDuck and Palamida having a fairly robust offering compared to OpenLogic (Palamida's later release had the better workflow engine).  The policies and policy rules were the most important to us and all of them had that ability.

Decision


At the end of the day, we decided to go with OpenLogic.  During the evaluation, we found pretty quickly that what we wanted was just a scanning tool to tell us what open source components were included.  We were not at the point to deal with a robust workflow engine with submission and approval workflows.  Nor, did we need a comprehensive library to store all our code as well as open source code our groups were using.  Also, we already have architects, configuration managers and enough engineering staff to be able to compare code snippets, so we didn't really need the analysis.  When OpenLogic came in with a reasonable price and an option for just the things were looking for, we decided they were the one to fit our needs.

I think BlackDuck and Palamida have a place in this space, but I'm not sure we as an organization were ready for what their strengths are.  We may be later, but for right now, we brought OpenLogic online and have done a scan and are happy with what we are seeing.  We will work with our Open Source Task Force to come up with a roll-out process and I'll update this blog when we progress down this path with any challenges and findings we have.

Translate Feature in Outlook 2010

We have a group in France who uses our tools, so periodically I have e-mail exchanges with their QA manager.  She speaks English very well, so there is not a need for me to know French, but as a courtesy, I try to include as much as I know in French (Bonjour, Merci, etc...).

So, in Outlook 2010, I was intrigued to see the Translate feature off the Review tools.  This isn't a bad tool, and so far most of the messages I have sent have been understood (not sure they are correct, but she has understood the meaning).

The translation isn't native to Outlook, but exports the word or text to an external Microsoft site to translate.  It has the ubiquitous pop-up saying you are transmitting data to an external site, so I wouldn't do it for anything confidential, but the site is pretty quick and as said, fairly accurate.

This is really helpful when receiving a message in another language, because the tool will translate it so you can read the message.  The issue is when you are trying to write something and use the translation tool to convert it to the receivers language, when you copy the converted text, it highlights it in this baby-puke yellow and you can't format it out in Outlook.  The reason it is highlighted is the translation tool displays a pop-up showing you the original text, so I understand why it is there, I just wish you could format it out.

Overall, it is a pretty neat feature and a nice addition.  Now if it could translate texting from my pre-teen, I would be jamming.

More on Security Development Lifecycle

In a previous blog, I talked about the methodologies and maturity for a security development lifecycle (SDL).  This is only one piece of the SDL program, and it takes people, process and technology to implement a SDL.  When we looked at these aspects, we tried to come up with a staged approach so we could implement security in our development process but in a way that didn't overrun people.

People
On the people front, we looked at the number of people and their roles for a centralized SDL team.  We quickly found we needed:

• 1-2 FTE for 1 yr to write the process and training materials
• 1 FTE for training coordinator
• 1 FTE for security tool administration (1/2 FTE for admin, 1/2 FTE for BA)
• Security buddies (Microsoft term) to help R&D groups take ownership of SDL in their business units

Because of that last one, we knew we needed to have more help from the business units and this would take a lot more people.  We also saw, that as we added business units, this could increase the number of people in the central roles, so we would assess our work as business units came on board.

Process
For process, it was defining what security means in each of the stages of the software development lifecycle.  The things we looked at for the major points of the lifecycle include:

• Requirements
• Include security non-functional requirement documentation that cover:
• access controls
• deployment consideration
• authentication
• authorization
• input validation
• logging & auditing
• error handling
• Design
• Threat model the design
• Include trained business unit security liaisons (BUSL) and whoever wrote the security non-functional specification
• Development
• Perform security checkpoint reviews after agreed upon code completion time
• Security review reports
• Static code analysis with training best practice
• Testing
• Dynamic code analysis with training best practice
• Deployment
• Determine how corporate security is integrated or help external customers with their application security checks

Technology
We have static and dynamic security testing tools, so it was a matter of rolling those out to all the business units and providing a central offering for those without the expertise.  This was probably the easiest of the pieces to tackle only because of what we currently have.

Deployment
To deploy this in a phased approach, we looked at the tasks for people, process and technology and grouped them into what we could accomplish without too much disruption.  With that, we came up with this model to help us plan our work.

*STRIDE approach is the Microsoft approach to threat modeling using Spoofing, Tampering, Repudiation, Information disclosure, Denial of service and Elevation of privilege.

How did it go
So how did it go after we planned all this?  Most of the items are being rolled out but I'm not sure it is as coordinated as I would do it.  Being in the tool business, my main focus was on technology and we have the static and dynamic code analysis tools in place, but we are assessing their value and determining if we need to replace or promote the tools.  I feel pretty comfortable in our space, and I'm sure the development groups are getting more comfortable putting SDL practices in place with their work.