HP QC10 and ALM11 on Windows 7

We are going through our Windows 7 beta program and I was selected as an early adopter.  Not sure why they selected me, but it was probably because I know how to write up a defect with somewhat accurate steps to reproduce.  I have a Dell D630 2GHz Dual Core with 2GB of RAM.  Kind of light machine from everything I read about Windows 7, but still should be adequate.

I loaded the corporate image of Windows 7 with Internet Explorer 8 and tested some of the basic apps I use everyday like Outlook, Office Communicator (OC), IE, Firefox, Ziepod (I'm not a big iTunes fan and all I listen to at work are podcasts), Yammer, Clarity, Hyperion and Business Availability Center (BAC).  Hyperion and BAC were giving me fits with IE8, so I downloaded the IE9 beta.  Still no luck getting Hyperion or BAC to work (we are only on BAC 7.5, so that is probably the reason), but the rest of my apps using IE work fine with IE 9, so I got a virtual desktop for Hyperion and BAC and went on with life.

I then accessed QC10 through IE9 and before it started the initial load, it gave me an error message to load a .Net component, with a link.  I went through the link and installed the component and then went on to installing QC.  I'm not sure the install was any longer than with XP, but it took time enough to notice.

Moving through QC10 was relatively painless.  I created requirements, tests, test folders, releases and test sets without any issues.  I ran some tests individually and through test sets, without any issue as well.  I didn't run any automated tests, but I'll do that at a later date.  I ran a couple of Excel reports and they ran to completion and was able to save them to my desktop.  I didn't go into the Dashboard because we don't have anything in there outside of what is delivered by HP, but I will need to look at that later as well.

While running QC10 on IE9, I also had Outlook 2010, OC 2007, Yammer, Firefox 3.6, Chrome 8.0.x and Ziepod running at the same time.  I really didn't have any issues running QC10 and swapping between other apps and it took about 100MB to 125MB of RAM by itself, with about 80% RAM use for all my apps (high, but still functional).

The reason I noticed the RAM use is when I then tried ALM11, with the same apps open, I started to get low memory errors pretty much right away.  I reviewed the memory and ALM11 was taking about 175MB to 200MB.  While it was a relatively significant increase for just the app, overall it should have been fine, until I noticed I was now pegging about 90 - 95% memory use, with spikes tapping out at 100%.

This got me to review the features in ALM11 and I noticed HP tried to improve speed with a feature where if you don't log off, then you come back to where you left without reloading the app.  This tells me HP is like everyone else and sees RAM as relatively cheap, so they are engineering their app to put more in RAM.  My 2GB of RAM, no longer works with Windows 7, so I got another GB (I would have added 2GB, but the D630 is a pain to add a second stick) and decided we may have to use a virtual desktop if we want to do extensive testing work in ALM11.

We will probably move the company to Windows 7 before we move everyone to ALM11, so we will need to make sure our QC/ALM users know they will need to beef up their machines or go to virtual desktops when they get Windows 7.  Overall, QC10/ALM11 works pretty well with Windows 7, just be prepared to check your machines so you don't run into any performance issues.

Thoughts on HP ALM 11.0

I was asked by HP to do a couple of press interviews in conjunction with their release of HP ALM 11.0.  I had a good time doing them, but was hoping to be able to expand a bit on what we found value in ALM 11.0  beyond the  couple of sentences included in print articles to help bolster their story.

Luckily, I got to do a follow-up podcast with Dana Gardner talking about how we believe ALM 11.0 will work within our company and the benefits of ALM tools in general.

Kind of weird hearing myself talk, but I think it came out pretty well.  Hope you find it informative and if you have any questions or comments, feel free to shoot them to me.

How did ALM become Agile?

One of my managers and I attended separate ALM related conferences recently and it was interesting how we both came away wondering how the ALM discussion morphed into an Agile discussion.

For those new to this area, ALM is Application Lifecycle Management, while Agile refers to a development methodology based on the Agile manifesto.  Both relate to how software is made and includes workflows and tools needed to manage the process.  But to me, ALM is bigger because it includes all the processes, methodologies and dimensions of the software lifecyle, while Agile is just one methodology for developing software.

When my group thinks of ALM, we take a three-dimensional look at how we build applications and use those dimensions to determine the workflow and tooling needs to help manage the application's lifecycle.  To me, that is the crux of the difference between ALM and Agile, because different methodologies than Agile, lend to different workflows or tooling.  

The three dimensions or criteria we look at are; the development methodology, the technology used and the maturity level of the organization (see pic below).  Depending on what option in which dimension is selected, we will determine the management level and tooling needs when making software.

1. Development Methodology - Agile is just one of the development methodologies we use, which also includes Waterfall and WaterScrumFall (waterfall requirements gathering, iterative construction, and then waterfall testing).  There are others like Extreme Programming (XP), but we don't use them, so we don't list them.  Depending on what development methodology is used, will determine the workflow needs or specific info needed during the lifecycle (i.e. waterfall has requirements while Agile has user-stories).

2. Technology Stack - because we provide tools, the technology stack is important to us and we broke it out into Microsoft technologies, the Java/Web 2.0 technologies and then the legacy technologies.  These could add more sub-dimensions, but this works for us because we found tools fit these technology stacks (i.e. Microsoft TFS for the .Net developers) and we didn't need other stacks.

3. Maturity - because we are healthcare services company, we have different levels of maturity based on if our application is regulated by the FDA or used for non-patient care use or just used for internal use (i.e. our intranet or employee portal page).  Depending on the maturity level will determine the application management needs, like an extended workflow that includes electronic signature.

There may be a fourth dimension which is how the application is deployed: internal use, external use (by customers) or for sale.  Sometimes this is covered by the maturity level, so it is debatable if it needs to be included. The main reason to include it is when groups need to account for deployment and operations into the ALM and they are including monitoring and defect reporting into their workflow decisions.

I'm thinking Agile is becoming such a hot topic that it is starting to take over discussion about the other parts of ALM.  But if you look at managing the application lifecycle, you will see there is more than just how to build the software, and just talking about Agile short changes the larger ALM discussion.

How Windows Server is like the Dallas Cowboys

Without going into too much detail, a question came up about how our users access our tools.  Most of our centralized tool infrastructure is based on a web/app or app server on Windows Server 2008 R2 machines and our databases are on either Windows Server/SQL Server or AIX/Oracle configurations.  A question came up about licensing and we usually license by processor for any of the 3rd party tools because we have a variable number of users hitting our applications and it is usually in the thousands.

During the conversation, the topic of Windows Server Client Access License (CAL) came up.  I told them I didn't think my users needed them (even though we have them) because my users don't access the server, just the app through either JBoss or IIS.  Oh, but I was wrong.  According to the Microsoft license specialist (I deleted the references to our company and bolding is mine):

• If the Windows Server is a server for internal use, every user/device that accesses the server directly or indirectly needs a Windows CAL.
• If the Windows Servers are used by external users, you need External Connectors.  External users are defined as users who are not employees or onsite contractors.
• If the Windows Server is hosted by someone else, the server needs to be properly licensed by the Hoster 

So how is Windows Server like the Dallas Cowboys or New York Yankees/ Mets /Giants /Jets or any other sports franchise trying to invent ways to get more money out of their stadium?  Because according to the stipulations above, Windows Server is the equivalent of a Personal Seat License (PSL) to house your application.

A computer is useless unless it has an operating system (well not totally useless, you could use it as a door stop), so you pay to put an OS like Windows Server on your web/app or database box and then install your app or database on top of that.  But according to the stipulations above, the end-users has to pay for a CAL (or ticket) to access the application even if they don't access the web/app or database box itself.  So even though we bought a server license (PSL), we still need a CAL (ticket) to use our application because the user is indirectly accessing the OS.

So why would I buy a Personal Seat (or Server OS) that I couldn't use unless I bought a ticket (or CAL)?  Because people like Jerry Jones think of these things (I hope you don't win another game Cowboys!)

Open Source Scanning Tool Eval

One of the risk areas for ISVs and others who make software for use outside of their company is the inclusion of open source packages in their software and the license obligations associated with those packages.  Most of our R&D groups have a pretty good handle on what open source packages they are using, however, our legal department and R&D management felt it would behoove us to have a scanning tool in order to verify everything included in our code.  So, with that in mind, the governance board kicked the tool evaluation over to my group and asked us to get an open source scanning tool.

Players


We found only a couple of players in the open source scanning space, BlackDuck, Palamida and OpenLogic.  There are a couple of other supporting groups like Veracode (they are mostly for security), Protecode (the engine used by OpenLogic), and  FOSSology (a community that develops a package to analyze code for open source software), but our evaluation stuck pretty much to BlackDuck, Palamida and Open Logic.


Overall, BlackDuck is the front runner in the field.  Gartner, Forrester and the like all recognize BlackDuck as having the largest structure, customer base and offerings of the three.  Palamida labels themselves as "application security for open source software", but their sales person dismisses the security talk and concentrate their message about their scanning capabilities.  OpenLogic is pretty small, but they have a decent scanning tool and a hosted offering to upload a fingerprint to analyze and then review the results.


Requirements


The requirements we looked at were:

1. Ability to review source code and binaries
2. Understand license obligations
3.  Provide software inventory (bill of materials)
4. Comprehensive library
5. Multi-user/Multi-role system
6. Common IDE interface
7. Report generation
8. Installation ease 
9. Ease of use
10. File comparison feature
11. Performance
12. Automation
13. Cryptography

Stakeholders

1. The following groups participated in the requirement gathering process and product evaluations:
1. Legal
2. Open Source Policy Subcommittee
3. Open Source Risk Assessment Subcommittee
4. IT Risk Management/GA Audit
5. Business Unit R&D Leadership

Outcomes


We did a POC with BlackDuck, Palamida and OpenLogic.  We used the same code package and gave each vendor a week to do the scan, reconcile the findings and do a presentation of their findings.  Both BlackDuck and Palamida came onsite to do their scans, while OpenLogic did theirs remotely.  Palamida brought their own box while BlackDuck had us install their software on a virtual machine in our development center.

All the scans came back with pretty much the same results and luckily our code didn't have any major violations.  We compared the found list and there were small variations in the number of exact hits, but between the exact matches and partials, each tool did a similar job of finding the open source components in our code.

All three vendors warned us about the large effort to analyze the scan after it was done and based on their results and presentation, they do have a point.  All three had delta scan ability, so the initial scan would have the majority of the work, but once a baseline was set, then later scans would be easier.

This may have been just our thinking, but it appeared to us BlackDuck and Palamida were trying to bundle their scanning software with their analysis service and when we tried to divorce the software from the service, the price of the software rose.  Also, the environment overhead for BlackDuck and Palamida was not overly large, but because OpenLogic was a hosted solution, the two were large in comparison.

Each of the tools had a customizable workflow engine, with BlackDuck and Palamida having a fairly robust offering compared to OpenLogic (Palamida's later release had the better workflow engine).  The policies and policy rules were the most important to us and all of them had that ability.

Decision


At the end of the day, we decided to go with OpenLogic.  During the evaluation, we found pretty quickly that what we wanted was just a scanning tool to tell us what open source components were included.  We were not at the point to deal with a robust workflow engine with submission and approval workflows.  Nor, did we need a comprehensive library to store all our code as well as open source code our groups were using.  Also, we already have architects, configuration managers and enough engineering staff to be able to compare code snippets, so we didn't really need the analysis.  When OpenLogic came in with a reasonable price and an option for just the things were looking for, we decided they were the one to fit our needs.

I think BlackDuck and Palamida have a place in this space, but I'm not sure we as an organization were ready for what their strengths are.  We may be later, but for right now, we brought OpenLogic online and have done a scan and are happy with what we are seeing.  We will work with our Open Source Task Force to come up with a roll-out process and I'll update this blog when we progress down this path with any challenges and findings we have.

Translate Feature in Outlook 2010

We have a group in France who uses our tools, so periodically I have e-mail exchanges with their QA manager.  She speaks English very well, so there is not a need for me to know French, but as a courtesy, I try to include as much as I know in French (Bonjour, Merci, etc...).

So, in Outlook 2010, I was intrigued to see the Translate feature off the Review tools.  This isn't a bad tool, and so far most of the messages I have sent have been understood (not sure they are correct, but she has understood the meaning).

The translation isn't native to Outlook, but exports the word or text to an external Microsoft site to translate.  It has the ubiquitous pop-up saying you are transmitting data to an external site, so I wouldn't do it for anything confidential, but the site is pretty quick and as said, fairly accurate.

This is really helpful when receiving a message in another language, because the tool will translate it so you can read the message.  The issue is when you are trying to write something and use the translation tool to convert it to the receivers language, when you copy the converted text, it highlights it in this baby-puke yellow and you can't format it out in Outlook.  The reason it is highlighted is the translation tool displays a pop-up showing you the original text, so I understand why it is there, I just wish you could format it out.

Overall, it is a pretty neat feature and a nice addition.  Now if it could translate texting from my pre-teen, I would be jamming.

More on Security Development Lifecycle

In a previous blog, I talked about the methodologies and maturity for a security development lifecycle (SDL).  This is only one piece of the SDL program, and it takes people, process and technology to implement a SDL.  When we looked at these aspects, we tried to come up with a staged approach so we could implement security in our development process but in a way that didn't overrun people.

People
On the people front, we looked at the number of people and their roles for a centralized SDL team.  We quickly found we needed:

• 1-2 FTE for 1 yr to write the process and training materials
• 1 FTE for training coordinator
• 1 FTE for security tool administration (1/2 FTE for admin, 1/2 FTE for BA)
• Security buddies (Microsoft term) to help R&D groups take ownership of SDL in their business units

Because of that last one, we knew we needed to have more help from the business units and this would take a lot more people.  We also saw, that as we added business units, this could increase the number of people in the central roles, so we would assess our work as business units came on board.

Process
For process, it was defining what security means in each of the stages of the software development lifecycle.  The things we looked at for the major points of the lifecycle include:

• Requirements
• Include security non-functional requirement documentation that cover:
• access controls
• deployment consideration
• authentication
• authorization
• input validation
• logging & auditing
• error handling
• Design
• Threat model the design
• Include trained business unit security liaisons (BUSL) and whoever wrote the security non-functional specification
• Development
• Perform security checkpoint reviews after agreed upon code completion time
• Security review reports
• Static code analysis with training best practice
• Testing
• Dynamic code analysis with training best practice
• Deployment
• Determine how corporate security is integrated or help external customers with their application security checks

Technology
We have static and dynamic security testing tools, so it was a matter of rolling those out to all the business units and providing a central offering for those without the expertise.  This was probably the easiest of the pieces to tackle only because of what we currently have.

Deployment
To deploy this in a phased approach, we looked at the tasks for people, process and technology and grouped them into what we could accomplish without too much disruption.  With that, we came up with this model to help us plan our work.

*STRIDE approach is the Microsoft approach to threat modeling using Spoofing, Tampering, Repudiation, Information disclosure, Denial of service and Elevation of privilege.

How did it go
So how did it go after we planned all this?  Most of the items are being rolled out but I'm not sure it is as coordinated as I would do it.  Being in the tool business, my main focus was on technology and we have the static and dynamic code analysis tools in place, but we are assessing their value and determining if we need to replace or promote the tools.  I feel pretty comfortable in our space, and I'm sure the development groups are getting more comfortable putting SDL practices in place with their work.