A Look at our R&D Tool Use

As a provider of centralized tool offerings, we periodically check with our users to see what is working and what can be improved with our portfolio.  It also gives us a chance to see what has run amok since the last time an inventory was done and what work we need to do to herd the cats back in the same direction.  With that in mind, we recently conducted interviews of our various business units about their R&D tool use and came away with some surprises and some not so surprises about the companies R&D tool use.  We talked to 100 people (out of the ~7500 who use our tools regularly) in 40 different office locations, in 27 states and 6 countries.  The interviews spanned most of the major divisions and business units, but did not include business units which did not develop software.  That may be a mistake, because we may have a solution for a business problem, like the Support group using Quality Center for their CRM (I know, don't ask), but we won't know if we don't ask.

Our current centralized tool offerings do not cover every aspect of the SDLC, but we have tools for design and modeling, requirement management, construction, testing (functional, performance and security), build and change management.  As we expected, even if we don't offer a tool in all phases of the SDLC, there is at least one being used in most phases somewhere in the company.  What probably surprised us the most was the number of tools used in some of the phases.  We are a company grown by acquisition, so we expected some variations and disparity of tools, but didn't realize the extent.

Below is a breakdown of the tool functions, the percentage of the teams using a tool for that function and the number of tools of that function.

The big things that stick out, at least to me, are:

1. 28 change management tools - I'm not sure we have 28 major development organizations, so that tells me some development groups are using multiple change request systems.
2. 72% IDE use - so 28% of groups are coding outside of an IDE?
3. 15% using Security Testing tools - this stood out until I realized most groups are still coding fat client applications, and there aren't really security testing scanning tools for those applications.  Talking with our Risk Management group, they are using threat modeling and secure coding techniques for most of those apps that cannot be scanned.
4. 1 Documentation Management tool used by 4% of the groups - I guess I better not let the SharePoint folks know about that one.

We presented this to the R&D Tools Task Force and they have given us some direction on what tools we should change in our portfolio and what areas we need a central offering.  We have some work ahead of us, but overall, this was a great exercise and I look forward to the change.  In future blogs, I'll talk more about our current tool portfolio, the tools we evaluate, and what we eventually settle on to get these variations down to a reasonable number.

Dusting Off Our Testing Methodology

I'm doing some volunteer work with a non-profit software company who makes an app for other non-profits to track the people they serve as well as data collected during their encounter.  Kind of a neat project and something I didn't really think about until I got connected with this group.  It's a pretty lean (not Lean) organization, with only a director, some services folks, a couple of developers, a couple of tester/support people, a business analyst/support person, and an admin assistant (who also happens to be my wife).  

I got connected with this company because my wife talked with the director of the company, and he needed to do some regression testing of their app and my wife told them about my role, and how we have R&D tools to support various stages in the SDLC, and I may be able to help. The company I work for supports various volunteer activities and I thought this could be a good one for some of the testing folks I know, instead of the traditional build a house or make care packages, so I was up to help and thought I could get others to help as well.

After talking with the director, what I found was not so much the need to do regression testing, they definitely need that, but they needed a testing methodology to tell the who,what,when,where, why and how of testing their app.  Luckily, I was instrumental in writing our testing methodology for the Verification & Validation piece of our CMMI appraisals about four years ago, so I decided to dust that off and see if it is transportable outside of our organization.

Looking at our methodology docs from afar, I'm not sure those fit the newer (at least new to us) development methodologies like Agile or Lean, but was made for Waterfall or what we call "WaterScrumFall" (waterfall in requirements, iterative in construction, waterfall in testing and release).  Our testing methodology concentrated on the different types of testing (unit, functional, UI, workflow, load, stress, etc...) and categorized those into testing categories (requirement, white box, black box, performance, etc...) and where they fit in the traditional development phases (inception, elaboration, construction and transition).  We then defined each of those and came up with a matrix to show which ones were mandatory or optional for each project type (major or minor release, service pack or hotfix).

We supported this matrix with a document that defined the different testing types, so everyone in the organization could speak the same language.  But after presenting it to the non-profit/smaller organization, I'm not sure lining them up to the traditional development phases will work for Scrums or Lean groups.  Inception, elaboration, construction and transition can be jumbled together and where one starts and one ends can be confusing. 


So with that in mind, I need to rework our methodology and adapt it to Agile and Lean development processes.  I think defining the test types is important, but I need to change when they should be run and what resources should do it.  As I go through that, I will update my blog on what we are doing and how it woks within and outside our organization.

New Place for My Blog

Back in May 2008, I started an internal company blog to talk about software quality and development tools.  I was reviewing the posts and noticed I haven't contributed anything to that blog for awhile and was undecided what to do next.  After giving it some thought and talking to folks I met in conferences and meetings, I thought I would bring it out into the public and open it up more to my new role of overseeing the centralized R&D tools for the various development groups we support.

So, what is the significance of the blog title "Just Don't Call It a Center of Excellence"?  That was the original title of a presentation I gave at HP Software Universe in 2009, but when the HP editors got a hold of it, they changed it to "Offering Common Tools Doesn't Require a One-Size-Fits-All Mindset".  To this day I don't think the new title did my presentation justice.  First, the whole "Center of Excellence"(COE) thing is kind of scary to people, because it implies consolidating work and eliminating positions, and no one likes that, so it is best not to even mention those words.  Also, the point wasn't about being flexible in deploying a common tool set, as much as it was about not forcing a standard on groups.  Instead, provide good tools that people believe in and can adapt to their development methodologies, so they eventually gravitate to a common tool set even though nothing was advertised as a COE.

So what will I write about in this blog?  I've got some ideas, but mostly I will write about the trends and topics I see in the various Software Development Lifecycle (SDLC) methodologies and processes and the Application Lifecycle Management (ALM) tools that support development.  I will also expound on the short takes I post on Twitter or anything I read or hear which may be interesting to most of the folks I know.  I'm not saying this will be the definitive take on software development or ALM tools, but will give you an idea of what is happening in my world in case you are looking for another view.

I look forward to writing again and I hope you enjoy it...