ITIL Release Management and Testing in Shared Environments

[PFCITIL]

I work for a North American insurance company with an in-house development shop, revenues ~600M per annum. We're just putting the finishing touches on a new set of shared testing environments for distributed apps in development. At the same time, we

[itsm_stephen]

I am at a fairly large staffing company in North America (we had nearly $5 Billion in revenue last year) and I run a team called Enterprise Services. We run the Help Desk technology (Remedy), Monitoring technology (HP OpenView), Alerting technology (AlarmPoint), Performance Testing technology (LoadRunner), and a few other things like TestDirector. We do not run QTP, WinRunner and other Functional type testing tools.

Testing is a big area and many people only think of testing in functional terms and not performance terms. There is also 'integration testing' - which essentially is a functional type issue but is typically not handled by functional testers because they seem to focus on one application or maybe a few applications as a specialty but with things like SOA or simply other business needs/requirements enterprise/system wide integration testing is important. It won't do me any good if my Active Directory works fine and my PeopleSoft works fine but due to the upgrade the integration has been broken and my Siebel application uses AD to authenticate but it is fed by PeopleSoft and now "new" users are not being populated into the environment causing some pains for the business.

At any rate, the point I have is, we have never shown things go into production "faster" due to testing - more likely to actually cause some delay, but the quality of the release has been improved. What has gone down is the number of post release incidents and issues. What systematic and disciplined testing has done for us is increased our ability to avoid problems once we go live but I don't think it has increased our speed from design to deployment.

There is an article on CIO Magazine called "11 Ways to Improve Software Testing"

Just a word of advice - something we have run into here - make sure you have good configuration management practices. We have found doing some Load Testing (Performance Testing) prior to going live a good way of uncovering potential problems.....but all of that is thrown out the window when you move to production with a different code set or hardware configuration than what you tested on. This has happened to us and so now we document what we tested on and so when there are production issues we can compare the configurations and see if the problem was with our testing methodology or due to "untested" configuration changes put.

[itsm_stephen]

Are you having delays due to testing because you do not have a good view/understanding of what your testing resources are working on? So, are they busy doing something already and so when a new deployment needs some testing done there is nobody available to handle it?

We don't quite have that problem here but we do run into some resource constraints. We do try to meet with the various PM's to understand their schedule so we can plan (have temporary resources if needed) accordingly.

Also, we have recently hired a Testing Lab Coordinator. His role is basically that of Release Management - except he does not handle Functional testing issues. His job is more about making sure that resources that are required for the Test Lab are made available - for instance if you need a firewall rule changed or a server built or you have some kind of problem with your testing environment you should see this guy first. It is his job to make sure the right people are involved and it is his job to prioritize what people are working on in the Test Lab (as far as - which server should be built first? Which DB needs to be exported/imported first?).

This role was only recently put in so it is a bit too early to tell what the benefits are but I think the hope is that this role will actually speed up some of the "testing" process - only in so much that if there is an infrastructure related issue he helps coordinate and prioritize the efforts to fix them.

Maybe this answers your question a bit better.