Rajith’s Column

Managing Software Projects - A Software Engineers Perspective.
Contrary to popular belief, many aspects of a software project could be proactively controlled and managed by an ordinary software engineer to make his/her life easier. Instead of whining about frequent requirement changes, looming deadlines ..etc, there are many things you can do to help yourself retire with enough hair left. The “Managing Software Projects” series is an attempt at exploring these aspects.

Be In A Position Of Strength When Changes Occur (By Maintaining Code In A Ready-To-Ship State).
Rarely does anything in life work according to plan and software projects are no exception. Simply put, changes are bound to happen and how you deal with them will determine how successful the project is. Most of these changes will be involuntary and unexpected and the programmers and testers are usually at the receiving end. With deadlines whizzing past you the pressure mounts and the vicious cycle continues. This is where a solid foundation is useful as explained here. If you have a solid foundation, then it becomes less of a burden to maintain code in a ready-to-ship state at all times. Without that you may not have the ability to respond to changes in a timely manner. Lets look at a few scenarios.

• Sudden requests for demos or alpha/beta builds by influential customers. If you have a continuous build system and you keep your builds reasonably green, then you could literally provide the last successful build. If not you would need to drop everything you are doing and try to figure out a way to get a stable build. If you have many test failures, you will probably need to fix them first as well.
• A team member resigns, or falls ill. Unless the project culture had insisted in tested, reviewed and documented code right from the get go, the person who have to take over will need to spend a significant amount of time in getting it going.
• And our favourite, the customer and/or marketing folks change their mind for the umpteenth time. Again if you have ready-to-ship code backed by an armada of tests (unit, integration, performance, interoprability ..etc) then you are in much better position to make the required changes with confidence. The tests will provide you instant feedback about the impact. The most important thing is to be able to give an accurate picture of the impact of such a change. If you have evidence to backup your claims then managers, marketing folks and the customer are more likely to listen to you

Have A Well Defined Process To Manage Change (No, I am not talking about the process that your organization already has)
Most organizations have a formal process for handling RFE’s and bug fixes etc. (There are also organizations that are completely at sea when it comes to this). Irrespective of whether there is such a process or not at the organizational level, the engineering team should have it’s own. Some points to consider from an engineering perspective (as alluded to in my previous post).

• If your organization doesn’t have a proper process already in place or does not follow the process, then the first thing to do is to work on getting that done. It’s sad to see that some organizations doesn’t have such a process, while others did have a process but was frequently ignored for short term convenience. The consequences were disastrous, especially for the poor engineering team, and not to mention that the ability to respond effectively decreases dramatically over time. How such a process should look like and what’s involved is a separate discussion that I hope to tackle later on.
• Branching policies for handling releases, hotfixes (these are notorious for making your life dreadful). In my humble experience, having this discussion right at the beginning (and on an ongoing basis) vs later in the project cycle (especially when everybody is under the gun) is whole lot more productive.
• Handling RFE’s/critical bug fixes for existing releases. - same as above. Do it when your head is relatively free.
• Figuring out how to evaluate change requests & who is responsible for what area to make that call While the team leader/manager is ultimately responsible for making a call, they are always not in a good position to make a good decision without input from others. Therefore it want hurt to have folks informally appointed (known within and outside the team) as subject experts in certain areas. Radical changes to the code base, taking on new functionality that directly/indirectly affects a particular area should not be decided without the input from these area experts. All though Apache Qpid is an open source project, most committers will discuss with the area expert (on the dev-list/JIRA) if the commit falls outside the committers core area of expertise. Not only it’s courteous and reduces friction/disagreements it also ensures things are done correct the first time which minimizes the impact it has on schedule etc..

Deal With RFE’s/Requirement Changes Methodically (instead of being emotional and kicking up a storm).
No matter how swamped your team is and/or how ridiculous the request/requirement is, emotional outbursts or rejecting them with “not enough time” ..etc is not going to help.
It’s in your best interest (and the interest of the customer) to listen, understand, analyse the impact and then compile your response backed with evidence.

• If you make an effort to listen & understand the change request,
  • You may find the real underlying requirement is something else. Which maybe easier to implement after all.
  • Provide an acceptable alternative
  • Provide an acceptable workaround until you fix it properly in the next release.
• If you provide a proper response backed by evidence
  • First of all people are likely to listen to you as you are presenting facts backed by evidence and not mere conjeture .
  • It helps people to understand the impact and the consequences (Ex half the functionality will not work and would need n extra days).
  • Even if your opinion is ignored and the change is forced upon, atleast for your own/teams benefit you know how much of an impact you have to deal with.

Your Credibility Is Important.
This could go a long way in ensuring your opinion is heard, hence being able to negotiate changes successfully!

• If you have provided sound judgement in the past, the managers are likely to listen to your opinion.
• If you have built credibility with the customers, then they are likely to listen to your opinion, accept your workaround or alternative solution.

Ability To Improvise
Sometimes it takes a bit of improvisation to deal with the changes and still beat the clock. Instead of complaining and whining about the changes, use them as an opportunity to show your worth.
And here are some stories..

• Once we had to deal with a customer who asked for frequent GUI changes on an application that ran in an embedded device. Suffice to say it was a real pain. At the time one of my colleagues was toying with the idea of expressing the screens in XML. We promptly adopted it and taught the customer how to customize the screens to their hearts content. My colleague eventually went on to lead the team.
• We needed an automated build system to run our test suites in different OS’s and aggregate results. As the project progressed the permutations grew. We looked around and didn’t find anything that fit our needs. Then a colleague of ours used our own product to create a distributed automated-build-system.
• While working on an application that ran on an embedded device (same as item 1), the QA team had a tough time testing due to the restricted nature of the embedded device. The long test cycle ate into our dev cycle and impacted our turnaround time. I finally wrote a simulation program during my free time that simplified a lot of the testing. I scored some brownie points there, which came handy during the next performance review

In summary, a software engineering team can do a lot to mitigate the risks posed by changes that happen throughout the project. How well you negotiate those changes will have a huge impact on the final outcome.

Managing Software Projects - A Software Engineers Perspective.
Contrary to popular belief, many aspects of a software project could be proactively controlled and managed by an ordinary software engineer to make his life easier. Instead of whining about frequent requirement changes, looming deadlines ..etc, there are many things you can do to help yourself retire with enough hair left. The “Managing Software Projects” series is an attempt at exploring these aspects.

It amazes me to see how little attention is paid to this simple truth when a software project is undertaken. There is a reason why it’s said “you reap what you sow”. More often than not the success of a software project (for that matter many things in life) depends on how well you have laid the foundation. But It seems far too many think that software projects are immune from simple realities in life. You don’t need to have elaborate plans or lengthy kick off meetings, but reasonable time and effort needs to be spent on getting the basics right. While none of us have the power to foresee every obstacle that might crop up during the project, there are many aspects that are within our control. And more often than not, these aspects are not given their due recognition, hence taken for granted !

The Way We Set It Up Is How It Ends Up.
What can be managed should be managed right from the start by following the proper process. We all know we need to test, use coding standards, provide documentation, do code reviews, follow process..etc, but how often do we think about these aspects right at the beginning of the project? These are things that we can control to a reasonable extent, but if not paid enough attention could spiral out of control and eat up enough cycles to cause major disruptions in your project later on.

• How many projects use a continuous build system right from the start? In the first couple of days or so you may not even have enough tests to run, but ensuring that at any given moment the code is compilable and tests are passing will set the tone for the project right at the beginning.
• We all think we could write documentation when we have enough functionality, but we soon find out that we have quite a bit to write with not enough time. Ditto for code review & coding standards.
• Same goes for writing tests too! By the time we decide to write the tests in earnest, we have done quite a bit of coding. All that time saved by not writing tests and a lot more is now wasted doing damage control.
• Using a bug tracking system right from the get go is better than using sticky notes:)
• How often do we slap in a quick ant file/make file ..etc to get it going soon, thinking we can improve it later as the projects gets more complex. Anybody who had gone through the pain of tinkering with the build system when the project is in full swing knows what I am talking about. Spending some time upfront designing a more robust build system would save you loads of time and frustration down the line.
• During the initial setup period, how often do teams discuss about the internal release process, branching policies, how to handle hotfixes, bug fixes for existing releases etc. If these points are discussed in earnest right at the start, then perhaps you could put in place structure that would accommodate them more easily vs having to bend over backwards at a latter stage of a project when we really need to do it.

Simply put, following proper process right at the start will determine how successfully your project ends.

This Applies Even When Your Are Doing A Proof-Of-Concept (And here’s why)
I have seen many proof-of-concepts being done without using any version control, without adhering to coding standards, using a bug tracking system or writing any sort of tests. The justification being it’s only a proof-of-concept, and we could do it properly when we get to the real thing. But how many times have we seen the proof-of-concept being used almost as it is as the initial code when the real project starts?

Now all of a sudden you have a lump of code, that is not reviewed, documented, tested or version controlled properly. Since no bug tracking was used, all the time spent in testing the POC is now wasted. While nobody cares about the POC being stable and forgive you readily when the system crashes several times during the demo, the same allowance is not given when you deliver the real system. But scarily enough the same unstable code is now being used as the base for the real system. Now is the time you lament about not following proper procedures during the POC development.

Ideally a POC should be what it is - a POC and nothing more. The real system should be started from scratch. But the problem is, we don’t live in an ideal world!
So if your organization has a history of turning POCs into real systems in double time, then follow process right from the start to avoid a stroke or a heart attack.

The Only Thing That Does Not Change, Is Change Itself. So Plan For It At The Beginning!
Changes are going to happen! Either the customer or the marketing dept or both will change their mind. These changes will happen for a variety of reasons and more often than not they will need to be accommodated. We cannot have an architecture that will withstand all the numerous RFE’s. We cannot plan for everything. But we could still have a plan for dealing with these changes! Having one right at the beginning will save you a load of trouble. You may need this kind of planing at different levels.

• Most organizations have a process for dealing with changes and RFE’s. But you may need to have a project specific criteria as well. What might work for one project may not work for another, so the corporate guidelines may be inadequate in covering all scenarios. Working that out right at the beginning and getting approval from all concerned parties may reduce a load of pain at the end.
• While you cannot architect for every conceivable change, we could still try to identify areas in the system that can potentially change. Not only this is useful in scheduling, it is also important in the architecture stage as you could try to isolate change as much as you can. Even though you have no control or visibility into how a particular feature might eventually pan out, you still maybe able to isolate it enough to minimize the impact the changes could cause in your system.Identifying areas that could change at the beginning and setting up your architecture in such a way will ensure how well you deal with inevitable changes.
• Similarly testing strategies should also be planned with changes in mind. Having this discussion right at the beginning rather than later will no doubt contribute positively towards the success of the project

Related to this topic is discussions about resourcing, requirements planning etc. These aspects are critical in properly setting up a project. However the above piece is written more from a software engineers perspective. In some ways the software engineer has very little influence (ex resourcing, requirements planning..) in determining the way a project is setup and in other ways quite a bit is within their sphere of control (ex. the aspects mentioned above). The idea is to maximize what you could manage/control and plan for what you cannot.