If you have a business of moderate to enterprise size, the chances are you have some custom software running your in core business systems. If that application, or suite of apps, has been running for more than a few years, it is also likely you have a regular team maintaining it and adding features as needed. After all, your business keeps moving, changing and reshaping your core Intellectual Property (IP). New regulations and data requirements pop up regularly that have to be addressed. There is nothing unusual about this situation, it is a normal part of custom application maintenance.
But let’s look ahead – what happens when that application reaches a point in the lifecycle more than five years down the road? Will the technology it is based on still support your needs? Probably – but for how much longer? At what point is the return on investment exhausted for your app? Seven years? At ten years, if you have kept up with maintenance and needed changes, is the code getting harder to manage? Are workarounds, unused code, and patches making it hard to follow how the app works? Are there still experienced resources in-house or available on the market that can work on the app? Is their cost still a reasonable trade-off against a rewrite? Can the software scale to meet your current requirements? Will it continue to over the near term?
To Rewrite or Not? A Continuing Assessment…
These are all questions that engineering teams are asked to evaluate on a regular basis. Most people will shy away from even considering a rewrite but time and technology can catch up with you regardless. In the best case, the target application is modular or a suite that can be attacked a piece at a time. But regardless, when the time comes that it is less risky and costly to rewrite an application than it is to continue with existing systems, the finances and plans have to be in place to assure that core business processes continue and the development effort can be fully supported. The skilled resources needed to do the job when the time comes is another matter. Assuming the rewrite is going to take place in parallel with production and maintenance of the existing application – can you spare your regular staff to take on the work? Do they have the skills and experience to work with the newer technologies you need to assure you don’t have to go back through this exercise too soon? Even if they do, is their value higher in their role of keeping the existing application reliable and on-course?
Of course, if the primary aim is a technology uplift, there are many tools that can help to convert source code from one version or code platform to another. None of the tools available can provide faultless target code, but they can speed the effort greatly, especially when large modular applications or suites are involved. But, to get the best return for your investment in the project, your team needs to be aware of the many tools and strategies available to them so they can find the best match for the situation. The project team also needs to have a good understanding of the architecture and operation of the source, features to be added or changed (which may be a moving target), organizational DevOps systems and procedures, and points of reference for business rules and technical integration. These areas of expertise, along with planning for testing and pre-release evaluation, are a lot for any team to take on, but if they are also responsible for on-going maintenance of the application in production, it is likely the project will either be too much to handle or the effort will go very slowly.
Even if the primary aim of the rewrite is to expand application capabilities and function, you will still want to consider upgrading technology where it is advantageous to do so. Rewrites are not light efforts and you don’t want to have to do them too often. But again, your resources need to have the skills and experience to know when a technology fits a situation and how to apply it. If your in-house team has been totally engaged in maintenance and incremental feature additions, they may not have the exposure to the wider field of technology necessary to make key decisions. You could hire specialists to address the weaknesses, but that can become a project in itself with developing acceptable profiles, recruitment, training, and integration into your team. And when you are done with the rewrite, scaling your team down for normal maintenance and feature development can be difficult as you deal with seniority, team dynamics, and individual career goals.
Outsourcing the Rewrite
So, let’s say you have moved down the road to the point where you have decided a rewrite of your application is the best answer. As you consider whether to use your internal team to do the rewrite or to continue to maintain the existing app while they gradually transition to the new one, you also have to consider the costs, your appetite for risk, and the time horizon you have in front of you to finish the project. Using an outsourced team to maintain your existing systems will require a great deal of knowledge transfer from the existing team, which will take them away from the new development effort. It will take considerable effort for the new maintenance team to reach full productivity, so their ability to tackle new requests will have to be carefully managed and prioritized. And of course, there is the element of risk to be considered. Assuming the new team does not know everything, all the history behind maintaining and extending the existing code, bugs can be expected that could cause production delays and outages. Depending on the operational value of the application to the organization, this could result in some serious situations and in the end, it could require pulling back the team working on the rewrite and delaying their work.
Because of the risk to continuing operations, most organizations will chose to keep their existing team in their positions, but with a requirement that they engage in a two-way knowledge transfer operation with the outsourced team working on the new application. The in-house team will provide the new team with the information they need about the existing app and the rewrite team will provide the knowledge the in-house team will need to maintain and continuously improve the new app after the project ends. This decision has several side effects that can be beneficial to the rewrite effort. Let’s assume you have chosen a service provider that has considerable experience in rewriting legacy applications and the technologies involved in your project:
- The rewrite team will have a wider view of available technologies and strategies to use for the rewrite project itself and in the new version of the application. Assuming a project organization that includes the in-house team, the resulting application will have a much better opportunity to remain compatible with other systems in the organization stack while it paves the way to new technical advantages.
- The rewrite team will have experience with organizing priorities during the project and maintaining a “big picture” view of the application in coordination with the in-house team. This is critical with larger projects as both new requests surface and unexpected opportunities will arise during development and need to be rolled into the project. Making decisions on what needs to be in release 1.0, what can wait until a later time, and what needs a base so it can be added later is critical for the success of any rewrite project. Assuring the in-house team is fully involved in these decisions is key to assuring the transition from new development to maintenance mode will go smoothly.
- The management of operations between the two teams is very important. Assuming a project using agile methodology, you will want to begin live testing of the new application against at least test data with real users as soon as possible. If your outsourcing team has experience with development using automated testing, continuous integration and source control with shared repositories, the opportunities can come relatively quickly because agile methodologies are predicated on releasing functional code to users frequently and incrementally. Even if your in-house team is not fully agile in its daily operations, the two teams should be able to leverage their experience to arrive at a release system that enables your test users to get on the new system and test it long before it nears production level.
- In the best case, your outsourced team and your in-house team should be able to communicate and work together in real-time, without a delay for opposite work cycles like often occurs with an offshore team. Real-time communication, without intermediaries or stops between the members of the development teams, is critical if you want to avoid time wasting delays for an answer to a question until the next day and workarounds that don’t really address core problems. And it builds trust and communication between the teams because they begin to understand each other and how they work.
- The outsourced team can be balanced to take care of the skills needed in this particular project. While most maintenance and update operations don’t get too involved in the UI of the application, the rewrite is likely an opportunity to improve the UI, making it more intuitive and streamlining entry processes. If you have a complex data structure or issues with data security, the outsourced team can add resources with skills that can address issues of data replication for testing and data migration. Look at the outsourced team as an opportunity to add skills you need but only for this project rather than as long term staff assets.
- At the end of the project, the outsourced team is no longer a cost and your team takes over. You may want agreements for a period of ongoing support in early stages after release from your outsourcing provider, but getting your team up to speed and productive needs to be your primary goal. If you have used the opportunity wisely, your team worked with the outsourced team enough to be able to take over without too many problems
So, thinking about those points leads to a few best practices for outsourcing code rewrites:
- Your outsourced team should have experience with conversion tools to address your source code and target technology, if you are not rewriting from scratch, and they also need experience in managing the knowledge transfer process necessary throughout the project. Rather than trying to understand every aspect of the application from the start, they should be able to “peel the onion” a layer at a time, getting down to the information they need quickly, without spending time on areas that are not necessary for the work at hand.
- Your in-house team should be tasked with assuring the outsourced team has the information they need whenever they inquire. Delays and incomplete information are as dangerous to the project as bad code. If possible, the outsourced team should be in the same time zone or with at least six overlapping work hours. You can work around the problem, but there is no substitute for being able to talk directly to each other rather than using emails and go-betweens.
- Your teams must be able to manage priorities and new feature requests strategically with joint decisions on what will be and won’t be addressed for the release of version 1 of the new app. Without that level of control, it is easy for the project to get out of hand with an unending queue of new requests and no end in sight. At the same time, cutting off requests entirely during development is often not sustainable, so the experience of both teams in estimating tasks and negotiating for the best outcome for everyone is critical, especially in longer projects.
- Your outsourced team should have the experience necessary to develop automated testing of the new application, continuous integration and shared code repositories. They should also be able to develop a plan for breaking the application into strategic pieces that can be attacked and finished incrementally, so that it can be released for user testing a piece at a time, rather than as a full production release.
- Between the two teams, they need to work out a process for data migration, replication in cases of higher security, and assurance that if the outsourced team cannot work directly with production data, they have a replacement with the same characteristics and a process for going to production that includes data migration and conversion if necessary.
- Although the in-house team must be the lead for technical and operational decisions, the two teams must be able to regard each other as peers. If one team feels subordinate to the other, it puts an artificial barrier between them that will keep new ideas from being put forward and issues from being fully discussed. It is a difficult area, especially if the in-house team feels threatened by the prospect of outsourcing more projects.
- At a technical level, there are some key areas that can cause friction if the two teams (existing app maintenance and new app development) don’t coordinate early on. These include:
- Integration with existing systems – Testing must begin early against systems that use or share data with the application under rewrite. The longer this task waits, the more likely it is to generate unexpected problems.
- Real data set availability – Regardless of how hard teams work to build representative replicate data sets for use in development and testing, there are almost always unexpected issues that crop up when an application is put against live data in production. Of course, there are many situations where because of regulations, IP and security concerns, the risk involved in using live data simply cannot be ignored. If that is a concern, it must be clearly understood by everyone that mitigation must be developed by the two teams and implemented when the new application first begins to use live data. Rigorous testing procedures and performance testing before deployment, along with frequent database snapshots during early production runs are among the many strategies that can be employed.
- Testing – Automated testing is among the most important quality assurance tools available – but of course it depends on the quality of the tests being developed. The outsourced team should be fully aware of their responsibility to develop quality testing and maintain coverage throughout development. User testing is also a critical area that can help find bugs and determine if usability and process flow meet user needs. In the end, acceptance testing is the point where things can fall apart if it is not carefully considered and part of the production process so that acceptance can proceed on incremental releases as they are available. Waiting too long to take on the acceptance process can result in costly rework and time losses.
- Procedures – It may seem obvious, but too many teams assume that because both teams use a similar methodology (agile for instance) their daily operations will also be compatible. It simply isn’t the case. Teams adapt their operational procedures to their environment and operational requirements and conflicts often arise when one team has a release ready but cannot proceed because critical systems are not ready. Planning sessions, workflow diagrams, walkthroughs and best of all – live working sessions where the two teams can experience the process directly – are all good approaches to assure that operational procedures integrate properly and are well understood. This is especially true in situations where the organizational standard is for a periodic release (perhaps monthly) and the new development team is agile and expects to be able to release with high frequency so they can continuously test against integration points and data services.
- Regardless of how experienced the outsourced team is, there is no substitute for the availability of regular, direct communications between teams. In the best projects, knowledge transfer is a two-way conversation about how the existing application works and how the new application will accomplish its goals. It is not a one-and-done process. It continues throughout the project as developers work through the layers of each application and consider new ways to deal with issues. In the best case, both teams would be collocated, but costs often make that option unsupportable. Teams within a region or nearshore teams offer a strong alternative however. Both teams can be working within regular hours and while they use all the tools of the Internet to talk, chat or share data and video and collaborate in real-time.
Application rewrites are an opportunity to improve your systems, making them more responsive and reliable in their role in your business operations. They can create serious issues if they are not carefully considered but there are many ways to approach these projects that can lower risks and assure a positive outcome. Scio Consulting has considerable experience with providing nearshore teams for rewrites of critical business applications. We would be glad to help you consider and plan your upcoming project.