I could start this discussion with a simple preface that would say something to the effect that if you are thinking this going to be a cool recipe for your lunchbox – move on. No lettuce, tomato, bacon and artisan cheddar on a crusty sourdough roll with a crisp pickle on the side – although that does sound good. I could say the same applies to those who think agile methodology is just a black and white, never-changing, small-team strategy for software development (period). It just isn’t so.
But please, even if either of the above profiles fit you – consider staying on for this article. Worse case, you can bookmark it so you can go to lunch now and satisfy the hunger the opening photo has firmly planted in your mind. Don’t worry – we will be here when you get back.
We mentioned the “Kanban sandwich” in the context of our article on Lean Product Development a couple of weeks ago. In the sense of product development, this pattern provides a general look at how a lean, product or service-based company with a significant investment in custom software serving core business needs might organize their development portfolio and projects. But in bringing it up in that article, we realized there was a lot more we could (and probably should) touch on when it comes to the scaling of agile and solutions within agile organizations.
So Many Patterns, Practices and Frameworks
You could implement all or some of the ideas behind this particular pattern but first you should know something about “methodology patterns and practices.” A pattern is the term used to describe a practice used to implement a methodology or act within an organization based on a methodology (like agile) to meet certain challenges. Patterns are often brought together in “pattern books” by organizations of practitioners that manage them as libraries of best practices that their members have found to be useful shortcuts or best practices. Why make the same mistakes that others have in finding their way when there are already some tried and proven shortcuts out there for you? Good examples are agile patterns and scrum patterns – but there are many more and we will mention some of them them as we go along. There are also “anti-patterns” – the mistakes that others have made and you should understand so you don’t fall into the same traps.
If you are in an organization that in the process of moving to a methodology like agile, DevOps, etc. you’re probably already aware of some of the organizations and patterns that are available. But if you are in an organization that is just beginning to explore methodologies – large, small or startup – these may be resources you aren’t aware of. As providers of outsourced, software engineering teams, we run into a lot of different situations and methodology implementations while working with our clients. We also use the patterns as reference points when we are working through many of the common situations that come up in our projects. They are good ways to bring combined teams together to work on getting past issues and moving on. When you realize that the problems you face aren’t entirely unique – it makes it much easier to adapt patterns to fit your needs.
As patterns around a methodology become richer and the organizations that bring them together become more formalized, they begin to identify frameworks of patterns that enable organizations to use them effectively. The Disciplined Agile Framework (formerly known as Disciplined Agile Delivery – DAD) is usually identified as the organization structure that is used to support DevOps implementations. It is a valuable resource for lean organizations that practice continuous improvement and with software assets that are deployed with continuous delivery.
When the pattern books and a community of practice becomes rich enough to recognize standard practices, they begin to evangelize and certify practitioners. Lean-Agile practices have reached this level in organizations like SAFe which freely provides the Scaled Agile Framework under their training and certification group – Scaled Agile.
Why Do (or Should) We Care?
When we, our clients or any organization hears about the Kanban sandwich or something like it that is a “cool idea” – we should also know that there is an idea, a body of practice and practitioners behind that pattern that have broken the ground, made it simple and repeatable. As I gathered references and ideas for this article, I came across an illustration from the Disciplined Agile Consortium (the group that trains and certifies for Disciplined Agile) that puts the “sandwich” in the context of an organization that practices agile and lean concepts fully.
As you can see from the illustration, “traditional projects” (not just software development projects – all traditional projects) bite off big problems with transformational solutions. Their strategy is to attack the (really-big-nasty) problem head-on and hit all the issues behind it with an entirely new set of solutions in a neatly wrapped package. It sounds like a great idea at the outset. “Let’s solve all our problems in one, big bang and start fresh!” This approach isn’t exclusive to enterprise projects either. Startups often attempt the same type of projects for the “transformational ideas” behind their business models, services and applications.
The problems that stem from this practice (you can call this an anti-pattern if you have been following along) come in two forms:
- Traditional enterprise projects are proposed based on a calculated Return on Investment (ROI). To actually realize the return or to know that the return is actually a loss, will require passing through a period of many months and in some cases – years. That is a long time to wait to see if your transformation is successful, it requires a lot of pressure to reach a successful conclusion and usually – it requires a lot of magic to prove the ROI was achieved. People who have been there and done that, simply cannot be dragged out to do it all again.
- The actual project strategy comes out of the fact that many of the people and key stakeholders involved in a traditional project will move on before the ROI is validated. So, what happens in practice is that when the really big project runs up against some (inevitable but..) unexpected problems and the project goes even longer than planned. The project then either stops short of its goals or reaches its end only to find that the problems have changed and now the benefits have melted away. The stakeholder support and goals are gone or no longer transformational. So, when people do go along with the really big idea project, many of them do with the sure knowledge that the ROI will never be realized, but they won’t be there to see it regardless.
It is the realization that this is a bad pattern to adopt if you want to be successful that has given rise to the Lean, agile and Lean Startup methodologies (among others) that capitalize on incremental and continuous change in a flexible framework to reach strategic goals. The pattern (as opposed to the anti-pattern we just outlined) is to recognize the “really big problem” or “pain” in the case of product-based companies, but not to try to solve it with an equally big project. Instead, you break the problem down into a number of smaller, achievable projects and prioritize them by the impact they will have and the speed with which they can be accomplished. If the impact is high but the time to complete and realize benefits is long, you again (like in the case of agile epics that actually contain main user stories) look for ways to break that smaller project into yet smaller projects and prioritize those. Along with this idea, comes the honest realization that as we burn down through the projects we have identified the goals and priorities that generated the list will change continuously. So – we need a way to pick our battles (projects), capitalize on our wins, and reassess our goals – continuously.
How Do We Pick Projects from Our Portfolio?
Kanban is a methodology adapted from the Toyota study of just-in-time and demand-driven, lean manufacturing systems. In knowledge work, it has evolved to aid decision-making concerning what to work to take on, when to execute, and how much to attempt to do at one time. The limiting factor is the size of the resource pool available to take on new tasks. With that limit, the active project portfolio is constantly managed and prioritized to provide a queue of relatively short and high impact projects that the resource pool can organize to pick from to work on.
In software development, Open Kanban is often used and is based on a set of principles that emphasize constant, collaborative improvement and feedback:
- Visualize the workflow – You cannot improve what you cannot see. Knowledge work needs a way to show progress. Kanban boards are one of the ways to display progress.
- Lead using a team approach – Without a team and leadership, nothing of significant value can be created or improved.
- Reduce the batch size of your efforts – Science has shown that when the batch unit of work is decreased, more can be accomplished. This principle goes beyond simply limiting Work-in-Progress to controlling the size of the batch (project) itself.
- Learn and improve continuously – This practice implies reflecting so that one can learn from experience, and it aligns with performing retrospectives and embracing change.
Applying this back to the problem of managing projects and software development projects in an organization specifically, we can see how the kanban sandwich works in practice. The portfolio (or project pipeline) is managed using kanban methods:
- Limit number of projects in the pipeline based on the size of the resource pool. Always have work available but not more than the amount the pool can (self-organize) to accomplish.
- Limit the size of projects by the time it takes resources to accomplish them. If the time gets too long, lower the size and complexity of the projects in the queue until a repeatable unit is achieved.
- Prioritize projects based on the impact they are assumed to have and the time they are assumed to require. Allow teams to select projects based on the skills and resources available at any given time.
- Constantly reassess the portfolio based on feedback from both development and stakeholders to assure projects will continue to provide value with a reasonable expenditure of time and effort.
When a project team comes together and selects a project from the kanban queue to work on, they use scrum within the project to organize the product backlog and sprints to organize task backlogs. Within the project itself, it is accomplished by a small, cross-functional team in collaboration with the product owner and key stakeholders to provide the solution needed.
After the initial release is pushed into production, the work of continuous development again adopts a kanban methodology, but in this case managing a feature portfolio and fixes with feedback, task sizing and prioritization coming from the user base, stakeholders and product management.
This might seem like a long, convoluted discussion for what at its core is a fairly simple concept to manage project work and drive achievement within an organization – but there is a lot going on in the field now. There are competitive organizations and methodologies, open and flexible approaches and the promise of better outcomes for everyone. From the perspective of a company providing outsourced software engineering teams, it is exciting and important to understand. We can’t simply impose “our way” of working on our clients – assuming it will be accepted and successful. The landscape is full of different roads to worthy goals. As a service, our customers depend on us to assess situations and find solutions that will be successful in their context – and not just technically.
If your organization is considering outsourcing a software development project, give some thought to how that project fits into your organization, its culture and its values. Would you be better to consider a portfolio approach rather than a single large project over several months or longer? If you take on a portfolio approach and you want to outsource your software development, can you select a vendor with a partner point of view that can provide a team that fits your needs, flexibly over the long-run?
Scio provides nearshore, outsourced software engineering teams to our clients in North America. Our teams have a wide range of skills and experience technology and lean/agile development methodologies. We partner with our clients to provide a flexible and scalable pool of skills that can take on their needs over the long run – wherever that take us. If this sounds like it might be a match for your needs – please contact us.