How to avoid spending $15 M on a large IT application you probably don’t need
by Timothy Clancy, Paul Varadian & Bruce Butler
(For a PDF copy of this article click here.)
Before embarking on costly large projects firms should first seek to understand the system-of-systems, which includes technical and human factors that drive performance issues. We use a rigorous methodology to help identify incentives and pressures that drive these behaviors and the interrelationships between individuals, technology, and performance. The insights produced by systems thinking can be incorporated into a solution design for the project – or sometimes – help avoid the need for an expensive and complicated project altogether.
A common problem.
We were approached to help a Fortune 100 company design a System-of-Systems analysis. They had already begun a solution design phase on a new expert-staffing application. Their problem was not unusual:
- Expert resources, billable on external projects, weren’t getting matched to new work fast enough.
- Client work was being delayed by days with missing resources causing lost revenue and dissatisfaction.
- Resources that were assigned weren’t always a match for the skillset needed.
- Highly-skilled resources sat on the bench underutilized.
As a result, no one was happy. Senior leadership had already approved an 18-month IT project to roll out a new application which included solution design, vendor evaluation, procurement activities, a pilot schedule. All the pieces, including change management, were falling into place.
But as any IT professional already knows – these projects are fraught with risk. According to research by the University of Oxford and McKinsey: “…half of all large IT projects—defined as those with initial price tags exceeding $15 million—massively blow their budgets. On average, large IT projects run 45 percent over budget and 7 percent over time, while delivering 56 percent less value than predicted. Software projects run the highest risk of cost and schedule overruns.” Armed with this knowledge, we were asked to provide a complementary systems-level perspective as part of solution design, to identify insights prior to vendor selection. They needed it fast – we only had a one-month window before vendor selection was scheduled to begin.
A different way to think of solution design.
With teams already focusing on the traditional elements of solution design: features, functionality, scope, budget, schedule – we took a different approach. We knew the ultimate solution would still be operating within the same system-of-systems that the current application was failing within. We worked with stakeholders, individuals and small focus groups, to create a “systems” map. Most participants – by habit and training – focused on technical wish-lists and features they wanted in a new app. But we also found within these discussions their ‘mental-models’, the perceptions they had about the system and how it worked was as much responsible for driving low tool adoption as any set of features themselves. Capturing this “human architecture” of the firm’s culture consisting of behaviors, incentives, pressures and habits that may be impacting performance is as critical as understanding the technical architecture.
Locating these mental-models on the system map (using the science of System Dynamics) offers a way of identifying inter-relationships and feedback drivers in complex systems. In addition to descriptive analytics, the map allows modeling prescriptive analytics – analyzing “what-if” scenarios and see what would result in a better staffing experience.
For example, one of our first observations was not just that staffing managers didn’t use the staffing tool, but why they weren’t using it.
Mapped in green – staffing managers had a common complaint – that the profile of expert-staff in the tool was usually out-of-date or inaccurate. Inaccuracies meant wasted time calling staff that weren’t available, or Staffing Managers were relying on their personal ‘tribal’ knowledge of departments and expert resources. It got the job done – placing staff on a project and reducing the pressure to staff, but the less they used the tool, the less they wanted to use the tool. This reinforced the behavior of avoidance.
This led us to speak with the expert staff themselves. When asked why they weren’t updating their profile – they responded that since they never received assignments through the staffing tool, what’s the point of updating? Although this made sense, we visualized it in the human architecture as a parameter driven by incentives in the blue structure below (green from staffing managers entering as potential scenarios.).
When assignments were made via tribal knowledge it reduced incentive to update the tool. This completed a vicious circle resulting in low tool adoption. Had expert staff received their assignments from the tool – they’d find benefit in keeping their profiles current. But because profiles weren’t current, staffing managers avoided the tool and used tribal social-networks to locate resources.
Behaviors are caused by a mix of incentives and pressures.
Incentive behavior like this is frequently the response to pressures in the system. We found the source of pressure when talking with senior leadership depicted in red below. Their pressure on reducing project start delays created incentives for staffing managers to use the quickest route – tribal networks. Senior leadership pressure on utilization resulted in two dynamics. Expert staff not wanting to waste their time on profile updates and managers (depicted in purple) answered staffing-managers calls by suggesting their own staff first. Behavioral response to incentives and pressures had shifted the system-of-systems around staffing from tool-use to a tribal social-network. Whether your manager knew a lot of staffing managers and vice-versa had more to do with expert staff utilization than the staffing tool did.
Contributing to the problem was leadership focus on a seemingly unrelated metric: sales pipeline accuracy. Sales staff (depicted in brown) would react to the constant requirement to update pipelines by limiting the information put in the pipeline tool to only deals already close to closing.
As depicted above by withholding awareness of upcoming projects until closing, the days remaining to project kickoffs reduced dramatically, and ironically, this was the critical time wherein staffing managers had to find resources without resulting in a days-delayed start. The system was now closed with all mental models connected in their feedback.
No one can be blamed. All actors were rationally responding to their own pressures and incentives. This local optimization came at the cost of global adoption of the staffing tool. But it’s hard to find these systemic issues when focusing on features and functions of a software program. They only become apparent when each participant’s mental model is connected in an end-to-end system map, such as the one shown below.
The map serves not only as a diagnostic to problems, but a prescriptive guide to design. This is because the map displays both relationship and feedback type, either reinforcing feedback (shown by a positive sign encircled by an arrow) or balancing feedback (indicated by a negative sign encircled by an arrow) as shown below.
In a snowball effect, the reinforcing feedback loops feed on themselves. The less that the expert staff received assignments from the staffing tool, the less incentive they had to update the tool profile and the less useful the tool became to staffing managers who turned to their social network and phone calls instead. As a result, the reinforcing feedback loops operate in either a “virtuous” or “vicious” method, either for, or against, the goal. Which direction a reinforcing feedback loop operates often depends on the balancing feedback loops in the system. These loops tend to drive behavior to some performance metric or goal, which once achieved, settles into a comfortable routine
When expert staff come to understand the pressure to maintain high utilization (balancing) they do not spend time updating a staffing tool which rarely provides billable assignments (reinforcing). This combination is an example of a vicious mode. If pressure on utilization was relaxed, or counting time spent updating the profile as utilization, this might flip the reinforcing feedback from vicious to virtuous.
How did this help in designing a new staffing tool application?
It is clear from the map that the problem was not the application itself. On examination, the state of the system was that every reinforcing feedback loop was in a vicious mode to act against the staffing tool. Since none of the pressures or incentives that occur originated in the tool itself, it is a challenge to see how incrementally improving the tool with features and functions would overcome them. Trying to change performance against a vicious reinforcing feedback loop is equivalent to rowing upstream – hard work without progress. However, if switching the behavior of the system from vicious to virtuous – the existing application might have been entirely sufficient. The reinforcing feedback now rows downstream – less work with better results. And the map guides us where to look – at the balancing feedback loops which govern the behavior of the reinforcing loops.
Not a typical solution
Our contribution to the overall solution design focused on the ‘staffing experience’ itself and design recommendations at the system level for behaviors that could improve performance. Although these behaviors could be enhanced by technical features most could be changed by management changes. It didn’t appear that a new tool was even needed if these were addressed adequately. Some of our suggestions:
- Count activities related to using the staffing tool as utilization (e.g. updating profiles, placing employees through the tool etc.)
- Don’t penalize Sales for putting in early stage deals that progress to fruition as this will give staffing managers a longer horizon in which to find staff.
- “Prime the pump” of expert staff by ensuring their assignments came *through* the tool. This would reverse the current incentive.
- Consider abandoning a “staffing” tool altogether and focus on building more connectivity between staffing managers, managers, and expert staff with internal social media tools since this seemed to be the natural state-of-the-system.
Although we saw little need for an entirely new application, we did recommend design-features that might work well within this human-architecture if the firm still wanted to pursue a technical project. Some of these included:
- Technology that leveraged existing social-networks (such as a LinkedIn model) over solutions that worked on a hierarchy or skills-based model only.
- Solutions that could harvest profile-information of expert staff would be more successful than those that required manual updates.
We warned however that without addressing the behavioral issues identified in the human-architecture system map, any technical improvements of a new application might be unable to overcome the reinforcing feedback loops of current culture.
Our belief is that too-often, solution design becomes a self-fulfilling effort to justify features and functionalities of an IT application without looking at the system-of-systems the solution will ultimately operate in. By understanding the human-architecture within this system, solutions can emerge that may bypass or limit the complexities, cost, and uncertainty of large IT projects. Even when an application project is still undertaken, the insights from a systems-thinking approach to solution design often steer major features to fit and work within the existing system rather than against it.
We know systems thinking can benefit a wide-variety of projects, large and small and not just in IT. Whether a firm is considering a new method of delivering healthcare, wondering how to introduce a new product into the market, or how to incorporate block-chain, understanding the system-of-systems is vital before getting too far down the road. It doesn’t take very long – only a few weeks – but can save tremendous time, cost and avoid dissatisfaction over the long run.