Why plans fail
As we enter the new year, our attention is naturally future-focused. New goals, new targets, new resolutions. The problem is, we’re terrible at predicting the future, so how can we plan better?
As humans we’re ridiculously uncomfortable with uncertainty. To reduce that uncertainty we spend a lot of time making plans and predictions for what will come next. Now, I’m a strong believer in goal setting. Like most of us I’ve spent the past few weeks of downtime resting, reflecting on 2024, and planning for the year ahead. I gave up new year resolutions a long time ago, but I do find that the beginning-of-year season is a fresh start to kick start habits and areas of growth to lean into.
On a personal level, progress towards achieving goals is easier to control by building habits. James Clear’s best selling Atomic Habits has proven that if we focus on achievable habits - like getting just 1% better, we’ll be more likely to achieve our goals. If I want to improve my running pace (which I always do), and work towards a faster marathon time, I will need to build a habit of incorporating weekly interval training.
However, on any industry or community level, planning and achieving goals is much harder. It requires collective systems and buy in. In their bestselling book ‘How Big Things Get Done’ Brent Flyberg and Dan Gardiner analyse hundreds of large scale projects from infrastructure to technology, and attempt to uncover and explain the fate of most projects. Their analysis found that a staggering 91.5% of projects go over budget, over schedule, or both. And than 1% are delivered on time and budget with their original intended benefits. Less than 1%!
A staggering 91.5% of projects go over budget, over schedule, or both.
- Brent Flyberg
So why is it that we are so terrible at predicting what’s next?
1. Future is evolving exponentially not linearly
As humans, we have a strong bias towards our past experience and current knowledge when it comes to making future plans. We naturally see change as linear, predictable and incremental.
Legendary computer scientist and voice of technological change Ray Kurzweil explains in his book, The Singularity is Nearer, that throughout our evolutionary history we’ve lived in times of much slower change: “our intuition about the future is linear because that is the way the world worked for most of history. Prey animals did not get exponentially faster”
“The future is widely misunderstood. Our forebears expected it to be pretty much like their present, which had been pretty much like their past.”
– Ray Kurzweil
However, the future is changing exponentially rather than linearly. That’s difficult for us to wrap our human brains around, and equally makes it difficult for us to predict what will happen, and when. It also means that we’re often met with surprise by the reality of what unfolds.
2. The Estimation Gap
Because exponential change is hard to see in the present, it’s hard to estimate future impacts from our current standpoint. This perhaps best summed up in Bill Gates’ famous quote that “Most people overestimate what they can do in one year and underestimate what they can do in ten years.” Like habits, we can take stock of the technology around us and apply it on a personal level, but fail to recognise its industry-changing impact.
“Most people overestimate what they can do in one year and underestimate what they can do in ten years.”
― Bill Gates
A great example of this is Horse Manure Crisis of New York City. In the late 1800s, population growth in New York City brought with it an unprecedented influx of horse-drawn transport. The volume of horse manure in the city streets soon reached crisis levels. It was predicted that, without intervention, the manure would pile up as high as first floor buildings of the city within a decade.
Of course this never happened, because by the 1920s the motorcar had begun to replace horse drawn transport, and the rest is history. Motor vehicles are another example of our failure to plan, but that’s another story.
3. Rigid Systems vs Flexible Systems
If we accept that we live in times of rapid change, then we need a different OS. Many of our organisational systems are designed to maximise efficiency and reduce risk by keeping everything exactly the same. Repetition and reducing complexity is good, but more often than not we end up with protocols that lack the flexibility required to adopt and embrace the kinds of technology and thinking which would ultimately improve efficiency.
Risk mitigation is critical. We need to make sure the bridge doesn’t fall down, or that we specify the right strength of railway steel. But if we try to anticipate every possible risk ahead of time and address it, the project can never move forward.
Equally if we bring that kind of mindset to organisational culture, we’re likely to miss the boat when it comes to adopting new technology and enabling teams to thrive through complex change.
Flyberg and Gardiner suggest breaking the project down into smaller chunks - applying agile thinking even to infrastructure projects like California’s High Speed Rail.
What’s the alternative?
If the future is so difficult to forecast or control, and we’re terrible at planning accurately, then what can we do? Planning still matters, a lot: failing to plan is planning to fail.
However, instead of planning for a linear, singular, predictable future that our human minds so want to believe, we need to look instead at the trends, signals and complexity that points to a future that may be different from the present.
But if goals are “good for planning your progress and systems are good for actually making progress”, as James Clear says, then we need different systems for understanding complex futures. This means:
Forecasting multiple scenarios: the good, bad, likely and even unlikely ones.
Gathering data - to identify signals that help us understand what this could be telling us about a different future scenarios
Testing assumptions through experiments and feedback. Something that tech companies do well but property, infrastructure, construction and other sectors I work with often struggle with.
Breaking things down into smaller goals and delivering outcomes. It means embracing an entrepreneurial mindsets that allow us to ask ‘what if’.
✨ Coming Up: 2025 Future Citizen Report ✨
Later this month I’ll be sharing the 2025 Future Citizen Report, a sneak peak at market trends and demographic signals to keep across this year, and strategic bets for changes we’ll see across people, property, housing and business and lifestyle. Subscribe to receive updates.