Change Management and Complexity Theory

Dean Styles
(dean.styles@gmail.com)
Sooty Solutions
Burnaby BC
http://www.sooty.ca

Abstract. Managing change is a difficult process in any organization. Change management involves controlling the three R's of change: Reward, Risk and Rate.  Change always brings with it unknown risks and uncertain benefits.  Change control systems can be classified as optimistic (reward focused), risk-adverse or rate-adverse as they attempt to find a comfort zone that balances the three R’s.   Without an honest review of the organization’s historical record at implementing change and in the absence of good analysis of each of the three R’s most organizations will be managing change at a rate that is not optimal for their situation.  In many cases the approach used for the next change is determined by the success or failure of the last change and a consistent balance is never attained. 

The Inevitability of Change

Change is a necessary outcome of a competitive environment.  Things that appear unchanging are merely at a point where there is a balance between growth and decline.  When change is aligned to our goals we say change is good, when it frustrates our goals we say change is bad, and if we are unwilling to control change we are simply participating in a change lottery.  If we let fate decide we are like the Lucky Man; “Sometimes I have good lucky and sometimes I have bad lucky but I am always a very lucky man”.

We can attempt to eliminate all the risks associated with change by slowing the rate of change.  Analyze every outcome, develop contingencies for every possible failure, plan the change down to the minutest detail, and drive the change process as if it were a heavy nail going into hardwood.  This approach ignores the fact that change occurs in a competitive environment.  While we strive to obtain absolute control in managing our changes the external environment is activating changes that are not aligned to our goals.  If the external changes are proceeding faster than our change process whatever we do accomplish will be quickly overtaken and rendered obsolete.

We can attempt to focus only on the rewards of change cutting our loses as soon as things start to turn bad.   Or we could use any number of gambler’s strategies.  For example “doubling up” translates to “we’re terrible at managing small changes so lets do everything as one big change”.   Increasing the rate of change to cover up previous change management failures is a recipe for disaster. 

The bottom line is that change is inevitable, change has little respect for our goals, and we are in a race to implement our changes before the outside world makes our systems irrelevant. Masters of change do not see themselves like a chess master moving pieces with predictable rules across a well-defined arena.  Instead they see themselves as a surfer skimming across surface of a big wave always in control of their surf board and always with a great deal of respect for the destructive power of process they have decided to engage. 

The Rewards of Change

The rewarding changes fall into two general categories “do something better” or “do something new”.  Doing something better is deceptively simple; if it takes ten hours to build a widget and you change the process so it only takes five hours the change will reward you with lower costs and increased capacity.  The deceptive part is in measuring whether there was actually a reward.   If you sacrifice quality in making your process faster, or burn out your staff, or run afoul of some environmental regulation your change may provide no gains at all.  For any change that promises improvement you must measure the situation before and after and must include all factors including the effects on both suppliers and customers.  Most important the measurement must be end-to-end otherwise you may just be shifting effort and improving subsystem performance at the expense of the overall system.

If you do something new measuring the rewards are simpler because the before condition is a zero state.  Moreover doing something new has a lot of intangible benefits; it may spark innovation in related areas, it may create products in services that people never knew the needed (and now cannot be without), or it may just break the ice and encourage people to accept change with less resistance in the future.  As with improvement type changes it is critical to quantify the rewards.  You must ask “are we better after the change than before” and answer the question with a measurement in dollars.

Many changes are promoted with an inflated promise of rewards.  Often this is not malicious or a conscious attempt to deceive.  Those proposing the change are naturally optimistic about the outcome.  They see resistance to their ideas as simply a resistance to change that can be overcome by selling harder.  Those resisting change will always minimize the reward and exaggerate the risks.  The negotiation process continues until the agents of change and those affected by the change find a balance that yields a credible understanding of the rewards and provides an acceptable level of risk.  

The Risks of Change

“If it ain’t broke don’t fix it” is advice handed down from generations of people who have experienced failed changes.   The risks of change are often difficult to assess or even foresee.  It is a psychological principle that a condition of uncertainly is felt as fear.  Fearful people are extremely resistant to change even if it is clear to them on an intellectual level that the change will improve their situation.  Fear of fire traps the victim in the burning building.  Fear of losing their job traps the employee in a dead end position.  The first law of risk management is that well informed staff are more willing to accept change but it is critical that the information leads to a reduction in uncertainty.

Pilots will tell you that every box on their checklist represents a crashed plane.  The risks of airline flight are perceived to be so great that the law mandates a rigid risk management process.  It is well known that per mile of travel more people die in automobile crashes than airplanes yet there is no checklist for you to when you make that trip to the grocery store.  In fact you are sixty times more likely to die on a motorcycle than in a car.  At least there are laws about seatbelts, energy absorbing bumpers, and airbags for cars.  On a motorcycle you are naked to the perils of the road.  The second law of risk management is that perceived risk determines behaviour more than actual risk.  The corollary is that risks are perceived as lower when they are under an individual’s control.

In complex systems processes are interconnected in ways that provide benefits and involve dependencies that can be hidden and mysterious.  A changed process may not provide the support or capitalize on the interconnections and therefore lose out on the resulting benefits.  Ecologists know about interconnected systems especially the classic case where spraying DDT on mosquitoes actually killed off all the cats, which lead to an explosion in rat population, and the fleas on the rats lead to an outbreak of bubonic plague.  An integrated supply chain with just in time deliveries is an environment that ecologists understand better than business analysts.  The third law of risk management is that interconnected systems exhibit exponentially larger risks than isolated systems.

The label on the bottle of cleaner insists you try on a small area to determine if the fabric is colorfast.  Pilot projects are a method to uncover hidden risks but only if the results of the pilot can be used to shut down the project.  If the pilot is designed to be stage one of a planned rollout the focus on the plan will overwhelm all else and the lessons from the pilot will be ignored.  The disciple of prototyping and pilots is that you must be prepared to assess the results objectively and beyond the reach of the enthusiasm for the plan.   The fourth law of risk management is that a pilot is a success if you learned something, it is even a bigger success if it avoided proceeding with a disastrous change.  
  
The Challenger disaster was probably caused by small voids introduced into the insulating foam during application.  Expanding gases in the void caused a small chunk of foam to fall off damaging a few tiles on the shuttle wing.  On reentry hot gases cut through the damaged tiles like a torch and destroyed the wing.  There are lots of examples of small changes that result in an amplifying cascade of events resulting in a disaster.  The fifth law of risk management is that there are no small changes.

Later it will be shown that the cascade of small changes into big problems is more likely to occur as a system moves past its optimum complexity and into a chaotic state.  In general it is impossible to change manage every small change and there will always be surprises.  Risk management can only rely on change management to provide an ordered change process.  To properly manage risk you also need a monitoring process that looks for cascading problems and can quickly take control of the change control system to quench the cascade.  The six law of risk management is that change control is only part of the solution.

The Rate of Change

There is an old saying “in chaos there is opportunity” and this is true for the lucky few.  Dropping your ice cream in the sand provides a wonderful opportunity for the ants but it is unlikely to be aligned to your goals.  You can start a revolution to end a repressive government and you may become a general in the new dictatorship.  On the other hand if you honestly assess the situation your outcome is more likely to be “first against the wall”.   Chaos chooses its beneficiaries at random and often it is the survivors of chaos that will be writing the glorious history.  Most of the participants will be victims.

Chaos is simply a situation where changes that result in destruction exceed changes that result in creation.  Chaos is marvelously exciting and if you ignore the destructive forces it can be described as a hugely creative time.  Many of our great books and art arise from chaotic times but we are often unaware of what was lost.  As the rate of change increases the destructive forces begin to dominate and productive efforts are quickly consumed in the turmoil.

There are those that recall a golden time when everything was perfect and wish such a time would last forever.  On closer examination such golden times always had a few things that needed to be fixed and even those who long for such times would want to tinker with it.  Museums are full of things that have been stopped in time.  In fact museums expend a great deal of effort in their attempts to stop time.  When there is no competition and any change is avoided because it is destructive the system enters a state of stagnation.  This is a very effective strategy for museums and it may also be effective for some business systems.  

Most business processes fall somewhere between chaos and stagnation.  In fact complexity theory provides a way of looking at what results we can reasonably expect from a process as we increase the rate of change from stagnation to chaos.

This is the shape of a shark’s tooth or a sand dune found in an excellent book on complexity by Mitchell Waldrop[2].  When there is no change there are no results.  Similarly when the rate of change is so high the forces of destruction completely dominate the forces of creation there are also no productive results.  Between stagnation and chaos there is an optimum point of complexity where the balance between creation and destruction maximizes the adaptive and productive capabilities of the system.

The sand dune shape has other things to tell us.  It takes a lot of change to get a stagnant system moving so it can be productive.  Moreover the benefits appear slowly and a great deal of risk must be suffered before increasing the rate of change starts to pay off.  If you are presented with a stagnant system that is becoming obsolete the risk of rapid evolutionary change (lots of new features in the next version) is almost as great as that of revolutionary change (replace the system completely).

At the optimum point a small increase in the rate of change can result in a precipitous drop into chaos.  Change cascades are commonly seen at this point.  A small change does not have time to diffuse its effects through the system before it is disrupted.  The disruption creates its own change cascade and the whole process rapidly becomes unmanageable.  Past the optimum point traditional prescriptive change management processes cannot keep up with the rate of change.  The system must be continuously monitored to watch for change cascades.
The chaotic region is very creative but not very productive.  To maximize the benefits groups operating past the optimum point must be in continuous contact with groups operating in more stable environments.  A good way to do this is have project leaders build core teams in the chaotic region then bring these teams back along the complexity curve as they move from research, to prototyping, to development, implementation and operation.  

Stagnant systems are also essential to business operations.  We keep seven year old financial records in a safe unchanging place just in case the tax department comes calling.  We keep medical records well beyond the death of the individual for similar reasons.  The overriding focus of archived information is data integrity.  Any change to historical records must overcome extensive reviews and large administrative barriers that are the key method of protecting this data.  Even data errors need to be protected from change.  Decisions made on the basis of these errors would be perplexing if the errors were repaired. 

Businesses will typically have systems that are operating over the range of the rate/complexity curve.  The problem is to tailor change management so it is appropriate to the environment.

Legacy Systems

From their location it is clear that Legacy system and archived data need very rigid change control procedures.  The first and only answer to many change requests is no.  Attempting to repair a system in this region will typically find no one willing to provide support.  Others will deny knowledge of the system just to avoid becoming trapped in a dead end job that will probably disappear when the legacy system is finally removed from service.

The most common problem with legacy systems is that many of these systems are never declared as legacy.  Critical support staff, those who keep the legacy system operating, are told the system will be kept operational forever.  The smart ones will escape the sinking system creating a huge dependency on those who remain.   Once the systems are final shut down the legacy support staff are often unfamiliar with current systems and their greatest fear is confirmed as they are forced to exit the company.

The best approach is to make an early declaration of legacy status for systems that are reaching the end of their life cycle.  The declaration brings with it a very bureaucratic change control process that slows the rate of change into stagnation.  This minimizes risks, cuts costs, preserves the integrity of the system, and provides stability for all the interconnections to other systems.

The legacy declaration will have a major impact on support staff.  Support staff often have knowledge of critical business processes and system connectivity that is difficult to recover if the staff leave the system.  The path for most of the support staff will involve moving them incrementally up the complexity curve to the next operational system.  They will quickly forget the legacy system and switch loyalties to their new system.

The few that are left behind with the legacy system must remain loyal to the maintenance of the legacy system or it will fail.  The natural home for these people is in research.  By placing these staff in research they will have no other operational system to capture their loyalty, their research activities are interruptible if a failure occurs on the legacy system, and they are rewarded for taking care of a dead system by training and exposure to leading edge technologies.  Of course if they prove themselves to be unable to learn the new technologies or adapt to the research environment you always have the option of laying them off when the legacy system is shut down.

Operational Business Systems

As we move up the complexity curve we get to systems that are main support of business operations.  These systems must be stable because without them the business will shut down.  They must also change at the same rate as the business.  Change management is a balancing act and these systems must be closely linked to the challenges and completive threats encountered by the business.   If the business changes the systems must change in lock step.

Change management policy in this region is heavily affected by the nature of the business.  If you sell paper products your business environment is relatively stable.  The lined three hole punched paper you are selling today is practically identical to that you sold thirty years ago.   Your rate of change will be determined by computer technology not the business environment.  If you manufacture cars your business environment is nearly chaotic.  New competitors, new suppliers, new markets, and new marketing strategies are appearing in rapid succession.  In this case your computer systems must be very nimble and your change control process must not interfere with getting the right system in place to meet the business challenges.
Software Development

The goal of software development is to build something new.  A secondary goal is to ensure the tools you use to build a new business system are absolutely current when you commission the system.   A two year development process starting with a two year old database product may find it can no longer get support for the four year old database software when the system is made operational.  Moving to the latest tools perhaps even beta versions of those tools is a fact of life in software development.  Any change control system must recognize this source of change and adapt to the disruptions and chaos it brings to the software development process.

Software development must also keep up with changes in the business environment.  Once again the rate of change in the business will determine the impact of this source.  For our paper supplier the impact will be minimal.  For our car manufacturer there is a serious risk that the software development team will deliver a new business system that is designed to operate in a business environment that no longer exists.

Software development must encourage creativity.  Building something new is a very creative activity.  If the change control process provides no outlet for creative expression the resulting systems will be less effective than they could be.  On the other hand if the change control process caters to creative individuals the result will be chaos.

Software development should operate at the sweet spot on the complexity curve to get the most out of the time and resources it consumes.   Change control managers need to develop a sixth sense of where their team is on the curve.  In slow moving business environment the change manager can allow more staff creativity and work with tools closer to the leading edge.  If the team is mostly junior and the business environment is chaotic then stricter more bureaucratic change control processes must be used.  If your team is technically senior and in tune with the business needs your change control procedures can very informal.  Change control manager must be most flexible in this region.

Research and Testing

Surprisingly there is a real need for change control in this region but its focus moves from controlling change to communicating changes.  A testing lab is a chaotic place where the rate of destruction is expected to approach the rate of production but that does not mean we cannot snatch the productive bits from the environment before chaos destroys them.

Even here chaos must proceed in an orderly fashion.  Lab systems must be allocated to development teams, equipment moves must be coordinated, teams must keep the lab clean of clutter, and licensing conditions of products must be maintained even as hardware platforms are wiped clean and reloaded.   Especially critical is a strict adherence to strategy of backing up and keeping safe all important test results and copies of in-house developed software tools.   Control policies are means used to keep the chaos contained and backup polices are the means used to snatch the results of creative effort out of the chaos.

The change control process is expected to fail on a regular basis in this region it is after all accepted as a region of chaos.  The goal of change control is to protect as much of the creative work as possible without getting in the way of the creative energy.  Focusing on change control failures will be the quickest way to cripple the creative engine.

Summary

The change manager must have respect for the force of change to be effective.  They must see themselves as a surfer skimming across the surface of a big wave in complete control of their surf board but always aware of the destructive force of the wave.  They must understand that productivity and results have a non-linear response to the rate of change.  Change control managers need to develop a sixth sense of where their team is on the curve and understand how to set a change rate that is appropriate for the team. 

The change manager must be flexible in their approach.  When the rate of change exceeds the ability of the system or the team to adapt the change manager must become bureaucratic and unbending.  When the team is not living up to its creative potential the change manager must allow an outlet for creative energy.  As far as possible the change manager needs to put the responsibility for managing change into the hands of individual team members by rewarding those who behave with increased freedom and punishing those who abuse their freedom with increased supervision.

Steps to evaluate your response to change:
1.	what are the expected rewards
2.	what are the expected risks based on a reasonable assessment of the risk
3.	what is the worst case scenario and can you survive it if you hit bottom
4.	where are you on the rate/complexity curve and where should you be
5.	what can you do to minimize risk as you move toward the optimum complexity point

The complexity curve gives an expected benefit for a given rate of change.  Reward versus effort analysis tells you if the change will be valuable.   Assessing risk as if you had to buy insurance to cover the risk will give you comparable risk cost.  Finally balancing the reward, risk and rate components should maximize the effectiveness of your change management process.

[2] Waldrop, M. Mitchell, Complexity: the Emerging Science at the Edge of Order and Chaos.  New York : Simon & Schuster, 1992.