The US Air Force’s Expeditionary Combat Support System (ECSS)

The initiative

The Air Force launched the Expeditionary Combat Support System (ECSS) programme in 2004, with the goal of creating a single, unified logistics and supply-chain management ERP system. It was intended to enable the organisation to track all of its physical assets from airplanes to fuel to spare parts. “The scope of the ECSS system included: advanced planning and scheduling; material management, contracting and logistics finance; configuration and bill of material; repair and maintenance; product lifecycle management; customer relationship management; order management; distribution and transportation; decision support; facilities management; quality control; document management and budgeting.”[3]

The ECSS programme was established through two separate contracts. The first, a contract with Oracle Corporation, was to supply the COTS [commercial off-the-shelf] software. The second, with Computer Sciences Corporation (CSC), was to integrate the COTS software into the existing Air Force infrastructure.[4]

Original estimates indicated that the project would take eight years to reach full deployment and would cost USD3 billion. Work was to be started in 2004 and completed by 2012, but due to contracting disputes with the various bidders, work did not begin until 2007. The project team grew quickly and at one point reached more than 1,000 team members, claiming at the time to be the world's largest ERP project. By 2010, signs of major problems had surfaced, and between 2010 and early 2012 the project went through at least three project "resets".[5]

The challenge

The US Air Force is one of the services within the Department of the Air Force, one of the three military departments of the Department of Defense (DoD). It has one of the largest and most complex supply chains in the world. Like commercial supply chains, it includes delivery, transport, maintenance, repair, procurement, inventory management and product lifecycle management of both inexpensive consumable items and very expensive equipment with long lifecycles.

To manage such logistics, the Air Force used a complex web of IT systems. These systems were built over many years, and formed a patchwork of components rather than a clearly thought through and structured IT architecture. By 2000, overlapping functions and disconnected databases meant that the Air Force was struggling to achieve the desired operational capabilities, efficiencies, and financial transparency. "The Air Force IT environment includes over 700 systems. Many are duplicative, standalone and ineffective. There is also a multitude of metrics with competing goals. Non-standardised reporting exists, causing credibility issues and time inefficiencies. In addition, there is limited visibility across the supply chain. No one knows what parts are available at different sites and personnel can't plan for maintenance.”[1]

In 2001, the Air Force initiated the Global Combat Support System-Air Force (GCSSAF) programme as a foundation for modernising the way it did business. The goal of the GCSS-AF plan was to consolidate these independent legacy systems into a more centralised, cohesive enterprise resource planning (ERP) system.[2]

The public impact

The project did not produce any of its intended impact, as it was cancelled after the expenditure of billions of (taxpayer) dollars:

• In 2012, the US Secretary of Defense cancelled ECSS after the Air Force had spent “over USD1 billion of taxpayer funds on the programme without it fielding any usable capability. In fact, at the time of the cancellation, ECSS would have cost an additional USD1 billion to yield only 25 percent of the capability the Air Force originally sought."[6]
• As of 2014, when a review report was launched by a committee on investigations, the Air Force was still unable to confirm how many legacy systems would have been phased out by implementing ECSS.
• After the DoD cancelled the ECSS programme, Air Force personnel reverted to using the legacy systems that the ECSS programme was supposed to replace and they continue to use these outdated systems.
• “The ECSS's failure resulted in a waste of USD1.1 billion, eight years of lost time, and the same insufficient legacy logistics system remained in place.”[7]
• A separate report from the Institute for Defense Analysis quotes the Government Accountability Office (GAO) October 2010 report of the estimated cost of the ECSS as having increased from USD3.0 billion in 2008 to USD5.2 billion. This was a much larger figure than reported in other sources.[8]

Stakeholder engagement

The programme was led by the DoD, through the Air Force, with Oracle and CSC as the main systems developers. The other main stakeholders were the ECSS users.

The design of the system was conducted after a competitive tendering process, where the DoD provided the guidelines and private contractors submitted their proposals. "The high level capabilities are documented by the DoD without any bias towards any ERP package. The end goal can be met if there are metrics to evaluate the improvements desired in the processes. The DoD followed a two-step acquisition process: a) COTS Package selection and b) Selection of contractor. The first step is the selection of COTS package after a list of desired capabilities is identified."[9]

The Air Force and CSC conducted surveys of the approximately 250,000 prospective users of ECSS, and they indicated a lack of user engagement. Responses suggested that training plans were flawed, and the respondents were also critical about the proposed software, for example:

• “Communication about ECSS is usually a bunch of generalisations, and fairly condescending without enough specifics to give anyone a warm fuzzy about the ECSS team. I haven't heard anything from the ECSS team that tells me what the real issues are."
• “Have heard ECSS will change policies and processes, but aren't really seeing many actual examples of what is changing.”[10]

An April 2011 survey found that only 37.7 percent of participating ECSS endusers felt informed about how they would use ECSS to do their jobs more effectively.

Political commitment

It is a good example of political commitment because the DoD saw a need for an ERP system in the Air Force and worked on the framework of the project in 2003 and 2004. The DoD spent around USD1 billion towards the project. Even after the ECSS strategy failed, the DoD maintained their efforts to implement their Defense Enterprise Accounting and Management System (DEAMS) to make the project successful.

The government had several practical reasons to push for the update of the system. Apart from modernising the Air Force's global supply chain, ECSS was also intended to:

• Help provide the core financial information required to meet a congressional mandate for auditable set of books by 2017.[11]
• Streamline processes and bring billions of dollars in savings.[12]
• Transform the way logistics was done in the Air Force, With ECSS as an enabling tool.[13]

Although ECSS was eventually cancelled, DEAMS continued to be an active Air Force programme. As of September 2013, the Air Force had received about USD427 million for developing DEAMS, with DoD approval for approximately USD1.6 billion more.[14]

Public confidence

Clarity of objectives

ECSS had broad operational objectives that were identified at the outset. However, for such a complex technology project these were neither clearly defined nor measurable, so there was little clarity on what the outcomes should be and when they would be delivered.

The system was intended to provide the Air Force with a single, integrated logistics system, including transportation, supply, maintenance and repair, engineering and acquisition, and with the financial management and accounting functions for its working capital fund operations.[15]

Although some challenges were identified initially, there was neither a stable set of system requirements nor a consistent approach to risk management. “The Air Force initially identified a number of risks associated with the ECSS programme, including lack of cultural acceptance of new business processes by Air Force personnel, as well as undefined programme requirements, which meant that the Air Force did not establish a stable set of objectives throughout ECSS's lifecycle. The Air Force did not effectively address those risks. Ultimately, many of the risks identified at the programme's inception came to fruition and ultimately contributed to ECSS's failure."[16]

Strength of evidence

Although there is mention of previous failed ERP initiatives within the Air Force before ECSS, there is no evidence of any examples being used in the development of this programme. It did integrate, however, a pilot programme in 2010, which provided some insights into the challenges of the project.

The Air Force Operational Test and Evaluation Center (AFOTEC) conducted an Early Operational Assessment (EOA) during a contractor-led developmental test event of Release 1, Pilot A of the Expeditionary Combat Support System (ECSS) in 2010. In the first release of the assessment, the AFOTEC conducted three pilots:

• Pilot A - Foundational Configuration and Tools and Vehicles Management
• Pilot B - Equipment Management
• Pilot C - Base Supply Chain

The data provided by the pilot was, however, insufficient to determine if the programme was developing successfully. “Due to the limited scope of Pilot A (with less than one-tenth of the planned Release 1 capability), AFOTEC was not able to collect sufficient quantitative data to determine if the programme was on track to deliver desired performance at the conclusion of Release 1. However, interviews with functional SMEs and analysis of the limited data enabled testers to identify several areas requiring attention, including data quality, data conversion, handheld scanner needs, interoperability, usability, information assurance, and requirements testability.”[17] After the completion of the EOA, the programme office took action to address identified shortfalls and conducted further tests to mitigate the concerns that had been identified.

Feasibility

The ECSS implementation faced very basic structural challenges from the outset, as it failed to baseline existing practices or establish effective measures for delivering the software. Similarly, resources and timelines were inadequate, and staff were not provided with the necessary support to adopt the changes to working practice.

The project estimates in terms of funding and time proved inaccurate. "Originally estimates indicated that the project would take eight years to reach full deployment and would cost USD3 billion. Work was to be started in 2004 and was to be completed by 2012. Due to contracting disputes with the various bidders, work did not begin in earnest until 2007. The project team grew quickly and at one point with more than 1,000 team members ECSS claimed to be the world's largest ERP project."[18]

The leadership of the initiative identified cultural resistance to the changes the project would bring about, and carried out risk management analysis and surveys to gauge the situation. "With over 250,000 users potentially affected by ECSS, the Air Force was aware that a potential risk area included its personnel resisting the transition to a new system. Yet it was unable to develop an effective plan to overcome that resistance. In accordance with business process re-engineering guidelines, the Air Force requested that training plans be developed by CSC to teach leadership and endusers about the benefits of transitioning to ECSS and its expected improvements to long-term operations.”[19] Feedback indicated that the resources and training made available to Air Force staff were inadequate.

Management

The DoD was responsible for the high-level oversight of ECSS, but it was implemented by two private contractors, and managed within the Air Force.

The programme's deviation from the DoD's business process guidelines was one of the main reasons identified for its failure. “ECSS's failure resulted, in large measure, from the Air Force's systemic deviation from widely-endorsed organisational guidelines. Those guidelines, which comprise a set of management principles called business process reengineering (BPR), are mandated by several legislative and internal DoD directives and are designed to ensure a successful and seamless transition from old methods to new, more efficient ways of doing business.”[20]

There was also a lack of leadership and continuity in the management of the project. “The Air Force lacked strong, continuous leadership, as called for by BPR. During the eight years that the Air Force tried to implement ECSS, this programme had six programme managers and five programme executive officers, which led to communication gaps and a loss of institutional knowledge about ECSS's progression through the acquisition process. Additionally, according to CSC, the Air Force permitted contractor staff to make programme decisions, which could not always represent the Air Force's views in the decision-making process."[21]

Frequent changes in management also created gaps in knowledge about the history of the initiative, leaving incoming staff less prepared to deal with challenges efficiently. “Programme managers who made key management decisions - rightly or wrongly - were constantly transitioned out of the ECSS programme, leaving other key decisions or their consequences to new personnel with less familiarity with, and historical knowledge of, the ECSS programme."[22]

Measurement

There was very little monitoring and control during the implementation of ECSS. Only very basic aspects, such as cost and effort, were recorded, but they were not measured against any targets. In addition, the lack of clarity regarding the programme gave little visibility even on the number of legacy systems that were being retired.

Normally for DoD programmes, it is common to use contract performance reports and contractor cost data reports to track the cost of work. However, these were not used for ECSS. “The ECSS programme manager and some other ERP experts argued that for an ERP it does not make sense to track progress relative to a work breakdown structure, as these reports require. Although we do not have a position with regard to that statement, the fact remains that there is currently almost no cost reporting in DoD ERP acquisitions, and there probably will not be any cost reporting unless the regulations governing these acquisitions are overhauled."[23]

A report on the programme's execution was particularly critical of the failure to measure the number of legacy systems targeted for replacement. "When the Air Force began planning for ECSS, it did not even know how many legacy systems the new system would replace. The Air Force has, on different occasions, used wildly different estimates on the number of existing legacy programmes, ranging from '175 legacy systems' to 'hundreds of legacy systems' to 'over 900 legacy systems'. Two years after the termination of ECSS, and after two major investigations, the Air Force was still unable to provide the exact number of legacy systems ECSS would have replaced."[24]

Alignment

There was weak alignment between the public and private sector organisations that were responsible for implementing ECSS, as well as between the developers and the staff who were the prospective endusers. The Air Force also caused redundancy and confusion by following two different programme governance schemes:

• The DoD Instruction (DODI) traditionally used for all acquisition programmes
• The Business Capabilities Lifecycle (BCL), a new structure that had been designed specifically for defence business system acquisition efforts.

Some of the main reasons identified for the lack of alignment between the developers and endusers included:

• “The hierarchical decision-making structures in the military were poorly aligned with the governance structure in use by the project (functional sponsors were at times lower in rank than the people whose groups were impacted by the changes).
• “Challenges in aligning and integrating functions across organisational boundaries and a failure to put in place a governance structure at a senior enough level to overcome those boundaries.
• “Lack of trust between groups.”[25]
• The Air Force and CSC failed to communicate the long-term objectives and benefits of the new programme clearly to the Air Force endusers.

This lack of communication, combined with a poorly-implemented training regimen, exacerbated the cultural resistance to changing business processes among Air Force personnel who were more familiar and comfortable with older legacy systems.[26]

Even after the programme was terminated, the organisations partnering with the Air Force in the ECSS programme - CSC and Oracle - held very different views about what (if any) capability the Air Force had gained from ECSS. Oracle contended that instead of serving as the foundation for the ECSS system, its commercial software was reconfigured for use with the existing legacy systems after the programme was terminated; CSC argued that it provided a number of capabilities the Air Force could use to build new ERP systems in the future.[27]

Bibliography

Air Force Programs: Expeditionary Combat Support System (ECSS), GlobalSecurity.org

DOD Business Transformation: Improved Management Oversight of Business System Modernization Efforts Needed, October 2010, United States Government Accountability Office

Expeditionary Combat Support System: Root Cause Analysis, October 2011, Institute for Defence Analyses

How to Avoid an ERP Implementation Failure, 22 August 2014, Envista

Modeling Government ERP Acquisition Methods Using System Dynamics - Working Paper, Farrah Tazyeen, June 2012, Massachusetts Institute of Technology

Senate Condemns US Air Force ECSS Program Management's Incompetence, Robert N. Charette, 21 July 2014, IEEE Spectrum

The Air Force's Expeditionary Combat Support System (ECSS): A Cautionary Tale on the Need for Business - Process Reengineering and Complying with Acquisition Best Practices, Staff Report, 7July 2014, Permanent Subcommittee on Investigations, United States Senate

US Department of Defense - U.S. Air Force “Catalogue of Catastrophe”, February 2011, International Project Leadership Academy