healthcare gov project management case study

Technology and Operations Management

Mba student perspectives.

• Assignments
• Assignment: Digitization Challenge

The Failed Launch Of www.HealthCare.gov

The US Government's failed launch of the Healthcare.gov website highlights issues with integrating technology into a large bureaucratic organization.

“I’m going to try and download every movie ever made, and you’re going to try to sign up for Obamacare, and we’ll see which happens first” – Jon Stewart challenging Kathleen Sebelius (former Secretary of Health and Human Services) to a race.

www.HealthCare.gov Overview

The affordable care act (increasing healthcare access to all Americans), signed by President Obama on 23 March 2010, required that the United States Department of Health and Human Services (HHS) launch HealthCare.gov on 1 October 2013.  This website going to be the official health care exchange that would allow residents to compare prices of health care plans, identify if they qualify for federal subsidies, and enroll in a chosen plan.  The Affordable Care Act gave states the right to create their own healthcare exchange or opt-in to the federal exchange (healthcare.gov).

Website Rollout

Healthcare.gov was officially launched on 1 October 2013 covering residents of 36 states that did not create and manage their own healthcare exchange.  Problems with the website were apparent immediately.  High website demand (250,000 users [5 times more than expected]) caused the website to go down within 2 hours of launch.  While website capacity was initially cited as the main issue, additional problems arose mainly due to the website design not being complete.  Users cited issues such as drop down menus not being complete and insurance companies cited issues with user data not being correct or complete when it reached them.

Pathways to Just Digital Future

In addition, the websites login feature (which is the first step to accessing the website) could handle even less traffic than the main website which created a huge bottleneck.  Due to poor planning, this same log in method was also used by website technicians, making it extremely difficult for them to log in and troubleshoot problems.

A total of 6 users completed and submitted their applications and selected a health insurance plan on the first day.

Through a large amount of troubleshooting, bringing in new contractors, and increased management, the website could handle 35,000 concurrent users at a time by December 1 and a total of 1.2 million customers signed up for a healthcare plan by 28 December, when the open enrollment period officially ended.

Root Causes

The myriad of problems experienced during the websites rollout were due to several factors, primarily:

• Lack of relevant experience. HHS employees and managers had a lot of experience with private insurance markets and maintaining large government projects, but did not have required experience in technology product launches.  Key technical positions were unfilled and project managers had little knowledge on the amount of work required and typical product development processes leaving very little time to test and troubleshoot the website.
• Lack of leadership. There was no formal division of responsibilities in place between the many government offices involved which caused a delay in key decision making or a lack of communication when key decisions were made.  For example, the contractor responsible for the log on system estimated a low demand because the initial website plan included the option to shop for products without creating an account or logging in.  However, due to technical delays, that functionality was removed from the initial website launch (so all users would need to log in) without increasing capacity.
• Schedule pressure. Since the launch date was mandated in the Affordable Care Act, HHS employees were pressured to launch on time regardless of completion or the amount (and results) of testing and troubleshooting performed.

Key Takeaways

The key issues discussed above resulted in the rollout of the healthcare.gov website ballooning the initial $93.7M budget to an ultimate cost of $1.7B.

It is easy to observe that the launch of healthcare.gov was a major failure, but this isn’t a particularly unique case.  Research has shown that over the past 10 years, 94% of large federal information technology projects were unsuccessful, more than 50% were delayed, overbudget, or didn’t meet expectations and a total of 41.4% were judged to be complete failures.  I contend that most of the root causes identified for healthcare.gov are not unique and it is not a surprise the federal government struggles with adapting to new technology.  A large, bureaucratic organization that has significant experience in core government policies is likely not adaptable to behaving like a “start-up” and successfully launching new technology.

Based on the number of government organizations that all have policy experience and will likely have to adapt to technology challenges in the future, the government should either create a new government office strictly responsible for handling technology products for the rest of the government or make a significant investment to ensure that all government offices have the experience, management tools, and resources required to integrate technology into their offices.

(782 words)

1 – Office of Inspector General, HealthCare.gov: CMS management of the federal marketplace: a case study, Available at: https://oig.hhs.gov/oei/reports/oei-06-14-00350.pdf [Accessed November 18, 2016].

2 – Goldstein, A., 2016. HHS failed to heed many warnings that HealthCare.gov was in trouble. Washington Post. Available at: https://www.washingtonpost.com/national/health-science/hhs-failed-to-heed-many-warnings-that-healthcaregov-was-in-trouble/2016/02/22/dd344e7c-d67e-11e5-9823-02b905009f99_story.html [Accessed November 18, 2016].

3 – Healthcare.gov. Wikipedia. Available at: https://en.wikipedia.org/wiki/Healthcare.gov [Accessed November 18, 2016].

4 – Johnson, C. & Reed, H., 2013. Why the Government Never Gets Tech Right. The New York Times. Available at: http://www.nytimes.com/2013/10/25/opinion/getting-to-the-bottom-of-healthcaregovs-flop.html [Accessed November 18, 2016].

Student comments on The Failed Launch Of www.HealthCare.gov

As you nicely pointed out, the launch of the national healthcare exchange website was a disaster. It really raises key questions as to how large projects within the civil service are run, and one must consider how the government can leverage the nation’s technological expertise in order to push forward, not backwards its agenda. For instance, could the government have worked with a Silicon Valley startup in order to develop a solution which was much more nimble at a fraction of the cost? Strong leadership and project management skills are undoubtedly key. Nonetheless, it is great that the exchanges ultimately worked. Whilst one can argue with certain points of implementation, the Affordable Care, by insuring 15 million previously uninsured people, undoubtedly had a positive impact on social justice in the USA. The recent election threatens this triumph, however one hopes that expanding healthcare accessing will remain a key agenda of the new administration.

“Based on the number of government organizations that all have policy experience and will likely have to adapt to technology challenges in the future, the government should either create a new government office strictly responsible for handling technology products for the rest of the government”

The USG did create a government office focused on digital services in the wake of the healthcare.gov debacle. The General Services Administration (GSA), a Federal agency, now houses 18F, a digital services agency build on the lean startup model. 18F recruits top talent via the Presidential Innovation Fellows program and acts as an in-house digital delivery team to help government agencies build effective, user-centric digital services. 18F runs on a cost recovery model where client agencies reimburse 18F for its work. See https://18f.gsa.gov/ for more information.

18F is well-suited to help agencies for projects on a smaller scale than healthcare.gov. I’d argue that for future high stakes, large scale roll outs, however, the U.S. government should contract out more of the work to experts in the private sector. The government currently does not have enough talent/expertise to effectively manage such projects from start to finish, and even if it did, internal bureaucratic roadblocks would likely stymie and frustrate that talent and contribute to project delays.

Great post — we don’t talk enough about the effects of digital transformation on the public sector. In the case of Healthcare.gov, this was a very visible event given the amount of attention and dialogue around the Affordable Care Act.

The first root cause you mention — lack of relevant experience — is spot on. It took a highly skilled team with a strong Silicon Valley pedigree to come in and address the many technical issues that occurred during the rollout. One reason is because there are only a handful of web properties that handle the same traffic volume and backend complexity that Healthcare.gov requires — and they are all based in the Bay Area. There was a great article about this in The Atlantic: https://www.google.com/amp/www.theatlantic.com/amp/article/397784/

That same team has gone on and founded Nava, which aims to solve many of the technical challenges facing the public sector. Along with Palantir, they represent companies with the relevant experience and nimbleness to attempt to fix such problems at a national scale. Perhaps the role of the Public-Private Partnership is key to the successful digital overhaul of the public sector: companies such as Nava leading the transformation, with the support and resources of the Federal Government.

Your point about better leadership is also key. The White House did not have a CTO until 2009, and did not have a CDO (Chief Digital Officer) until April 2015; and I speculate the latter appointment occurred in part due to the Healthcare.gov fiasco. Having the right leadership is not only important from an organizational and decision-making perspective, but also the position is very visible and with the right person in place, others from the technology sector may consider leading Silicon Valley to solve important problems in Washington DC.

I think the main reason a lot of these large IT projects fail is because organizations think that large project = large number of people working on project. In technology, you can have a small team launch to millions of people.

While the rollout of the healthcare.gov website was unquestionably a disaster, I can’t help but wonder what (if any) lessons were learned by the federal government following this situation. I find what was mentioned in the above comment interesting, that the government responded to this crisis by creating another office focused on digital services. While this department is meant to be built upon a lean startup model, I can’t help but be skeptical that the bureaucracy of the federal government will overwhelm any benefits that this group may be able to offer. Instead, I wonder if the government will ever look to outsource any future technology projects to the private sector/Silicon Valley. I believe these companies are better equipped to provide better technology at lower prices than any government office, which doesn’t have a core competency in developing and providing these services. Integrating digital technologies is vital as the government looks to lower costs and increase efficiency, and I hope that they will look to these sorts of key partnerships in the future so as not to repeat errors such as this.

Thanks for sharing such an insightful post on a relevant topic. I am surprised that any website made in 2010 have the problems you mentioned, even if made by the government. Every web developer should know that capacity planning and designing for key user journeys are among the most important aspects of creating a successful site. I am certain that Google or even a “random” startup in Silicon Valley could’ve done a better job in less time, especially given $1.7B for a single webservice.

As you mentioned. There is a big difference between private companies, for which every minute of downtime means lost revenue, and the government. People waiting online to sign up for the healthcare act does not, apart from bad reputation, have a direct impact on the people responsible for launching the site. Yet, these failures costs a lot for society, with all the manhours spent waiting online.

In your Key Takeaways, you also mentioned that the government should create a new office for technology products. Yet the separation or investment by itself does not guarantee better products. The Government faces the same challenge in attracting talent that a company like Google does. Good talent means good products, and what top engineering or design talent wants to work for the Government?

Again, very interesting post. Thanks for sharing!

Interesting post! I agree that a huge takeaway here is having the right resources and people in place for large technology projects, and that there could be great benefits from more public-private partnerships. Interestingly, Obama may have learned a bit from this situation and has been increasing his efforts to create connections between Silicon Valley and Washington ( http://www.nytimes.com/2016/10/30/magazine/barack-obama-brought-silicon-valley-to-washington-is-that-a-good-thing.html?_r=0 )

Great post, thanks for sharing! In conjunction with the initiatives you mentioned in setting up a technology office in government, I believe that the government should have a long-term plan to grow its technology office and decrease its reliability on technology consulting firms such as Accenture and other from Silicon Valley to handle crisis management once a mishap similar to this occurs. That is a typical example of how projects go over-budget and result in wasted taxpayer money.

Furthermore, as you mentioned a lack of leadership was clearly evident. For instance, the program could have been rolled out incrementally across the states, giving the technology team in the white house to trouble-shoot problems on a smaller scale.

Leave a comment Cancel reply

You must be logged in to post a comment.

Project Management Lessons From the HealthCare.gov Launch

By Yael Grushka-Cockayne

Newswise — When the Patient Protection and Affordable Care Act was passed in 2010, the government announced its vision for getting access to health care to millions of people. The goal of the HealthCare.gov project was to provide an easily accessible, web-based platform for U.S. residents to enroll in “Obamacare.” The site would allow people to compare different health care plans and ultimately enroll in one. The project was defined by the Secretary of Health and Human Services and was overseen by the Centers for Medicare and Medicaid Services, which would hire numerous outside contractors for the various supporting systems of the site. Over the course of the project, 60 contracts were awarded to 33 different companies.

Expectations and Risks

The initially declared budget of the project was $85.7 million. Early in 2012, the project deadline for launching the website was set to be 1 October 2013. This completion date was set with the goal of providing all U.S. residents six months to enroll without penalty. The official deadline was set to 31 March 2014.

As the project progressed, the timeline was updated and squeezed. Planning decisions aimed at saving time introduced significant risks to the scope of the project. A McKinsey report released in early 2013 highlighted some key risks the project was facing, in particular the insufficient time for testing and major concern surrounding the lack of a shared definition of success. In addition, there was an overall sense that further delays were expected, and that the exact requirements were still not properly defined.

Throughout the project, key milestones — such as the deadline for each state to declare partnership with the federal exchange and the establishment of regulatory guidelines — were postponed. The implications of such delays meant lack of clarity regarding the project scope.

Meanwhile, while timelines were updated and adjusted, deadlines surrounding the development and testing phase and the implementation phase of the project were not. The time allocated for these tasks was compressed from nine months to six months and from seven months to one month, respectively, essentially implying that if the project was to be complete by the 1 October target, there was little chance of achieving its full scope.

After years of planning and much political attention, millions tried to access the HealthCare.gov site the first week after its launch. The website suffered from thousands of defects and failed to handle the traffic in what nearly brought a quick end to the Affordable Care Act. And with a final cost of around $300 million [i] , the rollout of HealthCare.gov cost more than three times its original budget.

How could this have been avoided?

Project management frameworks can help shed light on the unfortunate launch of the site — and on its eventual fix.

In the planning and execution of projects, attention should be paid to the triple constraint principle of project management: Successful completion of a project depends on effective management of the connected traits of cost , time and scope .

Monitoring of the project progress and reflecting on the tradeoffs and priorities among the project’s objectives can offer project decision-makers improved capabilities. Given the clear warning signs, a project management framework suggests those leading the HealthCare.gov project should have either compromised on scope or time. A compromise on scope would have meant a staged rollout of the website, either by state or process phase. A compromise on time would have extended the deadline to a later date.

The site’s functionality was steadily improved post-launch, thanks to a dramatic change toward an agile approach to project management. Daily standup meetings and shorter development cycles, aimed at addressing bugs and gaining clearer understanding of the actual requirements, provided the developers and the project leaders the autonomy and motivation necessary for working toward a common goal.

Despite disappointing initial figures, when Health and Human Services released its May 2014 enrollment report, HealthCare.gov showed a significant uptick in enrollments, which was largely attributed to the site’s improved functionality and to increased awareness of the service. Between the site’s launch and the end of the open-enrollment period, approximately 8 million people selected a plan.

The above is based on the cases HealthCare.gov (A) (Darden Business Publishing), by Yael Grushka-Cockayne and Brian Ward (MBA ’15) and HealthCare.gov (B) (Darden Business Publishing), by Yael Grushka-Cockayne, Elizabeth Carnahan (MBA ’17) and Hyeongjin Kim (MBA ’17).

[i] “HealthCare.gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management,” United States Government Accountability Office Report to Congressional Requesters, July 2014, https://www.gao.gov/assets/670/665179.pdf .

About the Faculty

Yael Grushka-Cockayne

Grushka-Cockayne’s research and teaching activities focus on decision analysis, forecasting, project management and behavioral decision-making.

As an expert in the area of project management, she has served as a consultant to international firms in the aerospace and transportation industries. She is the secretary/treasurer of INFORMS Decision Analysis Society, a U.Va. Excellence... Learn More

The University of Virginia Darden School of Business delivers the world’s best business education experience to prepare entrepreneurial, global and responsible leaders through its MBA, Ph.D. and Executive Education programs. Darden’s top-ranked faculty is renowned for teaching excellence and advances practical business knowledge through research. Darden was established in 1955 at the University of Virginia, a top public university founded by Thomas Jefferson in 1819 in Charlottesville, Virginia.

MEDIA CONTACT

Type of article.

• Experts keyboard_arrow_right Expert Pitch Expert Query Expert Directory
• Journalists
• About keyboard_arrow_right Member Services Accessibility Statement Newswise Live Invoice Lookup Services for Journalists Archived Wires Participating Institutions Media Subscribers Sample Effectiveness Reports Terms of Service Privacy Policy Our Staff Contact Newswise
• Blog FAQ Help

Sunday, March 19, 2023

• Labels: Case Studies , Project Failure

Case Study 17: The Disastrous Launch of Healthcare.gov

Barack Obama was inaugurated on January 20, 2009, after defeating his opponent John McCain by 365 electoral college votes to 175. One of Obama's primary campaign issues was fixing America's healthcare system by providing affordable options to the 43.8 million uninsured Americans. 

In 2010, the year Obama signed the Affordable Care Act (ACA), the United States spent 17.6% of its GDP on health care, nearly double the OECD average of 9.5%, with the next closest developed nation, the Netherlands, spending 12%.

The 44th president was successful in introducing the ACA; however, the launch of the website that would connect Americans to the marketplace, Healthcare.gov, was a failure. While the platform would eventually enroll an estimated 10 million uninsured Americans in 2014, the rollout was a complete disaster that exposed the challenges the United States government faces in implementing technology.

According to a 2008 report by the Government Accountability Office (GAO), 48% of federal IT projects had to be restructured because of cost overages or changes in project goals. In addition, over half had to be restarted two or more times.

On the first day Heathcare.gov was launched, four million unique users visited the portal, but only six successfully registered. Over the next few days, the site experienced eight million visitors, but according to estimates, around 1% enrolled in a new healthcare plan. Even the users that did sign up experienced errors, including duplicates in enrollment applications submitted to insurers.

The trouble launching Healthcare.gov presents a seemingly reoccurring problem when the US government tech projects. Standish Group International Chairman Jim Johnson is on record praising the rollout based on the government's history of software failing by default. "Anyone who has written a line of code or built a system from the ground up cannot be surprised or even mildly concerned that Healthcare.gov did not work out of the gate. The real news would have been if it actually did work. The very fact that most of it did work at all is a success in itself."

However, there's far more to the failed launch of the federally facilitated marketplace (FFM). The agency responsible for the project, the Centers for Medicare and Medicaid Services (CMS), didn't follow many regulations in place to ensure transparency, proper oversight, and accountability. So was the project destined to fail from the start due to overwhelming layers of bureaucracy, or were the vendors tasked with developing the online marketplace to blame?

In this case study, we'll examine why Healthcare.gov failed to meet expectations and point out what CMS could have done differently to deliver a functioning FFM.

I f you are an executive sponsor, steering committee member, or a non-executive board member and want to learn what you need to do so that your project does not land on my list with project failures? Then my  (Non)-Executive Crash Course  is what you are looking for. If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for. If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here .

Timeline of Events

On March 23, 2010, President Barack Obama signed ACA, also known as Obamacare, into law. The legislation was the most comprehensive reform of the US medical system in 50 years and is still in place today.

Under the ACA, US citizens were required to have health insurance or pay a monthly fee. The law also required the establishment of online marketplaces that would allow individuals to compare and select health insurance policies by January 1, 2014. States could set up their own marketplace or use the FFM.

Each marketplace created under the ACA was intended to provide a seamless, single point of access for individuals to enroll in qualified healthcare plans and access income-based financial subsidies created under the new law.

Users were required to visit Healthcare.gov, register, verify their identity, determine their eligibility for subsidies, and enroll in a plan. The process appears straightforward; President Obama even touted the marketplace weeks before its launch by saying, "Now, this is real simple. It's a website where you can compare and purchase affordable health insurance plans, side by side, the same way you shop for a plane ticket on kayak… the same way you shop for a TV on Amazon."

However, building an identity verification platform on such a large scale alone is exceptionally challenging. The marketplace also required integration from databases in other government agencies. Once the user successfully was verified as an American citizen, income was determined, and they were filtered through state and federal government programs like Medicaid or the State Children's Health Insurance Program, then they would be matched with private health insurance plans.

The process was not simple and was far more complex than online shopping because it required integration with identification verification software, other government databases, and health insurance providers.

From day one, the project was underestimated. In addition, the requirements in the ACA that all citizens must enroll by January 1, 2014 or would be required to pay a fine created a hard deadline with economic and political consequences.

March 2010 - September 2011

Over a year passed between the ACA becoming a law and the CMS signing contracts with vendors who would build the FFM. During this period, problems directly affecting the launch were already beginning.

While the CMS was in charge of oversight of the project and hiring contractors, leadership was fractured across multiple government agencies. The project was headed by the CMS's Deputy CIO, Henry Chao, but the committee also included:

Todd Park – White House CTO

Jeanne Lambrew – Executive Office of Health Reform

Kathleen Sebelius and Bryan Sivak – Department of Health and Human Services

Members of the committee outside of the CMS executed a great deal of power and influence over the project; however, no one at the various agencies had visibility of the critical milestones that each group needed to reach to complete the project successfully.

The CMS awarded 60 contracts to 33 different vendors. The largest was granted to Conseillers en Gestion et Informatique (CGI), a Montreal-based IT company that employed more than 70,000 people. CGI grew to be worth more than $11 billion by 2013 by acquiring other companies, some of which handled US government contracts such as the 2004 purchase of American management Systems and the 2010 purchase of Stanley.

CGI’s contract consisted of developing the FFM and was valued at over $90 million. CGI was responsible for the most significant, user-facing aspect of the project but was not officially assigned a lead integrator role. CMS would later report they perceived CGI to be the project's lead integrator but didn't have written documentation outlining the agreement.

Representatives from CGI stated they did not have the same understanding at this point of the project.

US federal agencies are required to perform a procurement framework when awarding private companies with government contracts outlined by the Federal Acquisition Regulation (FAR). However, CMS failed to satisfy specific aspects of FAR, including:

> Preparing a written procurement strategy documenting the factors, assumptions, risks, and tradeoffs that would guide project decisions.

> Conduct thorough past performance reviews of potential contractors.

> Only used the federal government's contractor database (PPIRS) when evaluating bids on four of the six key contracts.

CMS leadership later claimed they were unaware that a procurement strategy was required. 

December 2011 – Summer 2013

Development of the Healthcare.gov project began in December 2011 consisting of four major components:

> Front-end website for the FFM (the marketplace)

>  Back-end data services hub

>  Enterprise identity management sub-system

>  Hosting infrastructure

CGI was responsible for the front-end website. The UI was developed with standard web tools, including Bootstrap, CSS, jQuery, Jekyll, Prose.io, and Ruby; however, it would later be revealed that common optimization features to aggregate and minify CSS and JS files were not used.

The back-end data services hub was developed by CGI and Quality Software Services, Inc (QSSI) using Java, JBoss, and the NoSQL MarkLogic database. The hub was responsible for orchestrating data and services from multiple external sources such as agent brokers, insurers, CMS, DHS, Experian, the IRS, state insurance exchanges, and the US Social Security Administration (SSA). While the integration was incredibly complex, utilizing data from multiple sources, the developers chose machine-generated middleware objects to save time.

Enterprise Identity Management (EIDM) was handled by QSSI but depended on the back-end to retrieve data from multiple sources. Before the launch, the EIDM was tested with an expected load of only 2,000 concurrent users.

The final major system component was the hardware infrastructure hosting of the website, FFM, data services hub, and EIDM. Akamai's CDN hosted Healthcare.gov's UI. The original back-end (it would be replaced after the failed launch) consisted of 48 VMWare virtual machine nodes running Red Hat Enterprise Linux and hosted on twelve physical servers in a Terremark data center. Some of the servers ran vSphere v4.1, with others running v5.1; the network was also running at 1 Gb/sec, far below its capacity of 4 Gb/sec.

Failures on all critical components of the project exemplify that CMS didn't have the personnel or experience to handle an IT project of this magnitude.

With only a couple of months left before the scheduled launch, CMS raised its concerns about CGI's performance but didn't take steps to hold the contractor accountable. In September 2013, CMS moved into CGI's offices for on-site support.

CMS delayed governance reviews that would have exposed the issues and did not receive the required approvals to move forward. However, they decided to continue with an on-schedule launch with all the platform's features, available to every American citizen needing affordable healthcare. The estimated project cost at this point was $500 million.

On October 1, 2013, the Healthcare.gov website was online. Most visitors experienced crashes, delays, errors, and slow performance throughout the week. By the weekend, the decision was made to take the site down because it was practically unusable.

Later in the month, HHS announced the following changes to the project:

Project management was centralized and led by Jeffrey Zients, former OMB director who had a reputation within the Whitehouse for solving tough problems and managing teams.

Todd Park, White House CTO, reorganized the technology leadership team, demoted some underperforming CMS employees and 3rd party contractors, and recruited top talent from Silicon Valley for a government sabbatical to save the site.

A Tiger team was formed with the narrow mandate of getting the FFM working properly.

The new team scrummed daily, triaged existential risks, and prioritized defects based on urgency. Over the next six weeks, the Tiger team resolved 400 system defects, increased system concurrency to 25,000 users, and improved the site's responsiveness to one second. The site went back online, and enrollment jumped from 26,000 in October to 975,000 in December.  

By Christmas, most problems had been fixed, but the site was still not fully operational.

CGI was replaced by Accenture as the lead contractor and awarded a $90 billion contract to replace the FFM.

The individual mandate requirement was pushed back to March 31, 2014, giving uninsured Americans more time to sign up without being penalized.  

In July, the GOA released a detailed report outlining the critical failures of the project.

According to the GOA's findings, FFM obligations increased from $56 million to more than $209 million. Similarly, data hub obligations increased from $30 million to nearly $85 million from September 2011 to February 2014. The study recommended that "CMS take immediate actions to assess increasing contract costs and ensure that acquisition strategies are completed, and oversight tools are used as required, among other actions. CMS concurred with four recommendations and partially concurred with one. "

In August, the Office of Inspector General released a report finding that the total cost of the Healthcare.gov website had reached $1.7 billion. A month later, Bloomberg News reported the cost exceeded $2 billion.

By November, open enrollment on Healthcare.gov began for 2015.

What Went Wrong?

A multitude of issues caused the failed launch of the Healthcare.gov website. While it is a clear example of the federal government's continuous struggle to implement functioning, secure software, the problems go beyond Washington's bureaucracy. Nevertheless, much can be learned about releasing digital solutions and managing large-scale projects with many moving parts in general.

Overconfidence

The project started with overconfidence and unrealistic expectations set by the White House. Obama's campaign staff had a reputation for being technologically savvy because they pioneered using social media and data mining in the 2008 presidential election. However, running a social media campaign and releasing a single point of contact that pulls from multiple government agencies and insurance companies aren't comparable.

Underestimated Scale of the Project

Due to overconfidence and unrealistic expectations, the project scale was drastically underestimated, resulting in mismanagement in organizational structure, leadership, accountability, and transparency.

As the deadline approached, the project scope grew, while CMS identified 45 critical and 324 severe code defects across FFM modules.

Launching large-scale software projects are extremely challenging but adding a hostile political climate made the rollout even more difficult. Not only did CMS not have the personnel or experience to handle an FFM, but they also experienced pressure and influence from outside the agency. One of the most significant examples came in August of 2013, the White House and executive Office of Health Reform insisted on requiring site user registration before shopping for insurance so that concrete user numbers could be shown as proof of the system's success.

Members of the project committee that weren't from the CMS exhibited a great deal of influence over critical decision-making while not having access to accurate progress reports. In addition, the 2012 presidential elections likely impacted delays. Polarizing decisions, such as final rules on private insurance premiums, coverage availability, risk pools, and catastrophic plans, were put off till after the election cycle. These rules had to be translated into software and tested before a successful rollout was possible.

Lack of technology understanding and experience at CMS

The CMS was not prepared to handle a technology project at this scale. Other government agencies, such as the DOD and NASA, had decades of experience navigating the institutional challenges required to develop, deliver, and operate reliable IT systems.

Throughout procurement, development, and launch, CMS made it clear that its personnel didn't understand the requirements necessary to oversee a large-scale technology project, let alone one that had additional regulatory hurdles set by government agencies.

Poor Project Management

The project committee was spread across various government agencies, including the CMS, the White House, the Office of Health Reform, and the Department of Health and Human Services. As a result, there wasn't an organizational structure standard on even small-scale projects. Fractured leadership contributed to the lack of project management, but CMS was primarily responsible. While the problem could have been lessened if CMS had followed the guidelines from FAR, the operators from the agency pleaded ignorance in the GAO report, strengthening the case that CMS didn't have personnel suitable for the project.  

Failed to Postpone Launch

All the problems accumulated, resulting in a failed launch. Typically, the release is delayed when a website or software isn't ready. The engineers perform more testing, fix the problems, and launch at a later time. Healthcare.gov was a unique project with consequences transcending a poor user experience. The time constraints pressured CMS to go forward with the launch rather than being transparent and communicating that the infrastructure couldn't handle millions of users.

Leading up to the launch, CMS was given more business rules and a broader scope. While they had no control over the pressure coming from the Whitehouse, the agency failed to be upfront, and instead of delaying the launch or releasing in stages, they scrambled to save the project.

How CMS Could Have Done Things Differently

CMS was in charge of the project, but the blame falls on the Obama administration. Appointing CMS to lead the project was the first mistake made in a number of shortcomings that led to billions of wasted tax dollars and US citizens being delayed health care coverage.

While appointing a different agency, one that had experience delivering technology projects, was the first significant error, analyzing the oversights made by CMS leading up to the launch is the most practical way to assess what could have been done differently.  

Preparation

An understanding of the scale of Healthcare.gov would have dramatically influenced CMS to prepare better and implement project management best practices. One way the leadership at CMS could have prevented a launch failure was to realize they couldn't handle heading the project. Assigning lead manager and integrator roles to an outside firm or the lead vendor would have been more sensible.

Procurement

Many institutional challenges out of the hands of CMS contributed to the failed launch of Healthcare.gov. However, the agency had complete control over the procurement of government contractors. Simply following the Federal Acquisition Regulation (FAR) procurement framework could have prevented many organizational problems that led to the failure.

Had CMS adhered to standard government agency procurement guidelines, issues like the confusion around who was lead integrator wouldn't have existed. Furthermore, decisions like depending on data provided by Experian, a data source that neither the government nor the other contractors could do any data quality work on, would have likely been questioned if not denied when submitted for approval.

Adopt Iterative Software Development Framework

The project would have benefited if an iterative system development philosophy and a lean software manufacturing process such as Scrum or Kanban were adopted. Project managers could have organized sprints driven by the top priorities, including the complete end-to-end testing of the technology solution. An Iterative Development Framework would have increased visibility across multiple contractors and federal agencies, improved development quality, and reduced time to market.

Strong Leadership

Leadership was a fundamental problem that affected every aspect of the Healthcare.gov launch. Distribution of authority, management, and accountability created an environment where a functioning FFM was impossible to deliver on time. The project steering committee should have elected one project executive to make final decisions, hold contractors accountable, and communicate with other government agencies.

See " Consensus Is the Absence of Leadership " for  for more insights on this topic.

Set and Guard Technical Standards

A rushed, unqualified, and fractured committee led to the development of poor technology. As a result, CGI and other contractors experience zero to very little oversight allowing for an unfinished UI, partially operating back-end, and unstable hosting. 

The developers from CGI delivered an unpolished UI with excessive typos, a bloated directory, sloppy code, and even Lorem Ipsum placeholder text on the web pages. In addition, best practices were not followed or ignored regarding file compression, causing the website to take eight seconds to load and 71 seconds for user account registration pages with client-side loading, according to a report by AppDynamics. 

A basic small business website delivered at this standard would be unacceptable, let alone the focal point of one of the most transformative pieces of legislation in recent US history.

CGI should have been held to higher standards and required to deliver a polished, functioning UI.

While the front end was a disaster, fixing HTML, CSS, and JS and optimizing webpages can be done overnight. Healthcare.gov's back-end was a different story requiring systemic changes to function. More oversight was needed on the database and server-side development, but it could have only gone smoothly with drastic changes to leadership and organizational structure.

Improve Security

Healthcare.gov requires an abundance of personal data to be submitted and collected. While security wasn't the primary failure of the project, the servers were hacked in July 2014. The malware was uploaded into the system but failed to communicate to any external IP addresses. In addition, multiple security defects were found, including the insecure transmission of personal data, unvalidated password resets, error stack traces revealing internal components, and violations of user data privacy.

Comprehensive security audits must be conducted before the launch rather than on the fly after a site is live.

Implement Testing and Bug Fixing Protocols

Adequate testing could have prevented many of the FFM's problems; however, CMS was well aware of the issues but failed to communicate with HHS and other government agencies. Still, testing protocols and a strategy to manage and fix bugs are necessary for an IT project of this scale.

CMS needed to coordinate unit and component testing by 3rd parties much sooner than a couple of weeks before launch. Testing conducted by teams outside of specific features of the project ensures there aren't biases and that the project works on various devices, browsers, and servers.

Another issue CMS encountered was communication with states implementing their own healthcare marketplaces. The date was pushed from November 2012 to December 2012, and some were confirmed as late as February 2013. Uncertainty of the traffic volume should have led to overpreparation and expanded capacity testing of concurrent users. Instead, a load of only 2,000 simultaneous users has been tested rather than the tens of thousands they should have expected.  

Phased or Staged Rollout

One way the project steering committee could have responded to the issues without pushing back the launch was to release the project in phases or stages. Below are multiple options the committee could have taken instead of the Big Bang launch approach:

> Released certain features that were ready or could have been prepared when prioritized before October 1, 2013.

> Roll out a beta version of the platform to a small number of applicants (by region, government employees, sample group, etc.) months before the deadline.

> Release the platform in strategic phases leading up to the deadline, e.g., encourage applicants to register early in August, request eligibility in September, and shop for plans in October.

> Limit the scope of the project months before the launch and focus on minimal requirements rather than continue expanding leading up to the hard deadline.

See " How Your Rollout in Waves Can End in a Tsunami " for more insights on this topic.

Face Reality

The problems facing the Healthcare.gov launch were apparent, and there's evidence in the GAO report that suggests CMS was well aware of the issues. In addition, a McKinsey report was released just a few months before the scheduled rollout in April 2013. The report highlighted the initiative's complexity and identified more than a dozen critical risks spanning the marketplace technology and project governance. 

The problems we've covered were clearly outlined in the report, as well as multiple definitions of success and concerns with a Big Bang launch approach. The McKinsey report also suggested several actionable methods to mitigate the risks. Still, the project steering committee did not act upon the McKinsey report's findings and recommendations before the system launch.

Whether CMS caved under the pressure of the deadline, the political consequences, or was just incompetent enough to expect a positive outcome is unclear. All the warning signs pointed to an unsuccessful launch and should have been taken seriously, resulting in making the necessary adjustments.

See " It Is Time to Face Your Project's Reality " for more insights on this topic.

Closing Thoughts

The Healthcare.gov project exemplifies just about everything that can go wrong with software integration between the federal government and the private sector. The project's failures are incredibly complicated due to the influence of multiple government agencies and a hostile political climate. When one problem is identified, more come to the surface with increasingly challenging solutions.

But was the project doomed to fail just because of the layers of government bureaucracy? I don't believe so.

Had the committee executed strong leadership with personnel who had experience working on institutional software, the project could have gone much differently. The remarkable aspect of Healthcare.gov is how fast it was turned around, given the state of the project after the launch. Once the Whitehouse was fully aware of the problems and competent leaders were put in place, the site was made functional in a few weeks and essentially operational in December 2013.

In a nutshell: Experienced leadership leads to realistic expectations, a healthy organizational structure, and strong communication. Never underestimate the importance of the person or group that is in charge.

> https://www.bloomberg.com/news/articles/2014-09-24/obamacare-website-costs-exceed-2-billion-study-finds

> https://oig.hhs.gov/oei/reports/oei-03-14-00231.asp

> https://hackernoon.com/small-is-beautiful-the-launch-failure-of-healthcare-gov-5e60f20eb967

> https://www.appdynamics.com/blog/product/technical-deep-dive-whats-impacting-healthcare-gov/

>  https://www.gao.gov/products/gao-14-694

October 30, 2013

Lessons learned from the Healthcare.gov project

Julian Pscheid

Flickr.com/pasa

Watching the bumpy rollout of online healthcare exchanges around the country—in particular Healthcare.gov—has provided a myriad of case studies to dissect. Tackling a large-scale systems development project—particularly a consumer-facing product—is never easy, and I feel for the development teams that are caught in the middle of this. There are some key lessons so far to be taken from this.

Put someone in charge that knows what they’re doing This might seem like the most obvious one, but it is also one of the toughest to fulfill when dealing with institutional or governmental bureaucracies. In the case of Healthcare.gov, the Centers for Medicare and Medicaid Services retained leadership over the project, and it turns out they had no technological expertise in-house  and so had no business running a complex software development project. This resulted in 55 separate contractors working on the project without clear leadership. If you don’t have technological leadership in-house, designate a partner to be the lead, as the State of Colorado did for their system’s implementation . Yes, it will probably increase your cost to have a partner designated as project lead, but consider it an insurance policy against your own ineptitude.

Throwing more money at a project alone isn’t a fix Even with a reported $170+ million budget , things didn’t go right with the Healthcare.gov rollout. Or maybe because of the $170+ million dollar budget things didn’t go right. In software development, lean design usually forces stakeholders and team members to make critical decisions along the way, adjusting scope and timelines to fit what’s possible, given the available budget. The risk of having too much money is that it creates the false impression that scope and timeline issues can be fixed by throwing more resources at them, when adding more resources usually only creates incremental efficiencies in software development.

Learn best practices from the new wave of software consumer products There are hundreds of tales of tech start-ups that have launched consumer-facing web-based software products in the last few years and have struggled through exponential growth phases. These startups had to deal with unexpected website traffic, application design challenges, and constricting 3rd party involvement, similar to issues that affected the launch of Healthcare.gov. Don’t let these lessons go to waste—many are readily available online.

If you’re going to compress a timeline, things will get missed It’s not unusual to have stakeholders on a project push for a “compressed timeline”. While on most projects there probably are some efficiencies that can be implemented, the unfortunate truth is that a lot of the time savings would originate from so-called “shortcuts”. And when you’re taking a shortcut, you are bypassing part of the planned route that would ensure a properly executed project. Quite often the quality assurance phases are affected by this as well. This was the case at Healthcare.gov, where due to the compressed timeline there was only room for two weeks of quality assurance . The best thing you can do is to be honest to yourself and your stakeholders upfront about the quality impacts a shortcut might have on the project.

Prioritize features, then plan for phased rollouts Everyone wants a perfect product right out of the gate, right? Unfortunately the more you pack into your day 1 launch, the more things that can and will go wrong. Soft-launches are your friend! Oregon’s approach was to release only a limited amount of functionality on October 1st , and phase in additional features over the coming weeks. The result was initial reports of 34 issues, only two of which were critical. The team could focus on the two critical issues right away, without being overwhelmed by media scrutiny and visitor traffic.

Even better: plan for a limited audience release. If you can limit your initial launch to 1-5% of the target audience (depending on how many users you expect to have in the end) this gives you time to flesh out issues with a smaller group of users, limiting negative exposure and providing a more controlled environment to release rapid updates. This is one reason that states which decided to manage their own exchanges have, for the most part, been more successful than Healthcare.gov.

Related Posts

• What to Look for in Website Accessibility: Audit of WhiteHouse.gov
• Do You Ask These Three Essential Project Stakeholder Questions?

Learn From Us

• First Name *
• Last Name *
• Country * United States Canada Afghanistan Albania Algeria American Samoa Andorra Angola Antigua and Barbuda Argentina Armenia Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia Bosnia and Herzegovina Botswana Brazil Brunei Bulgaria Burkina Faso Burundi Cambodia Cameroon Cape Verde Cayman Islands Central African Republic Chad Chile China Colombia Comoros Congo, Democratic Republic of the Congo, Republic of the Costa Rica Côte d'Ivoire Croatia Cuba Curaçao Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic East Timor Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Faroe Islands Fiji Finland France French Polynesia Gabon Gambia Georgia Germany Ghana Greece Greenland Grenada Guam Guatemala Guinea Guinea-Bissau Guyana Haiti Honduras Hong Kong Hungary Iceland India Indonesia Iran Iraq Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Kiribati North Korea South Korea Kosovo Kuwait Kyrgyzstan Laos Latvia Lebanon Lesotho Liberia Libya Liechtenstein Lithuania Luxembourg Macedonia Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Mauritania Mauritius Mexico Micronesia Moldova Monaco Mongolia Montenegro Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands New Zealand Nicaragua Niger Nigeria Northern Mariana Islands Norway Oman Pakistan Palau Palestine, State of Panama Papua New Guinea Paraguay Peru Philippines Poland Portugal Puerto Rico Qatar Romania Russia Rwanda Saint Kitts and Nevis Saint Lucia Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Sint Maarten Slovakia Slovenia Solomon Islands Somalia South Africa Spain Sri Lanka Sudan Sudan, South Suriname Swaziland Sweden Switzerland Syria Taiwan Tajikistan Tanzania Thailand Togo Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Tuvalu Uganda Ukraine United Arab Emirates United Kingdom Uruguay Uzbekistan Vanuatu Vatican City Venezuela Vietnam Virgin Islands, British Virgin Islands, U.S. Yemen Zambia Zimbabwe

• Artificial Intelligence
• Generative AI
• Business Operations
• Cloud Computing
• Data Center
• Data Management
• Emerging Technology
• Enterprise Applications
• IT Leadership
• Digital Transformation
• IT Strategy
• IT Management
• Diversity and Inclusion
• IT Operations
• Project Management
• Software Development
• Vendors and Providers
• Enterprise Buyer’s Guides
• United States
• Middle East
• España (Spain)
• Italia (Italy)
• Netherlands
• United Kingdom
• New Zealand
• Data Analytics & AI
• Newsletters
• Foundry Careers
• Terms of Service
• Privacy Policy
• Cookie Policy
• Copyright Notice
• Member Preferences
• About AdChoices
• Your California Privacy Rights

Our Network

• Computerworld
• Network World

6 Software Development Lessons From Healthcare.gov’s Failed Launch

The problems that plagued the launch of healthcare.gov -- the online data hub and insurance marketplace central to healthcare reform -- will someday fill a book. for now, developers can apply these six lessons from healthcare.gov's very public failure to their own software projects..

“I spent $174 million on a website and all I got was this bad press.”

— Someone, somewhere in the U.S. Department of Health and Human Services (HHS)

Well, at least someone should have said it.

Healthcare.gov is arguably the most public software failure of the decade. You may have read commentary by people who have never had to write or test code, never served on a software project, and likely don’t know how to right-click and “view source” and read the HTML. That’s OK; most journalists, most of the time, don’t need to be very technical.

This article tries to go further than the typical coverage of Healthcare.gov. The amazing thing about this story isn’t the failure. That was fairly obvious. No, the strange thing is the manner in which often conflicting information is coming out. Writing this piece requires some archeology: Going over facts and looking for inconsistencies to assemble the best information about what’s happened and pinpoint six lessons we might learn from it.

1. Sprints and Iterations Do Not an ‘Agile’ Project Make

One of the biggest arguments about the Healthcare.gov debacle is the development method. Pundits suggest that, since war room notes use terms such as sprints and story cards , the project used an agile approach, but agile approaches can not prevent all risk. Even the reduction of risks requires skill and judgement. Then again, CBS News claims that Healthcare.gov shrunk its schedule for testing from months to weeks, suggesting thast Healthcare.gov was never properly tested .

In agile software development, the terms “sprint” and “iteration” mean the time for a new, completed chunk of software development to be designed, coded and fully tested, end-to-end. The standard length for teams to start with iterations is two weeks; improvement beyond that generally means shortening the iteration.

Many teams use “sprint” or “iteration,” only to insert waterfall concepts. Language such as “three architecture sprints, six coding sprints, two test sprints and two hardening sprints” is usually a clue that something’s wrong.

But there’s something else more sinister in the HealthCare.gov project.

2. The System Produced the Outcome, Not the Lack of Testing

If you’ve worked on a waterfall project with a defined test phase, you know it’s not really a “test” phase at all. Once the first show-stopper bug is found, it’s actually a fixing phase.

To state that Healthcare.gov wasn’t fully tested implies that the test group either never found any show-stopper bugs or didn’t have the ability to halt the project. Given the legally required go-live mandate, the second option seems realistic.

One thing we do know for sure: The organization rushed ahead with coding before it knew answers to questions. That’s no recipe for success.

We know because at go-live the JavaScript code itself was still full of filler text that a non-decision maker typically insert onto a page or pop-up. Programmers know that something goes wrong but need to get the language approved, so they write lorem ipsum and move on. This is exactly the kind of thing that happens when people are under a deadline and can’t get answers to their questions.

Mike Adams, editor of NaturalNews.com, found dozens of critical JavaScript errors by simply viewing the source and following links. Here’s a small sample:

“TODO: add functionality to show alert text after too many tries at log in”

“make sure we don’t try to do this before the saml has been posted if (window.registrationInitialSessionCallsComplete)”

“Attention: This file is generated once and can be modified by hand”

“Fill In this with actual content. Lorem Ipsum”

“TODO: maybe modify the below to use a similar method instead”

People in the organization must have known the site was not ready. Perhaps they spoke up and were overruled. We don’t know.

The problem at Healthcare.gov wasn’t a lack of testing. It was a lack of critical thinking. Still, the site did have a prime testing contractor. Sure, the testers didn’t have months, but they did have weeks. What were they doing?

3. Testing Should Be Part of the Delivery Process

QSSI was the lead test contractor on Healthcare.gov , providing services it refers to as Independent Verification and Validation, or IV&V.

Here’s what QSSI has to say about its own IV&V Process (links added):

Our IV&V services are consistent with the latest systems engineering and process improvement models, and are derived from industry standards, including the IEEE Standard 1012 -2004 Standard for Software Verification and Validation and the CMMI process maturity framework. QSSI provides an objective assessment of products and processes throughout the project life cycle in an environment free from the influence, guidance, and control of the development effort. Services include: criticality analysis, requirements analysis and tracing, software design analysis, milestone reviews and metrics, code review and analysis, document inspection [and] defect Investigation, plus training evaluation, planning, execution, reporting and witnessing.

Look at the QSSI careers site and you see a lot of focus on these standards, policies and procedures — in particular, that IV&V involves tracing requirements to implementation.

That’s one aspect of product risk, but it’s certainly not a whole-process look. The agile manifesto changed the focus of software from “processes and tools” to “individuals and interactions” back in 2001. That same year, Joel Spolsky suggested it was possible to pass every functional test yet still have a product that isn’t usable.

The real reveal, however, comes from the 175 pages of war room notes from the early release of the website. The notes focus entirely on dealing with trouble tickets that occur in the field. There’s no connection to testing. At no point in the document — nowhere — does anyone go back and say, “Here’s the list of things deferred at go-live as known issues.”

The entire process is firefighting. Testing wasn’t seriously considered. If testing actually occurred — and real testing, not fake testing — it failed to make any meaningful connection to operations or support or project management. Testing failed to recommend a no-go decision.

If testing can’t impact the schedule or product in any way, it’s just waste. Better to eliminate it altogether.

(After repeated phone transfers, no one from QSSI was available to comment on its testing of Healthcare.gov.)

4. Do It Manually Before You Automate

After a great deal of failure and public fallout from Healthcare.gov, customers started to use the paper application process as an alternative. Sounds reasonable — until you realize the paper applications would be mailed to a customer service center using the same Web-based technology to determine eligibility.

In other words: If you give up on Healthcare.gov because you can’t purchase insurance, you can mail a paper application to a processing center where a human will try to enroll you manually in Healthcare.gov. That doesn’t work.

Neither the federal government nor any known contractor has any way of ensuring eligibility manually. If they had an 800-number to the IRS, and another to the Department of Homeland Security, the agencies could hire an army of temps to manage the order processing. (That idea’s not that farfetched; it’s essentially how the federal government conducts the census every 10 years.)

If you have a large-scale adoption and have to flip the switch, make sure you have a way to flip it back and can execute manually.

5. The System Had to Be Perfect, By Law, But It Wasn’t

Healthcare.gov must verify a would-be applicant’s eligibility from Homeland Security, IRS and Social Security systems, in real time, plus enroll members electronically to any registered insurance company in any of the 34 states that don’t have their own state health insurance exchange .

Information needs to be completely accurate, so every request needs to run-submit information, over Web services, to an insurance company. That means users click Submit, then wait for a back-end Web services call, and wait and wait and …

All this because the system had to work in “real time,” or synchronously. Except it didn’t.

Turns out that HHS created a spreadsheet with the basic rate table information. Healthcare.gov could have been a very simple process, then: Type in zip code, click Submit, select age and family factors, get quote and get number to call purchase insurance.

We know this smaller scope could have worked because it does work . A tiny company called Opscost took that spreadsheet and implemented a website doing just that in few than two person-weeks. The site is thehealthsherpa.com (shown below).

This comparison of six person-days to hundreds of millions of dollars is obviously unfair. HealthCare.gov had to do more, including end-to-end enrollment. Healthcare.gov customers were supposed to be able to sign up on a government Webform and have the transaction automatically flow into an insurer’s system.

Instead of an 80 percent system for a fraction of the cost, the customer insisted on 100 percent by force of law. Lacking the capability to actually develop such a system, the contractors made their best attempts and shipped whatever they had on the end date. The results, while tragic, aren’t exactly a surprise.

George Kalogeropoulos, customer support at OpsCost, says the company’s biggest surprise while validating the market was that a huge number of customers wanted to window shop. They wanted a quote without giving up personal information such as phone number or Social Security Number.

Analysis: Why Healthcare.gov May Be a ‘Black Swan’

More: Lawmakers Seek Answers on Obamacare Data Hub Security

At go-live, Healthcare.gov worked on the opposite model, forcing personal information before allowing users to see a quote. (This policy has since changed.) That allowed Healthcare.gov to force users to agree to terms of service, and to connect to backend servers at the IRS, Homeland Security and other agencies. Those are great features for the end of the process, but when your users want to window shop, perhaps less so.

6: Threat Modeling Matters

Ben Simo isn’t a black-hat hacker. He doesn’t spend his time trying to “crack” things. He’s a developer/tester who knows how to view source code and use the developer tab to look at back-end Web transmissions, mostly RESTful services. That’s exactly what Simo did with Healthcare.gov: Look at how his family’s information was transmitted as he tried to sign up.

Simo didn’t succeed in signing up — but he did find a site that transmitted his username and email in plain text , loaded insecure resources and could reveal user IDs and email addresses through login notification messages .

Perhaps worst of all, the password reset feature results in data combinations that could enable phishing attacks . An attacker logged in as you can get personal information from REST service responses, including name, address, date of birth and Social Security number. That’s exactly the kind of information identity thieves can use to get false credit cards or gain access to bank accounts.

Related: Healthcare.gov Granted a Waiver to Launch Despite High Risk Levels

More: Scramble to Fix Healthcare.gov Site Heightens Security Risks

Simo’s analysis has been quoted in TIME , on CNN and during HHS Secretary Kathleen Sebelius’ Congressional testimony . Simo is a former president of the Association for Software Testing , and I’ve served on its board of directors. We’re regular humans. How did one of us find not a single defect but a systematic mess?

It’s likely that security testing was faked, overlooked or ignored for Healthcare.gov, just like functional testing. Don’t let this happen on your watch.

Ultimate Lesson from Healthcare.gov: Take Incremental Approach

It’s early in the Healthcare.gov lifecycle. We know the project’s failure was systemic — multiple failures on multiple levels — but we don’t know exactly what did happen. Identifying all the issues might make for a good book someday, but it’s far too much for one article.

Putting together the six lessons here, though, and you’ll see a pattern: Healthcare.gov was a single, Big Bang rollout that couldn’t be stopped.

At this early stage, if you can take only one thing from Healthcare.gov, it’s this: Use incremental approaches such as betas, early testing and regular delivery of a completely tested system, in tandem with flexible scope to find risk before it finds you.

The saddest part about the Healthcare.gov failure isn’t that is so massive. The saddest part is that the failure was both predictable and greatly preventable.

Matthew Heusser is a consultant and writer based in West Michigan. You can follow Matt on Twitter @mheusser , contact him by email or visit the website of his company, Excelon Development . Follow everything from CIO.com on Twitter @CIOonline , Facebook , Google + and LinkedIn.

Related content

New uk government downplays ai regulation in program for the next year, china makes its bid for global ai governance, italian cios at the crossroads of transformation and sustainability, uk government’s pensions dashboards programme delayed, from our editors straight to your inbox, show me more, ai-powered mindfulness: the human element of innovation.

How to build a safe path to AI in Healthcare

Inside CIOs’ response to the CrowdStrike outage — and the lessons they learned

CIO Leadership Live ASEAN with Sandeep Pandey, Group Chief Technology and Operations Officer, FWD Insurance

CIO Leadership Live Australia with Sinan Erbay, Chief Information Officer, RMIT University

CIO Leadership Live with Anas Mosa, Director of IT and PIF Projects in KSA

CIO Leadership Live NZ with Eion Hall, Chief Information Officer, University of Waikato

How Saasteps automates sales process to speed revenue generation

Sponsored Links

• The future of identity is here. Unlock brand growth with Merkury
• Everyone’s moving to the cloud. Are they realizing expected value?
• Everybody's ready for AI except your data. Unlock the power of AI with Informatica
• The cloud shouldn’t be complicated. Unlock its potential with SAS.

A look back at technical issues with Healthcare.gov

Subscribe to the center for technology innovation newsletter, joshua bleiberg and joshua bleiberg assistant professor - university of pittsburgh darrell m. west darrell m. west senior fellow - center for technology innovation , douglas dillon chair in governmental studies.

April 9, 2015

Five years ago Congress passed the Patient Protection and Affordable Care Act into law. An  estimated 12 million people have enrolled in the exchanges created by the law. Despite these recent successes there is little doubt that the launch of HealthCare.gov was marred with many serious failures. A  recent report from the Government Accountability Office (GAO) provided some insights into the factors contributing to the launch issues and steps that are necessary to prevent future problems.

CMS lacked management

Prior to the launch of HealthCare.gov, the Centers for Medicare and Medicaid Services (CMS) eschewed four management practices recommended by the Software Engineering Institute and the GAO: scheduling, estimating the effort needed for project tasks, data management monitoring practices, and milestone project reviews. The initial CMS master schedule did not include many development activities or a design timeline. Even though the CMS Requirements Management Plan dictates that planning documents should estimate the effort needed to complete a project, it was missing from two major components of the project. Similarly, a guideline for managing project data was developed but not followed by CMS and its contractors when storing key development documents. Project progress and milestone reviews were incomplete.

More importantly, many of these issues remain today. After the failed launch CMS re-evaluated their development schedule, but the new plan still has key flaws. Project component deadlines weren’t properly aligned. In some cases the plan called for certain activities to be completed months before it was necessary. New policies and practices were developed to fix some of these problems but many of them were either not implemented or not officially documented.

HHS had a limited oversight role

The Health and Human Services (HHS) Secretary had appointed a Chief Information Officer (CIO) to manage information technology projects. The HHS CIO was supposed to work with an IT Investment Review Board to review, validate, and approve selected IT investments. In an August 2011 memorandum, OMB stressed that the CIO should assume responsibility for all IT projects.

However, the CIO claimed to have limited oversight role during the development of HealthCare.gov, arguing that the concerns about the project were not raised during the monthly meetings with senior leadership from each operating division. Moreover, the Investment Review Board hadn’t been active for years and the department was too large to manage effectively. HHS expanded its oversight role after the initial failure, but the CIO reiterated claims about lacking the authority to manage at the operating division level. In addition the review board is still not active.

OMB is responsible for analyzing, tracing, and evaluating the risk of major capital investments. Similarly to HHS, OMB did not take an active role in overseeing the development of HealthCare.gov. Prior to the launch, OMB only coordinated with CMS and other agencies, clarified policies, and oversaw the budget. In 2009 OMB launched the Federal IT Dashboard to evaluate major IT investments. Later OMB started to conduct TechStat sessions to intervene on low performance IT investments. Despite a designation on the IT Dashboard as a high-risk project, the HealthCare.gov initiative was not selected by OMB to be reviewed through TechStat. Officials in OMB claimed that it was HHS’s responsibility to select the project for TechStat for the more rigorous intervention.

OMB established U.S. Digital Service in August 2014 partially as a response to the HealthCare.gov initial failure. The U.S. Digital Service will improve and simplify the online interaction between individuals or private firms and federal government. Currently the service is working closely with the CMS system team to further support HealthCare.gov.  

Many oversight problems contributed to the initial failure of Healthcare.gov. Surprisingly the GAO report calls attention to several management issues that caused that disastrous launch. Unless CMS, HHS, and OMB address those weaknesses and fully implement best practices future federal IT projects may suffer a similar fate.

Yikun Chi contributed to this post

Governance Studies

Center for Technology Innovation

Darrell M. West, Roxana Muenster

August 5, 2024

Mark MacCarthy

July 31, 2024

Nicol Turner Lee, Darrell M. West

July 23, 2024

The HealthCare.gov project

• Teaching Case
• Published: 06 December 2016
• Volume 6 , pages 99–110, ( 2016 )

Cite this article

• Janis L Gogan 1 ,
• Elizabeth J Davidson 2 &
• Jeffrey Proudfoot 1  

335 Accesses

2 Citations

Explore all metrics

This case describes the development of the HealthCare.gov website front-end, systems and databases supporting the implementation of the Affordable Care Act. In late October 2013, US Health and Human Services Secretary Kathleen Sebelius and US Centers for Medicare and Medicaid Services Administrator Marilyn Tavenner appear before a Congressional subcommittee to apologize about system glitches. The case gives students an opportunity to consider project risks that affected this huge systems development effort, and to consider how to ensure that millions of uninsured or underinsured Americans would be able to sign up for affordable health insurance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Recent US Experience with Health ICT

Building National Healthcare Infrastructure: The Case of the Danish e-Health Portal

The Australian PCEHR or My Health Record: The Journey Around a Large-Scale Nationwide Digital Health Solution

GAO-13-601 (2013). Status of CMS Efforts to Establish Federally Facilitated Health Insurance Exchanges: U.S. Government Accountability Office, 2.

Gerencher, K. (2013). Get ready to enroll in exchanges. The Wall Street Journal , Eastern Edition 18 August: A3.

Gogan, JL, Fedorowicz, J and Rao, A. (1999). Assessing Risks in Two Projects: A strategic opportunity and a necessary evil, Communications of the Association for Information Systems 1(15).

Johnson, A. (2013). What you need to know about the Affordable Care Act. The Wall Street Journal , Eastern Edition 29 September: A1.

King N., Jr. and Prang, A. (2013). More voters turn on Obama. The Wall Street Journal , Eastern Edition 31 October.

Langley, M. (2013). ‘It’s tough to take these shots’: Health law’s rocky debut puts Sebelius in cross hairs. The Wall Street Journal , Eastern Edition 19 October: A1.

Mclean, B. (2013). Accounting for Obamacare: Inside the company that built HealthCare.gov. Vanity Fair , 22 December.

Parkinson, J. (2013). Obamacare website contractor Shirk Blame for foul-up. ABC World News 24 October [www document] http://abcnews.go.com/blogs/politics/2013/10/obamacare-website-contractors-shirk-blame-for-foul-up/ .

Radnofsky, L. (2013). How will the subsidies work? The Wall Street Journal , Online 30 August.

Radnofsky, L. and Weaver, C. (2013). Balky exchange set to get tech help. The Wall Street Journal , Eastern Edition 21 October: A6.

Schatz, A. (2013). Shutdown unlikely to curb Affordable Care Act funds. The Wall Street Journal , Eastern Edition 25 September: A4.

Schatz, A. (2013). Groups get $67 million to promote health exchanges. The Wall Street Journal , Online 15 August.

United States House of Representatives Committee on Energy and Commerce (2013). Oral testimony on Affordable Care Act implementation, 30 October, Washington DC: Government Publishing Office (statement of Kathleen Sebelius, Secretary, U.S. Department of Health and Human Services (HHS)) [www document] http://www.hhs.gov/asl/testify/2013/10/t20131030.html .

United States House of Representatives Committee on Ways and Means (2013). Hearing on Affordable Care Act implementation, 29 October, Washington DC: Government Publishing Office (statement of Marilyn Tavenner, Administrator, U.S. Department of Health and Human Services (HHS)) [www document] http://www.hhs.gov/asl/testify/2013/10/t20131029.html .

United States Senate Committee on Finance (2013). Health insurance exchanges: Progress report, 14 February, Washington DC: Government Publishing Office (statement of Gary Cohen, JD, Deputy Administrator and Director, Center for Consumer Information and Insurance Oversight, Centers for Medicare and Medicaid Services) [www document] http://www.hhs.gov/asl/testify/2013/02/t20130214.html# .

Weaver, C., Martin, T.W. and Radnofsky, L. (2013). Capital digs in for long haul: Health-law debut beset by tech woes as consumers clog exchange websites. The Wall Street Journal , Eastern Edition 2 October: A1.

Weaver, C., Ovide, S. and Radnofsky, L. (2013). Software and design defects cripple health-care website. The Wall Street Journal , Eastern Edition 7 October: A1.

Weaver, C. and Radnofsky, L. (2013). Health site stymied by lack of direction. The Wall Street Journal , Eastern Edition 28 October: A1.

Weaver, C. and Radnofsky, L. (2013). Health exchange’s flaws pinpointed. The Wall Street Journal , Eastern Edition 11 October: A4.

Download references

Acknowledgements

The authors acknowledge Ashok Rao, who provided invaluable insights as co-author of the first version of this case, which received a Best Case award from the Project Management Institute when presented at the Northeast Decision Support Systems Conference in March 2015. Rao opted out of the current version of the case. Janis Gogan acknowledges Bentley University, which provided a summer research grant for this case study.

Author information

Authors and affiliations.

Bentley University, Waltham, USA

Janis L Gogan & Jeffrey Proudfoot

University of Hawaii at Manoa, Honolulu, USA

Elizabeth J Davidson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Janis L Gogan .

Additional information

This case was prepared based on publicly available sources, to support class discussion rather than to illustrate either effective or ineffective handling of the situation. All characters and events are based on substantiated facts. Copyright 2016, Janis Gogan, Elizabeth Davidson, and Jeffrey Proudfoot.

Rights and permissions

Reprints and permissions

About this article

Gogan, J., Davidson, E. & Proudfoot, J. The HealthCare.gov project. J Info Technol Teach Cases 6 , 99–110 (2016). https://doi.org/10.1057/jittc.2016.2

Download citation

Published : 06 December 2016

Issue Date : November 2016

DOI : https://doi.org/10.1057/jittc.2016.2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• IT Project Risks
• Health-care IT
• Health care
• Find a journal
• Publish with us
• Track your research

More From Forbes

Healthcare.gov diagnosis: the government broke every rule of project management.

• Share to Facebook
• Share to Twitter
• Share to Linkedin

The Patient Protection and Affordable Care Act, better known as Obamacare, will probably be ... [+] remembered as President Obama's most important domestic policy initiative. However, inept federal management of the HealthCare.gov web-site that is central to implementing Obamacare has left many users with a negative first impression of the program. (Image credit: AFP/Getty Images via @ daylife)

After 400 software fixes and major hardware upgrades, the Obama Administration is claiming to have achieved its goal of transforming HealthCare.gov into a web-site that will operate smoothly for "the vast majority of users."  That's important, because the site is central to implementation of the most sweeping reform of federal healthcare services in half a century.  Whether the site is fixed or not, though, its faltering debut will probably be studied in business schools for decades to come as a classic example of inept project management .

Having made my career in the field of defense analysis, I have seen many such foul-ups in military acquisition projects.  It's a rare weapon system that gets delivered on time, on budget, and with all performance specifications satisfied.  The contractors always get blamed when weapon programs go awry, but usually it's the government customer who is really at fault, and that looks to be the case with HealthCare.gov too.  The Center for Medicare & Medicaid Services (CMS) overseeing HealthCare.gov appears to have violated every principle of sound project management.  Here's my personal list of what the government did wrong.

1.  Unrealistic requirements.  This is the first time anybody has ever tried to develop a single web-site where diverse users could (1) establish an on-line identity, (2) review hundreds of health-insurance options, (3) enroll in a specific plan, and (4) determine eligibility for federal subsidies -- all in real time.  The way people have traditionally accomplished these tasks is to go to an insurance agent and work through the possibilities over a period of several days (if not longer).  It probably wasn't realistic to expect that so many arcane functions could be accomplished in a couple of hours by users who lacked advanced computer skills or high-speed internet service.  The improved homepage of HealthCare.gov alerts users to other options for enrolling, like old-fashioned paper applications.

2.  Technical complexity.  As often occurs with poorly-planned weapon projects, unrealistic requirements for HealthCare.gov resulted in an extraordinarily complicated system that is difficult to maintain.  There are just too many moving pieces.  A typical user might have to navigate 75 screens to get to their goal of obtaining insurance, and the whole system contains over a thousand screens.  A total of 55 contractors were hired to produce the various pieces, and in order for all the steps to work CMS had to involve five federal agencies, 36 states, and 300 private-sector insurers offering well over 4,000 plans.

3.  Integration responsibility.  The government has difficulty maintaining organic expertise in information technologies, because the private sector often hires away the best talent.  An executive at a big tech firm engaged in government contracting once remarked to me that whenever he meets with government project leaders, he always knows he is talking to the people industry didn't want.  Despite weak organic IT capabilities, though, CMS decided it would take charge of integrating all the parts in HealthCare.gov, and testing the end product to assure functionality.  The results show why the military almost always hires outside companies to serve as lead integrator.  

4. Fragmented authority.  There seems to have been a great deal of infighting within CMS over how the web-site would operate and what the user experience would feel like.  With three different parts of the bureaucracy contending for control -- the IT shop, the policy shop, and the communications shop -- key decisions were often delayed, guidance to contractors was inconsistent, and nobody was truly in charge.  Government employees appear to have concealed critical information from each other, and on occasion mandated that certain features be implemented or suppliers be used despite contractor warnings that problems would result.

5. Loose metrics.  Perhaps the most important factor in keeping complex projects on track is for managers to utilize rigorous, unambiguous performance metrics in measuring progress.  The government said in a report released on Sunday that it has made "improvements in the site's key operating metrics over the last several weeks," which is a tacit admission that it didn't initially have adequate ways of measuring progress.  Absence of reliable metrics helps explain why federal officials didn’t realize until late in the game that HealthCare.gov might not be ready for prime-time.

6.  Inadequate testing.  The Washington airports authority announced this week that it would delay opening a new subway line to Dulles Airport so that additional testing of software could be conducted, stating that its overriding goal is "safety."  The people overseeing HealthCare.gov clearly had a different management philosophy.  Despite repeated warnings from contractors that more testing of system components was needed, CMS was determined to see the site go live on its planned debut date of October 1.  Because important decisions about the site were still being made only days before this date, there was almost no end-to-end testing of the site before it became operational -- which is why hundreds of software bugs had to be found and fixed later.

7.  Aggressive schedules.  You wouldn't think that standing up a web-site after literally years of planning might entail overly aggressive schedules, but in the case of HealthCare.gov the disorganized bureaucracy took so long to make design choices that the back end of the project was way too hurried for comfort.  When the Pentagon develops a missile-warning or weather satellite, it sometimes delays launches for years to make sure all software and hardware issues are resolved.  One vital satellite called the Space Based Infrared System was delayed for over a year due to concerns about software glitches; when the satellite finally reached orbit, though, it worked perfectly.  CMS chose to stick with its schedule even as problems multiplied, and got a site that didn't work.

8. Administrative blindness.  The Center for Medicare & Medicaid Services may not have had good management practices or metrics for identifying problems, but that doesn't mean it didn't get plenty of warnings about potential problems with HealthCare.gov.  Outside consultants and contractors on the project repeatedly warned government officials about functional difficulties with some features of the site, lack of adequate testing, poor protection of sensitive information, and the like.  Sometimes CMS listened, but much of the time it was in denial about how defective the site was.  It never adequately informed the White House about potential problems, and never subjected HealthCare.gov to systematic review until after the site went live and nearly collapsed.

These fundamental mis-steps explain why the government was able to competitively hire some of the best IT talent in the world, and still get sub-optimal results.  The stringent demands of the military acquisition system exist in large part because the Pentagon wants to avoid the kinds of mistakes that CMS made in standing up the Obamacare web-site.  It's likely that the administration will eventually fix the site or find work-arounds for its signature policy initiative, but it will probably never get back to where it could have been if competent government overseers had made sure HealthCare.gov worked right from Day One.

• Editorial Standards
• Reprints & Permissions

Enhancing Healthcare.gov

• What Accenture Did
• Value Delivered
• Related Capabilities

The Patient Protection and Affordable Care Act (ACA) created new health insurance exchanges, at both the state and federal level. These exchanges are public-private marketplaces where Americans can apply for a tax subsidy and shop for health insurance plans, across insurance companies. HealthCare.gov , the website for the federal exchange, is the front door for the Federally Facilitated Marketplace (FFM).

Many people are aware that the initial launch of HealthCare.gov had a number of challenges, which received extensive coverage and political, media and public scrutiny. People may not be aware, though, that the FFM is much more than just a website. It includes:

• A plan management system for loading healthcare plans onto the website
• Interfaces with state and federal systems
• Interfaces with insurance companies for enrollment, premium payment and reinsurance programs to support premium price stabilization and accurate payments
• Interfaces with the IRS for the tax subsidy
• The new Small Business Health Options Program (SHOP)

A rescue of the website began in November 2013, and in January 2014, the federal government hired Accenture Federal Services as the prime development contractor, with responsibility for stabilizing and improving the website, and finishing development of the additional systems and interfaces.

In just six weeks, Accenture mobilized more than 500 skilled professionals to transition the project at an unprecedented speed. Within eight weeks, Accenture delivered significant technical improvements to the website, stabilizing it during the peak of HealthCare.gov ’s initial enrollment period. This enabled millions of Americans to enroll in health insurance, many for the first time.

The Centers for Medicare & Medicaid Services, CMS, is part of the Department of Health and Human Services (HHS). The organization administers programs, including: Medicare, Medicaid, the Children’s Health Insurance Program (CHIP) and the Heath Insurance Marketplace.

What Accenture did

The initial launch of the HealthCare.gov website was historic—yet it had its share of challenges. CMS urgently needed help navigating to a successful outcome. Not only was CMS transitioning from an incumbent to a new contractor during a critical time, the agency also was transitioning to a new data center provider, the overall program was complex with multiple stakeholders, the customer base was demanding, there were issues with the underlying technology and the project was under high scrutiny.

CMS chose Accenture Federal Services to rescue, stabilize and enhance HealthCare.gov to build a positive experience for consumers. Accenture created a collaborative and comprehensive transition plan that mitigated the transition risk and enabled Accenture to begin hands-on delivery immediately.

CMS also needed Accenture to build new software for IRS tax forms, risk adjustment and re-insurance payments to insurance companies, and small business enrollment.

Quick Transition

Using an innovative workcell transitioning approach, Accenture ramped up more than 500 people in just over six weeks. Critical to the transition was successfully gathering knowledge from multiple organizations internal and external to CMS.

Accenture conducted more than 450 knowledge transfer sessions to capture over 700 knowledge artifacts. The team also hosted 16 days of intensive, requirements “lock-in” sessions that included Accenture developers, testers, business architects, technical architects and CMS business executives. The team captured 3,300 testable and traceable requirements.

This successful transition—unprecedented in its scale and sense of urgency—reduced risk and positioned the team to start hands-on delivery and rescue activities as soon as possible. The entire transition of the program from incumbent to Accenture took eight weeks…four weeks faster than originally proposed.

Successful Operations

To stabilize the site, Accenture completely took over maintenance and operations during the peak period of the 2014 open enrollment. The team tackled defects and delivered urgently needed fixes ahead of schedule, thereby maintaining operations throughout the peak enrollment period.

The effort included:

• Stabilizing the notices infrastructure and software to generate 2.9 million notices with a daily peak of more than 340,000 notices.
• Significantly improving Account Transfer performance, reducing the response time for inbound transfer by 99.8 percent.
• Stabilizing the FFM application in preparation for an expected enrollment surge. On March 31, 2014—the last day of the first open enrollment—the FFM supported a peak of more than 185,000 concurrent users with fast response times. For the second and third open enrollments, there was no unplanned downtime for any components for which Accenture was responsible. The Accenture team also accelerated online response times to under a quarter of a second.

To keep the system fully functioning, especially during peak enrollment, Accenture provided continual monitoring and reporting for HealthCare.gov. Each week, the team delivered multiple software releases, with significant enhancements.

Shop Delivery

Accenture quickly mobilized a team to take over Small Business Health Options Program (SHOP). In just eight months, the team started with requirements analysis and successfully delivered a commercial off-the-shelf (COTS) package.

This included customization, performance testing, functional testing, security auditing and integration with other CMS systems. Accenture collaborated closely with the COTS manufacturer to launch a new line of business for the marketplace in this very short time period, with a smooth go-live and very low defect count.

Innovative, Cloud-Based Solutions

The new federal and state health insurance exchanges created new marketplaces for insurance companies to sell new insurance products to new consumers. To stabilize the pricing of risk, the ACA included new reinsurance and risk adjustment programs.

The former helped offset larger-than-expected claims, and the latter helped transfer payments from issuers that took on lower-than-expected risk to those that took on higher-than-expected risk.

These programs required gathering confidential claims information from 800 different insurance issuers and then performing complex, risk stabilization calculations and analytics. CMS needed to provide a solution where issuers-maintained control of their confidential claims information, as input to the risk calculations, but CMS controlled the risk algorithms, software and reference data.

Accenture developed a cutting-edge solution that provides issuers a complete data processing environment, which each issuer owns and operates. The “EDGE” system uses Amazon Web Services (AWS) to connect with more than 800 issuers, to share and process claims information in the cloud according to the CMS analytical algorithms. Issuers maintain complete control of their proprietary claims and pricing data; CMS has visibility to the outputs, but not the inputs, of the algorithms. Additionally, 135 issuers elected to participate as AWS-deployed servers, using a fully automated environment provisioning process that has successfully and securely processed the issuers’ data without requiring internal infrastructure investment. The other issuers used an on-premise deployment model, which still took advantage of the same software images and upgrade processes.

The EDGE system enables CMS to create a level playing field for all issuers. It provides consistent software and data version management across the universe of independent installations. EDGE simplifies and expedites deployment for issuers, reducing time from several days in a standard software distribution and configuration model to as little as 15 minutes, while enabling hands-free software upgrades and execution of remote commands.

Program Integrity

Millions of Americans count on the integrity of their enrollments and tax subsidy calculations. And the 800 issuers count on the integrity of how their plans are displayed on HealthCare.gov, how their enrollments are processed, and how they get paid for reinsurance and risk adjustment.

Accenture worked with CMS to build in program integrity. We built an entirely new system for policy-based payments, to make sure the tax subsidy payments to issuers resolved to the penny, for each and every enrollment in the FFM. In addition, CMS hired an independent contractor and auditor to vet the EDGE system for reinsurance and risk adjustment payments. That contractor determined that a full 100 percent of the billions of dollars in payments was accurately calculated.

At a detailed level, we built in logging and auditing capabilities for all key transactions. And we worked with CMS to build a robust security program, which proactively detects potential vulnerabilities. Our joint security team achieved Authority to Operate (ATO) on schedule, for all of the systems that Accenture built and maintained.

Technology. Delivery. Discipline.

Rescuing the website required immediately instilling technology discipline, along with significant investment in automation and tooling. To improve the fully manual test and release process, the Accenture team created fully automated, regression test suites at all levels:

• 15,500-unit test cases in Java
• 1,400 functional tests, end-to-end, for the user interfaces
• 15,400 functional data validation and batch validation tests
• 990 transactions tested in a comprehensive, end-to-end performance suite, reflecting a real-world transaction mix.

This comprehensive regression suite runs on a regular basis, and provides automated emails and executive dashboards for monitoring. The automated functional testing consists of more than 300,000 test steps and is typically executed five times per week. The automation of this functional testing saves over 50,000 hours per year in manual testing. This high level of automation has allowed the team to dramatically improve the quality, reliability and speed of new software builds.

Accenture also created automated build and deployment scripts, not only for the software, but also for the software environments. Development and testing teams can now “click a button” to automatically provision new environments in AWS. They can also click a button to deploy software releases to those environments. This includes checking out the versioned code from the repository, building the software, checking code quality, running it through a battery of automated tests, validating performance and moving the software items into the environment. We have moved from fully manual processes, which are time-intensive and error-prone, to a fully automated Development and Operations (DevOps) capability.

Moreover, Accenture created a robust operations monitoring stack. The stack includes a suite of monitoring tools and dashboards that integrate with:

• Automated paging software for the operations team, with built-in, automated escalations if a team member does not respond quickly
• Detailed transaction and log information
• Interactive chat tools used by the operations teams, where the monitoring tools also posts automated updates

In addition to monitoring live traffic on the site, down to the level of individual transactions, the stack also monitors the performance of synthetic transactions. The synthetic transactions run every few minutes, around the clock, to proactively uncover production issues before consumers do.

Multi-Speed IT

Beyond just the technology, Accenture also brought methodology discipline to the program. This included introducing delivery techniques to make sure CMS realized business benefits incrementally, over time, without having to wait for a full-blown or big bang solution.

Additionally, Accenture and CMS together created an enterprise-ready agile approach, for new development. Accenture combined its proven, enterprise agile methods with CMS’ Expedited Life Cycle (XLC) system development process. This approach blends the benefit of agile with government requirements for documentation, auditing, traceability and independent verification. In sum, this approach enabled efficient delivery with sound risk management and proven software delivery practices.

CMS embraced agile, with actively engaged product owners, who represented multiple stakeholders and made decisions quickly. After each three- to four-week sprint, Accenture demonstrated the updated software for CMS acceptance, giving CMS real-time visibility into progress of the software build. This built confidence and allowed agile teams to incorporate feedback and re-prioritize along the way. The program’s use of agile has also helped improve collaboration between IT and the business on strategic system initiatives.

Architecture Simplification

HealthCare.gov , as originally built, had technical debt built up from the technical complexity of the software. The Accenture team innovated and simplified the technical plumbing of HealthCare.gov and other related systems.

This had multiple business benefits:

• Accenture saved CMS more than $4M per year in software licenses, data center costs and other labor; Accenture significantly reduced the number of servers required
• The website is now faster; response times went from roughly half a second to a quarter of a second
• The website is now easier to maintain and extend

This approach was so successful that another contractor is considering doing architecture simplification on its software, for similar benefits.

Cloud First Approach

Accenture delivered the new EDGE software in the cloud, as mentioned. That is not the only use of the cloud—Accenture has taken a “cloud first” approach, wherever using the cloud makes sense and meets CMS’ security needs.

For example, the suite of automated functional tests is run from servers in the cloud, which can scale up or scale down, as necessary. As another example, the new environments for new software capabilities are deployed in AWS. Accenture made it as simple as literally pushing a button: provisioning a new environment plus the COTS middleware plus the application software is fully automated.

Cloud-Based CRM

Accenture recommended using salesforce.com to quickly build a Customer Relationship Management (CRM) solution that would help improve collaboration and outreach with issuers. The initial prototype was deployed within weeks; the full solution was complete and operational in four months.

The unified CRM solution replaced dozens of disassociated email templates, contact lists and Excel spreadsheets. Email generation improved from 30 minutes to 10 minutes. Issuer lookup and update time improved from 20 minutes to 1 minute. Reporting time improved from 30 minutes to 2 minutes. These results all occurred on a platform that is now more accurate and easier to use.

Cloud-Based DevOps

Accenture implemented full DevOps processes and technologies to accelerate development and deployment. Automated tasks across environments include code check-out, build and packaging; code quality scanning; security scanning; unit testing, functional testing and initial performance testing; software deployment; and release smoke testing.

The team used tools such as Junit, github, Amazon Web Services, Splunk, JIRA Software (including Jira, Confluence and HipChat), Fortify, Python and Bash to support continuous delivery.

The DevOps approach has significantly reduced manual errors, improved software release quality and allowed the team to deliver faster.

Transforming HealthCare.gov required a shift in culture. Team members adopted a “one team, one goal” mindset and created a blame-free environment, where everyone would succeed or fail together.

It was not a client/vendor relationship that had clear lines of separation. Rather, it was one cohesive team that worked collaboratively and transparently. Everyone was open about schedules, risks and defects, and all worked together to share knowledge and solve problems proactively. Instead of finger pointing, people were rewarded during daily meetings for candidly acknowledging, “This isn’t working…and here’s what we need to do to fix it.”

CMS is now recognized as a leader among federal agencies for pivoting to the new: using agile techniques, DevOps and next-generation architectures in the cloud to deliver meaningful digital experiences for consumers.

Value delivered

The joint team, along with CMS, made significant progress quickly. Highlights of the work include:

Mobilized 500 people with the requisite industry, functional and technical skills in six weeks. We created a transition plan with the incumbent that included 450 formal knowledge transfer sessions.

Closed critical defects, resulting in a significant reduction of error rates.

Delivered 256 releases, 99 percent on time with the remainder delivered no more than 7 days after the planned release date.

Generated more than 18 million notices, 2.9 million inbound account transfers and 1.7 million outbound account transfers during the 2015 enrollment period.

Improved the loading time for healthcare plans by 98 percent for small business consumers; load time went from ~200 plans per day to ~420 plans per hour.

Implemented the first-ever automated policy-based payments process, for program integrity.

Worked with CMS and issuers directly to successfully on-board and conduct outreach for more than 800 issuers for the Risk Adjustment / Reinsurance program, using the EDGE solution.

Implemented Salesforce in a matter of weeks, ensuring more than 1,200 issuers received high levels of assistance in tasks related to their acceptance of marketplace policies and tools.

The stability of the website and the program contributed to year-over-year increases in enrollment. For 2016 open enrollment, HealthCare.gov enrolled & re-enrolled 9.6 million consumers in 38 states.

Related capabilities

Digital government transformation, applied intelligence for federal.

• Project Management
• Application Development
• Collaboration
• Cloud Virtualization
• Enterprise Apps
• Infrastructure
• News & Trends
• Case Studies
• Books for CIOs

Project Management Lessons From HealthCare.Gov

By Eric Thomas

President Obama recently admitted that Americans are facing difficulties attempting to sign up for insurance coverage via the healthcare.gov Website . His disappointment has been echoed by both proponents and opponents of the underlying legislation. The Department of Health and Human Services (DHHS) explained that users “have had trouble creating accounts and logging in to the site, while others have received confusing error messages, or had to wait for slow page loads or forms that failed to respond in a timely fashion.”

Ever since these problems first surfaced after the go-live date of Oct. 1, industry insiders and technologists have proposed a litany of reasons that could have led to the current situation. Reasons included everything from too much focus on external design and not enough focus on integration; poor testing practices; inefficient government contracting laws; not using the best resources; and overreliance on antiquated system development methodologies. And while all of these may be true, there are other, lesser reported project management lessons that CIOs can learn from the unsuccessful rollout of HealthCare.Gov.

Here are four key takeaways:

1.    Communicate the bad news early on. The project to develop HealthCare.Gov exceeded initial cost projections . And it was not just by a little bit, but perhaps by as much as 300 percent. Basic project management would flag this as a significant problem. However, the ultimate customers, the American people, were not informed. Proactively communicating overruns or other problems with the project may have damaged credibility and hurt politically in the short run, but it would have been better than what did transpire.

2.    Acknowledge your risks and manage them. Some of the biggest risks with this implementation would involve schedule delays, cost overruns, privacy and security issues, and functionality. Mitigating those risks would have to be the key priority for such a highly visible rollout. But with as many as 15 contractors working on the development, risks could get lost in the vast network of contractor support. Hopefully, the contractors themselves are not responsible for reporting and managing risks since that would present a blatant conflict of interest. (While an outside management consultant did warn of problems with the site, these warnings were largely unheeded.) One of the site’s core problems was inadequate management of these risks, which subsequently spiraled out of control.

3.    Beta can be your best friend. A beta site might not have been feasible for HealthCare.gov, but a minimally viable solution would have provided the environment needed for developers to test and improve the integration and the Website’s underlying technologies. Wherever possible, CIOs should take advantage of user willingness to freely test beta developments.

4.    Know who is in charge. The Website was managed out of the Centers for Medicare and Medicaid Services (CMS), which is an agency under DHHS. There is an administrator for CMS and a secretary for DHHS. There is an Office of Information Services within CMS and a secretary for information technology at DHHS. But don’t forget the White House has a CTO of its own. The Office of Management and Budget, which is part of the Executive Office, has a CIO. And the Government Accountability Office, which is part of the legislative branch, has a director of information technology management. In this authority-dense environment, it is no wonder there could be a lack of clearly defined leadership.

Ultimately, there should be a project management office or, at the least, a single point of contact through which all issues about scope, risk, cost and requirements can be collated, communicated and managed. These persons should be independent from the development team and, if necessary, have the authority and access to escalate issues to the key stakeholders. Hopefully, these project management lessons can be employed for the benefit of your next big rollout. 

Get the Free Newsletter!

Subscribe to Daily Tech Insider for top news, trends, and analysis.

Latest Articles

Storage vulnerabilities: the neglected cybersecurity frontier, 7 principles of quality management, domo vs tableau: which is the better bi solution, related articles, what do amazon, microsoft, meta, and ibm have in common tape storage, what does quantum computing mean for it, best business travel items: 11 business travel essentials, solving the video surveillance retention challenge .

CIO Insight offers thought leadership and best practices in the IT security and management industry while providing expert recommendations on software solutions for IT leaders. It is the trusted resource for security professionals who need to maintain regulatory compliance for their teams and organizations. CIO Insight is an ideal website for IT decision makers, systems integrators and administrators, and IT managers to stay informed about emerging technologies, software developments and trends in the IT security and management industry.

Advertisers

Advertise with TechnologyAdvice on CIO Insight and our other IT-focused platforms.

• IT Management
• IT Strategy
• Privacy Policy
• California – Do Not Sell My Information

Property of TechnologyAdvice. © 2024 TechnologyAdvice. All Rights Reserved Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

Integrating science‑based and local ecological knowledge: a case study of mangrove restoration and rehabilitation projects in the Philippines

• Marquez, Gian Powell B.
• Olavides, Ronald Dionnie

Mangrove forest is an ecosystem‑based solution for disaster risk reduction in the Philippines, but its historical deforestation has hampered its capacity to protect coastal communities. With the increasing occurrence of storm surge in the Philippines, mangrove reforestation projects have received renewed attention, but many have failed. Community participation was deemed to be essential in those projects that did well. Hence, this paper examines successful mangrove restoration and rehabilitation projects in the Philippines to find out how community participation contributed to the accomplishments. The study found that while the transfer of science‑based ecological knowledge from project managers to the community is an important factor in ensuring successful initial planning and implementation, its integration into existing local ecological knowledge—'localisation' of science‑based ecological knowledge or hybrid ecological knowledge formation—helped to facilitate long‑term community‑based mangrove management beyond project duration by empowering community members and enabling project acceptance and ownership. Still, continuous local institutional support is a necessary anchor for community resilience.要旨マングローブ林は、フィリピンにおける防災のための生態系ベースのソリューションだが、歴史的な森林伐採により、沿岸地域社会を保護するマングローブ林の力が弱まっている。フィリピンで高潮の発生が増加しているため、マングローブ再植林プロジェクトが再び注目を集めたが、多くは失敗に終わった。マングローブプロジェクトの成功には地域社会の参加が不可欠であると考えられていた。そこで筆者らは、コミュニティの参加がこの成功にどのように貢献したかを調べるために、フィリピンで成功したマングローブの修復および再生プロジェクトを調査した。科学に基づいた生態学的知識 (SEK) をプロジェクト マネージャーからコミュニティに移転することが、初期計画と実装を確実に成功させる上で重要な要素である一方で、SEK と既存の地域の生態学的知識 (LEK) の統合、つまり SEKまたはハイブリッド生態学的知識 (HEK) の形成の「ローカライズ」がコミュニティに力を与え、プロジェクトの受け入れと所有権を可能にすることで、プロジェクト期間を超えた長期的なコミュニティベースのマングローブ管理を促進するのに役立っていることが明らかになった。継続的な地域の制度的支援は、依然として地域社会の回復力に必要な基盤である。摘要红树林是菲律宾基于生态系统且减少灾害风险的解决方案, 但历史上的森林砍伐削弱了其保护沿海社区的能力。随着菲律宾风暴潮频繁发生, 红树林恢复造林项目重新受到关注, 但许多项目都以失败告终。社区参与被认为是对于红树林项目的成功至关重要。因此, 我们研究了菲律宾成功的红树林恢复和重建项目, 以了解社区参与如何促成这一成功。我们发现, 虽然将基于科学的生态知识 (SEK) 从项目经理转移到社区是确保成功进行初步规划和实施的重要因素, 但将 SEK 与现有的当地生态知识 (LEK) 相结合——SEK 的"本土化"或混合生态知识 (HEK) 的形成——有助于通过赋予社区权力并使项目接受和拥有权来促进项目持续时间之外的长期社区红树林管理。尽管如此, 当地机构持续的支持仍然是社区恢复力的必要支柱。