We’ve asked this question before, about other agencies and departments of the Executive Branch of the U.S. Government. Sometimes we imagine asking the question wryly, or archly, or sometimes with a cynical and knowing smirk. Other times we imagine asking the question in a plaintive or even despairing voice. We see Grandma Jane from Wichita or Bismarck, looking at the sky in supplication. We see Joe Six-Pack, wondering why his hard-earned taxes don’t get spent wisely.

(We are, perhaps, overly dramatic.)

In other words, this is the latest installment in an on-going series that (hopefully) probes the ability of the Federal government to manage its contractors and execute its programs. A quick review of our News Archive will lead you to other articles in the series, each starting with the word “Why”—which you can imagine in any voice you like.

Today is the Federal Bureau of Investigation’s turn in the barrel. What brings this respected law enforcement agency into our cross-hairs? For starters, try this article at FederalTimes.com. It reports that the FBI’s “Sentinel” project is “is $100 million over budget and nearly two years behind schedule,” according to a report released by the Justice Department’s Office of Inspector General (DOJ OIG), which has cognizance over the FBI.

Here’s a link to the DOJ OIG Audit Report.

As FederalTimes.com reported, the Sentinel project “is intended to replace the FBI's outdated Automated Case Management System. When fully implemented, it will provide FBI agents and analysts with a web-based case management system to manage evidence, automate document review and approval processes, and use expanded search capabilities.”

The FBI expected to implement Sentinel in four overlapping phases, each lasting 12 to 16 months. Each phase was intended to provide a stand-alone set of capabilities upon which subsequent phases would add further capabilities. The project’s first two phases were budgeted at $306 million; the FBI spent $405 million.

The entire project was scheduled to be completed by December 2009 at a total cost of $425 million. The project was rebaselined, and the at-completion estimate was increased by $26 million, based on project status in October 2007, when the first phase was completed. Now the project is scheduled to be completed in September 2011 for a total cost of $451 million.

In other words, the FBI has asserted (via its budget) that it can complete the final two phases of the work for between $20 and $46 million. And, in the word of the DOJ OIG, “we believe that the most challenging development work for Sentinel still remains.”

However, MITRE (who was hired by the FBI to independently assess the project) calculated that it will take the FBI another $351 million to complete the project—for an at-completion cost of $756 million (which would represent an overrun of 178% against the original project baseline budget of $425 million). Oh, and MITRE said it will take the FBI another six years to complete the project.

Naturally, the FBI doesn’t agree with MITRE’s assessment. According to the FederalTimes.com article—

‘We believe that the interim report does not accurately reflect the FBI's management of the Sentinel project, and fails to credit the FBI with taking corrective action to keep it on budget,’ according to a FBI news release. The agency noted that thousands of its employees are using the system to draft interview reports, send leads and manage their caseloads. … The FBI says Mitre estimate's assumes a ‘worst case scenario for a plan that we are no longer using.’

One of the planned corrective actions (according to the DOJ OIG) is that “the FBI will assume direct management of Sentinel development and significantly reduce the role of Lockheed Martin in developing Sentinel.” More on that innovative approach in a bit.

What went wrong? As usual, there were a number of decisions and factors that led the FBI (and its prime contractor, Lockheed Martin) to this point. Let’s look at the DOJ OIG report for some details.

On December 2, 2009, the FBI conditionally accepted delivery of Sentinel’s Phase 2, Segment 4, which included three of the eight electronic forms expected to be delivered in Segment 4, and their associated workflow. The FBI conditionally accepted this segment despite knowing that what was delivered had serious performance and usability issues and had received overwhelmingly negative user feedback during testing with FBI agents and analysts. As a result, the FBI did not deploy Segment 4 to the FBI’s agents and analysts when it conditionally accepted it in December 2009.

Then, on March 3, 2010, the FBI issued a partial stop-work order to Lockheed Martin for portions of Phase 3 and all of Phase 4, and also returned Phase 2, Segment 4 to the development phase from the operations and maintenance phase. FBI officials stated that the purpose of the partial stop-work order for Phases 3 and 4 was to focus Lockheed Martin’s efforts on delivering Phase 2, Segment 4 in a form that the FBI would find acceptable.

On July 26, 2010, the FBI deployed Segment 4 to FBI agents and analysts.

We can see from the foregoing that the project experienced an eight-month delay because of user satisfaction and system usability issues. But what we also see is a Government project management team that is indecisive and lacks leadership. First, they “conditionally accepted” the deliverables and three months later decided to “return” the accepted items back to “the development phase” for redesign and rework. Another five months passed before Lockheed Martin could rework the electronic forms to the satisfaction of the users—which indicates (to us) that there was some very extensive rework done. I.e., it was not a quick fix. So why were these forms initially accepted? One can only wonder.

What else did the DOJ OIG report?

As of August 1, 2010, the FBI had not decided on an approach for completing Sentinel, and FBI officials did not provide the OIG with detailed descriptions of the alternatives under consideration for completing Sentinel. At that time, however, the FBI Chief Technology Officer stated that the alternatives under consideration would allow the FBI to complete Sentinel within its $451 million budget by re-using portions of successful FBI IT projects, including Sentinel, taking advantage of technological advances and industry best practices, and increasing the reliance on FBI personnel to develop Sentinel. Yet, the Chief Technology Officer acknowledged that his estimate did not include the cost of maintaining Sentinel for 2 years after its completion – costs which had been included in all previous Sentinel budgets.

Yeah, about the FBI’s official at-completion estimate: We understand that one way to come in on budget it to forget to account for certain costs in the estimate; however, that’s not the way it’s supposed to be done.

Here are some other facts about the FBI’s project management to consider, as reported by the DOJ OIG.

• [The DOJ OIG] found that the FBI has either limited in scope or eliminated several project management activities that were designed to help it monitor the progress of Sentinel’s development. … For example, in December 2009, the FBI discontinued Sentinel’s Project Health Assessments. Performed by the FBI’s Enterprise Requirements and Assessment Unit, these assessments provided an independent assessment of Sentinel’s cost, schedule, and scope.

• The FBI stopped EVM reporting for Phase 2 in December 2009, and the EVM reporting for Phase 3 has not complied with OMB guidance since May 2010. EVM is an important risk management tool for major capital investments that measures the performance of a project by producing cost estimates, evaluating progress, and analyzing cost and schedule performance trends. … The FBI said that EVM and Project Health Assessments were discontinued as a result of the March 2010 partial stop work order because there was no schedule or baseline against which the FBI could measure its progress. While we [the DOJ OIG] agree there was not baseline to measure against, according to DOJ policy on implementing EVM, once the FBI realized that Sentinel was significantly behind schedule and over budget, the FBI should have established a new baseline for measuring Sentinel’s cost and schedule performance.

Going forward, the FBI told the DOJ OIG that it had a plan to assume more direct control of the project, so as to bring it to completion on-budget and on-schedule. According to the DOJ OIG—

Overall, the FBI plans to reduce the number of contract employees working on Sentinel from approximately 220 to 40. The FBI said that, at the same time, the number of FBI employees assigned to the project will also decrease from 30 to 12. The FBI asserted that this new, agile approach will streamline decision-making processes and allow the FBI to deliver Sentinel within budget.

The DOJ OIG enumerated nine concerns it had with the FBI’s proposed approach. Moreover, it reported that—

In September 2010 Carnegie Mellon’s Software Engineering Institute (SEI) performed an independent review of the FBI’s new approach for completing Sentinel. The SEI stated that the FBI’s decision to attempt an agile approach is ‘a positive step toward improvement over the prior development approach.’ However, SEI expressed similar concerns to ours, including the largely undocumented details of the FBI’s new plan, the FBI’s inexperience in using an agile development methodology, and the unknowns concerning the viable state of and path forward for the technical design of Sentinel.

To conclude, the DOJ OIG said that it had “significant concerns and questions about the ability of this new approach to complete the Sentinel project within budget, in a timely fashion, and with similar functionality as what the Sentinel project previously sought to provide.” Other than that, they were fine with it.

From our perspective, this is a cautionary tale about lack of project management skills, lack of rigor in measuring project status and using that information to make an accurate estimate-at-completion, and a corrective action plan that smacks of an alternate reality. Other than that, we’re fine with it.

On a more positive and less sarcastic note, this article is our 300^th blog post on this website. That’s about three novels worth of over-the-top strident and politically incorrect hyperbole. Thanks for your support.

< Prev		Next >