We’ve asked this question before, about other agencies and departments of the Executive Branch of the U.S. Government. Sometimes we imagine asking the question wryly, or archly, or sometimes with a cynical and knowing smirk. Other times we imagine asking the question in a plaintive or even despairing voice. We see Grandma Jane from Wichita or Bismarck, looking at the sky in supplication. We see Joe Six-Pack, wondering why his hard-earned taxes don’t get spent wisely.

(We are, perhaps, overly dramatic.)

In other words, this is the latest installment in an on-going series that (hopefully) probes the ability of the Federal government to manage its contractors and execute its programs. A quick review of our News Archive will lead you to other articles in the series, each starting with the word “Why”—which you can imagine in any voice you like.

Today is the Federal Bureau of Investigation’s turn in the barrel. What brings this respected law enforcement agency into our cross-hairs? For starters, try this article at FederalTimes.com. It reports that the FBI’s “Sentinel” project is “is $100 million over budget and nearly two years behind schedule,” according to a report released by the Justice Department’s Office of Inspector General (DOJ OIG), which has cognizance over the FBI.

Here’s a link to the DOJ OIG Audit Report.

As FederalTimes.com reported, the Sentinel project “is intended to replace the FBI's outdated Automated Case Management System. When fully implemented, it will provide FBI agents and analysts with a web-based case management system to manage evidence, automate document review and approval processes, and use expanded search capabilities.”

The FBI expected to implement Sentinel in four overlapping phases, each lasting 12 to 16 months. Each phase was intended to provide a stand-alone set of capabilities upon which subsequent phases would add further capabilities. The project’s first two phases were budgeted at $306 million; the FBI spent $405 million.

The entire project was scheduled to be completed by December 2009 at a total cost of $425 million. The project was rebaselined, and the at-completion estimate was increased by $26 million, based on project status in October 2007, when the first phase was completed. Now the project is scheduled to be completed in September 2011 for a total cost of $451 million.

In other words, the FBI has asserted (via its budget) that it can complete the final two phases of the work for between $20 and $46 million. And, in the word of the DOJ OIG, “we believe that the most challenging development work for Sentinel still remains.”

However, MITRE (who was hired by the FBI to independently assess the project) calculated that it will take the FBI another $351 million to complete the project—for an at-completion cost of $756 million (which would represent an overrun of 178% against the original project baseline budget of $425 million). Oh, and MITRE said it will take the FBI another six years to complete the project.

Naturally, the FBI doesn’t agree with MITRE’s assessment. According to the FederalTimes.com article—

‘We believe that the interim report does not accurately reflect the FBI's management of the Sentinel project, and fails to credit the FBI with taking corrective action to keep it on budget,’ according to a FBI news release. The agency noted that thousands of its employees are using the system to draft interview reports, send leads and manage their caseloads. … The FBI says Mitre estimate's assumes a ‘worst case scenario for a plan that we are no longer using.’

One of the planned corrective actions (according to the DOJ OIG) is that “the FBI will assume direct management of Sentinel development and significantly reduce the role of Lockheed Martin in developing Sentinel.” More on that innovative approach in a bit.

What went wrong? As usual, there were a number of decisions and factors that led the FBI (and its prime contractor, Lockheed Martin) to this point. Let’s look at the DOJ OIG report for some details.

On December 2, 2009, the FBI conditionally accepted delivery of Sentinel’s Phase 2, Segment 4, which included three of the eight electronic forms expected to be delivered in Segment 4, and their associated workflow. The FBI conditionally accepted this segment despite knowing that what was delivered had serious performance and usability issues and had received overwhelmingly negative user feedback during testing with FBI agents and analysts. As a result, the FBI did not deploy Segment 4 to the FBI’s agents and analysts when it conditionally accepted it in December 2009.

Then, on March 3, 2010, the FBI issued a partial stop-work order to Lockheed Martin for portions of Phase 3 and all of Phase 4, and also returned Phase 2, Segment 4 to the development phase from the operations and maintenance phase. FBI officials stated that the purpose of the partial stop-work order for Phases 3 and 4 was to focus Lockheed Martin’s efforts on delivering Phase 2, Segment 4 in a form that the FBI would find acceptable.

On July 26, 2010, the FBI deployed Segment 4 to FBI agents and analysts.

We can see from the foregoing that the project experienced an eight-month delay because of user satisfaction and system usability issues. But what we also see is a Government project management team that is indecisive and lacks leadership. First, they “conditionally accepted” the deliverables and three months later decided to “return” the accepted items back to “the development phase” for redesign and rework. Another five months passed before Lockheed Martin could rework the electronic forms to the satisfaction of the users—which indicates (to us) that there was some very extensive rework done. I.e., it was not a quick fix. So why were these forms initially accepted? One can only wonder.

What else did the DOJ OIG report?

As of August 1, 2010, the FBI had not decided on an approach for completing Sentinel, and FBI officials did not provide the OIG with detailed descriptions of the alternatives under consideration for completing Sentinel. At that time, however, the FBI Chief Technology Officer stated that the alternatives under consideration would allow the FBI to complete Sentinel within its $451 million budget by re-using portions of successful FBI IT projects, including Sentinel, taking advantage of technological advances and industry best practices, and increasing the reliance on FBI personnel to develop Sentinel. Yet, the Chief Technology Officer acknowledged that his estimate did not include the cost of maintaining Sentinel for 2 years after its completion – costs which had been included in all previous Sentinel budgets.

Yeah, about the FBI’s official at-completion estimate: We understand that one way to come in on budget it to forget to account for certain costs in the estimate; however, that’s not the way it’s supposed to be done.

Here are some other facts about the FBI’s project management to consider, as reported by the DOJ OIG.

• [The DOJ OIG] found that the FBI has either limited in scope or eliminated several project management activities that were designed to help it monitor the progress of Sentinel’s development. … For example, in December 2009, the FBI discontinued Sentinel’s Project Health Assessments. Performed by the FBI’s Enterprise Requirements and Assessment Unit, these assessments provided an independent assessment of Sentinel’s cost, schedule, and scope.

• The FBI stopped EVM reporting for Phase 2 in December 2009, and the EVM reporting for Phase 3 has not complied with OMB guidance since May 2010. EVM is an important risk management tool for major capital investments that measures the performance of a project by producing cost estimates, evaluating progress, and analyzing cost and schedule performance trends. … The FBI said that EVM and Project Health Assessments were discontinued as a result of the March 2010 partial stop work order because there was no schedule or baseline against which the FBI could measure its progress. While we [the DOJ OIG] agree there was not baseline to measure against, according to DOJ policy on implementing EVM, once the FBI realized that Sentinel was significantly behind schedule and over budget, the FBI should have established a new baseline for measuring Sentinel’s cost and schedule performance.

Going forward, the FBI told the DOJ OIG that it had a plan to assume more direct control of the project, so as to bring it to completion on-budget and on-schedule. According to the DOJ OIG—

Overall, the FBI plans to reduce the number of contract employees working on Sentinel from approximately 220 to 40. The FBI said that, at the same time, the number of FBI employees assigned to the project will also decrease from 30 to 12. The FBI asserted that this new, agile approach will streamline decision-making processes and allow the FBI to deliver Sentinel within budget.

The DOJ OIG enumerated nine concerns it had with the FBI’s proposed approach. Moreover, it reported that—

In September 2010 Carnegie Mellon’s Software Engineering Institute (SEI) performed an independent review of the FBI’s new approach for completing Sentinel. The SEI stated that the FBI’s decision to attempt an agile approach is ‘a positive step toward improvement over the prior development approach.’ However, SEI expressed similar concerns to ours, including the largely undocumented details of the FBI’s new plan, the FBI’s inexperience in using an agile development methodology, and the unknowns concerning the viable state of and path forward for the technical design of Sentinel.

To conclude, the DOJ OIG said that it had “significant concerns and questions about the ability of this new approach to complete the Sentinel project within budget, in a timely fashion, and with similar functionality as what the Sentinel project previously sought to provide.” Other than that, they were fine with it.

From our perspective, this is a cautionary tale about lack of project management skills, lack of rigor in measuring project status and using that information to make an accurate estimate-at-completion, and a corrective action plan that smacks of an alternate reality. Other than that, we’re fine with it.

On a more positive and less sarcastic note, this article is our 300^th blog post on this website. That’s about three novels worth of over-the-top strident and politically incorrect hyperbole. Thanks for your support.

We are bloggers, not journalists. And what’s more, we blog about things that your average Joe Six-Pack wouldn’t waste 30 seconds (the length of the average TV commercial) reading. That said, we also like to catch up on previous stories. Thus: the UPDATE prefix. Here are updates to three previous blog articles.

We reported on the suspension of GTSI Corporation here. GTSI was suspended because (in the words of the GSA), “… the evidence shows that GTSI was an active participant in a scheme that resulted in contracts set-aside for small businesses being awarded to ineligible contractors and with contracts not being performed in accordance with applicable law, regulations and contract terms.” Since GTSI derived roughly three-quarters of its total revenue from sales to the Federal government, this was (shall we say?) a kind of a big deal to the company.

About three weeks later, GTSI’s suspension was lifted. What did the company agree to in order to get the “death sentence” lifted? This article at WashingtonTechnology.com listed some of the terms of the agreement between GTSI and the Small Business Administration (SBA). It reported—

• Over the next three years, the SBA will have access to the company’s books, records, and other documents.

• An independent monitor will be appointed. The monitor will have “full access to inspect the company on an ongoing basis and report to SBA without interference from GTSI.”

In addition, the article reported—

… the agreement requires GTSI to give the monitor management-style office space and it must pay, among other things, all monitor fees, retainers and other reimbursements, including any legal fees….

Inside the company, GTSI must name an employee as ethics officer and adopt a code of ethics

The agreement demands other high-profile moves. More specifically, it forces out GTSI’s CEO Scott Friedlander and general counsel Charles DeLeon. It also suspends three top company employees: Tom Kennedy, vice president of civilian sales and general manager; Scott Schmader, senior sales manager; and Patrick Berg, program manager, until the agreement ends.

So the company’s CEO and general counsel have been terminated. Certain other executives have been “suspended” for three years (with pay). But that’s not all. The article noted that, “the SBA inspector general’s office will continue to probe SBA’s charges against GTSI for using small-business prime contractors as a front to funnel work and revenue back to itself.” In addition, “the government still reserves the right to extend the scope of the case if it comes across any additional revealing facts.” Moreover, “the agreement would remain in effect even if GTSI were to file for bankruptcy.” Which would have been a strong possibility, had the suspension continued for much longer.

A follow-up article by FederalTimes.com, link here, discussed the dire straits facing GTSI. It reported—

Even though GTSI believed that it was abiding by small-business procurement rules, the suspension was costing almost $2 million a day, [former CEO Scott Friedlander] said in an Oct. 21 interview. By the second week, alarm over the possible repercussions had spread from the firm's work force to its banks and corporate partners, he said.

‘We had to lift it [the suspension] to save the company or the company would have gone into financial ruin,’ he said, throwing some 530 people out of work. ‘I just think leaders have to lead and I had to do what I had to do.’ SBA officials never gave the company a chance to make its case, Friedlander said.

Next let’s talk about “Camera Guy”. We previously reported the sad case of John Feeney, Sr., here. Dubbed “Camera Guy” by one of our regular readers, Mr. Feeney pleaded guilty one count of mail fraud for using his position with BAE Systems Training Services to order nearly $500,000 worth of camera lenses and video equipment on the company’s tab, and then sold the gear on the internet “for a profit”. (Well, yes. Since his cost was zero any funds he got would be a profit. We are cost accountants. We can do that kind of math.)

Mr. Feeney is going to jail. He must have had a good attorney, because he was sentenced to serve a paltry 18 months in prison, according to this Department of Justice press release. In addition to serving his time, the DOJ release also reports that Feeney “was also ordered … to pay restitution of $464,819 to the Department of Defense and $11,604 to BAE Systems Training Services Inc. (BAE). In addition … Feeney [will] serve three years of supervised release following his prison term.”

Our final vignette in this trilogy concerns a more abstract problem, that of the dependence of the United States on China as the near sole supplier for rare earth metals. We reported this situation here, noting that an April 2010 GAO report was “rather alarming.” Even though the United States has rare earth ore deposits that it could mine and process, between 1985 and 2005, the U.S. essentially exited the rare earth production process, ceding the market in its entirety to China. As we said at the time—

We don’t want to be overly alarmist here—but this is a potentially very serious problem that could affect a number of major defense acquisition programs, from the DDG-51’s Hybrid Electric Drive Ship Program to the M1A2 Abrams Tank’s reference and navigation system.  We encourage DOD’s Industrial Policy Directorate to get moving on this potential supply chain ‘interruption’.

An October 20, 2010, article by Bloomberg.com reported that, “Rare-earth prices have jumped as Chinese export quotas crimped worldwide supplies for the elements used in the manufacture of disk drives, wind turbines and smart bombs.” The article reported—

Prices have climbed sevenfold in the last six months for cerium oxide, which is used for polishing semiconductors, and other elements have more than doubled, according to Metal-Pages Ltd. in London, which tracks rare-earth prices. … China reduced its second-half export quota for the minerals by 72 percent in July. It is now further restricting exports, according to industry participants.

Contributing to the rise in prices is an expectation of further restrictions.

‘It’s pretty frightening that there may be a gap where U.S. industry pays an extraordinary price,’ U.S. Representative Mike Coffman, a Colorado Republican, said in an interview. He said U.S. rare-earth mining isn’t likely to resume until at least late 2012 at a mine in Mountain Pass, California. …

Companies and government officials have already begun to react to the threat of a shortage of the elements. The U.S. rare-earth mine in Mountain Pass, California, shut down most operations in 2002. Molycorp, Inc., which owns the mine, plans to reopen it, and [its CEO] said this week that it may double the planned capacity to 40,000 metric tons. Glencore International AG, the world’s biggest commodities trader, also said this week that it would try to restart the Pea Ridge rare-earth mine in Missouri. …

In Germany, the government yesterday adopted a strategy to secure supply of raw materials including rare earths. Chancellor Angela Merkel said last week that it’s “urgently necessary” to boost European investment in eastern Europe and Central Asia to counter expanding Chinese interest in rare minerals.

We don’t have any witty conclusion to this collection of updates. As always, stay tuned for further developments.