Nixon admits faults, but defends VITA

The man tasked with reforming the Virginia Information Technology Agency is facing his greatest challenge. Sam Nixon, the former Delegate from Chesterfield has only been in charge of the Commonwealth’s IT structure for a few months, but he is already battling to defend its utility.

Last week, a memory board in the system’s main storage system failed. It’s back up, which runs redundant to the main system went down simultaneously.  The result was a catastrophic outage that shut down key operations at 26 state agencies. 13% of the state’s storage servers were knocked offline.

Nixon makes no excuses for the problem, calling it “unacceptable”.  Nixon said the problem was so rare that it only happens 1 out of every 1 billion hours of computing time.

At the center of the VITA outage is the state’s monster contractor, Northrop Grumman. The huge private company has been the source of controversy since they took over the centralized computer functions for the state. As a Delegate, Nixon was one of the most vocal legislators calling for reform of the VITA/Northrop Grumman arrangement. He got that reform in the spring and then was given the task of fixing what was wrong.

The system that failed was in place before Nixon arrived. It is one of many things he is still working to fix. He told me he is prepared to hold Northrup Grumman accountable. That includes demanding compensation for the Commonwealth if is determined the company’s negligence led to the outage.

Despite the controversy and the problems, Nixon remains committed to the idea that a public-private partnership that centralizes state information technology works. He compares the delivery of this service to the delivery of public utilities.  Nixon is prepared to make the case to anyone who feels otherwise. However, he realizes that his agency needs to live up to the expectations he and Governor Bob McDonnell have set.

You can read and watch my story with Sam Nixon on NBC12.com.

Meanwhile, Governor McDonnell has vowed to hold Northrop Grumman accountable.  Just like Nixon, he called the outage  “disappointing and unacceptable”.  His complete statement on the issue can be found after the jump:

————————————————-

VITA Recovery Efforts Move Towards Completion

All Agencies Now Have Access to Computer Systems

DMV Returns to Full Operations Tomorrow; Governor Directs Agency to Extend Hours in Weeks Ahead to Process Backlog in Applications

RICHMOND – The Virginia Information Technologies Agency continues to make progress in returning all state agencies to normal operating conditions in the wake of the recent disruption in service caused by the failure of a major data storage system.  The initial failure affected 26 of the Commonwealth’s 89 agencies and caused 13 percent of the Commonwealth’s storage servers to stop working.

As of this morning, all impacted state agencies have received access to their computer systems and files. The required pre-usage testing process is ongoing and found issues are being addressed. Specifically, the Department of Motor Vehicles will be back to normal operations by tomorrow. Details regarding the status of specific local DMV’s will be released on www.dmvnow.com. In order to accommodate Virginians inconvenienced by recent events, the governor has directed DMV to extend office hours over the weeks and weekends ahead in order to process the backlog in applications that has developed. DMV will provide specific information regarding the extended hours later today.

Speaking about the progress being made, Governor McDonnell remarked, “The recent computer outages impacting state agencies, and the citizens they serve, are disappointing and unacceptable. In the days ahead we will be launching a comprehensive effort to determine how this situation occurred and, most importantly, how we can ensure that it does not happen again. However, the foremost priority at this time is ensuring that all state agencies return to their normal operating conditions as quickly as possible. As of this morning all state agencies again have access to their computer systems and files. While some testing still is being done prior to a full return to normal, this is a positive development.”

The governor continued, “The recent outages have had a particularly disruptive impact on Virginians dealing with our Department of Motor Vehicles. To ensure that all problems are corrected as quickly as possible, I have directed officials at the agency to extend office hours over the weeks, and weekends, ahead. Details will be released by DMV later today. From this office to our agencies to our customer service centers, we are all committed to fixing these recent issues, returning all state operations to normal, and ensuring that outages of this nature do not take place in the future.”

###