Summary of Report 96-055

April 1996

Report Number 96-055

Overall Conclusion

The Post-Implementation Evaluation Reports (PIERs), as requested by the Quality Assurance Team, are working toward improving accountability for development of information systems which require such reports. Agencies not required to submit PIERs could benefit from similar post-implementation reviews. Without such evaluations, state leadership and management can not objectively know whether the benefits and objectives are met by new systems. We found that in some cases we are not achieving expected results and that in other cases where systems are not monitored by the Quality Assurance Team, we are not in a position to determine whether the State received the intended benefit for the dollars expended.

The guidelines for preparing the PIERs need improvement to provide even more accountability information to agency management to further lessen the risk of project failure or potential problems. For example, they can require that a comparison of budgeted costs to actual costs be reported. The development costs of the systems which we audited and surveyed totaled more than $88 million.

Key Facts And Findings

Four out of six PIERs and corresponding evidence for new systems audited indicate that functional objectives and benefits are met; however, two systems are not meeting their objectives yet.

For systems requested by the Quality Assurance Team to have PIERs, several improvements are needed. Agencies need to better follow the Quality Assurance Team's reporting guidelines, ensure more adequate system development methodologies are used, report quantifiable performance measures, and better track and report total costs. However, we found that inadequate PIERs did not necessarily indicate inadequate systems. Agency management and Information Resource Managers should ensure that improvements in these areas are made. The Quality Assurance Team can better monitor the adequacy of the PIERs.

Management at other state agencies surveyed do not always recognize the need for post-implementation reviews. There are a variety of system development methodologies which do not always include a final review of the project or the functionality of the system. The Quality Assurance Team and Department of Information Resources should realize the statewide potential for post-implementation reviews and better promote the benefits of them.

Contact the SAO about this report.

Download the Acrobat version of this report. (.pdf)

If you prefer an HTML version, follow this link to an Adobe site which converts PDF files to HTML.