Computerworld

Analysing Novopay bugs too much work for ministry

Ministry of Education doesn't have statistics on what proportion of bugs in Novopay have been introduced by previous fixes

The Ministry of Education does not have statistics on what proportion of bugs in Novopay have been introduced by previous hasty fixes.

Looking to establish a true estimate of the bug clearance rate for the troubled teachers’ payroll system, Computerworld asked under the Official Information Act: “Has any checking been done, when you find a bug, of whether it was present in previous versions?” Rebecca Elvy, associate deputy secretary of education payroll services replies: “Yes, [developer] Talent2 checks previous versions as part of analysing and resolving defects.” As a private organisation, Talent2 is not subject to the OIA.

The Ministry does not hold information at the level of detail required to quantify the approximate percentage of bugs that were introduced by other fixes, Elvy says.

“To produce the answer to this question would involve significant data collation and analysis. For this reason, this part of your request is refused under section 18(f) of the Act on the grounds that the information requested cannot be made available without substantial collation and research.”

Computerworld has approached Talent2, to see if it has the appropriate information and if it is prepared to release it. No reply was received by deadline for our print issue.

We also asked questions on the rate of bug clearance: What “probable remaining bugs” figure will be judged acceptable – that is, will result in termination of the special bug-detecting effort and return of Novopay to normal maintenance mode?

“This decision will be based on our assessment of the stability of the payroll system,” Elevy replies.

“The number of defects will be only one factor in making this assessment. Other factors that will be considered are the impact of any defects on the sector and other external parties, the effort required to fix known defects and the potential for further defects to be identified.

“In particular, we will be influenced by the recommendations in the Novopay Technical Review.”

Outlook for Novapay

Also, the ministry cannot say when Novopay is likely to be clear enough of bugs to be returned to a business-as-usual mode of maintenance.

“We have not yet decided when this will be,” Elvy says.

“It will likely be a gradual process as stability improves.”

“Very serious” defects have been reduced from 44 in early March, to 14 as at May 31, she says.

“Our aim is to reduce the number to less than 10 as soon as practically possible.”