FRAMINGHAM (03/22/2004) - Selecting an electronic data capture (EDC) vendor makes buying a pig in a poke seem worry-free. One can see if the pig has four legs. With clinical trial software, defects may not be revealed for years. Peering into the clinical poke, Forrester Research Inc. last month published an analysis ranking eight major EDC vendors on 160 criteria.
The report analyzes applications and companies seldom compared to each other comprehensively. Senior analyst David Shiple, who left the firm soon after finishing the report, didn't declare winners. (That may come in another report, due later this spring.) For now, it's clear that barriers to entry in EDC are surprisingly low. "You'll probably see more vendors, not fewer," Shiple says. "It is bad news for the sponsors," worried as they are about the financial strength of tiny software companies, often running on funds from venture capitalists.
While most EDC vendors seek the favor of the top 50 sponsors of clinical trials, Shiple says the juicier market may be the next 450 sponsors: "If you gave them an extra capability, like safety, a portal, and other clinical data management functions, you should be very strong," he says.
Individual vendors are likely to dispute the methodology of the Forrester study, which often juxtaposes well-established companies with fragile startups boasting a single customer.
Shiple did not have much sympathy for those with reservations about the software in the community of trial sponsors. "Whenever I tried to pinpoint the objections, the objections were not terribly compelling," he said. "I never found any of the problems they talked about as being intractable."
But he said he was also surprised by the percentage of trials that the vendors set up themselves -- perhaps as many as half. Asked Shiple: "If these systems are intended for data managers, people who are clinical trial experts, do they know how to program? Do they know how to debug the program?"
One tantalizing question is whether Shiple was gunned down by the industry he was trying to evaluate. He departed only days after the report was released, leading to questions about connections between his exit and the report. Forrester, for the record, says Shiple left to pursue consulting opportunities.
Off the record, however, EDC vendors express reservations about Forrester's methodology. Such griping may be well-considered or sour grapes -- or a mixture. Forrester's scoring did place highly disparate companies cheek by jowl, but the same techniques are used in other industries.
A more forward-looking, strategic evaluation will be published in April.