U.S. officials say the Healthcare.gov website is improving for its users, and response times are now less than a second. But the only way to know for sure is to put their claims to an independent test, and that's exactly what was done.
The chart shows the homepage load time of several health-related websites during testing Monday. Mayo Clinic topped the list at less than a second. Healtcare.gov ranked fourth. (Chart: SmartBear)
SmartBear, which makes site performance and testing tools including its AlertSite monitoring product, tested Healthcare.gov at the request of Computerworld. Healthcare.gov has faced availability and performance problems since its launch, and Monday was no different. There was a partial outage to a key site feature, the enrollment process during the test, but the site is also showing some strengths.
Government officials say they are repairing these problems and are providing day-by-day updates to underscore the urgency of their mission. In a press briefing Friday, Jeffrey Zients, the White House official in charge of ensuring the site gets fixed, touted the improvements so far: "The current page response times are now less than one second -- or "1,000 milliseconds."
The Healthcare.gov homepage, which doesn't have a lot of content, is loading quickly, ranging from 1.5 seconds to 4 seconds, according to SmartBear's testing. It was loading at an average of 3.04 seconds in this test, which was conducted during the workday on Monday.
Healthcare.gov was compared to a dozen other healthcare-related sites, which were also tested on Monday by SmartBear. It ranked well (see illustration), placing fourth in this list. The Mayo Clinic was at the top with load times under a second, and the AARP was last, at well past 10 seconds. The test were conducted from 10 cities, located coast to coast and north to south, as well as on multiple networks.
"The homepage of Healthcare.gov is certainly loading in-line with what people would expect," said Ken Godskind, the vice president of monitoring products at SmartBear, who called the homepage performance "pretty snappy."
But once testing drilled past the homepage to interior pages, performance issues arose. Notably, there was an approximately 1.5-hour period during Monday's test when users were unable to fill out an enrollment application. Instead, they got a page that said "The System is down at the moment" "Please try again later."
Users need to complete four steps to reach the "get started," button, the point at which they start enrollment. But that part of the site was unavailable from approximately 12:30 p.m. to 2 p.m. ET.
Federal officials acknowledged the outage to the enrollment section and blamed it on a load balancing problem. The White House has set an deadline for the end of this month to fix the site's issues.
The homepage was not affected by the outage.
Once the availability problems were fixed, Godskind said the data showed steady improvement in the four-step process it takes to reach the "get started" button. By midafternoon, the average completion time was 12 seconds. Earlier, it had taken as long as 36 seconds. "It seems like something was done today (Monday) to speed things up," Godskind said.
Godskind said Healthcare.gov has the hallmarks of a site built without considering performance until the end of the development lifecycle.
Godskind said the rollout sounds like "a typical old school deployment of a large scale system," with a lot of fixes post launch.
The issues facing Healthcare.gov have surprised few, said Forrest Morgeson, director of research at the American Customer Satisfaction Index, which benchmarks customer satisfactions across industries, including government. The research site has not benchmarked Healthcare.gov, but overall, government agencies have not done as well as the private sector in user satisfaction, he said
"Government agencies just haven't had as strong a grasp on the technology as the private sector," Morgeson said.
A complicating issue for government deployments is the lack of a centralized system for electronic government. As a result, the user experience on government websites can vary widely. Agencies "have rolled out their websites in a vacuum," Morgeson said.
This article, Benchmarking Healthcare.gov: A homepage in 3 seconds, but then a failure , was originally published at Computerworld.com.
Patrick Thibodeau covers cloud computing and enterprise applications, outsourcing, government IT policies, data centers and IT workforce issues for Computerworld. Follow Patrick on Twitter at @DCgov or subscribe to Patrick's RSS feed. His e-mail address is pthibodeau@computerworld.com.
See more by Patrick Thibodeau on Computerworld.com.
Read more about healthcare it in Computerworld's Healthcare IT Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.