Menu
IBM questions internal processes after Census outage

IBM questions internal processes after Census outage

Tech giant concedes failure to clearly communicate a DDoS mitigation plan to upstream partners

IBM managing director A/NZ, Kerry Purcell

IBM managing director A/NZ, Kerry Purcell

Credit: IBM

IBM has questioned its own dealings with an internet service provider and its upstream partner in its handling of the 2016 Census project, after the government portal was taken offline for 40 hours in early August after being hit by a series of DDoS attacks.

According to IBM engineer, Michael Shallcross, the company’s efforts to instruct Australian telecommunications provider, Nextgen, and its upstream partner, Vocus, in the implementation of a geo-blocking distributed denial-of-service (DDoS) prevention plan (named Island Australia) in the lead up to Census day, had failed.

“It’s apparent from the submissions brought by Nextgen and Vocus that perhaps the internal communications had not conveyed adequately the intent and instructions of and surrounding the implementation of Island Australia,” Shallcross told the senate committee investigating the 2016 Census project in a public hearing on October 25.

“We, as the prime contractor, dealt with both Telstra and Nextgen as our ISPs, and expected them, as large internet service providers, to be able to implement those instructions correctly.

"Telstra did [that] for us in 2011, and again in 2016. We had difficulty with the implementation by Vocus of those particular instructions.

“If there was one thing [I would do differently]…it would be trying to gain a greater certainty that the ISPs upstream partners correctly understood our instructions,” Shallcross said.

Nextgen has previously said it had "provided all possible assistance" to IBM to put in place the Island Australia plan, and that it complied with the technology company's framework to hold DDoS attacks at bay.

The IBM engineer also outlined a failure in the configuration in one of two routers it was using to channel data traffic to the Census site from its two ISP partners which, ultimately, led to a failed reboot after it was shut down following a data surge caused by a fourth DDoS attack - this left the router inoperable for more than an hour.

“We did, during the lead up to the Census, test the impact of a failure of that router and test that the failover mechanisms on the rest of the site worked effectively,” Shallcross said.

“But we tested that router failure by simulating it, which is relatively easy to do in a repeatable fashion. If we had our time again, we would probably test a hard ‘power-off, power-on’ that router. That would have discovered earlier that we had that reboot and configuration and loading problem,” he said.

The DDoS attack that ultimately overwhelmed the infrastructure put in place by IBM, which had been contracted by the Australian Bureau of Statistics in a $9.7 million deal to lead the 2016 Census project, and its ISP partners, was routed through Singapore.

According to IBM, the first of the four attacks notched up data rates of 3 Gbps. By the time the fourth attack was underway, the company said its staff noticed a “qualitative” difference in the traffic. It was this shift, and IBM's subsequent misinterpretation of the data, that prompted the company and the ABS to shut down the site in the interest of protecting it.

Since the inquiry began, IBM has claimed that the geo-blocking DDoS protection strategy was discussed in length with the ABS and signed-off by the agency.

However, it had not been formally approved by the Australian Signals Directorate (ASD), the national agency responsible for the collection and analysis of foreign signals intelligence, and the provision of cyber security for the government.

“It was discussed with ASD, but I’m not aware that the ASD ever passed a comment saying one way or the other that they endorsed it or not,” Shallcross told the committee.

The geo-blocking strategy was an appropriate approach to protecting the Census site from DDoS attacks, according to Shallcross, given that it was only meant to be used by people who were within Australia at the time.

“We stand by very firmly of the view that geo-blocking is an effective DDoS attack prevention mechanism,” IBM A/NZ managing director, Kerry Purcell, told the senate committee.

Purcell's stance on geo-blocking stands in stark contrast to comments by the special adviser to the Prime Minister on cyber security, Alastair MacGibbon, who told the committee that IBM's approach was flawed.

“There certainly were better alternatives, yes," MacGibbon said.

"The concept of the ‘Island Australia’ geo-blocking was to prevent internet traffic coming in from overseas to an Australian website. And on face value that might seem reasonably logical," he said.

But it is not uncommon for internet traffic sourced from Australia to look as though it has come from overseas, MacGibbon explained. In fact, IBM's own password reset system for the Census website relied on data coming in from overseas. As such, the geo-blocking approach was problematic, he suggested.

“There was a fundamental failure in the logic of an ‘Island Australia’. I could see it as part of a series of protections, adding some value. But to rely solely on it, clearly, was a failure," he said.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags ABScensusKerry PurcellDavid KalischMichael ShallcrossBill McLennan

More about Australian Bureau of StatisticsBillIBMIsland

Show Comments
[]