Hurricane Sandy exposed a decided lack of contingency planning on Wall Street, according to a report by a research firm.
With the Northeast still experiencing the effects of widespread storm damage, the business continuity plans of stock exchanges and financial services companies will continue to be put to the test in the days to come, IDC Financial Insights stated in a report released Tuesday.
Megastorms like Sandy, which may become more common in the future, are good reminders of the need to double-down on business continuity planning, the report said.
"Not only has Sandy exposed the susceptibility of the contingency plans devised by major market players (exposing a lack of foresight concerning the potential impact of a hurricane that can simultaneously hit the [New York-Connecticut-New Jersey] area), but it has also given insight into [financial market regulators'] apparent inability to monitor electronic markets," IDC stated.
For the first time since the Sept. 11, 2001, terrorist attacks, the New York Stock Exchange voluntarily shut down for two straight days, opening again on Wednesday. As of Friday of last week, the exchange had planned to remain open throughout the storm.
IDC Financial Insights faulted the U.S. Securities and Exchange Commission and the Financial Industry Regulatory Authority, saying the industry watchdogs were "all too happy for stock markets to shut down completely," making it clear that the regulators had little confidence in their ability to police an entirely electronic market.
According to IDC Financial Insights, "there is little evidence to suggest that business continuity plans across Wall Street have so far stood up to such [difficult] conditions."
For the past three years, the NYSE Euronext stock exchange has had a contingency plan -- known as "Print as N" -- which in the event of a disaster allows it to remain open as an electronic-only operation using its Archipelago (Arca) Exchange communications network. However, the NYSE changed those plans after talking to regulators and investment firms.
The exchange chose to shut down out of concerns about fairness -- it wasn't sure all traders would be able connect to the Arca network, according to an NYSE spokesman, who added that officials were also concerned about customer safety.
The NYSE's "Print as N" plan calls for using Arca as the primary market, filtering NYSE trades through it using the letter 'N' to designate NYSE transactions.
At no time were any of the NYSE's data centers offline because of the storm, the spokesman said. The exchange could have remained open had it chosen to do so, but it chose the path of common sense instead, the spokesman said.
The SEC could not be reached for comment at deadline.
"The systems were fine -- it was getting people to run the systems that was the issue, which in of itself is also an issue, as they were unwilling to run without human intervention," IDC Financial Insights analyst Marc DeCastro stated in an email response to Computerworld.
However, IDC's report criticized the NYSE's contingency plan, "or as it turns out, the lack of a continuity plan," for such a natural "black swan" event, because the plan called for the stock market to operate "as an electronic-only exchange for the first time in its history."
"Keeping the market open on a purely electronic basis, with the market having never operated this way even under perfect conditions, would only increase the chance of any minor malfunction to a high-frequency trading algorithm, causing potentially great disruption," IDC wrote.
IDC suggested that regulators and financial institutions need to discuss what can and cannot be done during widespread disasters.
In its report, titled " Sandy Surfaces the Importance of the 'Human Machine' on Wall Street," IDC stated that because of the sheer physical damage across lower Manhattan, it's expected that critical IT systems of many Wall Street institutions would not be fully functioning under full power.
Floodwaters from Hurricane Sandy shuttered two data center buildings in lower Manhattan when their backup diesel generators failed, according to a report on the news aggregation site Slashdot. A third Manhattan data center also reported it was without power.
News and media sites crashed under the weight of traffic overloads as New Yorkers and others throughout the U.S. overwhelmed the websites for weather updates and news about Sandy. The financial information site MarketWatch on Monday posted a notice on its home page saying, "We are experiencing technical difficulties. The full MarketWatch site will return shortly."
"Almost every major news outlet was negatively impacted in some way, but specifically, CNN, Weather.com, The New York Times, Bloomberg, The Wall Street Journal and The Los Angeles Times had the most severe slowdowns," reported Keynote, an Internet and mobile cloud testing and monitoring service.
The Federal Communications Commission reported 25% of cell towers in 10 states hit by Sandy were affected by the storm, and more than a quarter of land lines experienced outages along the Northeast coastline, from Virginia to Massachusetts. However, that figure had fallen to 22% by late Wednesday, the FCC said.
Keynote reported that "almost everyone in and around New York City" experienced either full-scale outages of their Internet connectivity or sporadic performance issues.
Keynote's services connect to the Internet through AT&T, Verizon, Sprint and T-Mobile networks. The monitoring service said it experienced significant slowdowns or complete service interruptions, as cell towers were affected by power outages and by a surge of voice and data traffic.
"This is true for both Internet users of both land line high-speed Internet connections or those accessing the Internet through their mobile phones," Keynote stated.
While overall the Internet weathered Hurricane Sandy, Keynote stated there are still "lessons to be learned."
"First, it is critically important to build robust business continuity plans around your online presence," it stated. "This means having your online content and websites available from multiple data centers, in different parts of the United States or world, as well as implementing technologies to allow for quick failover to those redundant data centers.
Keynote also suggested that businesses test contingency plans regularly, with a special emphasis on making sure users can connect to their sites via mobile phones and Web browsers.
"Finally, recognize the fragility of the mobile Web," Keynote stated. "In highly dense populations, even a single mobile cellular tower taken offline can wreak havoc for everyone in that area."
Lucas Mearian covers storage, disaster recovery and business continuity, financial services infrastructure and healthcare IT for Computerworld. Follow Lucas on Twitter at @lucasmearian, or subscribe to Lucas's RSS feed. His email address is lmearian@computerworld.com.
See more by Lucas Mearian on Computerworld.com.
Read more about disaster recovery in Computerworld's Disaster Recovery Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.