SAP rolls out 'smart' new way to do data warehousing

SAP is continuing to merge its HANA in-memory database platform with its Business Warehouse data warehousing software, with the latest update adding support for HANA's real-time data loading services.

Companies with large data warehouses often load information sets at off-peak times, such as in overnight batch jobs. But with the general availability of Business Warehouse 7.4, HANA's "smart data access" services can tap any source within or outside a company as it's needed. SAP is calling the approach an "in-memory data fabric."

The services don't actually physically move data into Business Warehouse; rather, the target sources are viewed as virtual tables. This services provide broader access to data sets, as well as the ability to keep frequently accessed information sets inside the core data warehouse while reaching out to ones that are needed only occasionally as desired.

SAP has been pushing the IQ database it gained through the acquisition of Sybase as an ideal option for coring such "cold" data sets.

Real-time data access also helps Business Warehouse users generate better BI (business intelligence) reports, said Neil McGovern, senior director of product and innovation marketing at SAP. "You're always getting the very latest data because you're reaching out to the source."

Monolithic, centralized data warehouses "are no longer viable," added Ken Tsai, vice president of HANA and data management product and innovation marketing at SAP.

Still, SAP is keen to show the scalability of HANA and Business Warehouse. SAP recently announced it had won a Guinness World Record for the largest data warehouse. It built a 12 petabyte system on Amazon Web Services spanning 111 HANA instances.

Queries on the cluster ran in 330 milliseconds, compared to 250 milliseconds for a single instance, "demonstrating the ability to scale massively with little impact on performance overhead," SAP said.

Beverage maker Molson Coors is a long-time Business Warehouse user and went live on Business Warehouse on HANA in November, said Pawel Mierski, senior BI development lead.

Molson Coors is still on version 7.3 but recently took part in a pilot project for 7.4, Mierski said. The newest release allows users to "model and design much quicker than before," even twice as fast as with 7.3, he said.

The company's pilot project didn't focus on real-time data loading or massive scalability, however. Rather, the project took advantage of the fact that 7.4 does away with previous versions' 60-character limitation on text fields.

Molson Coors contracts with a third-party vendor that collects social media feedback about the company. The information is analyzed for sentiment and then sorted into categories such as good, bad and neutral.

The 60-character limit meant Molson Coors was unable to load the social messages into Business Warehouse by their URLs in order to generate visual reports and develop KPIs (key performance indicators), leaving the company with only the .CSV file supplied by the vendor, Mierski said.

However, it's not clear whether Molson Coors will take advantage of 7.4's real-time data loading services. "It's something we are looking at," Mierski said.

But Business Warehouse on HANA has already provided some help in this area, Mierski said. "A main benefit we have with HANA is the speed of the nightly loads," he said. "The business was getting frustrated before because the data was not there before 10 or 10:30 a.m. [EDT]. We would get lots of emails and calls in the morning. No more emails, no more calls."

Chris Kanaracus covers enterprise software and general technology breaking news for The IDG News Service. Chris' email address is Chris_Kanaracus@idg.com