A year after the launch of Microsoft's ambitious Office 365, it's almost impossible for outsiders to get a grip on how the software-by-subscription program has done, analysts have admitted.
Microsoft kicked off Office 365 -- a revamped program for businesses and a new attempt to convince consumers to subscribe indefinitely rather than buy the software every few years -- on Jan. 29, 2013.
Since then, Microsoft has disclosed very few details about Office 365's revenue or subscriber performance, other than to occasionally tout the number of consumers who have signed up for the $100-per-year Home Premium.
"In its [SEC and earnings] disclosures, Microsoft talks about revenue at times, but the fact is a huge chunk [of Office 365] is really about revenue from desktop licenses that are just licensed by subscription," said Melissa Webster, an analyst with IDC. "In other words, they're moving revenue off one line and on another. That's not growth."
The last time Microsoft mentioned Office 365 revenue was in mid-2013, when it said the rent-not-buy program was at an annual "run rate" of $1.5 billion, meaning that at its then-current pace, it would generate that amount over the next 12 months.
The company has also pegged Home Premium subscriptions at various times in the last year, most recently last week when it said it had 3.5 million on that edition's rolls, an increase of 1.5 million from the last update in October that's worth $350 million in annual revenue.
Webster's point was that any numbers from Microsoft for enterprise Office 365 subscription revenue was misleading, since much of it was not new, but simply a change of licensing from the traditional "perpetual," where companies bought licenses with rights to use the software as long as they wanted, to subscription.
"At least half is not new revenue, and not a new revenue stream," Webster said, no matter how the media interprets Microsoft's rare disclosures with its headlines. "At least half is just desktop software licensed a different way." She estimated that 60% of the Office 365 revenue was derived directly from the desktop licenses included with most plans.
In a research note last year, Webster called the run rate increases that Microsoft boasted of as an "indifferent" boost to actual revenue, arguing that the real net-new income was approximately one-fourth of Redmond's number once migrations from earlier programs and the switch in licensing were taken into account.
However, she did acknowledge that Microsoft has been drawing some new customers to Office 365, many of them small- or medium-sized businesses, for the hosted services like Exchange and SharePoint that are offered alongside the desktop software they used already.
Wes Miller, of Directions on Microsoft, a smaller research firm that only tracks Microsoft's moves, took a different tack in evaluating Office 365's progress.
Rather than focus on numerical metrics, Miller said it was important to remember why Microsoft switched to a subscription model in the first place, and look beyond the obvious point that the change was meant to smooth out the revenue peaks and valleys, and provide a steadier stream of money to the company.
"The point of subscriptions is one, so that the software is there when you need it and it's constantly updated," said Miller. "Two, software by subscription is much more agile because it can use the iterative methodology of Google and the Web development we've become used to over the last 10 years. We've definitely seen that in the Office Web Apps."
Miller focused on Office Web Apps, the browser-based versions of Word, Excel and PowerPoint that recent reports claim will shortly be renamed "Office Online" -- because they demonstrated the benefit to customers.
From his perspective, the Office Web Apps, which were updated and improved numerous times last year and through the first month of 2014, were indicative of Office 365's success. That's because, if nothing else, that pace of change would have been impossible without a shift away from the deliberate three-year development cycle and toward a more frequent, if not constant, tempo.
"The company isn't monolithic in its product groups," said Miller, who like most at Directions, once worked at Microsoft. "The Office Web Apps group made a conscious decision to iterate in a velocity impossible before. I think it's really important to understand what the Office Web Apps, in particular, have done. They've released apps for iOS and Android around Office 365, [which before Office 365] would have taken three or four years."
In mid-2013, Microsoft did, as Miller noted, ship Office apps for Android and iPhone, making them available only to subscribers of Office 365. The company, however, has not released Office apps suitable for Apple's iPad, an omission that many analysts have criticized.
Miller also argued that while the revamped Office 365 is now a year old, it may still have its best days ahead of it, as larger organizations have yet to discard their on-premises back-end servers for Exchange, SharePoint, Lync and other services tied to Office.
"We're seeing the same thing as with Azure," Miller said, referring to Microsoft's cloud-computing platform. "For a bigger organization to take advantage of Office 365 is very complicated. They've invested in Exchange, SharePoint and Lync, and there's lot of heavy lifting to change. Many are still taking a wait-and-see position. And the revelations about the NSA [haven't helped anyone."
Microsoft did not recognize the one-year anniversary of Office 365 Wednesday, perhaps not surprising since it's been relatively low key about the subscription program's progress all along.
And the rent-not-buy model now faces its toughest test, said Miller. "The big metric will be the churn rate," he said, referring to how many new customers come in, how many existing ones leave or re-subscribe.
Whether Microsoft reveals the churn rate, or even hints at it in future disclosures of subscriber counts for Office 365, is another matter.
Webster, of IDC, believed that the lack of hard information on Office 365 for enterprises was due to the muddled origin of those subscriptions, with some coming from migrations from earlier programs -- she mentioned "BPOS," or Business Productivity Online Suite," several times -- some from companies switching simply from perpetual to subscription licensing, and the like.
Miller agreed up to a point, but added that the opaqueness of Office 365's performance was also likely due to Microsoft not having a compelling story to tell.
Gregg Keizer covers Microsoft, security issues, Apple, Web browsers and general technology breaking news for Computerworld. Follow Gregg on Twitter at @gkeizer, on Google+ or subscribe to Gregg's RSS feed. His email address is gkeizer@computerworld.com.
See more by Gregg Keizer on Computerworld.com.
Read more about web apps in Computerworld's Web Apps Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.