Don DeLoach

It's no secret that today's IT professionals need to help their organizations capture, track, analyze and share more information than ever before. From mass quantities of transactional data, Web data, and huge and growing volumes of "machine-generated" information, such as sensor and log data, volumes are expanding into the terabyte (and even the petabyte) range. At the same time, the way end users consume information is rapidly changing. Thanks to innovators like Facebook, Twitter and LinkedIn, social computing technologies are spreading like wildfire, and companies are starting to look at how to harness social networks, blogs, wikis and more to share business intelligence and collaborate more effectively. As the data center strains under the need for more storage and faster performance (all while keeping costs in check,) cloud computing, open source technologies and other emerging approaches are presenting compelling new ways to manage data and consume IT services. How can IT practitioners best navigate today's rapidly changing BI landscape?

Tap into the Cloud
While the data center is not going to disappear anytime soon, cloud computing is certainly democratizing information access. For example, smaller companies that previously couldn't afford to build huge server farms to process mass amounts of data can turn to providers like Amazon and Google to support large-scale analysis efforts. In addition, a number of innovative SaaS and "cloud-friendly" BI and analytic solutions are cropping up, which means that organizations can take advantage of the cloud to not only store their data, but also crunch it. There are, of course, some key considerations. Security and data privacy get the most press, but uptime, performance and openness/portability are also important. Depending on your organization's specific requirements, there's more than one flavor of cloud, ranging from public (affordable and highly scalable), private (offering greater security and control) and hybrid (combining aspects of both). The best approach will ultimately depend on what's most important to your organization. Is it massive scalability at minimal cost? Tight data security while optimizing IT resources? 24x7 uptime? Take the time to clearly define what the business needs to achieve before jumping on the bandwagon. And, when it comes to BI, cloud solutions will only be as good as the performance they can deliver.

Speed the Data "Acquisition-to-Action" Cycle
In the coming years, organizational competitiveness will be increasingly defined by how quickly companies can synthesize the many sources of information coming their way. To do this, they need to be able to master what I call the "acquisition-to-action" cycle. In other words, how fast can data be captured, stored, queried, analyzed, shared and acted upon? Traditional BI, massive data warehouses and databases that were originally designed to crank out pre-configured reports (like sales histories and financials) are simply not nimble enough to handle today's urgent analytic needs, especially in the relentlessly expanding domain of machine-generated data. Pre-calculated, batch-based solutions (Hadoop, Map Reduce and clusters, for example) have a role here, but their utility is limited when it comes to ad-hoc analysis (important for figuring out what to do now) and predictive analysis (essential for understanding what to do next.)

Fast, efficient and flexible query performance is the Holy Grail for today's information consumers. There's no time to index or partition data or perform other tedious forms of manual configuration just to create and run new queries. (Indexing and partitioning also increase database size, in some cases by a factor of two or more.) This demands solutions that are optimized to deliver quick analysis of large volumes of data, with minimal administrative effort needed to set up, change and customize analysis. Columnar databases and other newer analytic approaches have emerged that enable significant data compression and accelerated query processing (no indexes or partitioning of data required) and this is a particularly compelling capability in the "Big Data" era. In addition, there are a number of open source projects now focused on analytics, BI, data integration and more, which gives IT practitioners the opportunity to test drive more innovative analytic tools without risk. Open source tools can also offer the flexibility essential to optimizing rapidly changing query and reporting requirements. Finally, open source solutions are considerably more affordable than proprietary BI and database solutions.

If the Mountain Won't Come to Moses...
When we talk about information analysis, it's important to think about where the data actually resides. Much of the data that organizations need to look at is not necessarily "owned" by them - it exists within Twitter and Facebook feeds, it's hidden within Web logs, sensor output, and call detail records. Finding exactly what you need within such an enormous stream can be like finding a needle in a haystack. But imagine if you could define a set of queries and get the summary information needed in a much smaller, more digestible form. In other words, "If the mountain won't come to Moses, Moses must come to the mountain." This requires technology that's able to use knowledge about the data itself to intelligently isolate the relevant information and make queries more efficient. The goal is to minimize the need to access unnecessary data in order to resolve a query, even those that are fairly complex. It's a fundamentally different way of approaching analysis.

Get Social
The value of business intelligence lies in its ability to shape and enhance decision-making throughout the enterprise. Yet all too often, there's not enough context associated with the analyses generated by information management experts, leaving business end users unable to make sense of it all, or at least make sense of it quickly enough to take action. Today's social computing technologies offer a great opportunity for intelligence to be digested in a more collaborative atmosphere. Imagine combining analytics (a quarterly sales report tracking online purchases, for example) with capabilities like search, bookmarking, tagging, commenting and rating capabilities. Now imagine accessing the intelligence via a Web-based portal where any number of enterprise stakeholders can look at the report collaboratively and engage in a conversation that adds context to the content. This socially powered approach to BI can significantly speed decision-making. More important, it enables a far richer understanding of corporate data, which enables better decision-making as well. Collaborative BI solutions that integrate social technologies directly into the analytic environment are increasingly available, and smart companies would do well to take a look at how these can enrich their BI efforts.

As organizations continue to be bombarded by data, old BI strategies are increasingly giving way to more innovative approaches. The businesses that thrive will be those that succeed at adopting the best new ideas and technologies. When it comes to analytics, cloud computing and social collaboration, there a number of compelling options to choose from.