Data Discovery Applications

Over the last five years, the business intelligence (BI) space has undergone a huge transformation. The business user community has lobbied for data analysis tools that are easier to use, agile to deploy, and less expensive, thus encouraging the emergence of new products and vendors. These conditions, along with the acquisitions and mergers of software companies looking to offer new applications for analysis and decision-making support, are breeding a new set of innovative tools: the so-called data discovery applications.

What are they?

Data discovery tools are specifically aimed at connecting users to a wide variety of data source types (structured, semi-structured, and non-structured) and enabling users to freely explore the data within. There are no predefined data drill paths, so users can interact with data the way they want to and easily create visualizations that suit their own purposes. As such, they boast a flexibility and freshness that traditional BI solutions might find hard to match.

Taking advantage of these features, users can create prototypes fast and evolve them into more robust data exploration and visualization projects if needed. This way, costs and efforts can be kept to a minimum, so organizations of all sizes can benefit from data discovery tools, whether using them as a basis for creating more robust BI applications or maintaining them as simple solutions in themselves at low cost.

In the past, data discovery tools were limited in functionality and tied to strong data visualization features. They have at times been misunderstood—the true potential of data discovery applications has been neglected, and they are often considered a "lite" version of more serious BI applications. But certain innovative software providers such as Lyzasoft, QlikTech, and Endeca (now part of Oracle), and the research they have produced, support the principle of giving non-technical users the freedom to search/locate and explore information in a friendly but serious way to uncover insight.

Over time, as more and more organizations saw the potential of these tools to deliver quick and versatile data analysis services, data discovery applications were able to offer new features and functions that gave them entrance into the mainstream of the BI space.

Some of the main differences between traditional BI solutions and data discovery tools are as follows:

  • Traditional BI solutions consist of a complete stack of features covering a complete cycle—from specialized data movement tools (data integration, data quality, etc.) to data analysis and visualization. Data discovery tools use best-of-breed technologies to cover mainly analysis, exploration (search, drill through, etc.), and visualization.

  • Data discovery tools enable discovery "on the go," letting users explore data and discover insight without having formed a specific question. BI applications tend to follow a stricter methodology, with structured paths for exploring the data and following well-defined rules.

  • Deployment of data discovery tools tends to be faster than for traditional BI tools, which require more preparation to deploy and operate.

  • Data discovery tools are extremely user-oriented, while traditional BI tools take a corporate approach and so are somewhat more challenging technically to learn and apply.

With the concepts of ease of use, freedom to explore, and self-service at their basis, these business applications contain features that enable organizations to quickly gain access to a broad set of data sources, and start exploring data, analyzing it, and gaining insights from that information. Some of the highlights among the functionalities that data discovery applications offer include the following:

  • Easy-to-deploy data integration capabilities

  • Exploration capabilities of both structured and unstructured data

  • Rich drill-through capabilities

  • Easy and powerful visual data modeling and visualization tools

  • Features for enabling data mash-ups

  • Rich enterprise data and content search capabilities

  • Text enrichment and analysis features such as sentiment analysis

  • Team collaboration features

  • Availability of rich analysis tools such as regression, nonlinear modeling, etc.

  • Easy integration with Microsoft Office, especially Excel

  • Mobile analytics features

Organizations can benefit from these types of applications in the following ways:

  • Rapid deployment at a lower price than for corporate BI suites

  • Easier application of incremental and agile data analysis applications, such as prototyping and agile methodologies

  • Reduced need for supplementary code, or even eliminated altogether

Some interesting aspects of data discovery tools, if used properly, include their ability to uncover data relations and easily integrate dispersed corporate data, enabling a corporate snapshot and rapid analysis of information, potentially speeding up the time-to-decision cycle. (But it is important that these applications be deployed with the same rigor as any other software application within the organization to ensure that data is at all times safe from internal and external damage, and that it complies with regulations if necessary.)

Here are some currently hot data discovery products:

Many software providers of data discovery offerings provide both server and desktop options. Others, such as IBM, Oracle, and SAP, provide strong connection and integration of their data discovery tools with their existing BI platforms, giving customers both the flexibility to work on their desktop and the power of heavy-duty BI solutions if needed.

A final word

Nowadays it is hard to classify business analytics products as they touch upon so many different sets of functionality. Such is the case with QlikView, Tableau, and Endeca, which fulfill functions that might classify them in other areas of the business analytics space. Many vendors offer data discovery applications alongside their traditional BI suites.

Will data discovery applications replace corporate BI platforms? Maybe. For now, some organizations (especially large ones) rely on full-fledged traditional BI applications to cover all their BI needs, but data discovery tools are gaining presence in the small to medium business (SMB) market. They are often used as a solution by lines of business and can be an extremely useful complement to traditional platforms used in large corporations. (We'll have more on data discovery tools in our buyer's guide for BI and data management, coming this October.)

Data discovery applications can bring many benefits to an organization, enabling dynamic and versatile analysis of data, but to successfully meet the data analysis needs of your organization those applications need to be considered with the same rigor as you would employ in acquiring and implementing any other type of enterprise software.

What do you think? Are you using one of these products or some other data discovery tool? Are you a software provider with an offer we should now about? Please let us know.
comments powered by Disqus