Customer Data Integration: A Primer


Originally published - August 22, 2006


Implementing a customer data management system can be the difference between success and failure in terms of leveraging an organization's customer relationship management (CRM) system. Since customers drive profitability, organizations need a way to provide their employees with a single view of the customer and to provide that customer with above-average customer service. Unfortunately, this is not always the case. Disparate applications such as billing and call center systems do not always feed into one another, and even when they do, lack of data cleansing and management can cause employees to see only a portion of a customer's history, interactions, or profiles. A widely used example is that of an organization sending multiple marketing brochures to one customer because of inaccuracies and lack of customer data synchronization. A more alarming example is having more than one customer record for a specific customer, with the collections department calling that customer to collect on an account that is actually current.

Why do CRM initiatives fail? Because implementing a system to manage customers does not guarantee that CRM applications will work successfully within the organization. The old adage—"garbage in, garbage out,"—definitely applies to the realm of CRM. If organizations do not have clean, reliable, centralized data, their customer view will not be complete or accurate, and their business goals will not be achieved. Consequently, customer data integration (CDI) has become an essential component of an organization's management of data, along with any CRM initiative.

This article will provide an overview of CDI within CRM, and see how it differentiates itself from the general data integration industry. Additionally, the components of CDI will be explored, to identify the important areas that should be considered when implementing master data management (MDM) for CRM within the organization. Finally, key vendors in the industry and their key product features will be identified.

Defining Customer Data Integration

Within CRM, CDI is the management and consolidation of customer information from across the organization. This includes, but is not limited to, information stored in call centers, sales and marketing departments, and accounts receivables and payables. CDI ensures that each department requiring customer contact has access to timely data, to provide employees with a complete view of customer profiles or histories. This creates a standardized view of each customer and promotes positive customer interactions.

Most enterprise organizations have built or acquired their computer applications over an extended period of time, creating a series of complex systems that work independently or that interoperate with one another. Even if these systems have high interoperability, many times the business rules and data structures of each application and business unit have not been taken into account, as they were developed independently of one another. This means that data may be captured in different ways. For example, customer address information and name may be recorded in different formats within different business units. When data is pulled from one system to another, this particular customer information may not be synchronized.

CDI and Data Integration

CDI represents a consolidated view of customer data. Aside from MDM, which looks at the whole organization, data integration generally focuses on specific initiatives, and is the type of software used to perform data transfers, consolidations, etc. Thus, when an organization is looking to implement a CDI initiative, its focus should involve identifying the data integration vendors that specialize in CRM or that focus specifically on validating and consolidating customer data.

Data integration is defined as the act of bringing together or moving data from one or multiple locations to a centralized or replicated data store. The development of a data warehouse and the consolidation of information across the organization is an example of how data integration is applied in organizations. Sub-sets of data from disparate locations within the organization are loaded into the centralized structure of a data warehouse or dedicated database. This centralized structure creates a specified view of data to measure an organization's performance, to generate reports, to provide analytics, and so forth.

Not all data integration is equal when it comes to CDI. Different forms of data integration are used within different industries and for diverse initiatives. For example, when implementing a business intelligence (BI) solution, data mapping, data cleansing, and hourly data loads are likely the most important factors to consider. Also, different vendors within the data integration space may specialize in sub-categories such as data quality, and may partner with larger industry- or solution-specific vendors to have their solutions embedded within larger software packages. This gives organizations the ability to mix and match solutions based on their needs.

Considerations for CDI

CDI requires specialized data integration solutions. Aside from the general data integration requirements such as data extraction, data standardization, data transformation, and data load functionality, CDI solutions offer additional data cleansing, data profiling, and data mapping to guarantee that a universal view of the customer exists in a centralized structure. CDI solutions also match, merge, and link records, differentiating them from other data integration or data quality vendors.

  • Data cleansing and standardization is used to identify and define data definitions and standardize customer information. A common customer data standardization initiative includes creating a single view of customer name and address information across the organization. Data cleansing activities include removing duplicate records as well as fixing common spelling errors. Within CDI, the importance of cleansing and standardization activities takes center stage within integration efforts, since "a single version of the truth" is what organizations need to implement a successful CDI initiative.

  • Data profiling identifies statistics about the data available in existing databases. Two main aspects of data profiling that are essential for a successful CDI implementation are interdependency and redundancy profiling. Checking for interdependency among tables within different databases across the organization will create similar database structures. For example, a customer number should be attached to the customer table in order to link each customer appropriately based on order, billing, and call center information. Data redundancy profiling identifies duplicate records or overlapping values between tables. This eliminates the possibility of sending out multiply flyers to an individual customer.

  • Data mapping helps ensure the data elements are the same across disparate systems, and mapped where appropriate. For example, it is important that the customer first and last name link up the same way in each disparate system, to guarantee that the correct information is being merged. Included in data mapping are the linking and matching of customer records in order to confirm that the right data is being attached to the right customer when centralized in a hub or data store.

  • Data quality is a unique area within data integration. There are select vendors that compete solely in the data quality space, or that partner with broader data integration vendors to provide data quality functionality. The activities identified above are components of developing data quality management by profiling, standardizing, and monitoring the quality of data.

Implementing an initial CDI initiative is just the first step in providing the proverbial single customer view. The ability to keep data cleansed and to monitor changes in data quality over time to improve the process is a critical success factor of CDI.

Vendor Positioning

The following represents a general listing of vendors within the CDI industry and their product functionality. This list is in no way comprehensive, and organizations should use this as a general guide rather than a potential vendor short list.

  • Siperian Hub is a complete, integrated software platform for customer-centric master data management that creates real-time unified views of customers, organizations, and products, from disparate data silos. This allows organizations to create a unified customer view and framework. Siperian's solution is broken down into three separate modules. Siperian Master Reference Manager (MRM) is used to consolidate multiple customer profiles in order to identify customers uniquely across all channels of the organization. Some key features of MRM are template-driven data models, rules-based modules that are configurable, and built-in audit and historical lineage functionality. Siperian Hierarchy Manager (HM) creates a unified view of the multiple relationships that exist among customers and other entities across all applications to provide organizations with a consistent, complete view of the customer. Some key features include the ability to configure and to manage data relationship consolidation, rules and metadata maintenance for relationship unification across all organizational sources, and exception-handling capabilities. The third module, Siperian Activity Manager (AM), allows organizations to create relevant customer views that drive business actions based on the transactional data captured in the hub and distributed via analytical and operational activities.

  • Oracle's Siebel Customer Data Integration is comprised of three solutions: Universal Application Network, Data Quality, and Universal Customer Master. Oracle's Siebel Data Quality identifies duplicate customer records and provides pre-built integration to third-party data cleansing tools. Oracle's matching server functionality allows the organization to search, match, and identify duplicate customer records based on key customer attributes such as name and address. The data quality connector provides real-time and batch request capabilities that connect to an external data cleansing engine to eliminate duplicate customer records. Additional features of Oracle Siebel include pre-built integration functionality, and fuzzy searching for identifying variations in spelling and word sequence.

  • IBM, with its acquisition of DWL on August 31, 2005, provides a real-time transactional CDI solution. WebSphere Customer Center provides multiple interfaces to front- and back-office systems to access and manage a complete customer master record. It focuses on customer-data transaction management that is operational in nature, and its customer hub provides approximately 500 out-of-the-box services. IBM's CDI solution is implemented within a service-oriented architecture (SOA) and its business services, to help manage and maintain customer data. The CDI solution is a part of the IBM Master Data Management (MDM) suite of products, and integrates with IBM Information Integration and Entity Analytics products.

  • Initiate Systems provides CDI solutions to organizations to help control customer interactions related to sales, service, and customer relations. The Initiate Identity Hub software system focuses on CDI, enterprise master person index (EMPI), and entity resolution. It also leverages customer data in real time and enables organizations to find any data set, account, or transaction, based on person, household, or organization. The Initiate Identity Hub software also links duplicate and fragmented records within and across disparate data sources. Initiate Systems has several additional software components to complement the hub, and provides a complete CDI solution. Initiate's Data Federation compiles data maintained outside of the Initiate Identity Hub software, allowing organizations to overcome security restrictions that prevent a complete customer view. Initiate Synchronization functions as the central management point for all customer data, in order to keep data accurate. Initiate Data Profiling Report Pack helps monitor and improve data quality so that organizations can identify and understand the impact of data on business operations within the organization. Initiate Enterprise Integrator provides distributed access to the searching and linking capabilities offered by Initiate Identity Hub. With the integrator, the capabilities of legacy and operational applications can be extended to provide data integration services at the point of service, enabling customization and deployment at any stage of the enterprise that requires accurate customer identification.

  • DataFlux's CDI solution works by aggregating information from disparate applications, databases, and customer touch points into one centralized data source using an SOA. Using a combination of data quality and identity management technology, the DataFlux CDI Solution creates a master data reference file to consolidate information, and then feed those records and update information within a database or application as needed. Data problems are inspected and analyzed before being migrated to the CDI repository. Also, the results of the analysis are used to build targeted data quality routines to correct, standardize, and validate customer data. Some main features of the DataFlux CDI Solution include data profiling, access to multiple data sources, connection and access to multiple data sources to allow for easy and timely analysis, and matching and de-duplication functionality.


CDI is the act of consolidating data from across the organization to create a centralized view of the customer. Vendors have developed specific product offerings to meet the needs of organizations that require data integration initiatives within CRM.

CDI has become an important initiative within organizations that prioritize a consolidated view of customer information across the organization. CDI enables organizations to build a centralized customer data store, and to manage that process. Specialized considerations should be taken into account when implementing a CDI solution versus an overall data integration initiative. Key components to identify and match with vendor functionality are data cleansing and standardization, data profiling, and data mapping. These factors, when combined, allow data quality to increase and to be managed over time.

Organizations looking to implement a CDI solution should consider the key players mentioned above, based on the specific features and functionality they require. Each solution should be evaluated based on specified organizational requirements, and compared to the vendor offerings.

comments powered by Disqus