Enterprise Application Integration - Where Is It Now (And What Is It Now)? Part 1: What Is It Now?
Michael F. Reed
1: What Is It Now?
Since January 2000 when TEC last addressed the trends in Enterprise Application,
there have been massive changes in the overall direction of Application
Integration in general and EAI in particular.
great many of the players have changed in the vendor arena, new terminology
("buzz-phrases" like IAI, or "Inter-Enterprise Application Integration",
and "e-Business Infrastructure Enablement", or B2Bi), has arisen out of
the marketing machine, and customers are generally confused as to just
what vendors are offering and how it may (or may not) solve their business
problem. In addition, the strong downturn in the technology economy has
affected many vendors' prospects.
definition of EAI is amorphous at best, so TEC has sought to take the
broadest definition, which is as follows:
Application Integration is any process, or series of processes, which
enables business capabilities through the combination of data and business
logic obtained from separate systems. Whether the data is internal,
inter-divisional, customer or supplier facing is irrelevant. Vendors
can give it whatever term they choose, or sub-divide it into separate
applications, but to us, integration is integration. Plain and simple.
IT programmers have been doing it for over 20 years.
a two-part note on EAI trends. This part discusses the basic components
to achieve EAI. Part Two covers the Market Impact of the recent changes
and makes User Recommendations.
The basic components required to achieve EAI are the following:
Business Rule Component: to allow the applications to understand
your business processes.
Logic Modules (i.e., supply planning, sales order processing. Methods
for business process management). This is perhaps the most important
but least understood aspect.
tools (to define how to map data from one system to another)
Component: to allow access to the data.
- Data Source and Target Interfaces (i.e., Siebel, SAP,
PeopleSoft, ODBC, JDBC, Oracle, CICS, IMS, IBM
DB2 UDB). Note that the data acquisition component is crucial to
EAI success, and is tied to Data Cleansing (see below). Most vendors
refer to these interfaces as "adapters" or "connectors". Especially
to many large corporations, efficient access to legacy systems such
as ADABAS or IDMS may be required, and not via "flat file extracts"
which are expensive, time consuming, and bandwidth intensive. In addition,
Event-based data acquisition can be difficult to define, usually requiring
database triggers or database log analyzers. When the data source
is not a relational database (i.e., a text file), this process can
become extremely complicated.
Component: to ensure the data is correct.
Development Component: to allow programmers to design and test custom
tools (for business process design, data mapping and transformation
design, debugging, and testing)
- A published
API (Application Programming Interface) to the product so custom extensions
can be written as needed (and they are always needed)
Control Component: to allow the application to be monitored and controlled.
tools (for application-specific monitoring), preferably with support
for the Simple Network Management Protocol (SNMP) via a vendor-supplied
SNMP agent. TEC considers this the most important component currently
lacking in the major system management products (IBM Tivoli
TME, BMC Patrol, and Computer Associates
Unicenter). All three vendors have indicated that they understand
the need to help their customers track distributed units of work, especially
in heterogeneous EAI environments, but all admit they are not quite
there yet. It will be a daunting task indeed. The IT manager must be
able to know exactly where any process is at the time, similar to the
way JCL batch jobs on mainframes were monitored in the past. Questions
such as what step is the job in, how much CPU time has it absorbed,
etc. must be answered in real time.
control mechanisms (for control of business-level logical units of work)
- These mechanisms allow the systems involved to recover from failure
without corrupting the data in either system. This becomes much more
difficult when more than one type of database or file system is involved
in the logical unit of work.
support for metadata management, preferably distributed and bi-directionally
synchronized. This point also can not be emphasized enough. Without
distributed metadata management, IT personnel continually have to reinvent
the wheel, leading to errors and misleading results. Metadata repositories
from vendors such as Viasoft and Computer Associates
call be helpful in this regard.
Brokers (to control transactions, control security, and perform event
notification, e.g.,.IBM's MQSeries, or Microsoft's MSMQ
for Windows NT/2000) - The product should also include the capability
to "bridge" messages between different messaging systems (e.g. an IBM
MQSeries mainframe application that needs to communicate with a Microsoft
MSMQ application on a Windows platform).
for high-volume transaction throughput - We cannot put enough emphasis
on attention to scalability. It is almost impossible to know at implementation
time what the data volumes will be in the future.
for varying levels of fault tolerance, load balancing, and failover
for mission-critical systems
enablement (via SMTP, publish/subscribe capabilities, etc.) is a key
requirement to reduce latency between distributed processes. The product
should also have links to workgroup products such as Lotus Notes Domino
and Microsoft Exchange.
- A Web-based
portal for transparent, integrated information access
vendors overemphasize the importance of the portal. We believe that,
although the access method must be highly useable, the underlying technologies
are most important. A good example of a vendor overlooking things is
Computer Associates (see, Computer
Associates Jasmine ii: When is a Portal Not Just a Portal?). The
vendor has admitted that they may have underemphasized the importance
of the infrastructure in favor of the portal, at least temporarily.
- The ability
to work both inside and outside the corporate firewall (for instance
using HTTP sockets).
- Web Application
Servers (generally using Java 2 Enterprise Edition, or J2EE), also fit
loosely into this category.
To attempt to explain EAI, an understanding of data-oriented middleware
is necessary. In addition, the "portal" concept must be explained (see
glossary), as the web is the standard front-end to these distributed technologies.
The topology of this type of application can be split into two parts;
the first is the middleware that connects the disparate data sources,
the second is the business logic that provides a unified view of the data.
Given the complexity of this solution, consulting is required to integrate
all the pieces, as there is no "out of the box" solution that can provide
Time and Process
consideration must be given to the coupling of time and process. There
are three basic communication types which may be required for the middleware
provided by products such as IBM's MQ Series, used in situations where
the communicating applications don't need to be active at the same time.
This allows applications to be "loosely coupled in time". An example
of this would be a terminal application feeding a batch process.
provided by OMG's CORBA, Microsoft's COM, and the new emerging standard
SOAP (Simple Object Access Protocol, developed by Microsoft but now
being embraced by the industry since IBM now supports it), used where
the business logic requires that the applications communicate in real-time.
Generally the requesting transaction waits until it receives the result
set from the other application.
using transaction monitors such as IBM's CICS or BEA's Tuxedo. This
is required where the logical unit of work (defined as one business
transaction) spans multiple systems.
major requirement for an EAI system is security, typically in the form
of single sign-on (to provide access to all the required data without
multiple passwords.) Using EAI provides a business with the ability to
consolidate "stovepipes" of information from various transactional, legacy,
relational, and other sources with different security rules and requirements
into one unified and secure view. Security becomes even more crucial when
companies allow access to internal data by outside sources.
challenge is the need to rationalize the data in the different data sources.
For example, the definition and datatype of the "customer" field may vary
from one database to another. These dimensions need to be conformed in
order for any meaningful integration to take place
Part One of a two-part note on the Trends in EAI. Part Two discusses the
Market Impact of recent changes and makes User Recommendations.
ODBC: Open Database Connectivity. A database programming interface
from Microsoft that provides a common language for Windows applications
to access databases on a network. ODBC is made up of the function calls
programmers write into their applications and the ODBC drivers themselves.
Java Database Connectivity. A programming interface that lets Java applications
access a database via the SQL language. Since Java interpreters (Java
Virtual Machines) are available for all major client platforms, this allows
a platform-independent database application to be written. JDBC is the
Java counterpart of Microsoft's ODBC. Java was originally developed by
DB: OLE (Object Linking and Embedding) Database. A programming interface
for data access from Microsoft. It functions in a similar manner as ODBC,
but for every type of data source not just SQL databases. Applications
can use OLE DB to access ODBC databases as well. OLE DB for OLAP is used
to access OLAP databases. OLE DB is a COM object.
Component Object Model. A component software architecture from Microsoft,
which defines a structure for building program routines (objects) that
can be called up and executed in a Windows environment.
Interface: An interface written to a specific database API (Application
Programming Interface) which gives access to all of the features provided
by the database vendor. It is typically more robust and faster than using
ODBC, JDBC, or OLE DB (which are database translation interfaces.)
Gateway: A product that provides a connection to a database through
a proprietary interface. Typically gateways make the database being connected
to "look" like the gateway vendor's database (e.g.., Oracle's gateway
to IBM DB2 makes DB2 look like an Oracle database). Database gateways
translate SQL calls into a standard format known as Format and Protocol
(FAP). One of the most popular gateway architectures is IBM's Distributed
Relational Database Access (DRDA).
Common Object Request Broker Architecture. A standard from the Object
Management Group (OMG) for communicating between distributed objects (objects
are self-contained software modules). CORBA provides a way to execute
programs (objects) written in different programming languages running
on different platforms no matter where they reside in the network. CORBA
is suited for three-tier (or more) client/server applications, where processing
occurring in one computer requires processing to be performed in another.
CORBA is often described as an "object bus" or "software bus," because
it is a software-based communications interface through which objects
are located and accessed.
Intelligence Portal: A corporate portal that enables users to query
and produce reports on enterprise-wide databases. The term was coined
by Information Advantage, makers of the MyEureka software, which was the
first to combine BI software with a corporate portal. Information Advantage
was acquired by Sterling Software, which was in turn acquired by Computer
Secure Sockets Layer. A leading security protocol on the Internet.
When an SSL session is started, the browser sends its public key to the
server so that the server can securely send a secret key to the browser.
The browser and server exchange data via secret key encryption during
that session. There are also other key technologies in this area, such
as PKI and SET.