Understanding SOA, Web Services, BPM, BPEL, and More Part One: SOA, Web Services, and BPM

Introduction

The battle for the dominance in Service Oriented Architecture (SOA) and Web services has so far largely been a war of words without the clear winner yet (and not any time soon), as many underlying Internet-based standards have emerged only recently. Still, the advocates of both major platforms/frameworks, Java 2 Enterprise Edition (J2EE) and Microsoft .NET, agree on the future of Web services, and have been building similar technology frameworks for developers. Both the Java and .NET camps also rely on the same set of established standards such as:

• eXtensible markup language (XML), a language that facilitates direct communication among computers on the Internet, and, unlike the older hypertext markup language (HTML) cousin, which provides HTML tags giving instructions to a Web browser about how to display information, XML tags give instructions to a Web browser about the category of information;
  
  
• Universal description, discovery, and integration (UDDI), a Web-based distributed directory that enables businesses to list themselves on the Internet and discover each other, similar to a traditional phone book's yellow and white pages;
  
  
• Web services description language (WDSL), an XML-formatted language that UDDI uses, and which was developed jointly by Microsoft and IBM and used to describe a Web service's capabilities as collections of communication endpoints capable of exchanging messages; and
  
  
• Simple object access protocol (SOAP), an XML-based messaging protocol used to encode the information in Web service request and response messages before sending them over a network. SOAP messages are independent of any operating system or protocol and may be transported using a variety of Internet protocols, including simple mail transport protocol (SMTP), multipurpose internet mail extensions (MIME), and hypertext transport protocol (HTTP).

For information on the J2EE and .NET environments and their subtle differences, see Understand J2EE And .NET Environments Before You Choose.

Quite pertinent to the above, a well-publicized concept of service oriented architectures (SOA) should help developers further down the path of software componentization. The closer one can make the software map to the business processes and adapt over time, the better the applications will support business objectives (i.e., with an underlying agility).

A well constructed application that tightly integrates and yet loosely decouples a set of solid, yet customizable modules will certainly find customers in this highly assorted market. SOA is an application architecture in which all functions, so called "services", are defined using a description language and have evocable interfaces that are called to perform business processes. Processes, transactions, and special functional components all have to be exposed as services allowing composite, diverse applications to be exposed too. Each interaction should be independent of each and every other interaction and the interconnect protocols of the communicating devices (i.e., the infrastructure components that determine the communication system does not affect the interfaces).

Because interfaces are platform-independent, a client from any device using any operating system (OS) in any coding language can supposedly access or use the service. In simplified terms, SOA would be a set of services (which are, again, groups of software components executing certain business processes, such as processing a payment order, calculating or updating currency exchange rates, or authenticating users), on a network that can communicate to each other.

Though built on similar principles, SOA is not the same as Web services, which indicates a certain collection of technologies, such as the above-defined SOAP, UDDI, WSDL, and XML. In simpler terms, XML is used to tag the data, SOAP is used to transfer the data, and WSDL is used for describing the services available, while UDDI is used for listing what services are available. Used primarily as a means for businesses to communicate with each other and with clients, Web services allow organizations to communicate data without intimate knowledge of each other's information technology (IT) systems behind the firewall. Being Web-based applications that dynamically interact with other Web-based applications using open standards, Web services act analogically to electronic data interchange (EDI), with the difference of being an electronic process interchange instead.

On the other hand, SOA entails a much broader and more abstract notion, given it is more than a set of technologies and runs independent of any specific technologies. Web services promise non-vendor dependence through the use of emerging Internet standards for Web-based messaging, application access and interfacing, business process transactions, and other key SOA activities. In any case, given the opportunity of more widespread, systematic software reuse, reduced complexity, improved agility, and a faster way of integrating legacy systems with contemporary applications, SOA and Web services have a promise of "gluing together" heterogeneous environments found in most IT departments nowadays.

Thus, still maturing Web services technology is likely to increase the component-based applications concept's awareness and speed up its still fledgling adoption. Large vendors' endorsement of Web services technology might indeed help them make up for their latency of endorsing the component- or object-oriented technology several years ago. Web services do have a potential of becoming the latest evolution of application integration technology or a revolutionary new application design model by enabling developers to create or enhance applications by connecting granular components that are accessed via platform-independent Web protocols. While they leverage the aged concept of objects' reusability, they may finally offer that extra mile by adherence to standards that are taking hold. Further, they tend to be simpler in their nature, partly owing to the above adopted collaborative Internet standards, and they also tend to be higher-level abstractions (e.g., if an object is a purchase order, a Web service would be the process of closing a purchase order), which implies more likely platform independence and "mixing and matching" opportunity by developers.

Furthermore, the strategy will help the likes of SAP and Oracle further open or componentize their products, as standards like XML and extensible stylesheet language (XSL) make it possible to share data and have a common look and feel across an application, without necessarily dreadfully digging in the source code.

This is Part One of a two-part note.

Part Two will cover BPEL and make user recommendations.

Business Process Management (BPM) Technology

Closely related to SOA would be the evolving business process management (BPM) technology, which entails a broad set of services and tools that provide for explicit and complete process management, where the companies can relatively quickly change the way transactions, queries, and other communications are handled, and deal with exceptions or glitches. BPM typically entails

1) process analysis and modeling, using a graphical process designer targeted for business analysts;

2) definition;

3) execution;

4) monitoring process performance, its degree of completion and out-of-bounds conditions; and

5) administration for process termination and load balancing or rerouting, all including support for both human (manual) and application-level (automatic) interaction.

As the process flow is executed via a runtime execution engine, various enterprise applications (i.e., legacy, standard packaged, customized, third-party, and Web services) may be invoked, as will the tasks that humans have to complete or intervene. Therefore, BPM has emerged from many sources given the myriad of interconnected components that underpin a full fledged BPM system, such as workflow, enterprise application integration (EAI), middleware, process modeling, process monitoring, enterprise applications, collaborative tools, integration brokers, Web integration servers, application servers, applications development tools, rules engines and so on, which naturally creates a complex environment.

One of the most attractive promises of SOA is the potential of creating applications and systems using business models, that could even be visualized, because Web services can generally be described by their metadata, so that a person can construct or map the entire system by linking together the invocation of services in a given sequence (whereby even that sequence and its logic can also be described in metadata). The process of creating metadata to sequence the invocation of several services into a business process flow is referred to as orchestration, and interchangeably to composition, choreography or so. Whichever way referred to, it enables any technical complexity to be transparent and "technology-friendly" for non-IT savvy business users, and enables enterprises to achieve that long-coveted, but so far elusive goal of bridging the proverbial divide between business and IT worlds within an organization. The beauty of an SOA approach might be in forcing IT staff to devise enterprise architectures based on business processes rather than in mere technology terms so that resulting systems will reflect the business needs and practices of the organization.

One might note though, that smaller, mid-market vendors seem to be focusing less on complex routing and invoking automated processes across disparate systems (see BPM Weaves Data And Processes Together For Real-time Revenues), but rather on the BPM's aspects of handling exceptions and automating of simpler processes (i.e., technology or infrastructure services like authentication, alerting or queuing, as opposed to more involving activity services). The BPM term has long been used (and often misused) in the IT industry lingo, since much of the notion had initially been covered by the practice of workflow management technologies, only recently to be joined by the application integration vendors, which focused more on BPM from the aspect of mere technologies mentioned above.

SOA thus offers a promising design approach for making large IT systems more flexible and cost-effective. Namely, a plethora of extended-ERP applications designed to work in conjunction with the core ERP and back-office systems create a greater need for application and data integration, whereby any application must be able to communicate with external applications fairly easily and simply. Since a great deal of the cost of implementation is often actually that of integration, extensibility has significant implications on cost and performance. From the above discussion, applications that embrace the SOA concept and provide standard-based Web Services should significantly reduce the complexity and cost of integration. To that end, the .NET framework was built with SOAP as a core data transfer protocol, albeit this might not necessarily be the best choice, as SOAP is comprised of XML over HTTP, which is not the fastest data transfer protocol.

On the other hand, although J2EE was drafted prior to the advent and adoption of Web services, the market has responded with an enormous amount of Web services tools and applications. Consequently, at this point, applications developed with either .NET or J2EE can take advantage of SOA and Web services, and answer the extensibility question effectively. Furthermore, Web services may motivate vendors to more tightly couple integration with development early in the life cycle of software applications. Microsoft seems to have realized this through the ability of its BizTalk Server to utilize Visual Studio.NET (VS.NET) objects and combine them in a process-oriented manner with other application components. BEA's WebLogic, IBM's WebSphere, Oracle AS 10g, SAP NetWeaver, and other server platforms have been delivered along the same lines. Hence, instead of having to wade through the complexity of integration only after applications have been implemented and are up and running, enterprises can begin executing on integration strategy concurrently with development and deployment.

SOA and Web Services Challenges

But, one of the major challenges for the success of SOA and Web services in general is to provide an environment in which enterprises can leverage an available Web service to fulfill a specific business need. The key success factor will be that these users need to be insulated from the technical details and underlying technology and platforms. At their core, Web services are all about BPM, as the concept of Web services refers to the ability of combining portions of software applications (using a set of technology standards) to create new applications and business processes, in a manner analogous to city planning and shared infrastructure services like coordinating traffic lights, utilities, and road repairs in a municipality. While the ability to accomplish that combination is relatively easy, it becomes much more difficult when companies try to leverage that capability for creating new business processes. The business logic required for event-driven routing and matching information is typically more complex than traditional point-to-point data-based integration (e.g., demanding a service to send all domestic customer data to system A, and all foreign customer data to system B; or, when a purchase order is received, the order acknowledgement should be sent back).

If Web services offer the technical flexibility to better map software functionality to business needs, then business users must have an intuitive, easy-to-use tool for Web services that becomes part of their regular workspace. So far, no single integration solution from a vendor has been able to solve all complex enterprise integration requirements, and this limitation has led to the need for large organizations to implement numerous integration products from multiple vendors.

For this reason, there has been a proliferation of start-ups that have been providing the support infrastructure that surrounds Web services, which have not initially been provided by the major vendors in the market. For example, some functional areas (without clear boundaries and with many overlaps and crossovers) provided by only some mentioned Web services start-ups would be

• Development tools, which are used to create Web services interfaces and applications, from the likes of Exadel (formerly Eltegra), Mind Electric (now part of webMethods), Veridocs, Systinet, Logic Library, and Laszlo Systems.
  
  
• Management platforms, which are used to provide security layers and management of Web services applications, from the likes of Collaxa (now part of Oracle), Actional, Lombardi Software, Infravio, WebCollage, AltoWeb (now defunct), Fuego, Savvion, Talking Blocks (now part of HP), Ultimus, Chordiant, HandySoft, WEST Global, and AmberPoint.
  
  
• Web services networks, which provide infrastructure for Web service transactions outside of the firewall, from the likes of Allidex, Flamenco Networks (now part of Digital Evolution), and Grand Central Communications.
  
  
• Business process integration, which includes process modeling, loosely-coupled Web services-based integration, and web services messaging platforms, from the likes of Black Pearl, Juice Software, MetaMatrix, OmniHub, Sonic Software, Fiorano, IONA, Q-Link, Red Oak Software, RioLabs, and Metis Technologies.

Further, the likes of Sonic, Cape Clear, and PolarLake offer enterprise service bus (ESB) products, which are open standards-based distributed synchronous or asynchronous messaging middleware products that presumably provide secure interoperability between enterprise applications via XML, Web services interfaces, and standardized rules-based routing of documents. The multilingual and multi-platform design of an ESB allows enterprises to process data between applications from various sources, given ESBs leverage both J2EE and .NET environments. ESB can be considered an extension of traditional EAI for adding the following key functions:

• Transformation—the ability to transform XML documents from one data format into another so that the receiving party can interface with the data in an application format that is different from the one in which it is sent;
  
  
• Portability—the ability to share the data between different computers and OS environments;
  
  
• Load balancing and clustering—the ability to distribute processing among several devices so that no single device becomes overloaded; and
  
  
• Failover—the ability to transfer messaging functions to another server if one should fail during the data exchange.

However, as these diverse tools for a facilitated creation of Web services have swamped the market lately, enterprises have increasingly been striving to utilize prudent and standardized adoption or face infrastructure overload. Fairly soon, technology convergence and vendor consolidation will drive user organizations to rationalize the number of integration vendors and Web services-related products toward a core set of strategic tools that will provide the majority of the needed functionality.

This rationalization should benefit organizations by facilitating the usage of common integration tools that should, in the long term, lower training expenses and provide the opportunity to lower overall license, maintenance, and support fees. For the above reasons, many companies will opt for restraining their development till mainstream vendors catch up.

This concludes Part One of a two-part note.

Part Two will cover BPEL and make user recommendations.