SOA Testing saves Planet Earth

Every tree counts, and every page that we don't print matters. Now we all know that organizations are changing from paper-based processes to electronic processes by using data transmission standards such as MTOM mostly as a cost saving and improved services level measure. But a great side effect that is often not reinforced is the fact that we are improving planet Earth by not cutting down trees and creating undue waste.

By enabling enterprise to rapidly deploy technologies such as MTOM, we can collectively accelerate electronic processing of documents. It's a win-win for everyone: Companies save money and improve processing efficiencies and our environment improves.

SOA Testing tools such as SOAPSonar provide a rapid way of testing your MTOM deployments. The following article will get you jump-started on using and testing MTOM in your enterprise - and perhaps save some trees:

Introduction to MTOM: A hands-on Approach

SOA Test Tools -- InfoWorld Test Center Report

Rick Grehan wrote a comprehensive report titled "Clean up your SOAP-based Web services - The Test Center inspects five worthy tools for keeping your services squeaky clean." The report compares and rates commercial SOA Testing tools from the following vendors:

1. Crosscheck Networks
2. AdventNet
3. Mindreef
4. Parasoft
5. iTko

You can read the report at:

http://www.infoworld.com/article/07/11/26/48TC-web-services-test-tools_6.html

SOA Testing - It's gaining traction

STARWEST 2007 is the premier conference for QA/Testing professionals. With over 1000 attendees focused on learning the latest testing techniques, the conference serves as a benchmark for what is relevant to the testing community.

This year, we are thrilled to see the testing community starting to focus on and learn about the nuances of modern web services-based SOA Testing. The increasing number of SOA Testing related offering at STARWEST 2007 serves as a good bechmark for the increase in relevance and activity in this area. The following talks and courses are presented this year realted to SOA Testing:

The Coming SOA Revolution: What It Means To Testers -- Frank Cohen, PushToTest

Testing SOA Applications: What’s New, What’s Not -- Brian Bryson, IBM

Ensuring Quality in Web Services -- Chris Hetzler, Appolis Software

The Art and Science of SOA Testing -- Mamoon Yunus & Rizwan Mallal, Crosscheck Networks

This is the second event where we are presenting a day long course on SOA Testing. STAREAST 2007, hosted in Orlando, FL was the first day long course offered exclusively on SOA Testing in the industry. We are on v2 of this course with more structured Labs. I hope the attendees like it more than the v1 offering at STAREAST 2007 - although an 8.9 average from 28 attendees will be tough to beat.

SOA Testing Goes Academic

I came across a recent publication titled Testing in a SOA World presented at The Proceedings of the International Conference on Information Technologies (InfoTech-2007), September 21-23, 2007, Bulgaria.

Here's the Abstract:

"Abstract: Service-oriented architecture (SOA) is the latest attempt to better link the
business with technology. Testing a SOA applications become more and more complex,
as it should be continuous, not just in development and integration, but in deployment,
because an SOA by nature is never a static application. Even if each service in a SOA
application is tested thoroughly and carefully the quality of the final application may
suffer because testing is not enough after the integration of the services. This article
presents the challenges and problems that test teams experience when testing SOA
applications. It also summarizes how the testing of SOA is carried out now and gives
some ideas on further improvements."

The article provides a comprehensive overview of challenges facing the SOA Testing world. I am pleased to see the academic community getting involved in this area and hope that will innovate and produce exceptional original work in solving issues in SOA Testing. For the complete article, please see:

http://www.crosschecknet.com/soa_testing/TestingInAServiceOrientedWorld.pdf

SOA Testing Market Report -- the451group

Dennis Callaghan, analyst with The 451 Group, has written a comprehensive report on the state of the SOA Testing market. You have to be a member to access the report, however, here is a snippet from Dennis' piece:

"As enterprises increasingly deploy architectures where applications share and exchange data and information as services, demand will grow for tools that test these complex service interactions – a demand that traditional developer testing tools can't really meet. And so the SOA testing tools space was born and is currently populated by existing testing vendors that have developed new products to meet the particular needs of this architecture, as well as more nimble startups looking for ways to differentiate their offerings. Many of these startups, as they grow and prove the market for their software, should become attractive acquisition targets. This remains a nascent space and there has yet to be an acquisition of an SOA-specific testing company."

To read the full report, please visit www.the451group.com for a trial membership or click here - http://www.the451.com/apply/apply.php.

Adjusting for SOA Testing

SD Times recently published a good article by David S. Linthicum: Adjusting for SOA Testing. David argues that for SOA Testing, existing testing techniques and tools should not be tossed out but we need to rethink the concepts and technology behind SOA and "adjust" accordingly.

Depending on how much "adjustment" equates to a total toss out, perspectives may vary. We believe that adjusting the technology by slapping on service-orientation to testing products that were grounded in web-site testing is a severe adjustment.

How services are tested, require a clean, ground-up testing product that is built for testing services, their dependencies and re-use rather than testing web sites. The primary focus of SOA is re-use and testing tools focused on SOA need to be built with testing re-usable services in mind.

We humbly differ from David that adjusting the technology is sufficient. Tossing out web-site testing tools and adopting a SOA Testing Tool built ground-up for service testing is a better strategy that saves hours of test suite authoring for functional, performance, interoperability and vulnerability testing.

We do agree that the testing techniques need to be adjusted. One area of emphasis within SOA Testing is abstraction. Most likely, a modern SOA is built using web services with access to only a WSDL file and not the actual source code. This eliminates the ability to do White Box testing and restricts users to Blackbox or Gray Box testing. See for example: SOA Testing Tools for Black, White and Gray Box Testing.

Overall, David does a great job in bringing this discussion to everyones forethought - if quality and security are not addressed with a SOA deployment, the reuse of poor services are bound to proliferate and degrade the overall quality of a SOA.

Next Gen SOA Testing Tools: SOASOAPSonar 3.0

Crosscheck Networks recently released SOAPSonar 3.0 - the next generation of SOA Testing products with a broad array of new testing features and techniques tailored for SOA Developer, Testor and QA Professionals.

Enterprise ITPlanet Staff published a review of SOAPSonar 3.0 highlighting significant product areas and features.

http://products.enterpriseitplanet.com/security/security/1184252855.html

Web Application Security Testing - The father of SOA Security Testing

Watchfire, the maker of AppScan, a web application security testing with around $20M in annual revenue was acquired by IBM's Rational division for ~$100M. The exact price was not disclosed. Watchfire's competitor, SPI Dynamics, maker of WebInspect was subsequently acquired by HP. SPIDynamics had revenues of 18.5M for 2006 and was bought for ~$120M. Again the exact value of the deal was not disclosed.

What does this all mean for SOA Testing: The next logical step for HP and IBM is to extend web application testing into modern web services-based SOA testing. Although HP's acquisition of Mercury that had acquired Systinet, provides HP "SOA awareness," it makes startups like Crosscheck Networks, iTKO, and Mindreef prime candidates for a deeper relationship with such large vendors that are now poised to address SOA Testing.

SOA Magazine - On SOA Testing

Thomas Erl, a thought leader and specialist of everything SOA, author of Service-Oriented Architecture: A Field Guide to Integrating XML and Web Services" and "Service-Oriented Architecture: Concepts, Technology, and Design, " - both books international bestsellers, is the Site Editor and Series Editor for The SOA Magazine, a monthly online publication provided by SOA Systems Inc. and Prentice Hall/PearsonPTR and is officially associated with the "Prentice Hall Service-Oriented Computing Series from Thomas Erl."

Watch your SOA Blind Spots is published in the latest edition of The SOA Magazine. Web services testing techniques have been around for some time. However, with the increased utilization of Web services within service-oriented solutions, the demands and complexities placed on Web services are being taken to a new level. This is in sharp contrast to traditional RPC applications and integration architectures wherein the role of Web services was typically limited to point-to-point data exchanges. Now, Web services find themselves being reused across multiple service compositions and in the midst of dynamic and sophisticated runtime service activities and chains. This article raises a series of testing issues and provides recommended techniques and remedies for establishing robust Web services-based SOA implementations...

The SOA Magazine: http://www.soamag.com/default.asp

Watch your SOA Blind Spots: http://www.soamag.com/I8/0607-2.asp

Building SOA using Microsoft SQL Server 2005

Microsoft SQL Server 2005 provides one of the leading edge web services-aware Database. It is easy to install and configure and can rapidly expose a stored procedure as a web services WSDL operation. With an integrated HTTP stack, MS SQL Server 2005 does not require a separate web server like IIS.

For an overview of SQL Server SOA capabilities, see Jerry Dixon's article:

SQL Server and SODA
— Over the past year, I've been discussing some of the various technologies found inside SQL Server 2005. Three of these technologies are CLR integration, HTTP endpoints, and Service Broker. (Articles on these topics were published, respectively, in the November 2005, March 2006, and November 2006 editions of the DNDJ.) Each of these is a powerful tool in its own right, and can be used to great effect in almost any SQL installation. When used together, however, they become much more powerful.

For tutorial on setting up endpoints and exposing a stored procedure as web services operations, see Peter DeBetta's article:

New HTTP Endpoints Create SQL Server 2005 Web Services.

SOA Testing Microsoft SQL Server is simple. The WSDL generated is WS-I BP compliant and can easily be loaded into testing tools such as Crosscheck Networks SOAPSonar for Functional, Performance, Interoperability and Vulnerability Testing of the exposed endpoints.

How SOA Increases your Security Risk

Bret Latamore published an interesting piece in ComputerWorld on how SOA increases your security risk. The article emphasized what I can compress as follows: Flexibility is the enemy of Security.

Anytime one works towards an open, standards-based architecture to integrate internal and external systems, people and processes, the vulnerability target for attack vectors increases quadratically with the number of nodes that are "open."

The article highlights one of the most important aspects of SOA deployment: Identity Management. With chained web services where a web services may call a number of downstream web services, identity must be carried as a part of the content within the SOAP/XML message as a SAML assertions. Within such environments, each SAML assertions validity at every node has to be established. SOA Testing such environments with Identity requirements across chained web services is complex and requires specialized SOA Testing products.

Another important point highlighted in this piece is that legitimate XML traffic within SOA deployments may inadvertently carry malware that originated upstream, but because of the chained and interdependent nature of web services, this malware now gets to a place where it never got to before in siloed environments. Such malware propagation within SOA can be prevented by infrastructure from Crossbeam and Forum Systems.

For complete article, see "How SOA Increases your Security Risk"

http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9015145

STAREAST 2007: SOA Testing Tutorial

STAREAST 2007, the premier software testing conference is taking place in Orland, FL on May 14-18. For the first time, a new day long SOA Testing tutorial is being offered to attendees.

SOA Testing is becoming important to QA Professional and ramping up on web services concepts such as WSDL, SOAP, XML processing is essential in deploying reliable, scalable and interoperable web services within a SOA.

We are thrilled to see the leadership exhibited by the STAREAST 2007 organizers, especially Lee Copeland in recognizing the importance of SOA Testing.

Anyone interesting in learning about SOA Testing concepts is encouraged to register and attend.

SOA + Utility Computing

As Amazon EC2 exploded onto the IT scene summer 2006 with a beta "by invitation only," we were fortunate to get an account and by August 2006 start pounding on the first practical Utility Computing infrastructure that was more than a bunch of slides and marketing mumbo-jumbo.

Every astute marketing team has at some point used Utility, Grid and Virtualization in their power point slides and brochure-ware. I have been at many conferences and on many calls where I kept hearing such terms but couldn't figure out what all this hoopla was all about, I don't think the presenters knew either ;-) If I can't touch it, I don't understand it. If I can't play with it, it is just vaporware. So, I kept my eye on Grid computing but never really saw a commercial (non-academic) benefit of using grids and utility computing. As much as I am into Astronomy, I couldn't see myself ever using an academic grid to compute the Milky Ways expansion rate or some other esoteric problem.

Finally, Amazon EC2 exploded on the scene with a startling characteristic: a full web services-based provisioning interface. Starting up and terminating Linux instance is as simple as a SOAP call.

This characteristic is the most significant and disruptive aspect of the Amazon EC2 platform. All Hardware and Software product vendors take notice: Your products should be Amazon-aware for true scalability.

So we did what any geek would do, played with the EC2 platform day and night and then published a first paper on EC2: "Amazon EC2 and Oracle SOA Suite a Strong Combo" on Dr. Dobbs.

A few weeks later, SAP has taken notice. Perhaps it had something to do with the emails that we sent to 40 SAP folks about this concept. A couple of significant announcements/interviews from SAP this week:

InfoWorld: Q&A: SAP chief developer heads 'clouds'
Business Week: Opening Up to Collaboration (interview with Shai Agassi, President SAP Product Development)

We sincerely hope that large vendors such as SAP, Oracle, BEA, IBM, and Microsoft take advantage of platforms like EC2 for both their SaaS offerings as well as making their software components EC2-aware. We believe SOA Testing Tools will have a significant role to play in making SOA components run smoothly on Utility Computing Platform such as EC2.

SOA World Editorial - Getting on the Grid

On Jan 14, 2007, Dr. Dobbs published an article titled Amazon EC2 and Oracle SOA Suite a Strong Combo that highlights the convergence of SOA and Utility Computing. This article highlights a move towards Hardware as a Service (HaaS) and the merits of dynamically provisioned hardware based on crossing pre-set resource thresholds (CPU, Memory, TCP Connections, etc). Excerpts from this article are as follows:

Web services-based SOA has fundamentally changed how applications integrate. Add on top of that Amazon EC2 to host your business operations, and you get a potent combination. The significant, yet unnoticed breakthrough of Amazon EC2 is in its ability to spawn up a server instance by a mere web-service call. In addition to a command line interface, EC2 provides a detailed provisioning WSDL that can be used by any web-services application to dynamically control (e.g., run, terminate, authorize) Linux instances within the Amazon Cloud.... components which run business applications can also control dynamic provisioning and maintenance of the very physical infrastructure that they are deployed on. With Amazon EC2, for the first time, SOA components are aware of and in control of their host machines and can clone new instances of themselves based on environmental factors such as user load, available resources and cost.

On March 7, 2007, Ajax World Magazine Sean Rhody, Editor-in-Chief of SOA World Magazine, echoed our sentiments with a superb piece, SOA World Editorial — Getting on the Grid. It is always gratifying to see industry luminaries such as Sean see things in the same light. I liked the new term Sean uses, SOI: Service Oriented Infrastructure. Here are some excerpts from the article, it's a must read:

Grid computing, with the ability to bring capacity on line and to bear on a problem as needed provides another stunning opportunity to move from traditional means of operation to a service platform. Bringing CPUs to bear on a problem in a dynamic fashion, assigning additional network capacity to deal with peak loads, and allocating private connections on the fly in response to security needs are just a few of the capabilities that infrastructure vendors are building into their hardware and operating software.

Great minds do think alike - Kudos to Sean for appreciating and writing about the inevitable convergence of SOA & Utility computing and its impact on SOA Testing.

Intro to SOAP Headers in C#

Let's learn how to manipulate SOAP Headers in .NET C# with a simple web service producer-consumer example. .NET Framework SDK provides a sample producer that receives a SOAP Request with Header information, gets a handle on the Header and returns information from the Header as a part of the SOAP Response. To get familiar with manipulating SOAP Headers let's walk you through the steps on loading this in Web Matrix.

1. Get familiar with building simple web services in .NET. See Building and Testing your First Web Service in .NET
2. Download the C# SOAP Header sample from here.
3. Start a web services project in ASP .NET Web Matrix and copy the sample C# SOAP Header code from Step 2 above into the editor. See figure below.
4. Hit the save button and then the start button. This will bring up the browser with the service description. The WSDL file will be available at a location similar to http://localhost:9090/SOAPHeaders.asmx?WSDL (depending on your port setting and .asmx file name in Web Matrix). See Step 1 example.
5. You can now load the WSDL is a SOA Testing Tool that is SOAP Header aware. SOAPSonar is one such flexible SOA Testing Tool that you can download from Crosscheck Networks.
6. Load the WSDL into SOAPSonar. The parsed WSDL shown below has a header and a body inputs. Entering values for the Headers and Body results in a SOAP response from the SecureMethod.
   

Testing SOAP Headers requires test tools to properly parse WSDL and generate input fields for the SOAP Headers. SOAPSonar enables developers and testers to do this easily. With full SOAP Header control, you can now build authentication, routing, audit, and security schemes right into the header. It is strongly advised that standards-based header content is used and customization for header be restricted for maximum interoperability.

SOA Consolidation

Reactivity was purchased by Cisco for a cool $135MM, making it the largest acquisition in the SOA space. DataPower (aquired by IBM for ~$105MM and Systinet (acquired by Mercury for ~$102MM) were the other >$100MM acquisitions. Other smaller ones include Sarvega (acquired by Intel for ~$40MM), Conformative (acquired by Intel for ~$32MM), Infravio (acquired by webMethods for ~$38MM) pretty much consolidated the space leaving Forum Systems, Layer7, SOA Software and Amberpoint as the independent private companies. Further consolidation is inevitable.

The second wave of SOA Infrastructure startups in underway. Andrew Nash, CTO of Reactivity, started a company focused on XML identity. Sonoa Systems is the Next Gen XML Infrastructure Company still in stealth mode and is funded by SAP Ventures, Norwest Ventures, and Bay Partners.

Ofcourse, all XML/SOA infrastructure companies will require extensive SOA Testing to make their products enterprise ready. They will have to deal with the extensive web services/XML standards such as WS-Security, its profiles such as User Name, X.509, Kerberos, SAML tokens as well as SOAP signatures and encryption. Starting XML/SOA infrastructure companies from scratch will enable such new startups to build ground up on relatively newer and useful standards such as WS-Policy, WS-SecureConversation, etc. The second cycle will be shorter with exits taking 3-4 years unlike the 5+ years needed by the 1st wave of SOA consolidations.

For an interesting interview on Sonoa by Eric Knorr, see this video clip.

Testing SOA Applications: App Labs Article

AppLabs Technologies is a Sequoia Funded testing company based in Hyderabad, India. AppLabs services encompass Functional, Performance, and Quality Testing with a distinctive SOA Testing focus. They published a good article on Approach to SOA Testing Applications. This article does a great job on detailing issues associated with Functional and Performance Testing.

On functional testing, it points out that most testing tools are focused on unit testing and are incapable of building composite interdependent tests across technology platforms, languages and systems.

On the performance testing front, this article takes the position that:

"Once the appropriate performance scenarios have been defined, multiple test tools/techniques are required because of the presence of different platforms and technologies. During test execution, monitoring application performance and collating data would be a challenge since there is no “one stop shop” tool which gives insight into the overall big picture."

Although obtaining a composite functional & performance picture may require source code access to figure out root bottlenecks, in modern, web services-based SOAs, the atomic web service "producer" API is what is being tested for performance characteristics. The web service operation internals are a black box operation that may internally call other web services. If all dependent contracts are advertised and available to a SOA Tester, then overall big picture performance characteristics are readily available from existing SOA Testing Tools.

The visibility does stop at the WSDL API level that is provided to the SOA Tester however. For more detailed view into performance and functional characteristics, white box testing is required. This may not be feasible with the proliferation of SaaS. The next closest view into functional and performance characteristics of a web service may be obtained through Grey Box testing. See SOA Testing using Black, White, & Gray Box Techniques for details.

With the proliferation of web services-based SOA within Enterprises and SMBs, AppLabs is focused on the right space and is positioned to capitalize on the increasing testing needs of complex SOA deployments.

Testing SOA Applications and Services

Mercury/HP published a good article on SOA Testing highlighting some facets of "non-GUI" testing. Mercury solution seems to address Functional and Performance testing for SOA deployments. These 2 aspects of web services-based SOA testing are part of the 4 pillars of SOA testing necessary for comprehensive SOA test coverage. Interoperability and Vulnerability testing form the other two necessary Pillars of SOA Testing.

One of the features that stands out is Mecury's "stub-simulation" capability that allows testers to simulate services to build test suites without requiring access to the target production system.

For complete article, see Testing SOA Applications and Services (registration required).

Watch your SOA Testing Blind Spots

This latest article by Crosscheck Networks' R&D team highlights various common SOA Blind Spots that SOA testers experience in real-life deployments.

The Blind Spots and remedies focus on the following areas:

1. Performance
2. Security
3. SOAP Attachments
4. WSDL
5. Interoperability

For these 5 areas of SOA Testing, the article describes how to identify common SOA blind spots and techniques for avoiding them.

The full article (PDF), can be downloaded here: Watch your SOA Testing Blind Spots

Mashup Camp, MIT

I enjoyed attending a few sessions at Mashup Camp. Overall, there is significant innovation and electricity around what the Mashup community is putting together. I enjoyed Jinesh Varia's (Amazon) presentation on building your own YouTube (iTube!). The use case involved uploading-> storing(S3)-> queuing(SQS)-> processing(EC2)-> hosting(ECS)-> Filtering(MTurk)-> Searching(perhaps Hadoop on EC2) video/images. The use case was intriguing since it logically and convincingly utilized parts of the Amazon Infrastructure. This highlights the point in my earlier blog that the cost of innovation is dramatically lower and even when scalability is required, the cost will align with specific computation & storage needs.

I also enjoyed looking at Mashups developed by LignUp, VOIP-SOA company. LignUp provides a comprehensive platform for controlling and integrating VOIP traffic within an enterprise network. The use case that I found most intriguing is having voice tied into CRM systems. This would give a fuller picture on communication between a company and its customers beyond just email archives.

I bumped into Julio Burgo from Carmun, a web portal for students to share and store academic research papers. I love their tag line: "Students of the world Unite!"

For a list of all the attendees, see Mashup Attendees.

Amazon EC2 and Oracle SOA Suite a Strong Combo

With Mashup Camp (Boston) right around the corner, I was happy to see Dr. Dobbs Journal publish an article that I have collaborated on with David Shaffer, Sr. Director Product Mgmt, Oracle Integration and Rizwan Mallal, from Crosscheck Networks, R&D. The article is written to drive home an important set of shifts that are a result of Amazon's Utility Compute Platforms, EC2, S3, SDS, etc. and it's intersection with web services products such as Oracle SOA Suite 10g.

In this article, we focused on how the Oracle SOA Suite can be used beyond its conventional business process role and more as a core SOA infrastructure traffic management platform for content-based routing, load balancing and fail over type functions for Amazon EC2 Linux instances.

http://www.ddj.com/dept/webservices/196900803

Prediction: In the next 2-3 years, perhaps sooner, Amazon EC2 in combination with SOA Suites will cause a fundamental shift in corporate computing. Also, EC2 & SOA will unleash unprecedented low cost innovation where startups will get off the ground with far lower capital than that required in the pre-Amazon and pre-web services era.

Web Services Testing SugarCRM

SugarCRM is cost-effective, powerful and flexible open source CRM system built on PHP and MySQL Database. Its a great CRM system for cost-conscious companies that need a solid CRM platform. At $40/user/month for a hosted On-Demand offering, its priced for the frugal.

Besides its great CRM functionality and reasonable pricing, what I really like about SugarCRM is its extensive web services API. This enables the SugarCRM platform to be easily integrated with the IT ecosystem that is increasingly becoming web services aware.

If you a have a web site portal that captures customer registration information, and you'd like to integrate that with a Leads Capture Mechanism, I highly recommend that you consider SugarCRM. Its SOAP API makes the integration process between your web portal and the SugarCRM Leads Module simple. The fields in the Leads Module are highly customizable and all your customized fields are available via the SOAP API as well.

For a Quick Start on SugarCRM Web Services Functionality, see the following article:

Getting Started with SugarCRM Web Services using SOAPSonar

RadView's PdM & Dev blog: SOA closing the gap between functional and performance testing

The following blog by the Radview Product Management team shows how quickly companies are adapting to SOA requirements. I think they have clearly identified customer needs both from a testing and traffic management perspective. With their products becoming web services aware, Radview will meet customer needs for web services testing.

RadView's PdM & Dev blog: SOA closing the gap between functional and performance testing