Building SOA using Microsoft SQL Server 2005

Microsoft SQL Server 2005 provides one of the leading edge web services-aware Database. It is easy to install and configure and can rapidly expose a stored procedure as a web services WSDL operation. With an integrated HTTP stack, MS SQL Server 2005 does not require a separate web server like IIS.

For an overview of SQL Server SOA capabilities, see Jerry Dixon's article:

SQL Server and SODA
— Over the past year, I've been discussing some of the various technologies found inside SQL Server 2005. Three of these technologies are CLR integration, HTTP endpoints, and Service Broker. (Articles on these topics were published, respectively, in the November 2005, March 2006, and November 2006 editions of the DNDJ.) Each of these is a powerful tool in its own right, and can be used to great effect in almost any SQL installation. When used together, however, they become much more powerful.

For tutorial on setting up endpoints and exposing a stored procedure as web services operations, see Peter DeBetta's article:

New HTTP Endpoints Create SQL Server 2005 Web Services.

SOA Testing Microsoft SQL Server is simple. The WSDL generated is WS-I BP compliant and can easily be loaded into testing tools such as Crosscheck Networks SOAPSonar for Functional, Performance, Interoperability and Vulnerability Testing of the exposed endpoints.

How SOA Increases your Security Risk

Bret Latamore published an interesting piece in ComputerWorld on how SOA increases your security risk. The article emphasized what I can compress as follows: Flexibility is the enemy of Security.

Anytime one works towards an open, standards-based architecture to integrate internal and external systems, people and processes, the vulnerability target for attack vectors increases quadratically with the number of nodes that are "open."

The article highlights one of the most important aspects of SOA deployment: Identity Management. With chained web services where a web services may call a number of downstream web services, identity must be carried as a part of the content within the SOAP/XML message as a SAML assertions. Within such environments, each SAML assertions validity at every node has to be established. SOA Testing such environments with Identity requirements across chained web services is complex and requires specialized SOA Testing products.

Another important point highlighted in this piece is that legitimate XML traffic within SOA deployments may inadvertently carry malware that originated upstream, but because of the chained and interdependent nature of web services, this malware now gets to a place where it never got to before in siloed environments. Such malware propagation within SOA can be prevented by infrastructure from Crossbeam and Forum Systems.

For complete article, see "How SOA Increases your Security Risk"

http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9015145

STAREAST 2007: SOA Testing Tutorial

STAREAST 2007, the premier software testing conference is taking place in Orland, FL on May 14-18. For the first time, a new day long SOA Testing tutorial is being offered to attendees.

SOA Testing is becoming important to QA Professional and ramping up on web services concepts such as WSDL, SOAP, XML processing is essential in deploying reliable, scalable and interoperable web services within a SOA.

We are thrilled to see the leadership exhibited by the STAREAST 2007 organizers, especially Lee Copeland in recognizing the importance of SOA Testing.

Anyone interesting in learning about SOA Testing concepts is encouraged to register and attend.

SOA + Utility Computing

As Amazon EC2 exploded onto the IT scene summer 2006 with a beta "by invitation only," we were fortunate to get an account and by August 2006 start pounding on the first practical Utility Computing infrastructure that was more than a bunch of slides and marketing mumbo-jumbo.

Every astute marketing team has at some point used Utility, Grid and Virtualization in their power point slides and brochure-ware. I have been at many conferences and on many calls where I kept hearing such terms but couldn't figure out what all this hoopla was all about, I don't think the presenters knew either ;-) If I can't touch it, I don't understand it. If I can't play with it, it is just vaporware. So, I kept my eye on Grid computing but never really saw a commercial (non-academic) benefit of using grids and utility computing. As much as I am into Astronomy, I couldn't see myself ever using an academic grid to compute the Milky Ways expansion rate or some other esoteric problem.

Finally, Amazon EC2 exploded on the scene with a startling characteristic: a full web services-based provisioning interface. Starting up and terminating Linux instance is as simple as a SOAP call.

This characteristic is the most significant and disruptive aspect of the Amazon EC2 platform. All Hardware and Software product vendors take notice: Your products should be Amazon-aware for true scalability.

So we did what any geek would do, played with the EC2 platform day and night and then published a first paper on EC2: "Amazon EC2 and Oracle SOA Suite a Strong Combo" on Dr. Dobbs.

A few weeks later, SAP has taken notice. Perhaps it had something to do with the emails that we sent to 40 SAP folks about this concept. A couple of significant announcements/interviews from SAP this week:

InfoWorld: Q&A: SAP chief developer heads 'clouds'
Business Week: Opening Up to Collaboration (interview with Shai Agassi, President SAP Product Development)

We sincerely hope that large vendors such as SAP, Oracle, BEA, IBM, and Microsoft take advantage of platforms like EC2 for both their SaaS offerings as well as making their software components EC2-aware. We believe SOA Testing Tools will have a significant role to play in making SOA components run smoothly on Utility Computing Platform such as EC2.

SOA World Editorial - Getting on the Grid

On Jan 14, 2007, Dr. Dobbs published an article titled Amazon EC2 and Oracle SOA Suite a Strong Combo that highlights the convergence of SOA and Utility Computing. This article highlights a move towards Hardware as a Service (HaaS) and the merits of dynamically provisioned hardware based on crossing pre-set resource thresholds (CPU, Memory, TCP Connections, etc). Excerpts from this article are as follows:

Web services-based SOA has fundamentally changed how applications integrate. Add on top of that Amazon EC2 to host your business operations, and you get a potent combination. The significant, yet unnoticed breakthrough of Amazon EC2 is in its ability to spawn up a server instance by a mere web-service call. In addition to a command line interface, EC2 provides a detailed provisioning WSDL that can be used by any web-services application to dynamically control (e.g., run, terminate, authorize) Linux instances within the Amazon Cloud.... components which run business applications can also control dynamic provisioning and maintenance of the very physical infrastructure that they are deployed on. With Amazon EC2, for the first time, SOA components are aware of and in control of their host machines and can clone new instances of themselves based on environmental factors such as user load, available resources and cost.

On March 7, 2007, Ajax World Magazine Sean Rhody, Editor-in-Chief of SOA World Magazine, echoed our sentiments with a superb piece, SOA World Editorial — Getting on the Grid. It is always gratifying to see industry luminaries such as Sean see things in the same light. I liked the new term Sean uses, SOI: Service Oriented Infrastructure. Here are some excerpts from the article, it's a must read:

Grid computing, with the ability to bring capacity on line and to bear on a problem as needed provides another stunning opportunity to move from traditional means of operation to a service platform. Bringing CPUs to bear on a problem in a dynamic fashion, assigning additional network capacity to deal with peak loads, and allocating private connections on the fly in response to security needs are just a few of the capabilities that infrastructure vendors are building into their hardware and operating software.

Great minds do think alike - Kudos to Sean for appreciating and writing about the inevitable convergence of SOA & Utility computing and its impact on SOA Testing.

Intro to SOAP Headers in C#

Let's learn how to manipulate SOAP Headers in .NET C# with a simple web service producer-consumer example. .NET Framework SDK provides a sample producer that receives a SOAP Request with Header information, gets a handle on the Header and returns information from the Header as a part of the SOAP Response. To get familiar with manipulating SOAP Headers let's walk you through the steps on loading this in Web Matrix.

1. Get familiar with building simple web services in .NET. See Building and Testing your First Web Service in .NET
2. Download the C# SOAP Header sample from here.
3. Start a web services project in ASP .NET Web Matrix and copy the sample C# SOAP Header code from Step 2 above into the editor. See figure below.
4. Hit the save button and then the start button. This will bring up the browser with the service description. The WSDL file will be available at a location similar to http://localhost:9090/SOAPHeaders.asmx?WSDL (depending on your port setting and .asmx file name in Web Matrix). See Step 1 example.
5. You can now load the WSDL is a SOA Testing Tool that is SOAP Header aware. SOAPSonar is one such flexible SOA Testing Tool that you can download from Crosscheck Networks.
6. Load the WSDL into SOAPSonar. The parsed WSDL shown below has a header and a body inputs. Entering values for the Headers and Body results in a SOAP response from the SecureMethod.
   

Testing SOAP Headers requires test tools to properly parse WSDL and generate input fields for the SOAP Headers. SOAPSonar enables developers and testers to do this easily. With full SOAP Header control, you can now build authentication, routing, audit, and security schemes right into the header. It is strongly advised that standards-based header content is used and customization for header be restricted for maximum interoperability.

SOA Consolidation

Reactivity was purchased by Cisco for a cool $135MM, making it the largest acquisition in the SOA space. DataPower (aquired by IBM for ~$105MM and Systinet (acquired by Mercury for ~$102MM) were the other >$100MM acquisitions. Other smaller ones include Sarvega (acquired by Intel for ~$40MM), Conformative (acquired by Intel for ~$32MM), Infravio (acquired by webMethods for ~$38MM) pretty much consolidated the space leaving Forum Systems, Layer7, SOA Software and Amberpoint as the independent private companies. Further consolidation is inevitable.

The second wave of SOA Infrastructure startups in underway. Andrew Nash, CTO of Reactivity, started a company focused on XML identity. Sonoa Systems is the Next Gen XML Infrastructure Company still in stealth mode and is funded by SAP Ventures, Norwest Ventures, and Bay Partners.

Ofcourse, all XML/SOA infrastructure companies will require extensive SOA Testing to make their products enterprise ready. They will have to deal with the extensive web services/XML standards such as WS-Security, its profiles such as User Name, X.509, Kerberos, SAML tokens as well as SOAP signatures and encryption. Starting XML/SOA infrastructure companies from scratch will enable such new startups to build ground up on relatively newer and useful standards such as WS-Policy, WS-SecureConversation, etc. The second cycle will be shorter with exits taking 3-4 years unlike the 5+ years needed by the 1st wave of SOA consolidations.

For an interesting interview on Sonoa by Eric Knorr, see this video clip.

Testing SOA Applications: App Labs Article

AppLabs Technologies is a Sequoia Funded testing company based in Hyderabad, India. AppLabs services encompass Functional, Performance, and Quality Testing with a distinctive SOA Testing focus. They published a good article on Approach to SOA Testing Applications. This article does a great job on detailing issues associated with Functional and Performance Testing.

On functional testing, it points out that most testing tools are focused on unit testing and are incapable of building composite interdependent tests across technology platforms, languages and systems.

On the performance testing front, this article takes the position that:

"Once the appropriate performance scenarios have been defined, multiple test tools/techniques are required because of the presence of different platforms and technologies. During test execution, monitoring application performance and collating data would be a challenge since there is no “one stop shop” tool which gives insight into the overall big picture."

Although obtaining a composite functional & performance picture may require source code access to figure out root bottlenecks, in modern, web services-based SOAs, the atomic web service "producer" API is what is being tested for performance characteristics. The web service operation internals are a black box operation that may internally call other web services. If all dependent contracts are advertised and available to a SOA Tester, then overall big picture performance characteristics are readily available from existing SOA Testing Tools.

The visibility does stop at the WSDL API level that is provided to the SOA Tester however. For more detailed view into performance and functional characteristics, white box testing is required. This may not be feasible with the proliferation of SaaS. The next closest view into functional and performance characteristics of a web service may be obtained through Grey Box testing. See SOA Testing using Black, White, & Gray Box Techniques for details.

With the proliferation of web services-based SOA within Enterprises and SMBs, AppLabs is focused on the right space and is positioned to capitalize on the increasing testing needs of complex SOA deployments.

Testing SOA Applications and Services

Mercury/HP published a good article on SOA Testing highlighting some facets of "non-GUI" testing. Mercury solution seems to address Functional and Performance testing for SOA deployments. These 2 aspects of web services-based SOA testing are part of the 4 pillars of SOA testing necessary for comprehensive SOA test coverage. Interoperability and Vulnerability testing form the other two necessary Pillars of SOA Testing.

One of the features that stands out is Mecury's "stub-simulation" capability that allows testers to simulate services to build test suites without requiring access to the target production system.

For complete article, see Testing SOA Applications and Services (registration required).

Watch your SOA Testing Blind Spots

This latest article by Crosscheck Networks' R&D team highlights various common SOA Blind Spots that SOA testers experience in real-life deployments.

The Blind Spots and remedies focus on the following areas:

1. Performance
2. Security
3. SOAP Attachments
4. WSDL
5. Interoperability

For these 5 areas of SOA Testing, the article describes how to identify common SOA blind spots and techniques for avoiding them.

The full article (PDF), can be downloaded here: Watch your SOA Testing Blind Spots

Mashup Camp, MIT

I enjoyed attending a few sessions at Mashup Camp. Overall, there is significant innovation and electricity around what the Mashup community is putting together. I enjoyed Jinesh Varia's (Amazon) presentation on building your own YouTube (iTube!). The use case involved uploading-> storing(S3)-> queuing(SQS)-> processing(EC2)-> hosting(ECS)-> Filtering(MTurk)-> Searching(perhaps Hadoop on EC2) video/images. The use case was intriguing since it logically and convincingly utilized parts of the Amazon Infrastructure. This highlights the point in my earlier blog that the cost of innovation is dramatically lower and even when scalability is required, the cost will align with specific computation & storage needs.

I also enjoyed looking at Mashups developed by LignUp, VOIP-SOA company. LignUp provides a comprehensive platform for controlling and integrating VOIP traffic within an enterprise network. The use case that I found most intriguing is having voice tied into CRM systems. This would give a fuller picture on communication between a company and its customers beyond just email archives.

I bumped into Julio Burgo from Carmun, a web portal for students to share and store academic research papers. I love their tag line: "Students of the world Unite!"

For a list of all the attendees, see Mashup Attendees.

Amazon EC2 and Oracle SOA Suite a Strong Combo

With Mashup Camp (Boston) right around the corner, I was happy to see Dr. Dobbs Journal publish an article that I have collaborated on with David Shaffer, Sr. Director Product Mgmt, Oracle Integration and Rizwan Mallal, from Crosscheck Networks, R&D. The article is written to drive home an important set of shifts that are a result of Amazon's Utility Compute Platforms, EC2, S3, SDS, etc. and it's intersection with web services products such as Oracle SOA Suite 10g.

In this article, we focused on how the Oracle SOA Suite can be used beyond its conventional business process role and more as a core SOA infrastructure traffic management platform for content-based routing, load balancing and fail over type functions for Amazon EC2 Linux instances.

http://www.ddj.com/dept/webservices/196900803

Prediction: In the next 2-3 years, perhaps sooner, Amazon EC2 in combination with SOA Suites will cause a fundamental shift in corporate computing. Also, EC2 & SOA will unleash unprecedented low cost innovation where startups will get off the ground with far lower capital than that required in the pre-Amazon and pre-web services era.

Web Services Testing SugarCRM

SugarCRM is cost-effective, powerful and flexible open source CRM system built on PHP and MySQL Database. Its a great CRM system for cost-conscious companies that need a solid CRM platform. At $40/user/month for a hosted On-Demand offering, its priced for the frugal.

Besides its great CRM functionality and reasonable pricing, what I really like about SugarCRM is its extensive web services API. This enables the SugarCRM platform to be easily integrated with the IT ecosystem that is increasingly becoming web services aware.

If you a have a web site portal that captures customer registration information, and you'd like to integrate that with a Leads Capture Mechanism, I highly recommend that you consider SugarCRM. Its SOAP API makes the integration process between your web portal and the SugarCRM Leads Module simple. The fields in the Leads Module are highly customizable and all your customized fields are available via the SOAP API as well.

For a Quick Start on SugarCRM Web Services Functionality, see the following article:

Getting Started with SugarCRM Web Services using SOAPSonar

RadView's PdM & Dev blog: SOA closing the gap between functional and performance testing

The following blog by the Radview Product Management team shows how quickly companies are adapting to SOA requirements. I think they have clearly identified customer needs both from a testing and traffic management perspective. With their products becoming web services aware, Radview will meet customer needs for web services testing.

RadView's PdM & Dev blog: SOA closing the gap between functional and performance testing

NuSOAP: An easy to use SOAP Stack for PHP

If you need to make your PHP-based application make SOAP calls into an application with a WSDL-based interface, consider using NuSOAP. It is simple to use and will get you up and running within minutes. The best part is that you dont have to install any new PHP Extensions. All you have to do is to download and unzip the NuSOAP files in your PHP-enabled web server and you are ready to go.

A couple of good resources for getting started with NuSOAP are:

1. NuSOAP Home.
2. Introduction to NuSOAP.

JavaWorld: Naked Web services

JavaWorld published a piece titled "Are your Web Services Naked or Covered?" I co-authored this article to highlight the impact of exception handling techniqes on web services-based SOA. You will learn exception handling constructs that ensure your web services do not leak sensitive information to the consumer.

"Bottom line: For externally facing web services exposed to a large number of public consumers, only recoverable exceptions should be communicated. All other exceptions should be controlled, cleansed, or suppressed."

http://www.javaworld.com/javaworld/jw-12-2006/jw-1220-wsexcept.html

Building & Testing your first Web Service using .NET

Alright - it this takes more than ten minutes for you to do, then I will change the instructions and try to simplify things so that you can build & test your first web service in less than 10 minutes. The over all steps are as follows:

1. Download and install .NET WebMatrix.

2. Goto: Start > All Programs > Microsoft ASP .NET Web Matrix > ASP .NET Web Matrix.

3. You will be prompted for with a screen shown below. Fill in the information as shown. Select "Web Services" in the left panel and the "Simple" template in the right panel. I created a C:\WebServices folder to store the .asmx file and picked C#.

3. This will auto-generate a web service for you with the following code:

4. Hit the play button in the IDE and it will prompt you for the start the web application on port 9090. Make sure your local firewall is turned off.


5. A web browser with operation Add will appear. You can click this and start playing with your first web services.

Within 5 simple steps, you should have successfully setup a web services.

Note: Web Matrix cannot be accessed remotely so all your testing has to be local. For remote accessibility you can deploy your web services file (MathWS.asmx) on an IIS server.

SOA testing and building is as easy as 1,2,3 -- and 4, 5 :-)

Oracle BPEL Process Manager

The following article published in SOA Web Services Journal provides a good glimpse into interoperability issues between Microsoft .NET WCF and Oracle BPEL Process manager. It highlights a couple of interesting items:

1. WS-Addressing is required for endpoint representation especially for long running transactions where the server invokes a callback to the client for sending the response once the processing is complete. Oracle BPEL Process Manager supports a different version on WS-Addressing than Microsoft .NET WCF. So out-of-the-box, the WS-Addressing will not work. Luckily, Oracle BPEL Process Manager supports WS-BPEL extensions (bpelx:headerVariable) that provides the BPEL process access to header information sent by the client. With a handle on header information, the BPEL process manager can manipulate WS-Addressing information and ensure that the the Process executes and that the call-back is compliant with what the .NET WCF client is expecting.

2. When a server is WS-Security enabled and is expecting a User Name Token, an X.509 token, or a SAML assertion in the SOAP Header, the BPEL Process Manager can again use WS-BPEL extensions such as (bplex:inputHeaderVariable) to stuff SOAP Headers with tokens before invoking an endpoint that expects this information.

Really Cool Stuff!

Web Services and SOA
— People sometimes ask what a service-oriented architecture enables today that could not have been done with the older, proprietary integration stacks of the past 5 to 15 years, such as those from Tibco, IBM, or Vitria. One such ability is the greater degree of interoperability between heterogeneous technology stacks that is made possible by the standards SOA is built on, such as Web services and BPEL. Although interoperability is only one facet of the SOA value proposition, it is one that has become increasingly more important, due in large part to the evolving IT environment, merger and acquisition activity, and increased partner connectivity.

CSI 33 Orlando

I like the fact that Orlando Airport offers a free wireless connection and that I am able to make this blog entry in front of Terminal 73 while I wait for my flight back to Boston.

CSI 33 is a Computer Security Conference where security professional gather to stay current with IT security related issues. This is my second CSI and just by the attendance at my talk, I can see the rapidly increasing interest amongst Security Professional in testing, discovering and remediating SOA security related issues. CSI 33 had an entire Web Service Track with four sessions around SOA Security and Threat related issues. I chose to present under the Attacks and Countermeasures track and was pleasantly surprised that the room was almost packed for an early morning session. I was also impressed by the level of questions coming in from the audience. One individual from Salesforce.com was focused on security issues for the web services based AppExchange interface with over 400 third party applications developed against their API.

More so than ever, I believe that Web Services & SOA security is something security professionals are not just aware of, but now see as their responsibility. Security Professionals are making great strides in understanding the nuances of web services security and how it is a logical extension of their domain given that they are already dealing with application security issues within HTTP(S), HTML, Cookies and the HTTP header in general. Now they have to go deeper in the SOAP packets and make sure that the back end systems are tested and secured for SOAP-borne vulnerabilities.

BEA WebLogic 9.2: Testing SOAP Encryption

One of the most powerful capabilities of application servers such as BEA's new WebLogic 9.2 is message-level security. Such features make WebLogic a natural component for a SOA deployment.

In the article, posted on BEA's dev2dev.bea.com website, the message level encryption capabilities of WebLogic are explored and a step-by-step guide is provided from downloading the server to setting up you first policy.

http://dev2dev.bea.com/pub/a/2006/10/message-level-encryption.html


Give WLS 9.2 a spin, its simple, easy and powerful.