Click here to close now.




















Welcome!

Microservices Expo Authors: AppDynamics Blog, Elizabeth White, Tim Hinds, Liz McMillan, Trevor Parsons

Related Topics: Microservices Expo

Microservices Expo: Article

SOA World Product Review — Testing...1...2...3...Testing

A review of SOAtest 5.0

Few topics evoke more groans and eye rolling from software engineers and Web developers than the dreaded "TESTING." Testing falls into the same category as documentation, refactoring code, dusting, and visiting the dentist. Put it off until the last minute, do as little as possible, do it quickly, and move on to something else. I must confess that I have the same visceral reaction to the thought of 'testing' as others do. Consequently, I approached the prospect of reviewing a testing tool with the loathing of visiting the dentist. I was very relieved to discover that Parasoft's SOAtest 5.0 took a lot of the pain, frustration, and busy work out of the testing experience.

Parasoft's SOAtest 5.0 is a comprehensive testing and analysis tool suite tailored to the unique testing and validation needs of Service Oriented Architectures. It supports functional testing, scenario-based testing, stress testing, client testing with a mock service, and a whole range of validation capabilities (XML Schema, WSDL, WS-Security, BPEL, etc.). SOAtest 5.0 then further supports the creation and automated execution of regression test suites. It supports a broad range of SOA specifications and standards, and is designed from the ground up to support the dynamic and evolving nature of service-oriented systems.

Getting Started with SOAtest 5.0
Installing SOAtest was a breeze. The process is as follows:

  • Run the setup executable from media or by downloading it from www.parasoft.com,
  • Follow the on-screen prompts.
  • Decide if you want to set up SOAtest as a Windows service.

Once the software is installed, you're ready to begin using SOAtest. The first time that you launch the software, you'll be prompted either to input your individual license information or point to an available license server.

After providing licensing information, you will be confronted with an option to create a New Project, Open an Existing Project, or access the SOAtest Tutorial. The tutorial is well-written and provides a nice tour of the tool's major features.

Creating Test Cases
Tests can either be created individually or as a part of a larger test suite. The tool seems to drive you toward creating test suites rather than individual tests, which is nice for reuse, organization, and best of all - regression testing a collection of tests. SOAtest supports the creation of test suites from a wide range of sources including:

  • Web Services Description Language (WSDL) files
  • Business Process Execution Language (BPEL) files
  • Universal Description, Discovery & Integration (UDDI) registry end-points
  • Web Services Inspection Language (WSIL) files
  • BEA Aqualogic Enterprise Repository end-points

The test suite creation wizard will ask you to designate a file, URL, or end-point to query and also ask you to designate what tests you want to create. Upon completing the wizard, you'll have a whole set of tests automatically generated and ready-to-run as is or customized prior to execution.

Functional Testing
I was very impressed with the functional verification testing capabilities. SOAtest supports the following functional verification features:

  • Check schema validity against a WSDL
  • XML-aware diff engine that flags only true XML structure changes
  • Surgical inclusion/exclusion of XML message elements in the verification process via XPATH (see Figure 1)
  • A graphical rules engine for managing assertions

Every one of these features is enabled through simple intuitive menu options, dropdown lists, check boxes, and XML tree structures. The interface is simple to navigate and although there's a wealth of options, they're organized to avoid the feeling of being overwhelmed.

Executing one or more functional tests multiple times based on an external data set (database or spreadsheet) was another feature that I sought out. SOAtest came through in this area as well supporting the ability to point at a data source (CSV, Excel, relational databases, etc.) and run through a battery of tests that pull values out of the data source and send service requests containing the extracted data values. This lets you define a single test case or suite of tests and then automatically test the full range of data values that the test case needs to support without creating additional tests for each value.

Testing with Mock Services
In a truly test-driven environment, I'd create service interfaces and corresponding suites of test cases before any implementation code is developed. Although this is good in theory, without a supporting toolset, it's not very realistic. SOAtest provides all the capabilities needed to support this kind of development model. When creating a new project/test suite, simply point the tool at a WSDL file and check the "Generate Server Stubs" radio button. The server stubs will run on SOAtest's embedded Tomcat server and let you create and run testing scenarios before you've written a single line of implementation code. This way, when you're ready to implement the service interfaces, you have a defined a suite of tests for verification. This helps to control the scope (when you meet all the tests, stop working) and provides one or more test suites that can be incorporated into automatic regression testing (see below) to ensure that functionality isn't compromised as the project progresses.

Of all the features that I've worked with, I found the service mock-up process the most cumbersome. The other aspects of SOAtest were intuitive and easy-to-work with, whereas I had to wrestle a while to get the mock service capability working on anything other than the tutorial walk-through. In the end, the functionality of this capability was excellent, albeit a bit difficult to initially configure.

Regression Testing
Running your tests once is nice. Running your tests regressively is better. Running your tests regressively and automatically at night is divine. SOAtest supports all three scenarios. Any of the test suites that you define in a SOAtest project can be converted to regression test suites. Furthermore, using XPATH you can selectively indicate which portions of the test cases may change from test to test and which values should never change. Once you have a set of regression tests that you're happy with, SOAtest provides a command-line mechanism to kick off your test suites automatically. Thus in an agile, continuous integration environment you can run regression testing at night, at lunch, or every hour on the hour to ensure that you find bugs early and often.

Reporting
SOAtest provides several reporting features (auto-generated reports for nightly regression tests, on-demand, detailed summary reports for test suites, and WS-I interoperability reports). I found the quality and readability of the reports developed by SOAtest to be quite good. Figure 2 provides a snapshot of part of the SOAtest detailed report. The WS-I interoperability reports were pretty low quality, difficult to navigate, and provided information overload. In fairness to SOAtest, those reports are copyrighted by WS-I and seem to be auto-generated by one or more WS-I tools. Consequently, I'm not sure that Parasoft has any control over the quality of these reports. Nonetheless, I would have liked a WS-I conformance report of the same quality and user-friendliness as the native SOAtest reports.

Advanced Testing Features
The palette of testing and analysis features in SOAtest is extensive. In the interest of not filling up this entire magazine with feature descriptions, I'll list several of the compelling features that I've not covered already:
- Scenario-based testing where subsequent test cases depend on data returned from previous test cases
- Stress/load testing your SOA and specifying Quality of Service parameters
- Testing non-XML services (JMS, MQ, TIBCO, EJB, REST, Binary, Text, etc.)
- Validate SOAP security using WS-Security (encryption, digital signatures, and authentication)
- Asynchronous service testing
- Custom scripting with Python, JavaScript, or Java (Figure 3)

Drawbacks
As I mentioned earlier, the mock service feature is a bit awkward to work with initially. Also, the WS-I conformance report was not up to the same quality standards as the native SOAtest reports. With SOA's strong integration and interoperability play, there are a lot of enterprises that have Java and other services that all need to be involved in the same business process and testing scenario.

Summary
In spite of my general dislike for software analysis and testing, I found Parasoft's SOAtest 5.0 to be a well-designed tool that took a lot of the pain and work out of testing and validating a SOA. The tool isn't perfect, but it is easily one of the best SOA testing tools that I've ever worked with.

Product Snapshot

Target Audience: SOA architects, developers, QA/testers, and analysts

Level: Beginner to Advanced

Pros: Powerful, intuitive UI, robust testing and analysis tools

Cons: The mock service feature is a bit awkward, only supports Java clients for now

Testing environment: Dell Inspiron 640m, 1.6GHz Intel Core-Duo, 2GB RAM, Windows XP Pro with SP2

Platforms: Windows 2000/XP, Linux, Solaris

More Stories By Kyle Gabhart

Kyle Gabhart is a subject matter expert specializing in strategic planning and tactical delivery of enterprise technology solutions, blending EA, BPM, SOA, Cloud Computing, and other emerging technologies. Kyle currently serves as a director for Web Age Solutions, a premier provider of technology education and mentoring. Since 2001 he has contributed extensively to the IT community as an author, speaker, consultant, and open source contributor.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
SOA News 04/04/07 04:11:45 PM EDT

Few topics evoke more groans and eye rolling from software engineers and Web developers than the dreaded 'TESTING.' Testing falls into the same category as documentation, refactoring code, dusting, and visiting the dentist. Put it off until the last minute, do as little as possible, do it quickly, and move on to something else. I must confess that I have the same visceral reaction to the thought of 'testing' as others do. Consequently, I approached the prospect of reviewing a testing tool with the loathing of visiting the dentist. I was very relieved to discover that Parasoft's SOAtest 5.0 took a lot of the pain, frustration, and busy work out of the testing experience.

@MicroservicesExpo Stories
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Opening Keynote at 16th Cloud Expo, S...
Software is eating the world. The more it eats, the bigger the mountain of data and wealth of valuable insights to digest and act on. Forward facing customer-centric IT organizations, leaders and professionals are looking to answer questions like how much revenue was lost today from platinum users not converting because they experienced poor mobile app performance. This requires a single, real-time pane of glass for end-to-end analytics covering business, customer, and IT operational data.
Approved this February by the Internet Engineering Task Force (IETF), HTTP/2 is the first major update to HTTP since 1999, when HTTP/1.1 was standardized. Designed with performance in mind, one of the biggest goals of HTTP/2 implementation is to decrease latency while maintaining a high-level compatibility with HTTP/1.1. Though not all testing activities will be impacted by the new protocol, it's important for testers to be aware of any changes moving forward.
"ProfitBricks was founded in 2010 and we are the painless cloud - and we are also the Infrastructure as a Service 2.0 company," noted Achim Weiss, Chief Executive Officer and Co-Founder of ProfitBricks, in this SYS-CON.tv interview at 16th Cloud Expo, held June 9-11, 2015, at the Javits Center in New York City.
You often hear the two titles of "DevOps" and "Immutable Infrastructure" used independently. In his session at DevOps Summit, John Willis, Technical Evangelist for Docker, covered the union between the two topics and why this is important. He provided an overview of Immutable Infrastructure then showed how an Immutable Continuous Delivery pipeline can be applied as a best practice for "DevOps." He ended the session with some interesting case study examples.
The Internet of Things. Cloud. Big Data. Real-Time Analytics. To those who do not quite understand what these phrases mean (and let’s be honest, that’s likely to be a large portion of the world), words like “IoT” and “Big Data” are just buzzwords. The truth is, the Internet of Things encompasses much more than jargon and predictions of connected devices. According to Parker Trewin, Senior Director of Content and Communications of Aria Systems, “IoT is big news because it ups the ante: Reach out ...
"We got started as search consultants. On the services side of the business we have help organizations save time and save money when they hit issues that everyone more or less hits when their data grows," noted Otis Gospodnetić, Founder of Sematext, in this SYS-CON.tv interview at @DevOpsSummit, held June 9-11, 2015, at the Javits Center in New York City.
Auto-scaling environments, micro-service architectures and globally-distributed teams are just three common examples of why organizations today need automation and interoperability more than ever. But is interoperability something we simply start doing, or does it require a reexamination of our processes? And can we really improve our processes without first making interoperability a requirement for how we choose our tools?
At DevOps Summit NY there’s been a whole lot of talk about not just DevOps, but containers, IoT, and microservices. Sessions focused not just on the cultural shift needed to grow at scale with a DevOps approach, but also made sure to include the network ”plumbing” needed to ensure success as applications decompose into the microservice architectures enabling rapid growth and support for the Internet of (Every)Things.
How do you securely enable access to your applications in AWS without exposing any attack surfaces? The answer is usually very complicated because application environments morph over time in response to growing requirements from your employee base, your partners and your customers. In his session at @DevOpsSummit, Haseeb Budhani, CEO and Co-founder of Soha, shared five common approaches that DevOps teams follow to secure access to applications deployed in AWS, Azure, etc., and the friction an...
Digital Transformation is the ultimate goal of cloud computing and related initiatives. The phrase is certainly not a precise one, and as subject to hand-waving and distortion as any high-falutin' terminology in the world of information technology. Yet it is an excellent choice of words to describe what enterprise IT—and by extension, organizations in general—should be working to achieve. Digital Transformation means: handling all the data types being found and created in the organizat...
This week, I joined SOASTA as Senior Vice President of Performance Analytics. Given my background in cloud computing and distributed systems operations — you may have read my blogs on CNET or GigaOm — this may surprise you, but I want to explain why this is the perfect time to take on this opportunity with this team. In fact, that’s probably the best way to break this down. To explain why I’d leave the world of infrastructure and code for the world of data and analytics, let’s explore the timing...
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin,...
JavaScript is primarily a client-based dynamic scripting language most commonly used within web browsers as client-side scripts to interact with the user, browser, and communicate asynchronously to servers. If you have been part of any web-based development, odds are you have worked with JavaScript in one form or another. In this article, I'll focus on the aspects of JavaScript that are relevant within the Node.js environment.
Discussions about cloud computing are evolving into discussions about enterprise IT in general. As enterprises increasingly migrate toward their own unique clouds, new issues such as the use of containers and microservices emerge to keep things interesting. In this Power Panel at 16th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the state of cloud computing today, and what enterprise IT professionals need to know about how the latest topics and trends affect t...
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
Containers are changing the security landscape for software development and deployment. As with any security solutions, security approaches that work for developers, operations personnel and security professionals is a requirement. In his session at DevOps Summit, Kevin Gilpin, CTO and Co-Founder of Conjur, will discuss various security considerations for container-based infrastructure and related DevOps workflows.
Microservices are hot. And for good reason. To compete in today’s fast-moving application economy, it makes sense to break large, monolithic applications down into discrete functional units. Such an approach makes it easier to update and add functionalities (text-messaging a customer, calculating sales tax for a specific geography, etc.) and get those updates / adds into production fast. In fact, some would argue that microservices are a prerequisite for true continuous delivery. But is it too...
Summer is finally here and it’s time for a DevOps summer vacation. From San Francisco to New York City, our top summer conferences list is going to continuously deliver you to the summer destinations of your dreams. These DevOps parties are hitting all the hottest summer trends with Microservices, Agile, Continuous Delivery, DevSecOps, and even Continuous Testing. Move over Kanye. These are the top 5 Summer DevOps Conferences of 2015.
Countless business models have spawned from the IaaS industry. Resell Web hosting, blogs, public cloud, and on and on. With the overwhelming amount of tools available to us, it's sometimes easy to overlook that many of them are just new skins of resources we've had for a long time. In his General Session at 16th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, broke down what we've got to work with and discuss the benefits and pitfalls to discover how we can best use them to d...