Part II: Solutions for Web API Testing

In this section, we’re going to look at different ways and tools we can test a Web API. It’s because a different tool, something that isn’t an automation framework, might actually be more suitable for your needs. Learning a new Java technology, building your own framework, all of that requires time and effort, and what if building an automation framework is actually not the optimal solution for your circumstances? So let’s first briefly discuss for the other ways and tools of testing an API, and you might just as well decide that these other ways are actually better for you. In other article, I will describe WHY it is necessary to know other ways for testing API (cheat-sheet: to make sure your code reflects the reality).

Manual Testing: cURL & Automated Testing: PostMan.

We’l use these tools to test the Web API. To do that, we’ll have to define a couple of tests, and then we will execute these tests, to get a feeling for those tools. But before that, let’s see where the Web API testing sits in the grand scheme of quality assurance.

Testing Pyramid

If you work or somehow are related to quality assurance in automation, you may have heard by now that there are many types of testing: unit testing, functional testing, system testing, integration testing, component testing, etc. They’re all useful and they don’t compete against one another; they complement each other. So let’s take a look at this classing testing pyramid.

Testing Pyramid

We start at the bottom: unit tests. A Web API is a piece of software and if you imagine it as a black box, then you could say that the unit tests written by developers exercise small, separate chunks of code within that box. If that black box has 100 little functions, then unit tests will work with each of these 100 functions separately. Unit tests are great, but they can’t guarantee that software works correctly with other software. In other words, these tests can’t guarantee that the box will work correctly when talking to the outside world, to a client. This is where component tests come in. Functional component tests are mostly about sending input to the black box and verifying the output. Send a request, get a response. We don’t care about the magic that happens inside that black box, as long as the output is correct. This is the kind of testing for “API”. For the completion of the picture, let’s mention also the integration tests. Integration tests come into play when software is big and is split into multiple modules. Suppose you only have two modules so you get two black boxes that talk to each other. One module is done by developer A and the second by developer B. They both finished their work and now someone has to verify that they worked together as expected. That’s integration testing. One way you could test that is to use an external client, the one on the left pictured as a laptop, to send a request to the first box, the first box talks to the second box, which sends some output all the way back. Finally, the user interface or UI testing. That’s usually the easiest to understand, but generally more difficult to get right. Let’s say you have a shopping website and with UI automation, you can automate all of the clicking that the user can do. Browse, choose an item, enter your credit card details, and pay for the item. As you might have guessed, this kind of testing doesn’t really apply to Web API testing. It simply doesn’t have a user interface because it’s a machine-to-machine application. So no interaction with humans and no buttons to click. Although it is worth noting that you can actually create a user interface with buttons that make HTTP calls to a web service, but we won’t be doing that. That’s an extra layer between you and the web API, and in this section, we’re talking to the API directly. Now, let’s define some tests and start with the component testing.

Defining Tests

We’ll use behavior-driven approach or BDD that uses the format given when then.

Given: <some condition>
When: <action>
Then: <expected output>

*Anything else will mean the test failed.

Let’s try and formulate several tests.

Test #1
Given: API url is http://api.github.com
When
: User sends a GET request
Then: Response status code is 200.
Test #2
Given: API url is http://api.github.com/nonexistingendpoint
When: User sends a GET request
Then: Response status code is 404.

Enough for now. We will have many more once we start automating, but that’s good enough to explore curl and PostMan.

Manual API Testing with cURL

So what is curl? Well, it’s just a command line tool, that’s it. You type the commands with some arguments, execute those and see what you get as an output on the screen.

cURL syntax

Fair enough, we need to specify url. Try this: curl https://api.github.com. Will work out! But just a body of the response. Will not see status codes or any header. What we need to do is add the verbose flag, or just -v. Try this: curl -v https://api.github.com. Now we have a lot more output. Before we look at the response, I’d like to take a look at what we sent in the request. We sent this to api.github.com server. Alright.

curl HTTP Request

We used the Http GET method. We didn’t specify that, but that’s the curl default, but there is more. There’s also this User-Agent and Accept header. We didn’t specify those but it does make sense for a tool to have sensible defaults because Web APIs expect those. For example, a Web API can serve both JSON and XML, and as a client, you might specify, hey, I only want XML or I only want JSON. A wildcard means, I don’t care, give me whatever you have. Manipulating these headers allows you to pretend to be a different client in any single test, which is pretty useful. Let’s move on to the response.

curl HTTP Response

There are a lot of headers containing a lot of interesting information. Just what we’d expect from a real Web API, but our test should focus on one thing only at a time. So let’s just check the status code. Yes, it’s 200. Good. Let’s move on to the next test and type in nonexistingendpoint: curl -v https://api.github.com/nonexistingendpoint, and there we see a 404 – Not Found:

negative scenario – curl HTTP Response

So, since we’re writing code anyway, can’t we just write mode code such as this and automate with curl? Yes, we could, but it gets ugly fairly quickly. Here’s an example of the command if we want to get just the status code line:

curl -v –silent https://api.github.com/ 2>&1 | grep Status

This will give you the whole line, not the integer 200, and if you want just the integer 200, then your command becomes something similar to this:

output=$(curl -v –silent -I https://api.github.com/ | head -n 1| cut -d $” -f2)

If you’re comfortable with Bash, that’s great, but to everybody else, this gets pretty unreadable fairly quickly, and this is just the most simple get request where we get the status code. No posting, no deleting, setting custom headers, making complex verification. We’re just trying to get a simple integer. Now compare this to what we could write in Java:

HttpGet get = new HttpGet(url);
response = client.execute(get);
int code = response.getStatusLine().getStatusCode();

Create a get, execute that, get the status code. Not as short as the curl command, that’s true, but much, much more readable. This resembles human language so it’s easier to read and understand, and that’s one of the top priorities for any kind of test, readability.

So perhaps you can now see why curl isn’t great for automating hundreds of complex tests. It doesn’t mean that it should be completely ignored though.

Curl advantages:

  • Ubiquitous. It’s a simple command line tool, so just about anyone can use it. No complex setup is a good thing as well.
  • Smoke Tests. Making a simple quick request to see if the API is up and running is a good case to use curl.
  • Easy to reproduce. If you have failing test and it’s fairly simple, you can write a curl command, give it to the developers to literally just copy/paste into their terminal, and let them reproduce the issue quickly and conveniently.

Manual API Testing with POSTMAN

PostMan started as a Chrome extension, but eventually as it became very popular, it became a standalone desktop application. It’s powerful, it’s easy to get started with, and has a nice user interface. If the API you have to test is relatively simple or you have any reason to believe that starting an entire automation framework could be an overkill, or there is a very tight deadline, PostMan might actually be your better choice.

It’s very easy to get started. Open PostMan, choose your HTTP method, type in the URL and click Send. You’ll see the response body in one tab, headers in another. We can even see the status code separately and has tool-tips with helpful documentation.

The method selection indicates why you can’t really use the browser for proper API testing. Yes, you can send requests and inspect the response with a browser, but when you type in a URL in the address bar and you hit Enter, you use Get and Get only. You can’t use POST or DELETE for Options or manipulate cookies easily, or set your headers.

Anyways, it all feels pretty manual still. This is where the Tests tab comes in, and in this tab, you have to write JavaScript; not Java. So you still need to write code, but it’s actually rather simple and PostMan has a great documentation on how to write these tests. All of these tests will run after we send the request and receive a response. See:

Before we click Send (write tests in JS)
After we click Send (test results)

Boom! That took less than a second and all of our tests executed within that time. That’s where you start seeing the real benefit of automated tests. You may invest some time into learning and coding, but the return on that investment definitely pays off. PostMan wouldn’t be a complete automation tool without the possibility of organizing tests into test sets. PostMan calls them collections, and with those, you can organize your tests in any way you liike:

PostMan Collections

Everything we discussed is just the tip of the iceberg, and you can do much more with PostMan. There’s a number of tools to explore, and I’ll just mention one. One of the top players is SoapUI. It’s a desktop application and it acts as a client talking to the Web API, which means it does the same thing that we’ve been doing all this time; sending HTTP requests and evaluating HTTP responses. It is considered as a complete solution to all your API testing needs. It has a free version and the paid version. Even the free one, does pack a lot of functionality. You can write scripts with it, but in a lot of cases you don’t even have to. You can create tests just by clicking around the user interface.


Summary

We looked at a lot of things that have nothing to do with writing Java code, but we did this for a reason.

  1. Manual Testing. First, we looked at the manual testing with curl. We saw that we could just type in commands and get back all the necessary information, but that code becomes fairly complicated very quickly, and as a general rule, almost no one likes maintaining tests. Even less so if they are difficult to read. Thousand of tests were thrown out to window just because nobody wanted to spend a lot of time understanding them, and really long curl commands don’t help in that respect.
  2. Automation tools. Next, we took a look at one automation tool, PostMan, and we saw it’s really user friendly, easy to get started with, a top choice for any quick and dirty testing, but it does require JavaScript knowledge to actually write automated tests. This can be a good choice if the API you’re testing is not huge and complicated, but do remember that PostMan has a free version and a paid version. Meaning the free version has limitations, and you might hit those limitations and then you’d have to pay for the extras. This is not a concern if you write your own solution.
  3. Own Framework. Finally, we glanced very briefly at what your own code might look like, which I’d say wins in readability over curl and that has no limitation in terms of what you can do compared to third-party software. It’s more difficult to write your own solution, but in big projects, complete freedom can really pay off.

Source: "Getting Started with Web API Test Automation in Java" by Andrejs