As the name of the section suggests, for now we will be writing simple get requests only, but I can assure you, we will still learn a lot of interesting things.

Before we start writing tests, let’s set up our project and import some dependencies that we will heavily rely on, TestNG and Apache HTTP. Also, check in cmd for mvn and java -version commands.

Project Setup

1. Create a Project in an IDE
2. Import TestNG and Apache dependencies

First, we already have a defined structure: main and test. That’s the benefit you get when using maven. We will put all of our tests into the test folder, and if we come up with any helper functions, helper methods or anything else, that will go to main.

Second thing is this xml file called pom; that comes from maven. This is where we declare our dependencies, meaning if we want to use other people’s code packaged into jar files, we declare them here. Maven being a smart tool will pull it from the central repository from the internet.

Last thing, notice the External Libraries, and it looks pretty empty as well for now. We only see stuff that’s part of java 8 itself there, if expanding the folder.

Now, let’s create our first test, named Get200.java that will work with GitHub API:

Notice we have a lot of red. That @Test annotation is red, and the HttpGet object is also red, meaning this code won’t even compile. Well, that’s because test annotation is something that is supported by TestNG, and the HttpGet is part of the Apache library, and we don’t have this libraries imported. They’re not part of our project. To get those:

Go to https://mvnrepository.com > Search for TestNG and click on the first link > choose the (latest) version > copy the xml snippet > Go to the pom file and create a tag <dependencies></dependencies> and paste the snipped inside.
Annotation @Test now looks fine, and the External Libraries should be updated too. Great!

Notice also the small green arrow on the left. This is the integration of IntelliJ and TestNG and it enables a really convenient way to run our tests with a single mouse click:

Now, let’s do something about the HttpGet. In the https://mvnrepository.com search Apache HttpClient and repeat the same steps, just paste the xml snippet under the previously added. Now, notice our External Libraries have even more jar files.

Well, it’s time to finish with the kindergarten and write our first test – 200 OK!

I. Test for Status Codes

To execute all of our tests, we need a client. Until now, our client has been a browser, the curl tool, and PostMan, but from now on, a Java object will be our client. So let’s create a client that does the sending. To create it, use the class HttpClientBuilder, which is available since version 4.3.

HttpClient client = HttpClientBuilder.create().build();

All the versions have different ways to create a client. For example, we can create an object DefaultHttpClient, but IntelliJ will map it as deprecated; that’s because developers of Apache HttpClient marked it with annotation deprecated, so we won’t be using that.

So now our client object needs to know what kind of request to send; a get request, a post request, or a delete request? Let’s create a get request object. This object requires a parameter, the destination or the URL of the API. For now, we’ll just hard code that.

HttpGet get = new HttpGet("https://api.github.com");

Actually, it’s going to be the same BASE string everywhere, so let’s refactor it immediately into a constant: Select the string > Refactor tab > Refactor This:

Let’s name the new constant variable BASE_ENDPOINT. Let’s tell the client to execute that:

HttpResponse response = client.execute(get);

Let’s get a step back and bring up our slide from another section:

We created a client object on the left. That will do the same job as our browser or curl tool. We created an HttpGet object, specifying the location, and we told the client to execute that, send it to the server where the API is located. The API processes that request, and sends the response back, and we save that response to a variable.

Like I said before, we send some input, and we verify the output.

This response variable, like anything else in Java, is an object, which contains all of the necessary information about the output. Now we need to dig inside and extract the status code. The thing is, getting the status code is such a frequent thing to do, that the Apache developers created convenience methods for us:

int actualStatus = response.getStatusLine().getStatusCode();

The last thing we have to do is to compare what we got with what we expected. Assert the two values, actual and expected, are indeed equal. This assertion mechanism is provided by TestNG as well, and we will be using such assertion expressions quite a lot.

Assert.assertEquals(actualStatus, 404);

I chose 404 over the 200 because wanted to negatively test, first. The test should fail.

Perfect. Now we can be sure that the test won’t be passing by chance. It actually executes the intended verification. It is a good practice to make the test sometimes fail before making it pass. The tests you see both failed and passed, are generally more trustworthy.

Now, let’s change 404 to 200 and make it pass. Awesome! Our first passing test.

Test Setup and Teardown

Let’s select another endpoint from https://api.github.com/, ones that does not require authorization. For example, rate_limit_url and repository_search_url.

Let’s reuse the code from the previous test and change the url in our get object to point to a different endpoint. And the test name, ofc. Both should pass.

HttpGet get = new HttpGet(BASE_ENDPOINT + "/rate_limit");

Now, same for the third test. Run from class level and see – the third one is pending! The reason this is happening is we’ve used the number of concurrent connections. This means that when we ran our first test, it opened up a connection to the API, but it never closed it. It keeps it open. Then this second test runs, opens up another connection, and then never releases it. The default in Apache HttpClient is two connections, which means that the third test isn’t going to be allowed to have its own connection.

We could increase the number of the default connections, but if we’re going to have 1000 tests, are we going to try and have 1000 concurrent connections from our localhost? That’s not what we want. To solve this problem, we need to open a separate connection before each test, and we need to close that connection after each test.

TestNG comes with @BeforeMethod and @AfterMethod functionality, which is extremely useful. With this, we will be creating a new client before every test, and will close the client after every test. But the close method is in red, so this won’t compile:

Let’s go inside the HttpClient class. Ah! It’s not even a class, but an interface, and it says here that this gives us only the basic contract for an HTTP request execution. If we want more functionality that is actually implemented, we want to use an implementing class:

Old Value:                        New Value:
HttpClient client;                CloseableHttpClient client;
HttpResponse response;            CloseableHttpResponse response;

Result: All 3 tests passed.

Next step is categorizing classes. I have created a Get404 class for testing a nonexisting endpoint, and a Get401 class for testing endpoints that require authorization (user, user/followers and notifications). All of them are negative tests, and all of them passed.

Refactoring Tests

The number of our tests is starting to grow and so is code duplication. It’s important to refactor early, so let’s do just that. You’ll see the benefit it brings once we finish.

First thing, the base URL. If it changes and we have it in a hundred places, we need to change it a hundred times, and no, search and replace isn’t good. It’s easy to make a mistake or miss one.

I created a public class named BaseClass, cut the BASE_ENDPOINT constant declaration from the other tests, placed it into the new class and changed the modifier to protected. All of our test classes will extend and inherit from it. Ideally, it should be in the Properties file, so if you have multiple test environments, you won’t need to change the code, but for our purposes, this is good enough.

Next, assert, and that can be a static import. Replace All the Assert with the assertEquals static import by using IDE shortcuts. Now things are shorter and more readable.

Old: Assert.assertEquals(actualStatus, 401);  
New: assertEquals(actualStatus, 401);

There’s one more thing I’d like to show you, and that’s the power of the data provider. Look at these tests, they’re almost identical:

The expected result is the same, everything is the same except the endpoint. It would be nice to have this code once, and just pass five different values, one at a time.

Go to the Get401 test, leave just one and delete the other two 401 tests. Declare a dataProvider to the only remaining test. Add a parameter of type string called endpoint, and create the data provider that will pass all the values we want to the method, one by one.

@DataProvider
private Object[][] endpoints(){
    return new Object[][]{
            {"/user"},
            {"/user/followers"},
            {"/notifications"}
    };
}

Let’s trigger our test, and I expect it to run 3 times; in other words, one test should have three iterations. It passed! If you API under test has 20 different endpoints that behave the same way when, say, you’re unauthorized, you don’t need to write 20 tests. You only need one single test and pass it 20 different values. It’s a pretty awesome possibility, right? 🙂

Time to move on. We’ll be testing other headers now, and we’ll start with content type, which is very important.

II. Test for Content-Type Header

Let’s highlight something important before we continue. There are three different terms; MIME Type, Media Type and Content-Type. For our purposes, it’s the same thing. MIME Type or Media Type is what we get in the response; plain text, an image, a file, XML, JSON, you name it. And in the world of HTTP, the header is called Content-Type, which tells us the Media Type plus the encoding. In our case, the media type is JSON.

Create another class called ResponseHeaders and paste the code from another class. So now it’s the same as before: send a request and save the response to a variable. But this time, we need to fetch the media type somehow. Well, again, because it’s such an important header, the Apache library developers gave us a ready-made solution.

Header contentType = response.getEntity().getContentType();

We got to a point where we have a very specific header, the content-type saved to a variable. As you might remember, the entire header contains also the encoding. So:

assertEquals(contentType.getValue(), "application/json; charset=utf-8");

Run the test, and it passes!

This can be good enough if you’re happy to test both in one pass, the contentType and the encoding, but you might want to test just the media type without the charset. There are two options for this. One way, is you can take that string and slice it in half and use just the first bit in your assertion, but there is a better way.

Go back to the response variable and use a method called getEntity. That’s an object that contains the media type. To dig in it, we need to put it in another method like so:

ContentType ct = ContentType.getOrDefault(response.getEntity());

This method returns an object content type and here we can conveniently query either the charset or the MIME type. So let’s use that and also write an assert:

assertEquals(ct.getMimeType(), "application/json");

Awesome! Another green test.

III. Test for other Headers

Let’s try and validate another header. This time, let’s assert that the header server has the value GitHub.com. So we will again reuse some code from the previous test, and there’s no magic get server methods.

The thing is, a lot of the headers aren’t mandatory. They don’t have to be there. We can be sure that status code will always be there, but server or ETag?..

Developers of web APIs have a lot of control what they set in the response. So no library can give you a magical set of methods that gets you all possible headers.

That’s why at this point we just need to write our own header extractor code. It’s easy and once we write it, we will use it for all other headers. So let’s write a getHeader that takes two parameters: the HTTP response that we want to inspect, and the string – the header that we want to find.

String headerValue = getHeader(response, "Server");
assertEquals(headerValue, "GitHub.com");

The method does not exist yet. Create a getHeader method & implement the following steps:

1. Get all the headers
2. Loop over the headers list
3. If no header found – throw an exception
4. Return the header

Test should pass. Excellent! We now have a universal header getter, and since we want to reuse it, let’s move it to a util class in the main folder.

So now, we have our first util method and it’s available to use in any test we like. 🙂

Get Headers Java 8 Way

There’s an even simpler way to find a header using Java 8.

Previously, we wrote a lot of boilerplate code, looping with an if statement inside the loop. Java 8 gives us strings and lambdas and if you haven’t used them before, I definitely recommend you learn how to use them. In this video, I’ll we’ll to use Java strings and lambdas to find a header.

Declare another method, getHeaderJava8Way that has the same parameters. The first line is the same as well – getting all the headers that the response contains, but the rest is going to be very different.

Explanation. We take a list of headers and we stream them. We make the headers flow in a stream one by one. As they flow past, we filter for a header that we care about, and after that, we invoke findFirst: find and return the first header that matches the filtering criteria, and if we don’t find any, throw an exception. Finally, return the string value of the header.

The syntax might appear awkward at the beginning, but once you get used to it, you will never look back.

Let’s write another test to see if this works. This time we will check the value of X-RateLimit. Notice the X at the beginning. This is supposed to tell us that this is not a standard header. It’s something that the developers created themselves.

This again shows that you can’t possibly have an Apache or any other HTTP library that supplies you with all the methods that gets you all possible headers. You just can’t know that API developers will come up with, and the value of this header is 60.

Passed. Great! If you don’t want to compare numbers as strings, that’s fair enough. You’re welcome to convert it to an integer and compare the values as integers. 😉

IV. Test for presence of a Header

Well, we’ve learned how t o test for a specific header value, but sometimes we don’t know in advance what the value is going to be.

For example, ETag. It helps Web APIs with caching, meaning it helps to determine if the client still has a fresh copy of the content. If no, send the content with code 200; if yes, send a 304 saying, use your own local copy.

So we don’t know that the value of ETag is going to be. It seems to be a harsh string that is going to be different every now and then. So we can’t test exact value, but we can and should test that ETag header is present on the response. So we will have to write another util method, and this one’s easier than the last one.

Let’s call it headerIsPresent and make it return a Boolean. It will take the same two arguments we are familiar with, the response we want to inspect, and the header we are interested in. This time, only wrote the Java 8 implementation. Let’s test now:

Boolean headerIsPresent = ResponseUtils.headerIsPresent(response, "ETag");
assertTrue(headerIsPresent);

It’ssss green. Great! That’s a job well done.

Summary

Time to sum things up. We learned a great deal in this section:

1. Set up the Project
   We started off by setting up our project using Maven and an IDE. We imported several dependencies we need to comfortably run our tests, especially Apache HTTP Client. Without it, we couldn’t communicate with the API.
2. Status Code and Content-Type Headers
   Wrote several simple tests for status codes and the Content-Type header, and that was pretty easy. These headers are so common and so universal that the Apache library provides us with convenience methods to get those out of the response object.
3. Other Headers
   This can be the case for all possible headers. There are just too many of them. Furthermore, API developers can create their own, so it’s much more practical to just write your own methods that are suitable for all possible headers.

So far, we have been limiting ourselves to the headers, the metadata of the response object. Let’s not forget is also has a body, the actual data, and that’s what we are going to test in the upcoming section. 😛