Testing a java spring boot rest api with karate

Testing a Java Spring Boot REST API with Karate

Learn how to write web service tests with Karate, a new framework for scripting HTTP sequences specifying expectations.

Brought to you by

Semaphore

Introduction

This tutorial will show you how to write web service tests with the Karate framework. We will start with a short introduction about the basic features, then we will explore the domain-specific language (DSL), and learn how to apply it to verify a web service's response and the returned JSON structures.

As integration with existing frameworks is important, we will demonstrate how to integrate well-known tools like JUnit, TestNG, and Maven to execute our tests and generate reports that may be used for integration-servers/services.

This tutorial will also cover other interesting aspects of this framework like configuration files, data tables, switching HTTP client implementations or more complex use cases like testing multipart file uploads.

Finally, we will show you how easy it is to set up a build plan on Semaphore for Karate tests in a few seconds.

Prerequisites

We assume that you posses general knowledge of the HTTP protocol, the JSON notation, basic Java β„’ knowledge and experience in using build tools like Maven.

To integrate Karate in our project using Maven as build tool, we only need to add the following two dependencies if JUnit is our test framework of choice:

<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-apache</artifactId>
  <version>0.4.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-junit4</artifactId>
  <version>0.4.1</version>
  <scope>test</scope>
</dependency>

Integrating with TestNG as alternative is convered later on in this tutorial.

About Karate

Karate is a new web services testing framework build on top of the popular Cucumber library. It eases the testing process allowing developers without Java β„’ knowledge to script sequences of HTTP calls and specify expectations for the web services response easy and fast, offering a custom domain-specific language.

The project itself is quite new as the first commit was in February 2017 but its author and main contributor, Peter Thomas is very busy adding new features and pushing new releases.

Features of Karate

In addition to the features mention above, the Karate library offers a variety of additional features:

  • Cucumber based, Gherkin syntax supported β€” IDE support and syntax-highlighting are supported,
  • 'Native' JSON and XML support including JsonPath and XPath expressions,
  • Re-usable scripts and feature files that may be called from other scripts and feature files,
  • Embedded JavaScript engine that allows to write reusable functions in Javascript,
  • Re-use of payload-data and user-defined functions across tests,
  • Configuration switching/staging support,
  • Multi-threaded parallel execution support,
  • Ability to invoke Java classes from a test,
  • Select between Apache and Jersey HTTP client to avoid conflicts with existing dependencies, and
  • Simple latency assertions may be used to validate non-functional requirements in respect of performance requirements.

Many other features are listed on the GitHub project.

Implementing Karate Tests

We're now ready to write our first tests. Tests consist of a Java β„’ class that might be recognized by the designated test framework used and a feature file that describes our interaction with the service under test.

Directory Structure

The test classes are save in src/test/java/feature and our feature files in src/test/resources/feature β€” this follows the naming conventions and allows the Java β„’ class to find its corresponding feature file.

└── src
    └── test
        β”œβ”€β”€ java
        β”‚Β Β  └── feature
        └── resources
            └── feature

Feature Files

Karate features are written in a DSL that we'll be covering in the following examples.

Our features are generally stored in resources/feature so that feature files and Java β„’ tests are matched by their name and package structure.

JUnit Integration

All we need to do to integrate JUnit is to create a test class addressing our corresponding JUnit runner, so that a minimal approach could look like the following class.

package feature;

import org.junit.runner.RunWith;

import com.intuit.karate.junit4.Karate;

@RunWith(Karate.class)
public class SomeFeatureTest {}

If we need to set up instrumentation for our tests, we can use JUnit's lifecycle mechanisms like @Before, @BeforeClass, @After, @AfterClass, test rules and so on.

Simple Hello World Example

We're now ready to start with a simple example.

Assuming that we've got a simple RESTful web service running on localhost (e.g. http://localhost:3000/quote) that returns a random quote when called in a GET request.

The response is returned using the JSON format and looks like this one:

{
    "quote":"Hello world, use Karate! :)"
}

We're starting by writing our expectation in a new feature file named quote.feature saved in src/test/resources/feature/quote:

In this feature file, we're defining a new feature named "Quote generator", and setting a global setting for our base url (named url).

Afterwards we're adding one scenario for fetching random quotes and assure that the response status code is 200 and that the quote in the response's JSON structure is not null using a special marker.

The following markers are currently supported:

  • #ignore Ignores the field.
  • #null Value must be null.
  • #notnull Value must not be null.
  • #array Value must be JSON array.
  • #object Value must be JSON object.
  • #boolean Value must be true or false.
  • #number Value must be a number.
  • #string Value must be a string.
  • #uuid Value must match the UUID format.
  • #regex Value matches a regular-expression.
  • #? EX Javascript expression EX must evaluate to true.

More detailed information about the rich syntax of Karate's DSL can be found in the project's wiki.

Feature: Quote generator

Background:
* url http://localhost:3000

Scenario: Fetch random quote

Given path '/quote'
When method GET
Then status 200
And match $ == {quote:'#notnull'}

In the next step, we're adding a Java β„’ class so that JUnit's test-runner executes our test.

Following the naming conventions, we're creating a the following class named QuoteTest in src/test/java/feature/quote:

package feature.quote;

import org.junit.runner.RunWith;

import com.intuit.karate.junit4.Karate;

@RunWith(Karate.class)
public class QuoteTest {}

We may now run our test using Maven in the command line as follows:

$ mvn test                               
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running feature.quote.QuoteTest
11:56:31.648 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - init test class: class feature.quote.QuoteTest
11:56:31.756 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - loading feature: /data/project/karate-bdd-testing/target/test-classes/feature/quote/quote.feature
[..]
1 > GET http://localhost:3000/quote
[..]
1 < 200
1 < Content-Type: application/json
1 < Server: Jetty(9.2.13.v20150730)
1 < Transfer-Encoding: chunked
1 < Vary: Accept-Encoding, User-Agent
{ "quote": "Hello world, use Karate! :)"}
[..]
1 Scenarios (1 passed)
5 Steps (5 passed)
0m0.866s

Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.874 sec

Results :

Tests run: 6, Failures: 0, Errors: 0, Skipped: 0

Or alternatively using our IDE of choice:

Running Karate Tests in Intellij IDEA

Data Tables Example

Data tables allow us to add datasets in a tabular form as examples for our tests, and also use this information to fill our tests and assertions.

Feature: Name transformation

  Background:
    * url 'http://localhost:3000'

  Scenario Outline: Transform multiple names

    Given path '/name'
    And request {name:'<name>'}
    When method POST
    Then status 200
    And match $ == {length:'<length>'}

    Examples:
      | name  | length |
      | Tim   | 3      |
      | Liz   | 3      |
      | Selma | 5      |
      | Ted   | 3      |
      | Luise | 5      |

Again, we can run these tests using Maven in the command line like this:

$ mvn test                            
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running feature.name.NameTest
13:43:23.567 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - init test class: class feature.name.NameTest
13:43:23.688 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - loading feature: /data/project/karate-bdd-testing/target/test-classes/feature/name/name.feature
[..]
1 > POST http://localhost:3000/name
1 > Content-Type: application/json
{"name":"Tim"}
[..]
1 < 200
1 < Content-Type: application/json
1 < Server: Jetty(9.2.13.v20150730)
1 < Transfer-Encoding: chunked
{"length":"3"}
[..]
5 Scenarios (5 passed)
30 Steps (30 passed)
0m1.288s

Tests run: 35, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334 sec

Results :

Tests run: 35, Failures: 0, Errors: 0, Skipped: 0

Or alternatively using our IDE of choice:

Using datatables running in IntellJ

Complex Example

To demonstrate Karate's capabilities when dealing with file uploads and Multipart requests, we're going to write tests for a RESTful web service for file management.

This service allows us to upload files that are stored in a temporary directory, and also to list all stored files.

Throughout its APIs, the JSON format is used for the server's response messages.

For demonstration purpose, we have hacked together a quick and ugly implementation using Spring Boot.

File Store Service

This is our simple file store application written with Spring Boot:

package com.hascode.tutorial;

// imports ..

@SpringBootApplication
public class Application {

  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }

}

This is our RESTful web service implementation:

package com.hascode.tutorial;

// imports ..

@Controller
public class FileUploadController {

  // setup, temp-dir etc ..

  @GetMapping("/")
  @ResponseBody
  public ResponseEntity<List<FileDto>> listFiles() {

    return ResponseEntity
        .ok()
        .contentType(MediaType.APPLICATION_JSON)
        .body(Arrays.stream(store.listFiles()).map(FileDto::of).collect(Collectors.toList()));
  }

  @PostMapping("/")
  @ResponseBody
  public ResponseEntity<Map<String, Object>> uploadFile(@RequestParam("file") MultipartFile file)
      throws IOException {
    // store file

    Map<String, Object> response = new HashMap<>();
    response.put("error", false);
    response.put("bytesUploaded", file.getBytes().length);

    return ResponseEntity
        .ok().contentType(MediaType.APPLICATION_JSON).body(response);
  }

  @ExceptionHandler(FileNotFoundException.class)
  public ResponseEntity handleStorageFileNotFound(FileNotFoundException e) {
    return ResponseEntity.notFound().build();
  }

  // simple DTO
  private static class FileDto {

    private final String name;
    private final long sizeInBytes;

    private FileDto(String name, long sizeInBytes) {
      this.name = name;
      this.sizeInBytes = sizeInBytes;
    }

    public static FileDto of(File file) {
      return new FileDto(file.getName(), file.length());
    }

    public String getName() {
      return name;
    }

    public long getSizeInBytes() {
      return sizeInBytes;
    }
  }
}

Starting the RESTful Web Service

We may run our service now using our IDE of choice or by using Maven in the command line as follows:

$ mvn spring-boot:run                    
[..]
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.5.3.RELEASE)
[..]
06-09 13:53:39.316  INFO 14498 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
[..]
2017-06-09 13:53:39.792  INFO 14498 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/],methods=[POST]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> com.hascode.tutorial.FileUploadController.uploadFile(org.springframework.web.multipart.MultipartFile) throws java.io.IOException
2017-06-09 13:53:39.793  INFO 14498 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/],methods=[GET]}" onto public org.springframework.http.ResponseEntity<java.util.List<com.hascode.tutorial.FileUploadController$FileDto>> com.hascode.tutorial.FileUploadController.listFiles()
[..]
2017-06-09 13:53:40.270  INFO 14498 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2017-06-09 13:53:40.274  INFO 14498 --- [           main] com.hascode.tutorial.Application         : Started Application in 2.463 seconds (JVM running for 4.887)

Accessing the Service

We are now ready to access our RESTful web service and practice some Karate with it.

First, let's have a short introduction of the APIs exposed and how to access them using regular tools like curl or Postman.

Storing a New File

We may upload files to the service that are stored in a temporary directory.

In the following examples, we're uploading sample PDF files to our webservice.

Uploading a file with curl:

curl -vi -XPOST -F file=@test.pdf http://localhost:8080/

Uploading a file with Postman:

File upload using Postman

Listing Stored Files

The second API allows us to list stored files and receive a JSON response containing the file's names and size (in bytes).

List stored files with curl:

 $ curl -vi -XGET http://localhost:8080/

Response:

Assuming that we have uploaded two files named test.txt and test1.txt the following JSON structure is returned.

 [
   {
     "name": "test1.txt",
     "sizeInBytes": 53
   },
   {
     "name": "test.txt",
     "sizeInBytes": 41
   }
 ]

List stored files with Postman:

Listing uploaded files with Postman

Karate Test for the File Service

To demonstrate how easy it is to test file-uploads and/or file-multipart requests, we're going to write a quick test for the file server above.

Test Configuration

In the first step, we're using a separate configuration file to load the base url from a file and support staging.

For this purpose, we're adding a file named karate-config.js to the directory src/test/resources:

function() {   
  return {
    baseUrl: 'http://localhost:8080'
  }
}

From now on, we're referencing our base-url in our feature-files like this:

Background:
* url baseUrl

Feature File

This is our feature file that describes how we're interacting with our webservice's upload and file-listing functions.

Feature: Uploading a file

Background:
* url baseUrl

Scenario: Upload file

Given path '/'
And multipart field file = read('test.pdf')
And multipart field name = 'test.pdf'
When method POST
Then status 200
And match $ == {error: false, bytesUploaded:'#notnull'}

Given path '/'
And multipart field file = read('test1.pdf')
And multipart field name = 'test1.pdf'
When method POST
Then status 200
And match $ == {error: false, bytesUploaded:'#notnull'}


Scenario: List uploaded files

Given path '/'
When method GET
Then status 200
And match $.length() == 2

Directory Structure

Our test directory structure should now look similar to this one:

src/test
β”œβ”€β”€ java
β”‚Β Β  └── feature
β”‚Β Β      └── file
β”‚Β Β          └── FileTest.java
└── resources
    β”œβ”€β”€ feature
    β”‚Β Β  └── file
    β”‚Β Β      β”œβ”€β”€ file.feature
    β”‚Β Β      β”œβ”€β”€ test1.pdf
    β”‚Β Β      └── test.pdf
    └── karate-config.js

Running the Test

We may run our tests again using Maven in the command line or using our IDE of choice.

$ mvn test
[..]
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running feature.file.FileTest
17:05:46.548 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - init test class: class feature.file.FileTest
17:05:46.671 [main] DEBUG com.intuit.karate.cucumber.CucumberRunner - loading feature: /data/project/karate-bdd-testing/target/test-classes/feature/file/file.feature
[..]
1 > POST http://127.0.0.1:45074/
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Type: multipart/form-data; boundary=fgmmzFn4TdUJNb-aFXVLkGTwHY9GCma2fZ
1 > Host: 127.0.0.1:45074
1 > Transfer-Encoding: chunked
1 > User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_45)
17:05:47.105 [main] DEBUG com.intuit.karate - 
1 < 200
1 < Content-Type: application/json
1 < Date: Thu, 06 Jul 2017 15:05:47 GMT
1 < Transfer-Encoding: chunked
{"bytesUploaded":6514,"error":false}
[..]
2 > POST http://127.0.0.1:45074/
2 > Accept-Encoding: gzip,deflate
2 > Connection: Keep-Alive
2 > Content-Type: multipart/form-data; boundary=YqMIBXLu-C1HQkMo0AuggtF2vcd3-XuWRJ
2 > Host: 127.0.0.1:45074
2 > Transfer-Encoding: chunked
2 > User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_45)

17:05:47.170 [main] DEBUG com.intuit.karate - 
2 < 200
2 < Content-Type: application/json
2 < Date: Thu, 06 Jul 2017 15:05:47 GMT
2 < Transfer-Encoding: chunked
{"bytesUploaded":6514,"error":false}

17:05:47.170 [main] DEBUG com.intuit.karate - response time in milliseconds: 6
17:05:47.225 [main] DEBUG com.intuit.karate - 
1 > GET http://127.0.0.1:45074/
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: 127.0.0.1:45074
1 > User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_45)

17:05:47.239 [main] DEBUG com.intuit.karate - 
1 < 200
1 < Content-Type: application/json
1 < Date: Thu, 06 Jul 2017 15:05:47 GMT
1 < Transfer-Encoding: chunked
[{"name":"test.pdf","sizeInBytes":6514},{"name":"test1.pdf","sizeInBytes":6514}]

17:05:47.240 [main] DEBUG com.intuit.karate - response time in milliseconds: 14

2 Scenarios (2 passed)
18 Steps (18 passed)
0m1.106s
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.585 sec

Results :

Tests run: 20, Failures: 0, Errors: 0, Skipped: 0

The Karate demo section offers a test-setup using an embedded Spring Boot instance, too as well as other interesting examples.

Using TestNG Instead of JUnit

To use TestNG instead of JUnit, we simply need to change our Maven setup to use the following two dependencies:

<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-apache</artifactId>
  <version>0.4.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-testng</artifactId>
  <version>0.4.1</version>
  <scope>test</scope>
</dependency>

Choosing the HTTP Client

Since Karate 0.3.0 we may select which HTTP client to use for running our tests, by adding the designated Maven coordinates β€” at the time of writing this article, Apache and Jersey implementations are supported.

For Apache we're adding the following dependency to our project's pom.xml:

<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-apache</artifactId>
  <version>0.4.2</version>
  <scope>test</scope>
</dependency>

For Jersey we're using this dependency:

<dependency>
  <groupId>com.intuit.karate</groupId>
  <artifactId>karate-jersey</artifactId>
  <version>0.4.2</version>
  <scope>test</scope>
</dependency>

Reports

Karate uses the surefire-plugin to generate standard reports in xml format that may be processed by standard mechanisms of common integration servers and providers like Semaphore.

target/surefire-reports
β”œβ”€β”€ feature.user.UserManagementTest.txt
└── TEST-feature.user.UserManagementTest.xml

Running Karate Tests on Semaphore

As Karate integrates well with known tools like Maven, JUnit or TestNG, setting up jobs or build plans for integration servers is an easy task. We'll use Semaphore for continuous integration and demonstrate how to set up a build plan within a few seconds.

First, sign up for a free Semaphore account if you don’t have one already.

First create a new project and authorize with GitHub or Bitbucket depending on the fact where your Git(https://git-scm.com/) repository is located.

Afterwards you may search and add the designated repository:

Selecting the Git repository on Semaphore

In the next step, you select the repository's target branch:

Selecting the target branch on Semaphore

Having done that, Semaphore pulls the repository and analyses its structure.

Semaphore analyzing the repository

Depending on this analysis, Semaphore recommends a build setup so that we just need to modify a few steps.

In this example we switched the Java β„’ version to 1.8 and reduced the Maven tasks to a simple execution of mvn test.

Our build plan on Semaphore

Having created the plan we may now lean back and enjoy watching our build plan being run by Semaphore within a few seconds.

Semaphore running the build plan

Finally we're able to see that our build plan ran successfully within 40 seconds

Semaphore build plan success

Conclusion

We hope that this tutorial helped you to get a basic understanding of Karate's features and possibilities and that you now have another utility in your tool box for testing web services of all kinds.

As the project is in an early stage, please feel free to contribute or file new feature requests β€” the speed in which new features are implemented is quite impressive and Karate's author, Peter Thomas responds very fast.

If you found this testing framework interesting, please feel free to share this tutorial. You're also welcome to leave any comments or questions here!

Happy testing :)

871fffe6cd70d8352e3b7c14a3932823
Micha Kops

Software engineer and consultant, highly interested in software architecture, agile methods and continuous learning and curiosity. More of his work can be found on his blog and website.

on this tutorial so far.
User deleted author {{comment.createdAt}}

Edited on {{comment.updatedAt}}

Cancel

Sign In You must be logged in to comment.