Integration testing is as popular among software developers as security and when you realize you need to do it, it’s almost too late.

But what is a good testing strategy, and what kind of test do really you need?

Well, like always, it depends. It depends on a variety of things, but also on your mindset and what you think about testing.

We have 100% UnitTest Coverage! Isn’t that enough?

BTW: Check out the code for this post: https://github.com/peterquiel/karate-getting-started-guide/tree/master/karate-docker-and-mocks

I heard very often that a test where you use a real database isn’t a unit test. Well, I believe that testing a database repository without a real database doesn’t have much value, don’t you think? Uncle bob thinks the same way:

We need unit tests and that way of testing software is my first choice, but in times of highly integrated applications, I heavily rely on Integration Tests and I use the term Integration Test as defined in the Spotify Testing of Microservice post. The post differentiates between integration and integrated tests.

An integration test fails or passes based on the correctness of systems you own.

And

An integrated test that will pass or fail based on the correctness of another system.

Suppose you spin up three of your own services, a MongoDB, and PostgreSQL database in your test case. If that test fails, it’s pretty likely that the test is wrong or you have a bug in one of your services, don’t you agree?

If you want to own the services used in your tests, you ultimately conclude that you have to create mocks for service you don’t own.

This leads to the question of how you manage the life-cycle of your Mocks, your Services, and needed third-party Services like databases?

Tool to Manage Integration Test Setup

I use Karate Mock Server to develop my mocks because it’s easy and powerful and it creates a separation of mock and test code by nature.

I use Docker (well,… who doesn’t?) to start third-party Services and I prefer to start Services I own with Docker if I ship these Services as a Docker Image as well, because that’s closer to the production system.

Testcontainers is a Java library and has become the tool of choice to start, manage and stop Docker Containers during test cases. I used Docker Compose in combination with Gradle to manage the Docker Container life-cycle in my tests before Atomfrede showed me Testcontainers. BTW: Check out his Blog.

This post is about developing API Mocks with Karate and I will cover Karate Mocks as Docker Image and setting up the whole thing with Testcontainers in successive posts.

Karate Mock

Karate uses the same syntax to write mock as for writing API tests. That means that you create a feature file as you would when writing a test case and you write a Background and Scenarios. However, there are new concepts you have to understand.

First, Karate starts a server with the feature file as a parameter. The server interprets the feature file and runs as long as you stop the server.

Second, the server life cycle has two steps. The first step is the background evaluation and is used to initialize a global state. You can read files, set up a JSON array that is used for a simple collection of mock data, and so on. The server executes the background only once at startup and not before every scenario (request). This is a major difference compared to the behavior when writing tests.
In the second step, the server waits for requests to handle.

Third, every scenario title is an expression (a predicate) that is evaluated for every request. If that predicate is true that Scenario is responsible to handle the request. The server does the check from top to down.
The first thing I do when I start with Karate Mock development is to place a catch-all scenario at the end of the feature file that logs the request. Then I write a Karate Test for that mock to check if the server starts correctly. It’s a Test-Driven Approach that makes fun because you see the result immediately.

Again: What Are the Differences Between Karate Tests and Karate Mocks?

The fundamental differences between a Karate Mock and a Karate API Test are:

  1. 2-step feature file execution by the server.
  2. Each scenario title is a JavaScript expression that is used to match whether a Scenario should handle a request.

Example

Let’s look at the simple predicate in the Scenario Title:

Scenario: pathMatches('/greeting') && methodIs('get')
    * def response = { content: 'Hello World!' }
    * def responseStatus = 200

As mentioned before, the scenario title is JavaScript and Karate comes with many functions that you can use to check how you want to handle the request.
In this example, we only check if the URI path matches /greeting and the HTTP-Method is get. If that is the case, this scenario handles the request.

You can find a list of all Karate functions that you can use in a scenario title in the Karate Mock Request Handling guide.

Path Parameter

The pathMatches provides a way to extract path parameters directly by wrapping the name in curly braces.

The path /greeting/{pathVariable} would match any path, starting with greeting an arbitrary following path segment. Karate stores that path segment into the variable named pathVariable.

Wildcard Path Match

pathMatches does not support any wildcard style as you may know from the ant style pattern. You can use requestUri.startsWith('foo/bar/) to match the foo/bar resource and any sub-resource.

Catch All Scenario

I start every Karate mock feature with a catch-all Scenario that prints out request information that helps me to debug the mock.

A catch-all scenario has no title at all:

  Scenario:
    * print 'No dedicated scenario matches incoming request.'
    * print 'With Headers:'
    * print requestHeaders
    * print 'With Request Parameters'
    * print requestParams
    * print 'And Request:'
    * print request

The catch-all Scenario has the lowest possible matching score. Therefore, it only handles the request if there is no other matching scenario.

Response Definition

To define the response and status that you want to return, define variables named response and responseStatus respectively.
If you want to set response headers, you simply define a variable responseHeaders and assign a JSON array representing your HTTP-Headers.

Response definition is fairly easy and you could use the same patterns for response creation you already apply to create requests when testing APIs.

Before I let you read the Response-Building-Guide I want to mention two useful concepts.

First, you can delay the response by defining the duration in milliseconds to a variable name: responseDelay. 

Scenario: pathMatches("/delayed/response")
  * def responseDelay = 4000

This is very useful to simulate network problems. You could increase the delay with every request to verify client timeouts and that your service doesn’t pass on the delay to its consumers.

Second, Karate provides a Proxy Mode that I haven’t used yet but seems quite useful for more integrated test cases.

Ok, have an overview of how a Karate Mock works, and the comprehensive Karate documentation should help you whenever you don’t know how to match a request or to build a request.

Next, I will show you how you can easily start and test a Karate Mock.

Start Karate Mock Server

You can start a Karate Mock in a different way and each way has its advantage.

The good thing: this doesn’t affect your feature mock file itself. You write your mock and you can start the server in different ways. Depending on your needs.

Start Karate Mock Server Directly from Feature

You can start a Karate Mock directly from a Feature and that is very useful if you want to test your Karate Mock and execute that test directly from your IDE. Most IDE support running Karate Scenarios directly. Read my Karate Getting Started article if you don’t know that.

Put the following code into your Background and that will spin up the Karate Mock Server before a Scenario executes:

Background:
  * def start = () => karate.start('demo-mock.feature').port
  * def port = callonce start
  * url 'http://localhost:' + port


The start function starts the Karate Mock server for the feature file and returns the port of the server. The server will select a free port.
It’s possible to define a port and I will use that feature in the following section, where we start the Karat Mock server via a JUnit test case.

I use the callonce feature to execute the start function, because I don’t want to start a new Karate Mock Server for each Scenario we run.

Use JUnit Runner to Start Karate Mock Server

You can start a Karate Mock server in your JUnit with the Java API using the following code:

public class DemoMockTestRunner {
  private static com.intuit.karate.core.MockServer mockServer;

  @BeforeAll
  static void startMockServer() {
    final File mockFile =   ResourceUtils.getFileRelativeTo(DemoMockTestRunner.class, "demo-mock.feature");
    mockServer = MockServer.feature(mockFile).build();
  }
  ...

This example starts with one MockServer for all TestCases. The Karate Mock would share the state between test cases. If that’s not what you want, you simply have to move the Code into a @Before non-static method.

One caveat of the MockServer.feature() method. It accepts a path as a String to the feature file, but the method doesn’t support a relative path. That’s the reason I use the ResourceUtil to look up the feature file relative path.

Don’t forget to stop the Karate Mock in a @AftreAll annotated method:

@AfterAll
static void stopMockServer() {

    if (mockServer != null) {
        mockServer.stop();
    }
}

Spice Up! Karate Mock Server as JUnit5 Extension

I used the former @Rule and @ClassRull JUnit occasionally to extract startup and tear down logic into a separate reusable class. JUnit5 doesn’t support these annotations anymore and comes with a more generic and consistent Extension API.

The KarateMockServerExtension.java is a JUnit5 extension and makes it easy to start and stop a Karate Mock Server.

The following code shows the DemoMockTestRunner using that extension:

public class DemoMockTestRunner  {

    @RegisterExtension
    static KarateMockServerExtension mockServerRule = KarateMockServerExtension
        .create(DemoMockTestRunner.class, "demo-mock.feature");

    @Karate.Test
    Karate testDemoMockServer() {

        final Karate runner = Karate.run("demo-mock-test");
        runner.builder()
            .systemProperty("mock_server_port", String.valueOf(mockServerRule.getPort()))
            .relativeTo(getClass());
        return runner;
    }
}

The @RegisterExtension is the JUnit5 way of adding a JUnit5 Extension to the Jupiter Engine in a programmatic way. JUnit5 offers extension registration via annotation and java service loader. These are a suitable match if you don’t need to configure your extension. Configuration via annotation gets can get clumsy quickly.

Don’t forget to make the KarateMockServerExtesion a static field. There is an implicit contract between a programmatic registered extension implementing AfterAllCallback and a modifier of the field. If the field is not static JUnit won’t call the after-all callback.

Start Karate Mock Server from Command Line

You can start a Karate Mock Server directly from the command line with the Karat.jar (there is no dedicated karate standalone jar anymore since Karate version >= 1. x): 

java -jar karate.jar -m src/test/java/com/stm/karate/mock/demo-mock.feature -p 8081

A useful feature if you want to build Docker Images for your Karate Mocks and enable other teams to use your mocks as well.

General Note: The mock server won’t evaluate karate-config.js. Therefore, I recommend making your Karate Mocks “karate-config agnostic” and using system properties to configure them – if you need configuration at all.

Testing a Karate Mock

Testing a Mock, why should I do that?

Fast feedback is crucial in software development.

I want to have the confidence that my code works as I expect, and I want to have that confidence for my API Mocks as well. Fast feedback is key to gaining confidence and I get that fast feedback by writing tests and executing these as often as possible.

Imagine the opposite. You write code for a long time and then you execute it after – let’s say two hours – the first time. I’m always afraid of executing that code for the first time. Does it work? Did I do something wrong? Do you have this feeling too?

Execute it often to get confidence that you are on the right path.

That’s the reason I create a Karate Mock with a catch-all Scenario along with a Karate Test where I start that mock and make my first test.

The initial Karate test for mock looks like this:

Feature: Testing our demo mock server with a feature that startes the demo mock server in first

  Background:
    * def start = () => karate.start('demo-mock.feature').port
    * def port = callonce start
    * url 'http://localhost:' + port

  Scenario: Test catch all
    * path "does", "not", "exist"
    * method get
    * status 200
    * match response.status == "OK"

I start the mock server using karate.start on a free port. I use calonce to start the mock server only once and not for every feature and point the url to the mock server.

The scenario just calls a non-existing mock path and checks if status=OK is returned by the catch-all rule. Check out the mock server and the mock server test.

Conclusion

Developing and testing API Mocks with Karate is simple and I like the separation of java code and mock declaration Karate enforces because of its nature.

Short turnarounds are king for testing and integration testing often lacks fast execution especially when you have to spin up many services, databases, and API mocks. Testing your mocks speeds up integration test development and ensures that you trust your API mocks.

Having all the Karate power available for Mock development and for API testing speeds up my productivity and makes it easy to deal with complicated JSON requests and responses.

The next post is about using Testcontainers together with Karate mocks in your integration tests to create a reliable setup that you can use for local development and for integration tests as part of your CI/CD pipeline.

Want to know when I publish these posts? Subscribe to my Newsletter!