7 New Features in Karate Test Automation Version 1.0

Karates’ journey started roughly four years ago and these four years haven’t been easy, how could they? Imagine all the time it takes to drive an open-source project all the way until it gets noticed, used, and finally, got listed on the Thought Works Technologie Radar - even if it’s not listed on the current round.

And today, after 4 years, version 1.0 is in reach. Peter Thomas already released the third candidate, and there isn’t much left.

That’s a reason for me to take a closer look at the upcoming version, and here it is, my list of the most favored features of Karate Automation Framework version 1.0.

Fell free to download the code examples from: https://github.com/peterquiel/karate-getting-started-guide/tree/master/karate-standalone-setup/src/examples/users

Support for all Java versions 8 and above#

Karate relies on the JavaScript Engine Nashorn that runs on the JVM. Oracle deprecated that engine in JDK 11 and JDK 15 ships without Nashorn. This results in the fact that Karate won’t run on JDK 15 or above.

Think of the situation you want to use Karate in your Tool Stack and you know that Karate isn’t future proof. This would be a major showstopper for me if I have to decide.

Peter Thomas must have been aware of that, and he invested a lot to make this happen. He refactored major parts and lifted Karate on top of the GraalVM JavaScript Implementation.

GraalVM? Do I have to use another JDK to run karate now?

Not really. The karate-core comes now with all the dependency you need - this is actually another improvement - to run it on every JDK version 8 and above, and that includes JDK version 15 in particular. Needless to say, it’s JDK 16 ready as well!

Another idea that pops up in my mind when I hear GraalVM is: native image support.
Native Image is a technology to ahead-of-time compile Java code to a standalone executable, called a native image. 

These executables don’t need a JDK at all and run faster, but I ran into a few issues with creating a native image, and since this isn’t a blog post on how to create a karate native image, I didn’t dig deeper into this topic.

On the other hand, Karate starts and runs fast enough on top of a JVM and doesn’t need a performance boost necessarily.

Improved error details for match failures#

Karates match command with its == , contains and contains any operators are very powerful. The JSON schema expressions add another level of dense expressiveness, and that confused me sometimes when I try to understand why this particular case didn’t work.

It looks like I wasn’t the only one having problems understanding why a matcher’s expression didn’t match or match.

Karate matcher,… talk to me!

The new matcher talks to you in a more detailed way to explain the

Examples#

This will fail, because c2 is 2 and not 3.

1
2
3
4
5

Scenario: Pretty Printed Matcher

* def actual = {a: 1, b: 2, c : {c1: 1, c2: 2}}
* match actual == {a: 1, b: 2, c : {c1: 1, c2: 3}}
  

Before Karate 1.0:

With Karate 1.0:

Another example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

Scenario: Matcher pretty, type dosen't match

* def actual =
  """
{
  "id": 1,
  "email": "Sincere@april.biz",
  "username": "Bret",
  "company": {
      "bs": "harness real-time e-markets",
      "catchPhrase": "Multi-layered client-server neural-net",
      "name": "Romaguera-Crona"
  }
}    
  """
* match actual contains {id :"1"}

Before improvement:

After improvement:

HTML Templating Engine - Thymeleaf Integration#

Along with Karate 1.0 release comes a Thymeleaf

Thymeleaf integration for report rendering. This is what you always wanted to generate human-readable reports.

This is a really valuable feature you want when you integrated Karate into your testing environment and you use Karate testing results as a communication medium or as a QS report.

Using templates is easy, and the integration in your test case is straightforward.

First, you create a simple HTML template file like this one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

<table class="table table-striped">
    <thead>
        <tr>
            <th>Id</th>
            <th>Name</th>
            <th>Email</th>
            <th>Company</th>
        </tr>
    </thead>
    <tbody>
        <tr th:each="user: users">
            <td th:text="${user.id}"></td>
            <td th:text="${user.name}"></td>
            <td th:text="${user.email}"></td>
            <td>
                <div th:text="${user.company.name}"></div>
                <div th:text="${user.company.catchPhrase}"></div>
            </td>
        </tr>
    </tbody>
</table>

Second, you render that template using the already known doc command:

1
2
3
4
5
6
7

Scenario: Html Templateing
Given url 'https://jsonplaceholder.typicode.com'
* path 'users'
When method get
Then status 200
* def users = response
* doc {read : 'users.html'}

The resulting report looks like this:

One other important thing to note about the new integrated Thymeleaf Engine is that Karate is actually dogfooding this for its own reports, as you can see from the karate-step.html template.

When Peter told me about that dogfooding the next thought that came to my mind is: “Can you configure the template karate should use?” and the answer is: “Configuration isn’t part of the current release candidate 0.9.9.RC3, but will be in 1.0.”

So, in the future, you will be able to customize the report as you like and to load your customized templates from a jar for instance. This enables you to share your reporting templates across teams and your company.

I could even imagine that third parties publish their own report templates to the maven repository, and you can easily take advantage of that.

Distributed Testing#

Often you decide to introduce a new tool into your tech stack and you may have evaluated that tool using a comparison matrix, but you can’t think of every use case and for scaling your stack if often gets dirty.

It’s too late to step back, and it has to work.

API testing isn’t the fasted tool you have on your test workbench, and automated browser tests are even worse. While during API tests the machine mostly waits for IO and has nothing to do, browser testing becomes quite CPU intense.

Sometimes you want to test how fast a website renders on a slow machine, and you, therefore, you want to render the website with CPU reduced setup.

Distributed Testing helps you to fulfill these requirements when you want to improve test execution time. It’s not an easy setup, it adds complexity, it will increase your false-positive rates, because some of your tests might not run in parallel, but having this opportunity is better than none.

Karate 1.0 comes with Distributed Testing support, and Peter Thomas asks for your support since it’s difficult to test and debug.

I haven’t tried it. It’s like reactive programming, it’s fun to play with, but you don’t want to use it in production until you have to.

Nevertheless, it’s an important feature if karate wants to be enterprise-ready!

ECMAScript 2015 / ES6 and Improved Error Details for JavaScript Failures#

I don’t know how often I used the => operator in Karate before I learned that Karate isn’t ES6 ready.

Supporting ES6 may look like a slight improvement. I mean, you don’t write loads of JS code in Karat, do you?

But since almost every JS environment runs ES6, it would feel like Karate is falling behind without that support. If you don’t move, you fall behind.

Just a little example, that shows pretty much of the new stuff:

1
2
3

Scenario:  ES6 Support
* def log = x =>  karate.log(`${new Date()}: ${x}`)
* log("test")

Output:

1

13:27:17.579 [main]  INFO com.intuit.karate - Mon Jan 25 2021 13:27:17 GMT+0100 (CET): test

The improved JS failures are worth mentioning as well because in the past I have faced different situations where I changed JS code in small incremental changes to find the reason.

I removed the e from new Date() in the code above and when I try to execute that, I will get this improved error message:

1
2
3
4
5

failed features:
js failed:
01: log("test")
org.graalvm.polyglot.PolyglotException: ReferenceError: "Dat" is not defined
.:=>(Unnamed:1)

JS Functions for Data Driven Testing#

Data-Driven test cases have been a really helpful tool for me. I used this feature often in combination with reading test data from a CSV file that karate turns automatically into a JSON.

With Karate 1.0 comes the feature to use a JavaScript function to generate your test data. The following example simply creates a bunch of users using the iteration index as part of the email:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

Scenario Outline: Js Function in Data Driven Test Case: Creating User <email>
* def user =
"""
{
    "name": "Test User",
    "username": "testuser",
    "email": #(email),
    "address": {
        "street": "Has No Name",
        "suite": "Apt. 123",
        "city": "Electri",
        "zipcode": "54321-6789"
    }
}
"""

    Given url 'https://jsonplaceholder.typicode.com/users'
    And request user
    When method post
    Then status 201

    Examples:
    | i => i > 2 ? null : ({email: `test_${i}@user.com`}) | 

Report:

This is a simple example that shows the expressiveness of the newly introduced ES6 support.

But is that all?

One thing that pops up in my mind that is much easier to use with this new feature is Property-Based Testing.

Property-Based Testing is an interesting approach. Unfortunately, I did not manage to make it a tool of my daily use. The simplicity - and its limitation - of imperative test cases are too strong.

I hope this feature enables others to experiment more!

Single Dependency#

Peter Thomas restructured the dependencies and now karate-core comes with all its dependencies in one jar. This might look like an anti-pattern if you think of modularity.

But Karate isn’t part of an enterprise application that consists of 100ths of jar files. Principles applied in that situation don’t necessarily apply here. It’s an interpreter for your .feature files, and a single dependency makes it easy to execute your feature file.

Simple execution is the driver of this improvement, and Peter Thomas provides a simple example of how jbang helps you.

In this Karate Java API showcase Peter Thomas demonstrates how you can execute a Karate test case using a simple command:

1

jbang https://github.com/ptrthomas/karate-showcase/blob/main/javadsl.java

Conclusion#

Releasing Version 1.0 is something special, it’s the state of your software as you imagined it when you started your journey.

The journey of Karate started ~4 years ago, and it gained a lot of attention during this period of time. Many people are using it every day and some people don’t like it at all. Some of them think, it’s BDD for people who can’t program or it’s just too different and they want to stay in their comfort zone. It’s easy to judge from the sidewalk.