Karates journey started roughly four years ago and these four years haven’t been easy, how could they? Imagine all the time it takes to drive an open source project all the way until it gets noticed, used and finally, got listed on the Thought Works Technologie Radar – even if it’s not listed on the current round.

And today, after 4 years, version 1.0 is in reach. Peter Thomas already released the third candidate, and there isn’t much left.

That’s a reason for me to take a closer look at the upcoming version, and here it is, my list of the most favoured features of Karate Automation Framework version 1.0.

Fell free to download the code examples from: https://github.com/peterquiel/karate-getting-started-guide/tree/master/karate-standalone-setup/src/examples/users

Support for all Java versions 8 and above

Karate relies on the JavaScript Engine Nashorn the runs on the JVM. Oracle deprecated that engine in JDK 11 and JDK 15 ships without Nashorn. Which result in the fact that Karate won’t run on JDK 15 or above,.

Think of the situation you want to use Karate in your Tool Stack and you know that Karate isn’t future proof. This would be a major showstopper for me if I have to decide.

Peter Thomas must have been aware of that, and he invested a lot to make this happen. He refactored major parts and lifted Karate on top of the GraalVM JavaScript Implementation.

GraalVM? Do I have to use another JDK to run karate now?

Not really. The karate-core comes now with all the dependency you need – this is actual another improvement – to run it on every JDK version 8 and above, and that includes JDK version 15 in particular. Needless to say, it’s JDK 16 ready as well!

Another idea that pops up to my mind when I hear GraalVM is: native image support.
Native Image is a technology to ahead-of-time compile Java code to a standalone executable, called a native image

These executables don’t need a JDK at all and run faster, but I ran into a few issues of creating a native image, and since this isn’t a blog post on how to create karate native image, I didn’t dig deeper into this topic.

On the other hand, Karate starts and runs fast enough on top of a JVM and doesn’t need a performance boost necessarily.

Improved error details for match failures

Karates match command with its == , contains and contains any operators are very powerful. The json schema expressions adds another level of dense expressiveness, and that confused me sometimes when I try to understand why this particular case didn’t work.

It looks like I wasn’t the only one having problems to understand why a matcher expression didn’t match or match.

Karate matcher,… talk to me!

The new matcher talks to you in a more detailed way to explain you the.

Examples

This will fail, because c2 is 2 and not 3.

  Scenario: Pretty Printed Matcher
  * def actual = {a: 1, b: 2, c : {c1: 1, c2: 2}}
  * match actual == {a: 1, b: 2, c : {c1: 1, c2: 3}}

Before Karate 1.0:

Matcher fail text before Karate 1.0

With Karate 1.0:

Matcher fail text with Karate 1.0

Wow, what an improvement. Spotting mismatches in leafs can’t be easier.

An other example:

  Scenario: Matcher pretty, type dosen't match
  * def actual =
  """
{
  "id": 1,
  "email": "Sincere@april.biz",
  "username": "Bret",
  "company": {
    "bs": "harness real-time e-markets",
    "catchPhrase": "Multi-layered client-server neural-net",
    "name": "Romaguera-Crona"
  }
}    
  """
    * match actual contains {id :"1"}

Before improvement:

Matcher fail text before Karte 1.0

After improvement:

Matcher Fail Text with Karate 1.0

HTML Templating Engine – Thymeleaf Integration

Along with Karate 1.0 release comes a Thymeleaf integration for report rendering. This is what you always wanted to generate human readable reports.

This is really valuable feature you want when you integrated Karate into your testing environment and you use Karate testing results as a communication medium or as a QS report.

Using templates is easy, and the integration in your test case is straightforward.

First, you create a simple html template file like this one:

<table class="table table-striped">
    <thead>
        <tr>
            <th>Id</th>
            <th>Name</th>
            <th>Email</th>
            <th>Company</th>
        </tr>
    </thead>
    <tbody>
        <tr th:each="user: users">
            <td th:text="${user.id}"></td>
            <td th:text="${user.name}"></td>
            <td th:text="${user.email}"></td>
            <td>
                <div th:text="${user.company.name}"></div>
                <div th:text="${user.company.catchPhrase}"></div>
            </td>
        </tr>
    </tbody>
</table>

Second, you render that template using the already known doc command:

Scenario: Html Templateing
    Given url 'https://jsonplaceholder.typicode.com'
    * path 'users'
    When method get
    Then status 200
    * def users = response
    * doc {read : 'users.html'}

The resulting report looks like this:

Karate report with Thymeleaf template
Karate Report with Customized Table Content

One other important thing to note about the new integrated Thymeleaf Engine is that Karate is actual dogfooding this for its own reports, as you can see from karate-step.html template.

When Peter told me about that dogfooding the next thought that came to my mind is: “Can you configure the template karate should use?” and the answer is: “Configuration isn’t part of the current release candidate 0.9.9.RC3, but will be in 1.0.”

So, in future you will be able to customize the report as you like and to load your customized templates from a jar for instance. This enables you to share your reporting templates across teams and your company.

I could even imagine that third parties publish their own report templates to maven repository, and you can easily take advantage of that.

Distributed Testing

Often you decide to introduce a new tool into your tech stack and you may have evaluated that tool using a comparison matrix, but you can’t think of every use case and for scaling your stack if often gets dirty.

It’s too late to step back, and it has to work.

API testing isn’t the fasted tool you have on your test workbench, and automated browser tests are even worse. While during API tests the machine mostly waits for IO and has nothing to do, browser testing becomes quite CPU intense.

Sometimes you want to test how fast a website renders on a slow machine, and you therefore you want to render the website with CPU reduced setup.

Distributed Testing helps you to fulfill these requirements when you want to improve test execution time. It’s not an easy setup, it adds complexity, it will increase your false positive rates, because some of your tests might not run in parallel, but having this opportunity is better than none.

Karate 1.0 comes with Distribute Testing support, and Peter Thomas asks for your support, since it’s difficult to test and debug.

I haven’t tried it. It’s like reactive programming, it’s fun to play with, but you don’t want to use it in production until you have to.

Nevertheless, it’s an important feature if karate want’s to be enterprise ready!

ECMAScript 2015 / ES6 and Improved Error Details for JavaScript Failures

I don’t know how often I used the => operator in Karate, before I learned that Karate isn’t ES6 ready.

Supporting ES6 may look like a slight improvement. I mean, you don’t write loads of JS code in Karat, do you?

But since almost every JS environment runs ES6, it would feel like Karate is falling behind without that support. If you don’t move, you fall behind.

Just a little example, that shows pretty much of the new stuff:

    Scenario:  ES6 Support
    * def log = x =>  karate.log(`${new Date()}: ${x}`)
    * log("test")

Output:

13:27:17.579 [main]  INFO  com.intuit.karate - Mon Jan 25 2021 13:27:17 GMT+0100 (CET): test 

The improved JS failures are worth mentioning as well, because in the past I have faced different situations where I changed JS code in small incremental changes to find the reason.

I removed the e from new Date() in the code above and when I try to execute that, I will get this improved error message:

failed features:
js failed:
   01: log("test")
 org.graalvm.polyglot.PolyglotException: ReferenceError: "Dat" is not defined
 .:=>(Unnamed:1)                                       

JS Functions for Data Driven Testing

Data Driven Testcases have been a really helpful tool for me. I used this feature often in combination with reading test data from a csv file that karate turns automatically into a json.

With Karate 1.0 comes the feature to use a JavaScript function to generate your test data. The following example simply creates a bunch of users using the iteration index as part of the email:

Scenario Outline: Js Function in Data Driven Test Case: Creating User <email>
    * def user =
      """
      {
        "name": "Test User",
        "username": "testuser",
        "email": #(email),
        "address": {
          "street": "Has No Name",
          "suite": "Apt. 123",
          "city": "Electri",
          "zipcode": "54321-6789"
        }
      }
      """

    Given url 'https://jsonplaceholder.typicode.com/users'
    And request user
    When method post
    Then status 201

    Examples:
    | i => i > 2 ? null : ({email: `test_${i}@user.com`}) | 

Report:

Karate report of data driven test case

This is a simple example shows the expressiveness of the newly introduced ES6 support.

But is that all?

One thing that pops up my mind that is much easier to use with this new feature is Property Based Testing.

Property Based Testing is an interesting approach. Unfortunately, I did not manage to make it a tool of my daily use. The simplicity – and its limitation – of imperative test cases are too strong.

I hope this feature enables others to experiment more!

Single Dependency

Peter Thomas restructured the dependencies and now karate-core comes with all its dependencies in one jar. This might look like an anti-pattern if you think of modularity.

But Karate isn’t part of an enterprise application that consists 100ths of jar files. Principles applied in that situation don’t necessarily apply here. It’s an interpreter for your .feature files, and a single dependency makes it easy to execute your feature file.

Simple execution is the driver of this improvement, and Peter Thomas provides a simple example on how jbang helps you.

In this Karate Java API show case Peter Thomas demonstrates how you can execute a Karate test case using with a simple command:

jbang https://github.com/ptrthomas/karate-showcase/blob/main/javadsl.java

Conclusion

Releasing Version 1.0 is something special, it’s the state of your software as you imagined it when you started your journey.

The journey of Karate started ~4 years ago, and it gained a lot of attention during this period of time. Many people are using it every day and some people don’t like it at all. Some of them think, it’s BDD for people who can’t program or it’s just to different and they want to stay in their comfort zone. It’s easy to judge from the sidewalk.

So, Karate isn’t a perfect tool, but perfection won’t get you done.

I don’t like everything about Karate, and in particular I don’t like the small little inconsistent language decisions, like the switch between JSON and JavaScript expressions or the simple Scenario replacement – using <name> pattern – in data driven tests.

Update (01.02.2021): Peter Thomas just told me, that there is a PR in the pipeline to support JavaScript String interpolation in scenario names.

So, Karate isn’t a perfect tool, but perfection won’t get you done and done is better than perfect. I have had so many brilliant ideas and none of them made to public, because every brilliant, perfect idea gets dirty and rusty when facing reality. Most people quit when this uniform optimism transforms into an informed pessimism and the cost of reality becomes apparent.

Peter Thomas, the major force behind karate, didn’t quit, nor did he pay too much attention to the awful voices. His consistency and discipline lead to what karate is today.

A beautiful, imperfect and powerful tool I don’t want to miss in my toolbox.

Congratulations Karate to your first version!