Outside-In TDD a Consumer

Consumer Driven Contracts

Consumer Driven Contracts (CDC) is a pattern that takes Outside-In TDD across boundaries, and is frequently used in Micro Service Architectures. Note that TDD is not primarily a testing process, but a design technique. The same is true for Consumer Driven Contracts where Consumers drive the capabilities of Producers by expressing a clear need. This ties service evolution to business value. Consumers start by writing Contract Tests and define what a Producer’s API should be like. The Producer is replaced by a Mock Object whose configuration generates the Contract. The interactions described in the Contract are later replayed against the Producer in a test harness. Both, the Consumer and the Producer are tested in isolation and evolve independently.

Driving Client Code Via Consumer Tests

Another thing that the Consumer’s Contract Tests are good for is they can help emerge the client code. We aim for a client code that hides the technical details such as HTTP messaging and JSON serialization from us. The client code should represent the remote service within our boundary and hide the fact that it is remote.

Defining A First Contract

We’re building a Book Store, so we need a Book Catalogue as a Producer. It should serve the Book’s information. Let’s start with a happy path for finding a single Book. This is what the Producer Mock would look like using Pact JS. The definition of this Mock as well generates the Contract. I’m skipping the boilerplate, you can look it up in my github.

const requestASingleBook = {
    state: "two books",
    uponReceiving: "a request for retrieving the first book",
    withRequest: {
        method: "GET",
        path: "/books/1"
    },
    willRespondWith: {
        status: 200,
        headers: {"Content-Type": "application/json"},
        body: {
            "self": "/books/1",
            "title": "Title"
        }
    }
}

Choices, Choices, Choices

We can now start writing the actual Consumer Test. But where do we start? What do we assert?

We could try to design the client code by wishful thinking.

it("finds a single book", () => {
    // represents the producer
    const books = new Books() 

    // executes the http call and does json deserialization
    const book = books.byId(bookId) 
    
    expect(book).to.deeply.eq(expectedBook)
})

This could be a first failing test, and a nice design that hides the technical details as desired. But it certainly is a bit premature. We haven’t even put an API’s baseUrl. Neither did we think about potential errors. All the HTTP handling code would have to go to the not yet existing Books object. How would we even go about making a HTTP request? How would we deserialize the content? There are many things coming up, and it already feels like a huge leap.
There has to be a better way.

Making Smaller Steps

We could just ignore the design for now and focus on the technical details. Meaning we would start and implement the HTTP call within the test. This approach puts us closer to the technical details. It allows us to fiddle around, and prevents us from designing prematurely. Later, when we experience duplication, we can extract the implementation code from the test to emerge a more mature design. Some would call this TDD as if you meant it (TDD aiymi).

A Failing Test

We start by defining our final expectation, that is the book data as a result. We don’t yet know how to get there. All we know is the url. Note how we don’t assert on any technical details like the HTTP Status Code and Content-Type. These details don’t concern us, they will be hidden once we’re finished.

const requestASingleBook = {
    state: "two books",
    uponReceiving: "a request for retrieving the first book",
    withRequest: {
        method: "GET",
        path: "/books/1"
    },
    willRespondWith: {
        status: 200,
        headers: {"Content-Type": "application/json"},
        body: {
            "self": "/books/1",
            "title": "Title"
        }
    }
}

// ... boilerplate ...

it("finds a single book", async () => {
    const url = "http://localhost:1234/books/1"
    
    const result // ??

    expect(result).to.deep.eq({self: "/books/1", title: "Title"})
})

Run the tests => Red.

Make it Green

Let’s find the simplest way to make the test green by writing all implementation code inside the test.

it("finds a single book", async () => {
    const bent = require("bent") // bent is just a http client
    const getJSON = bent("json")

    const result = await getJSON("http://localhost:1234/books/1")

    expect(result).to.deep.eq({self: "/books/1", title: "Title"})
})

Run the tests => Green.

Fabulous!

Refactor

As it works we can now think of improving it. Let’s create a type for the book.

it("finds a single book", async () => {
    const bent = require("bent")
    const getJSON = bent("json")

    type Book = {
        self: string
        title: string
    }

    const result: Book = await getJSON("http://localhost:1234/books/1")

    expect(result).to.deep.eq({self: "/books/1", title: "Title"})
})

Run the tests => Green.

Sticking To The Robustness Principle

⚠ The Robustness Principle says that we should be conservative in sending stuff but liberal in receiving it. The goal of this is to reduce the risk for messages to fail.

Consumers should be tolerant to API change, able to survive most of it. They should only be concerned with the Resources, Methods and Fields they actually consume, and treat even them with care.

In the existing solution a new arriving field would cause the message to fail.
Let’s imagine the server would add a new field: description. Our client code would not expect it and break. We need to make it more robust and prevent this from happening.

But, should we not write a test first proving that it would break?

Totally! However, we are currently in the writing of a Consumer Test. It generates the Contract for the Producer. Adding the description field to the Contract is not what we want. We only want to add the fields that we consume. So let’s

  1. add the field temporarily to prove that it breaks,
  2. fix the implementation to not break anymore,
  3. and finally remove the field again.
it("finds a single book", async () => {
    const bent = require("bent")
    const getJSON = bent("json")

    type Book = {
        self: string
        title: string
    }

    const decodeBook = (object): Book => ({
        self: object.self,
        title: object.title
    });

    const response = await getJSON("http://localhost:1234/books/1")

    const book: Book = decodeBook(response)

    expect(book).to.deep.eq({self: "/books/1", title: "Title"})
})

Run the tests => Green.

The decodeBook function makes the code robust against any change to fields that we don’t consume.

Aiming For Duplication

Already fiddling around a lot, aren’t we? Let’s aim for some duplication by implementing the not found case. We can just copy and paste the earlier test and adapt it. Shouldn’t have to change much, should we?

const requestANonExistingBook = {
    state: "two books",
    uponReceiving: "a request for a non existing book",
    withRequest: {
        method: "GET",
        path: "/books/3"
    },
    willRespondWith: {
        status: 404
    }
}


// ... boilerplate ...


it("finds nothing", async () => {
    import {none, Option} from "fp-ts/lib/Option"
    const bent = require("bent")

    type Book = {
        self: string
        title: string
    }

    const decodeBook = (object): Book => ({
        self: object.self,
        title: object.title
    });

    const getStream = bent("http://localhost:1234/books/3", 200, 404)
    const stream = await getStream()
    let book:Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    }

    expect(book).to.equal(none)
})

Run the tests => Green.

That escalated quickly. The implementation changed quite a bit. We had to change the way we use bent to support the 404 status code. Also, we decided to introduce the Option type to make explicit a book might not be returned. The two tests look quite different now.

Finding The Common Denominator

Before we can extract it as production code we need to adapt both tests to match each other.

it("finds a single book", async () => {
    import {none, Option, some} from "fp-ts/lib/Option"
    const bent = require("bent")

    type Book = {
        self: string
        title: string
    }

    const decodeBook = (object): Book => ({
        self: object.self,
        title: object.title
    });

    const getStream = bent("http://localhost:1234", 200, 404)
    const stream = await getStream("/books/1")
    let book:Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    } else {
        book = some(decodeBook(await stream.json()))
    }

    expect(book).to.deep.eq(some({self: "/books/1", title: "Title"}))
})

// ...

it("finds nothing", async () => {
    import {none, Option, some} from "fp-ts/lib/Option"
    const bent = require("bent")

    type Book = {
        self: string
        title: string
    }

    const decodeBook = (object): Book => ({
        self: object.self,
        title: object.title
    });

    const getStream = bent("http://localhost:1234", 200, 404)
    const stream = await getStream("/books/3")
    let book:Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    } else {
        book = some(decodeBook(await stream.json()))
    }

    expect(book).to.equal(none)
})

Run the tests => Green.

Extract Duplication

Ok, great. The implementation code is now the same in both cases: Finding a book, and not finding it. We can finally start and extract the duplicated parts out of the tests. Let’s begin with the Book type, and the decodeBook function.

import {none, Option, some} from "fp-ts/lib/Option"
const bent = require("bent")

type Book = {
    self: string
    title: string
}

const decodeBook = (object): Book => ({
    self: object.self,
    title: object.title
});

// ...

it("finds a single book", async () => {
    const baseUrl = "http://localhost:1234"
    const path = "/books/1"
    
    const getStream = bent(baseUrl, 200, 404)
    const stream = await getStream(path)
    let book:Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    } else {
        book = some(decodeBook(await stream.json()))
    }

    expect(book).to.deep.eq(some({self: "/books/1", title: "Title"}))
})

// ...

it("finds nothing", async () => {
    const baseUrl = "http://localhost:1234"
    const path = "/books/3"
    
    const getStream = bent(baseUrl, 200, 404)
    const stream = await getStream(path)
    let book:Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    } else {
        book = some(decodeBook(await stream.json()))
    }

    expect(book).to.equal(none)
})

Run the tests => Green.

Note how I as well extracted both the baseUrl and path variables inside the tests. This allows us to further extract the function that does the actual request.

import {none, Option, some} from "fp-ts/lib/Option"
const bent = require("bent")

type Book = {
    self: string
    title: string
}

const decodeBook = (object): Book => ({
    self: object.self,
    title: object.title
});

async function findBook(baseUrl: string, path: string): Promise<Option<Book>> {
    const getStream = bent(baseUrl, 200, 404)
    const stream = await getStream(path)
    let book: Option<Book>

    if (stream.status !== 200) {
        log("received " + stream.status + " " + await stream.text())
        book = none
    } else {
        book = some(decodeBook(await stream.json()))
    }
    return book
}

// ...

it("finds a single book", async () => {
    const baseUrl = "http://localhost:1234"
    const path = "/books/1"

    const book = await findBook(baseUrl, path)

    expect(book).to.deep.eq(some({self: "/books/1", title: "Title"}))
})

// ...

it("finds nothing", async () => {
    const baseUrl = "http://localhost:1234"
    const path = "/books/3"

    const book = await findBook(baseUrl, path)

    expect(book).to.equal(none)
})

Run the tests => Green.

Final Touch

The tests became nice and short. However, I would prefer having a bookClient object that had the baseUrl baked in. This would allow me to add more functions later. So I refactor it further, and extract the module. The final result looks like this:

// content of book-client.ts
import {none, Option, some} from "fp-ts/lib/Option"
const bent = require("bent")

export function createBookClient(baseUrl: string) {
    type Book = {
        self: string
        title: string
    }

    const decodeBook = (object): Book => ({
        self: object.self,
        title: object.title
    });

    return {
        requestBook: async (path: string): Promise<Option<Book>> => {
            const getStream = bent(baseUrl, 200, 404)
            const stream = await getStream(path)
            if (stream.status !== 200) {
                log("received " + stream.status + " " + await stream.text())
                return none
            }
            return some(decodeBook(await stream.json()))
        }
    }
}

// content of tests
import {createBookClient} from "./book-client"

// ...

const client = createBookClient("http://localhost:1234")

it("finds a single book", async () => {
    const book = await client.requestBook("/books/1")

    expect(book).to.deep.eq(some({self: "/books/1", title: "Title"}))
})

// ...

it("finds nothing", async () => {
    const book = await client.requestBook("/books/3")

    expect(book).to.equal(none)
})

Run the tests => Green.

And we are finished.

Conclusion

We drove a clean client code from writing Consumer Tests. We used the tests to fiddle around with technical implementation detail. Once we had tested more cases, we aligned their implementations to extract common code. The design we arrived at is a little different from the one we had originally anticipated.

TDD leaves us with a choice when to design.
We can do it upfront by wishful thinking - in the Red Phase.
Or we can defer it to a later stage - in the Refactoring Phase.
The latter makes it easier to fiddle around with the details and may lead to a more mature design.

You can find the code on my github.


to the comments

becoming a keyboard ninja


Hi, i’m Gregor, and I want to help you become better at coding. Here are some tips to improve your keyboard utilization. If you are already very good at it, this might not be for you.

keep your eyes on the street

It is our most important tool because of it’s relatively high throughput. This throughput not only determines our speed, it also affects the quality of our output. Let me explain: As coders we’re switching between different views and modes. We’re mostly looking at the code from a bird’s eye perspective, just reading and thinking. As soon as we have decided on the changes we want to make, our focus tightens. We’re now more concerned which button to click next, which menu to open. Using the mouse is terribly slow. It breaks our flow of thought. We’re not watching the street, we’re looking at the stick shift. Fluent keyboard utilization can help us keep our eyes on the street. It frees our minds and helps us make fewer mistakes. We can not only drive faster, but also safer.

getting better is easy!

Just lower your expectations. There is no need to get to 0% mouse usage right away. Start small, but start now! The following tips should help you get started.

tip #1: buy a keyboard you’ll want to use

Do yourself a favour and buy a good keyboard of your choice. After all, you’re a programmer and this is your most important tool. It will be much easier to get better, once you own a keyboard you enjoy working with. One you’ll want to use. Maybe a mechanical one, maybe a slim one, whatever suits you.

tip #2: use a powerful editor

It should have support for code navigation, refactoring and other coding related stuff. Could be an IDE, could be a lightweight editor that has a healthy plugin ecosystem.

tip #3: plugins may help

Search for plugins that assist you in learning keystrokes. Like the Key Promoter X for IntelliJ. There is also Mousefeed for Eclipse, and Keypromoter for Netbeans, which do a similar thing. They will assist you by showing the keystroke you could have pressed, whenever you used the mouse. Also they will count and show you how often you used the mouse for that command.

tip #4: start small

Don’t try and learn everything at once. You gotta start small. Choose ~4 keystrokes or combos you want to learn first, and focus only on them while working on the code you are usually working on. You will soon start doing them unconsciously. This is when you are ready for the next ones.

tip #5: prepare a cheat sheet

Prepare yourself a cheat sheet, so you can quickly look up the shortcuts you are working on. Maybe you’ll find a nice print out online, or even create one yourself. There is the code cops shortcut search for example.

tip #6: be gone, mouse!

If it does not work out, and you keep grabbing the mouse out of habit, move it to the other side of your keyboard. So when you try and grab it, it won’t be there. This is also a good tip if you’re advanced and you feel like ditching the mouse anyhow.

tip #7: pair with others

You can’t practice tricks you didn’t know existed. Pair- and mob- programming will help you find inspiration. I learned some of the most valuable shortcuts from pairing with others.

tip #8: RTFM

Also, it might be worth it to study your editors documentation. Some of the actions I adopted are ones i found there. For example: The ‘Search and Replace Duplicates’ refactoring.

tip #9: extend your keymap

Some of the actions you frequently use might not even have a keystroke assigned by default. I for example like to split my tabs when doing TDD. So I can see my tests on the left, and production code on the right. The ‘Move Tab to the Right’ action however did not have a keystroke assigned by default. I had to do this myself. Expect your editors keymap to be imperfect. Think outside the box, and adapt the tool according to your need.

I hope these tips will help you utilize your keyboard more. So you can keep your eyes on the street, and increase both your throughput and quality of output.


to the comments

maybe a different view on object orientation

information/state and behaviour

I like to think of software as a universe of state and behaviour. State, whether mutable or immutable, is represented in the form of parameters, return values, vars, fields, files and so on and so forth. Those have many different traits to them. For example: vars, fields and files are location based, while parameters and return values are flow based. But they are all state.

Behaviour is represented in the form of functions that operate on that state. There is no behaviour without state. Try to imagine a program that had no state. No Vars. No Input. No Output. Not even lambdas… they generate state. We can’t even print a damn thing without passing a string state.

cohesion

Some state is related. If we’d increase the cohesion and move related state closer to each other and behaviour towards the state it operates on, groups are emerging. Groups of related parts. Groups of state and behaviour.

encapsulation

We can wrap them in capsules, which come in many different forms. Functions, classes, packages, modules, processes and even servers are all kinds of capsules. For each capsule we define a minimum viable interface. The encapsulation serves the purpose of defining usable units that hide their internals which are irrelevant to the outside world.

objects

We are defining Objects - behavioural capsules that interact with each other at runtime. So a function is a form of an object. An instance is a form of an object. A module is a form of an object. And even a microservice is a form of an object.

coupling

Each object is in some way coupled to the neighbours it interacts with. Coupling prevents independent changeability. Objects which are strongly coupled to one another cannot be changed independently of each other. The size of an objects public interface correlates with the strength of this coupling. The more we close the capsule, the more details we hide, the smaller the interface will be, thus the weaker the coupling will turn out.

feature envy

Sometimes, an object accidentally operates on the state of another object. We call that a ‘Feature Envy’, as the object envies the other object. This increases the coupling between the two. We can resolve this situation by moving the behaviour to the object that holds the state.

dependency

When an object knows about another, we call that a dependency. But don’t confuse ‘knows’ with ‘controls’, those are different things.

dependency inversion

An object can control another object without even knowing it. This is called dependency inversion. Think about a AAA battery. It provides power, but it has no idea who it provides power to. The form of the battery, AAA, is its interface. The flashlight knows the battery, it implements its interface. And even though the battery knows nothing about the flashlight, the former controls the latter.


to the comments