No More Seat Costs: Semaphore Plans Just Got Better!

    18 Mar 2015 · Software Engineering

    Testing Clojure With Expectations

    5 min read
    Contents

    Introduction

    Clojure is shipped with clojure.test — a library with basic features for writing tests. However, there are a few alternatives that aim to make writing tests more pleasant or more suitable for BDD. Expectations by Jay Fields is one of them, described as “a minimalist’s unit testing framework” with the slogan “adding signal, removing noise”.

    This article explains how you can install Expectations in your Clojure project. We will also show what Expectations brings to the table, that’s not already present in clojure.test.

    Expectations Setup

    The easiest way to get started is to add Expectations and lein-expectations to your project.clj:

    (defproject expectations-playground "0.1.0-SNAPSHOT"
      :description "Playground for exploring Expectations"
      :license {:name "Eclipse Public License"
                :url "http://www.eclipse.org/legal/epl-v10.html"}
      :dependencies [[org.clojure/clojure "1.6.0"]
                     [expectations "2.0.16"]]
      :plugins [[lein-expectations "0.0.8"]])

    lein-expectations is a Leiningen plugin to run tests written using the expectations library. After that, you can run tests with lein expectations.

    Expectations also integrates well with several editors and development environments, such as Emacs and IntelliJ. More information about installation and setup can be found in the library documentation.

    Adding Signal, Removing Noise

    A simple example that compares clojure.test and Expectations already reveals few interesting details:

    ; clojure.test
    (deftest equality-test
      (testing "Is 'foo' equal 'fooer'"
        (is (= "foo" "fooer"))))
    
    ; expectations
    (expect "foo" "fooer")

    Running the test with clojure.test gives:

    $ lein test
    
    lein test expectations-playground.core-test
    
    lein test :only expectations-playground.core-test/equality-test
    
    FAIL in (equality-test) (core_test.clj:7)
    Is 'foo' equal 'fooer'
    expected: (= "foo" "fooer")
      actual: (not (= "foo" "fooer"))
    
    Ran 1 tests containing 1 assertions.
    1 failures, 0 errors.
    Tests failed.
    

    While running the test with Expectations gives:

    $ lein expectations
    
    failure in (expectations_test.clj:4) : expectations-playground.expectations-test
    (expect "foo" "fooer")
    
               expected: "foo"
                    was: "fooer"
    
               matches: "foo"
               diverges: ""
                      &: "er"
    
    Ran 1 tests containing 1 assertions in 42 msecs
    1 failures, 0 errors.
    

    We already see Expectations delivering on the slogan “adding signal, removing noise”. The test is shorter, but it gives us more informations in a nicer way. Output is colored by default, format is a bit easier on eyes and it’s more precise about the failure — it says what’s different about strings. It also shows how much time the test took to run, which can be useful when optimizing tests.

    Minimal and Consistent Syntax

    It’s interesting to see how minimal and consistent the Expectations syntax is. At first glance, there is not much besides expect. It can get you a long way:

    (expect 2 (+ 1 1))
    
    (expect "foo" (str "f" "o" "o"))
    
    (expect [1 2 3] (conj [1 2] 3))
    
    (expect {:a 1 :b 2} (assoc {:a 1} :b 2))
    
    (expect #"Expect" "Expectations")
    
    (expect empty? [])
    
    (expect 3 (in [1 2 3]))

    With expect you can compare integers, strings, types, regular expressions and collections and test for function output.

    Even More Signal

    Expectations really shines when it comes to testing collections as it tests not only equality, but also contents of collections:

    $ lein expectations
    
    failure in (expectations_test.clj:4) : expectations-playground.expectations-test
    (expect [1 2 3] [2 3 1])
    
               expected: [1 2 3]
                    was: [2 3 1]
    
               in expected, not actual: [1 2 3]
               in actual, not expected: [2 3 1]
               lists appear to contain the same items with different ordering
    
    failure in (expectations_test.clj:6) : expectations-playground.expectations-test
    (expect [1 2 3] [1 2 3 4])
    
               expected: [1 2 3]
                    was: [1 2 3 4]
    
               in expected, not actual: null
               in actual, not expected: [nil nil nil 4]
               actual is larger than expected
    
    failure in (expectations_test.clj:8) : expectations-playground.expectations-test
    (expect [1 2 3] [1 2])
    
               expected: [1 2 3]
                    was: [1 2]
    
               in expected, not actual: [nil nil 3]
               in actual, not expected: null
               expected is larger than actual
    
    Ran 3 tests containing 3 assertions in 73 msecs
    3 failures, 0 errors.

    Here you can easily see what’s different about expected and real value. This is especially important when comparing long or nested collections.

    Even Less Noise

    Expectations tries aggressively to remove noise from test output. For example, it trims long stack traces, leaving only the important part.

    Example stack trace using clojure.test:

    ERROR in (bad-aritmetic-test) (Numbers.java:156)
    Uncaught exception, not in assertion.
    expected: nil
      actual: java.lang.ArithmeticException: Divide by zero
     at clojure.lang.Numbers.divide (Numbers.java:156)
        clojure.lang.Numbers.divide (Numbers.java:3731)
        expectations_playground.core_test/fn (core_test.clj:6)
        clojure.test$test_var$fn__7187.invoke (test.clj:704)
        clojure.test$test_var.invoke (test.clj:704)
        clojure.test$test_vars$fn__7209$fn__7214.invoke (test.clj:722)
        clojure.test$default_fixture.invoke (test.clj:674)
        clojure.test$test_vars$fn__7209.invoke (test.clj:722)
        clojure.test$default_fixture.invoke (test.clj:674)
        clojure.test$test_vars.invoke (test.clj:718)
        clojure.test$test_all_vars.invoke (test.clj:728)
        clojure.test$test_ns.invoke (test.clj:747)
        clojure.core$map$fn__4245.invoke (core.clj:2559)
        clojure.lang.LazySeq.sval (LazySeq.java:40)
        clojure.lang.LazySeq.seq (LazySeq.java:49)
        clojure.lang.Cons.next (Cons.java:39)
        clojure.lang.RT.boundedLength (RT.java:1654)
        clojure.lang.RestFn.applyTo (RestFn.java:130)
        clojure.core$apply.invoke (core.clj:626)
        clojure.test$run_tests.doInvoke (test.clj:762)
        clojure.lang.RestFn.applyTo (RestFn.java:137)
    ...
    

    This is the equivalent stack trace using Expectations:

    failure in (expectations_test.clj:4) : expectations-playground.expectations-test
    (expect 1 (/ 1 0))
    
      act-msg: exception in actual: (/ 1 0)
        threw: class java.lang.ArithmeticException - Divide by zero
               on (expectations_test.clj:4)
               on (form-init1230342255835259211.clj:1)
    
    Ran 1 tests containing 1 assertions in 67 msecs
    0 failures, 1 errors.
    

    Expectations output is much shorted, with more useful information.

    Conclusion

    Expectations certainly delivers on promise “more signal, less noise”. But the library also provides a few additional tricks that are not covered here — like testing side effects and freezing time. For more informations, visit the Expectations website.

    Leave a Reply

    Your email address will not be published. Required fields are marked *