DDevelopment & Design

Clojure – Fascinated, Disappointed, Astonished

brackets-banner I’ve had a pleasure to work with Piotrek Jagielski for about two weeks on Clojure project. I’ve learned a lot, but there is still a lot to know about Clojure for me. In this post I’ll write what fascinated, disappointed and astonished me about this programming language.

Clojure & InteliJ IDEA tips

Before you start your journey with Clojure: * Use Cursive plugin for InteliJ IDEA. In ’14 Edition it was not in the standard plug-in repository (remove La Clojure plug-in and Cursive repository manually). For IDEA ’15 it is in repository. * Colored brackets help me a lot. You can find configuration for colored brackets on Misophistful Github.

Fascinated

Syntax

For many people Clojure brackets are reasons to laugh. Jokes like that were funny at first: “How many brackets did you write today?” I have to admit, that at the beginning using brackets was not easy for me. Once I’ve realized that the brackets are just on the other side of the function name, everything was simple and I could code very fast. After few days I’ve realized that this brackets structure forces me to think more about the structure of the code. As a result the code is refactored and divided into small functions. Clojure forces you to use good programming habits.

Data structure is your code

Clojure is homoiconic, which means that the Clojure programs are represented by Clojure data structures. This means that when you are reading a Clojure code you see lists, maps, vectors. How cool is that! You only have to know few things and you can code.

Do not restart your JVM

Because Clojure code is represented as data structures, you can pass data structure (program) to running JVM. Furthermore, compiling your code to bytecode (classes, jars) may be eliminated.

For example, when you want to test something you are not obligated to start new JVM with tests. Instead you can just synchronize your working file with running REPL and run the function.

Traditional way of working with JVM is obsolete.

clojure-repl

In the picture above, on the left you can see an editor, on the right there is running REPL.

The same way you can run tests, which is extremely fast. In our project we had ~80 tests. Executing them all took about one second.

Simplicity is the ultimate sophistication – Leonardo Da Vinci

After getting familiar with this language, it was really easy to read code. Of course, I was not aware of everything what was happening under the hood, but consistency of the written program evoked sense of control.

Disapointed

Data structure is your code

When data structure is your code, you need to have some additional operators to write effective programs. You should get to know operators like ‘->>’, ‘->’, ‘let’, ‘letfn’, ‘do’, ‘if’, ‘recur’ …

Even if there is a good documentation (e.g. Let), you have to spend some time on analyzing it, and trying out examples.

As the time goes on, new operators will be developed. But it may lead to multiple Clojure dialects. I can imagine teams (in the same company) using different sets of operators, dealing with the same problems in different ways. It is not good to have too many tools. Nevertheless, this is just my suspicion.

Know what you do

I’ve written a function that rounds numbers. Despite the fact that this function was simple, I wanted to write test, because I was not sure if I had used the API in correct way. There is the test function below:

(let [result (fixture/round 8.211M)]
  (is (= 8.21M result))))

Unfortunately, tests were not passing. This is the only message that I received:

:error-while-loading pl.package.calc-test
NullPointerException   [trace missing]
(pst)
NullPointerException

Great. There is nothing better than a good exception error. I’ve spent a lot of time trying to solve this, and solution was extremely simple. My function was defined with defn-, instead of defn. defn- means private scope and test code, could not access testing function.

Do not trust assertions

Assertions can be misleading. When tested code does not work properly and returns wrong results, error messages are like this:

ERROR in math-test/math-operation-test (RT.java:528)
should round using half up
expected: (= 8.31M result)
 actual: java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.math.BigDecimal

I hadn’t got time to investigate it, but in my opinion it should work out of the box.

Summary

It is a matter of time, when tools will be better. Those problems will slow you down, and they are not nice to work with.

Astonished

The Clojure concurrency impressed me. Until then, I knew only standard Java synchronization model and Scala actors model. I’ve never though that concurrency problems can be solved in a different way. I will explain Clojure approach to concurrency, in details.

Normal variables

The closest Clojure’s analogy to the variables are vars, which can be created by def.

(defn a01 []
  (def amount 10)
  (def amount 100)
  (println amount))

We also have local variables which are only in let scope. If we re-define scope value of amount, the change will take place only in local context.

(defn a02 []
  (let [amount 10]
    (let [amount 100]
      (println amount))
    (println amount)))

The following will print:

100
10

Nothing unusual. We might expect this behavior.

Concurrent access variables

The whole idea of concurrent access variables can be written in one sentence. Refs ensures safe shared access to variables via STM, where mutation can only occur via transaction. Let me explain it step by step.

What is Refs?

Refs (reference) is a special type to hold references to your objects. As you can expect, basic things you can do with it is storing and reading values.

What is STM?

STM stands for Software Transactional Memory. STM is an alternative to lock-based synchronization system. If you like theory, please continue with Wikipedia, otherwise continue reading to see examples.

Using Refs

(defn a03 []
  (def amount (ref 10))
  (println @amount))

In the second line, we are creating reference. Name of this reference is amount. Current value is 10. In the third line, we are reading value of the reference called amount. Printed result is 10.

Modifying Refs without transaction

(defn a04 []
  (def amount (ref 10))
  (ref-set amount 100)
  (println @amount))

 

Using ref-set command, we modify the value of the reference amount to the value 100. But it won’t work. Instead of that we caught exception:

IllegalStateException No transaction running  clojure.lang.LockingTransaction.getEx (LockingTransaction.java:208)

Using transaction

(defn a05 []
  (def amount (ref 10))
  (dosync (ref-set amount 100))
  (println @amount))

To modify the code we have to use dosync operation. By using it, we create transaction and only then the referenced value will be changed.

Complete example

The aim of the previous examples was to get familiar with the new operators and basic behavior. Below, I’ve prepared an example to illustrate bolts and nuts of STM, transactions and rollbacks.

The problem

Imagine we have two references for holding data: * source-vector containing three elements: “A”, “B” and “C”. * empty destination-vector.

Our goal is to copy the whole source vector to destination vector. Unfortunately, we can only use function which can copy elements one by one – copy-vector.

Moreover, we have three threads that will do the copy. Threads are started by the future function.

Keep in mind that this is probably not the best way to copy vectors, but it illustrates how STM works.

(defn copy-vector [source destination]
  (dosync
    (let [head (take 1 @source)
          tail (drop 1 @source)
          conj (concat head @destination)]
      (do
        (println "Trying to write destination ... ")
        (ref-set destination conj)
        (println "Trying to write source ... ")
        (ref-set source tail)
        (println "Sucessful write " @destination)))))

(defn a06 []
  (let [source-vector (ref ["A" "B" "C"]) destination-vector (ref [])]
    (do
      (future (copy-vector source-vector destination-vector))
      (future (copy-vector source-vector destination-vector))
      (future (copy-vector source-vector destination-vector))
      (Thread/sleep 500)
      @destination-vector
      )))
Execution

Below is the output of this function. We can clearly see that the result is correct. Destination vector has three elements. Between Sucessful write messages we can see that there are a lot of messages starting with Trying to write. What does it mean? The rollback and retry occurred.

(l/a06)
Trying to write destination ...
Trying to write source ... 
Trying to write destination ...
Trying to write destination ... 
Sucessful write (A)
Trying to write destination ...
Trying to write destination ...
Trying to write source ...
Sucessful write  (B A)
Trying to write destination ...
Trying to write source ...
Sucessful write  (C B A)
=> ("C" "B" "A")
Rollback

Each thread started to copy this vector, but only one succeed. The remaining two threads had to rollback work and try again one more time.

stm-rollback

When Thread A (red one) wants to write variable, it notices that the value has been changed by someone else – conflict occurs. As a result, it stops the current work and tries again whole section of dosync. It will try until every write operation succeed.

Pros and cons of STM

Cons: * Everything that happens in dosync section has to be pure, without side effects. For example you can not send email to someone, because you might send 10 emails instead of one.
* From performance perspective, it makes sense when you are reading a lot from Refs, but rarely writing it.

Pros: * Written code is easy to read, understand, modify. * Refs and transactions are part of standard library, so you can use it in Vanilla Java. Take a look at this blog post for more examples.

Summary

There is a lot that Java developers can gain from Clojure. They can learn how to approach the code and how to express the problem in the code. Also they can discover tools like STM.

If you like to develop your skills, you should definitely experiment with Clojure.

Blog from Michał Lewandowski personal blog.

— Previous article

Easy Hoogle usage from bash

Next article —

Formatting Java Time with Spring Boot using JSON

You May Also Like
DDevelopment & Design

Spock basics

Spock (homepage) is like its authors say 'testing and specification framework'. Spock combines very elegant and natural syntax with the powerful capabilities. And what is most important it is easy to use.

One note at the very beginning: I assume that you are already familiar with principles of Test Driven Development and you know how to use testing framework like for example JUnit.

So how can I start?


Writing spock specifications is very easy. We need basic configuration of Spock and Groovy dependencies (if you are using mavenized project with Eclipse look to my previous post: Spock, Java and Maven). Once we have everything set up and running smooth we can write our first specs (spec or specification is equivalent for test class in other frameworks like JUnit of TestNG).

What is great with Spock is fact that we can use it to test both Groovy projects and pure Java projects or even mixed projects.


Let's go!


Every spec class must inherit from spock.lang.Specification class. Only then test runner will recognize it as test class and start tests. We will write few specs for this simple class: User class and few tests not connected with this particular class.

We start with defining our class:
import spock.lang.*

class UserSpec extends Specification {

}
Now we can proceed to defining test fixtures and test methods.

All activites we want to perform before each test method, are to be put in def setup() {...} method and everything we want to be run after each test should be put in def cleanup() {...} method (they are equivalents for JUnit methods with @Before and @After annotations).

It can look like this:
class UserSpec extends Specification {
    User user
    Document document

    def setup() {
        user = new User()
        document = DocumentTestFactory.createDocumentWithTitle("doc1")
    }

    def cleanup() {

    }
}
Of course we can use field initialization for instantiating test objects:
class UserSpec extends Specification {
    User user = new User()
    Document document = DocumentTestFactory.createDocumentWithTitle("doc1")

    def setup() {

    }

    def cleanup() {

    }
}

What is more readable or preferred? It is just a matter of taste because according to Spock docs behaviour is the same in these two cases.

It is worth mentioning that JUnit @BeforeClass/@AfterClass are also present in Spock as def setupSpec() {...} and def cleanupSpec() {...}. They will be runned before first test and after last test method.


First tests


In Spock every method in specification class, expect setup/cleanup, is treated by runner as a test method (unless you annotate it with @Ignore).

Very interesting feature of Spock and Groovy is ability to name methods with full sentences just like regular strings:
class UserSpec extends Specification {
    // ...

    def "should assign coment to user"() {
        // ...
    }
}
With such naming convention we can write real specification and include details about specified behaviour in method name, what is very convenient when reading test reports and analyzing errors.

Test method (also called feature method) is logically divided into few blocks, each with its own purpose. Blocks are defined like labels in Java (but they are transformed with Groovy AST transform features) and some of them must be put in code in specific order.

Most basic and common schema for Spock test is:
class UserSpec extends Specification {
    // ...

    def "should assign coment to user"() {
        given:
            // do initialization of test objects
        when:
            // perform actions to be tested
        then:
            // collect and analyze results
    }
}

But there are more blocks like:
  • setup
  • expect
  • where
  • cleanup
In next section I am going to describe each block shortly with little examples.

given block

This block is used to setup test objects and their state. It has to be first block in test and cannot be repeated. Below is little example how can it be used:
class UserSpec extends Specification {
    // ...

    def "should add project to user and mark user as project's owner"() {
        given:
            User user = new User()
            Project project = ProjectTestFactory.createProjectWithName("simple project")
        // ...
    }
}

In this code given block contains initialization of test objects and nothing more. We create simple user without any specified attributes and project with given name. In case when some of these objects could be reused in more feature methods, it could be worth putting initialization in setup method.

when and then blocks

When block contains action we want to test (Spock documentation calls it 'stimulus'). This block always occurs in pair with then block, where we are verifying response for satisfying certain conditions. Assume we have this simple test case:
class UserSpec extends Specification {
    // ...

    def "should assign user to comment when adding comment to user"() {
        given:
            User user = new User()
            Comment comment = new Comment()
        when:
            user.addComment(comment)
        then:
            comment.getUserWhoCreatedComment().equals(user)
    }

    // ...
}

In when block there is a call of tested method and nothing more. After we are sure our action was performed, we can check for desired conditions in then block.

Then block is very well structured and its every line is treated by Spock as boolean statement. That means, Spock expects that we write instructions containing comparisons and expressions returning true or false, so we can create then block with such statements:
user.getName() == "John"
user.getAge() == 40
!user.isEnabled()
Each of lines will be treated as single assertion and will be evaluated by Spock.

Sometimes we expect that our method throws an exception under given circumstances. We can write test for it with use of thrown method:
class CommentSpec extends Specification {
    def "should throw exception when adding null document to comment"() {
        given:
            Comment comment = new Comment()
        when:
            comment.setCommentedDocument(null)
        then:
            thrown(RuntimeException)
    }
}

In this test we want to make sure that passing incorrect parameters is correctly handled by tested method and that method throws an exception in response. In case you want to be certain that method does not throw particular exception, simply use notThrown method.


expect block

Expect block is primarily used when we do not want to separate when and then blocks because it is unnatural. It is especially useful for simple test (and according to TDD rules all test should be simple and short) with only one condition to check, like in this example (it is simple but should show the idea):
def "should create user with given name"() {
    given:
        User user = UserTestFactory.createUser("john doe")
    expect:
        user.getName() == "john doe"
}



More blocks!


That were very simple tests with standard Spock test layout and canonical divide into given/when/then parts. But Spock offers more possibilities in writing tests and provides more blocks.


setup/cleanup blocks

These two blocks have the very same functionality as the def setup and def cleanup methods in specification. They allow to perform some actions before test and after test. But unlike these methods (which are shared between all tests) blocks work only in methods they are defined in. 


where - easy way to create readable parameterized tests

Very often when we create unit tests there is a need to "feed" them with sample data to test various cases and border values. With Spock this task is very easy and straighforward. To provide test data to feature method, we need to use where block. Let's take a look at little the piece of code:

def "should successfully validate emails with valid syntax"() {
    expect:
        emailValidator.validate(email) == true
    where:
        email }

In this example, Spock creates variable called email which is used when calling method being tested. Internally feature method is called once, but framework iterates over given values and calls expect/when block as many times as there are values (however, if we use @Unroll annotation Spock can create separate run for each of given values, more about it in one of next examples).

Now, lets assume that we want our feature method to test both successful and failure validations. To achieve that goal we can create few parameterized variables for both input parameter and expected result. Here is a little example:

def "should perform validation of email addresses"() {
    expect:
        emailValidator.validate(email) == result
    where:
        email         result }
Well, it looks nice, but Spock can do much better. It offers tabular format of defining parameters for test what is much more readable and natural. Lets take a look:
def "should perform validation of email addresses"() {
    expect:
        emailValidator.validate(email) == result
    where:
        email           | result
        "WTF"           | false
        "@domain"       | false
        "foo@bar.com"   | true
        "a@test"        | false
}
In this code, each column of our "table" is treated as a separate variable and rows are values for subsequent test iterations.

Another useful feature of Spock during parameterizing test is its ability to "unroll" each parameterized test. Feature method from previous example could be defined as (the body stays the same, so I do not repeat it):
@Unroll("should validate email #email")
def "should perform validation of email addresses"() {
    // ...
}
With that annotation, Spock generate few methods each with its own name and run them separately. We can use symbols from where blocks in @Unroll argument by preceding it with '#' sign what is a signal to Spock to use it in generated method name.


What next?


Well, that was just quick and short journey  through Spock and its capabilities. However, with that basic tutorial you are ready to write many unit tests. In one of my future posts I am going to describe more features of Spock focusing especially on its mocking abilities.
DDevelopment & Design

Rapid js + css development

BackgroundLast time I had some work to do in OSGi web module written in Spring MVC. If we have application splitted to well-designed modules, back-end development in this framework run in OSGi environment is quite fast because after some modification w...