Clojure web development – state of the art – part 2

This is part 2 of my “Clojure web development” series. You can discuss first part on this reddit thread. After reading the comments I must explain two assumptions I had writing this series:

  • Keep things easy to understand for people from outside Clojure land, especially Java devs. That’s why I use REST/JSON in favor of transit and component as a “dependency injection” implementation which could be easily explained as Spring framework equivalent. The same goes with Om which is a bit verbose, but in my opinion it’s easier to understand for a start and has wider adoption than the other React wrappers.
  • Keep things easy to bootstrap on a developer machine. This is a hands-on walkthrough and all the individual steps have been committed to GitHub. That’s why I use MongoDB, which could not be the best choice for scaling your application to millions of users, but it’s perfect for bootstrapping – no schema, tables, just insert data and start working. I highly recommend Honza Kral polyglot persistence talk, where he encourages starting simple and optimize for developer happiness at the start of a project.

In previous post we bootstrapped a basic web application serving REST data with (static for now) Clojurescript frontend, all fully reloadable thanks to reloaded repl and figwheel. You can find final working version of it in this branch.

Today we’re going to display contact list stored in MongoDB. I assume you have MongoDB installed, if not – it’s trivial with docker.

Serving contact list from database

OK, let’s start. In the backend we need to add some dependencies to project.clj:

:dependencies ... [org.danielsz/system "0.1.9"] [com.novemberain/monger "2.0.0"]]

monger is an idiomatic Clojure wrapper for Mongo Java driver and system is a nice collection of components for various datastores, including Mongo (a bit like Spring Data for Spring).

In order to interact with a data store I like to use the concept of abstract repository. This should hide the implementation details from the invoker and allows to switch to another store in the future. So let’s create an abstract interface (in Clojure – protocol) in components/repo.clj:

(ns modern-clj-web.component.repo) (defprotocol ContactRepository (find-all [this]))

We need this as a parameter to allow Clojure runtime dispatching correct implementation of this repository. Mongo implementation with Monger is really simple:

(ns modern-clj-web.component.repo (:require [monger.collection :as mc] [monger.json])) ... (defrecord ContactRepoComponent [mongo] ContactRepository (find-all [this] (mc/find-maps (:db mongo) "contacts"))) (defn new-contact-repo-component [] (->ContactRepoComponent {}))

Things to notice here:

  • mc/find-maps just returns all records from collection as Clojure maps
  • ContactComponent gets injected with mongo component created by system library, which adds Mongo client under :db key
  • Since this component is stateless, we don’t need to implement component/Lifecycle interface, but it still can be wired in system like a typical lifecycle-aware component
  • Requiring monger.json adds JSON serialization support for Mongo types (e.g. ObjectId)

Ok, it’s now time to use our new component in the endpoint/example.clj:

(:require ... [modern-clj-web.component.repo :as r]) (defn example-endpoint [{ repo :contact-repo}] (routes ... (GET "/contacts" [] (response (r/find-all repo)))

The {repo :contact-repo} notation (destructuring) automatically binds :contact-repo key from system map to the repo value. So we need to assign our component to that key in system.clj:

(:require ... [modern-clj-web.component.repo :refer [new-contact-repo-component]] [system.components.mongo :refer [new-mongo-db]]) (-> (component/system-map :app (handler-component (:app config)) :http (jetty-server (:http config)) :example (endpoint-component example-endpoint) :mongo (new-mongo-db (:mongo-uri config)) :contact-repo (new-contact-repo-component)) (component/system-using {:http [:app] :app [:example] :example [:contact-repo] :contact-repo [:mongo]}))))

In short – we use system’s new-mongo-db to create Mongo component, make it a dependency to repository which itself is a dependency of example endpoint.

And finally we need to configure :mongo-uri config property in config.clj:

(def environ {:http {:port (some-> env :port Integer.)}} :mongo-uri "mongodb://localhost:27017/contacts"})

To check if it works fine, restart repl, type (go) again and make a GET to http://localhost:3000/contacts.

curl http://localhost:3000/contacts []

OK, so we got empty list since we don’t have any data in Mongo database. Let’s add some with mongo console:

mongo localhost:27017/contacts MongoDB shell version: 2.4.9 connecting to: localhost:27017/contacts > db.contacts.insert({firstname: "Al", lastname: "Pacino"}); > db.contacts.insert({firstname: "Johnny", lastname: "Depp"});

And finally our endpoint should return these two records:

curl http://localhost:3000/contacts [{"lastname":"Pacino","firstname":"Al","_id":"56158345fd2dabeddfb18799"},{"lastname":"Depp","firstname":"Johnny","_id":"56158355fd2dabeddfb1879a"}]

Sweet! Again – in case of any problems, check this commit.

Getting contacts from ClojureScript

In this step we’ll fetch the contacts with AJAX call on our ClojureScript frontend. As usual, we need few dependencies in project.clj for a start:

:dependencies ... [org.clojure/clojurescript "1.7.48"] [org.clojure/core.async "0.1.346.0-17112a-alpha"] [cljs-http "0.1.37"]

ClojureScript should be already visible by using figwheel, but it’s always better to require specific version explicitly. cljs-http is a HTTP client for ClojureScript and core.async provides facilities for asynchronous communication in CSP model, especially useful in ClojureScript. Let’s see how it works in practice.

To make an AJAX call we need to call methods from cljs-http.client, so let’s add this in core.cljs:

(ns ^:figwheel-always modern-clj-web.core (:require [cljs-http.client :as http])) (println (http/get "/contacts"))

You should see #object[cljs.core.async.impl.channels.ManyToManyChannel]. What madness is that???

This is the time we enter the core.async. The most common way to deal with asynchronous network calls from Javascript is by using callbacks or promises. The core.async way is by using channels. It makes your code look more like a sequence of synchronous calls and it’s easier to reason about. So the http/get function returns a channel on which the result is published when response arrives. In order to receive that message we need to read from this channel by using function. Since this is blocking, we also need to surround this call with go macro, just like in go language. So the correct way of getting contacts looks like this:

(:require ... [cljs.core.async :refer [ >! chan]]) (go (let [response ( (http/get "/contacts"))] (println (:body response))))

Adding Om component

Dealing with frontend code without introducing any structure could quickly become a real nightmare. In late 2015 we have basically two major JS frameworks on the field: Angular nad React. ClojureScript paradigms (functional programming, immutable data structures) fit really nice into React philosophy. In short, React application is composed of components taking data as input and rendering HTML as output. The output is not a real DOM, but so-called virtual DOM, which helps calculating diff from current view to updated one.

Among many React wrappers in ClojureScript I like using Om with om-tools to reduce some verbosity. Let’s introduce it into our project.clj:

:dependencies ... [org.omcljs/om "0.9.0"] [prismatic/om-tools "0.3.12"]

To render a “hello world” component we need to add some code in core.cljs:

(:require ... [om.core :as om] [om-tools.core :refer-macros [defcomponent]] [om-tools.dom :as dom :include-macros true])) (def app-state (atom {:message "hello from om"})) (defcomponent app [data owner] (render [_] (dom/div (:message data)))) (om/root app app-state {:target (.getElementById js/document "main")})

What’s going on here? The main concept of Om is keeping whole application state in one global atom, which is Clojure way of managing state. So we pass this app-state map (wrapped in atom) as a parameter to om/root which mounts components into real DOM (

from index.html). The app component just displays the :message value, so you should see “hello from om” rendered. If you have fighweel running, you can change the message value, and it should be updated instantly.

And finally let’s render our contacts with Om:

(defn get-contacts [] (go (let [response ( (http/get "/contacts"))] (:body response)))) (defcomponent contact-comp [contact _] (render [_] (dom/li (str (:firstname contact) " " (:lastname contact))))) (defcomponent app [data _] (will-mount [_] (go (let [contacts ( (get-contacts))] (om/update! data :contacts contacts)))) (render [_] (dom/div (dom/h2 (:message data)) (dom/ul (om/build-all contact-comp (:contacts data))))))

So the contact-comp is just rendering a single contact. We use om/build-all to render all contacts visible in :contacts field in global state. And most tricky part – we use will-mount lifecycle method to get contacts from server when app component is about to be mounted to the DOM.

Again, in this commit should be a working version in case of any problems.

And if you liked Om, I highly recommend official tutorials and Zero to Om series.

You May Also Like

How to use mocks in controller tests

Even since I started to write tests for my Grails application I couldn't find many articles on using mocks. Everyone is talking about tests and TDD but if you search for it there isn't many articles.

Today I want to share with you a test with mocks for a simple and complete scenario. I have a simple application that can fetch Twitter tweets and present it to user. I use REST service and I use GET to fetch tweets by id like this: http://api.twitter.com/1/statuses/show/236024636775735296.json. You can copy and paste it into your browser to see a result.

My application uses Grails 2.1 with spock-0.6 for tests. I have TwitterReaderService that fetches tweets by id, then I parse a response into my Tweet class.


class TwitterReaderService {
Tweet readTweet(String id) throws TwitterError {
try {
String jsonBody = callTwitter(id)
Tweet parsedTweet = parseBody(jsonBody)
return parsedTweet
} catch (Throwable t) {
throw new TwitterError(t)
}
}

private String callTwitter(String id) {
// TODO: implementation
}

private Tweet parseBody(String jsonBody) {
// TODO: implementation
}
}

class Tweet {
String id
String userId
String username
String text
Date createdAt
}

class TwitterError extends RuntimeException {}

TwitterController plays main part here. Users call show action along with id of a tweet. This action is my subject under test. I've implemented some basic functionality. It's easier to focus on it while writing tests.


class TwitterController {
def twitterReaderService

def index() {
}

def show() {
Tweet tweet = twitterReaderService.readTweet(params.id)
if (tweet == null) {
flash.message = 'Tweet not found'
redirect(action: 'index')
return
}

[tweet: tweet]
}
}

Let's start writing a test from scratch. Most important thing here is that I use mock for my TwitterReaderService. I do not construct new TwitterReaderService(), because in this test I test only TwitterController. I am not interested in injected service. I know how this service is supposed to work and I am not interested in internals. So before every test I inject a twitterReaderServiceMock into controller:


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {
TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

def setup() {
controller.twitterReaderService = twitterReaderServiceMock
}
}

Now it's time to think what scenarios I need to test. This line from TwitterReaderService is the most important:


Tweet readTweet(String id) throws TwitterError

You must think of this method like a black box right now. You know nothing of internals from controller's point of view. You're only interested what can be returned for you:

  • a TwitterError can be thrown
  • null can be returned
  • Tweet instance can be returned

This list is your test blueprint. Now answer a simple question for each element: "What do I want my controller to do in this situation?" and you have plan test:

  • show action should redirect to index if TwitterError is thrown and inform about error
  • show action should redirect to index and inform if tweet is not found
  • show action should show found tweet

That was easy and straightforward! And now is the best part: we use twitterReaderServiceMock to mock each of these three scenarios!

In Spock there is a good documentation about interaction with mocks. You declare what methods are called, how many times, what parameters are given and what should be returned. Remember a black box? Mock is your black box with detailed instruction, e.g.: I expect you that if receive exactly one call to readTweet with parameter '1' then you should throw me a TwitterError. Rephrase this sentence out loud and look at this:


1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }

This is a valid interaction definition on mock! It's that easy! Here is a complete test that fails for now:


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {
TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

def setup() {
controller.twitterReaderService = twitterReaderServiceMock
}

def "show should redirect to index if TwitterError is thrown"() {
given:
controller.params.id = '1'
when:
controller.show()
then:
1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }
0 * _._
flash.message == 'There was an error on fetching your tweet'
response.redirectUrl == '/twitter/index'
}
}

| Failure: show should redirect to index if TwitterError is thrown(pl.refaktor.twitter.TwitterControllerSpec)
| pl.refaktor.twitter.TwitterError
at pl.refaktor.twitter.TwitterControllerSpec.show should redirect to index if TwitterError is thrown_closure1(TwitterControllerSpec.groovy:29)

You may notice 0 * _._ notation. It says: I don't want any other mocks or any other methods called. Fail this test if something is called! It's a good practice to ensure that there are no more interactions than you want.

Ok, now I need to implement controller logic to handle TwitterError.


class TwitterController {

def twitterReaderService

def index() {
}

def show() {
Tweet tweet

try {
tweet = twitterReaderService.readTweet(params.id)
} catch (TwitterError e) {
log.error(e)
flash.message = 'There was an error on fetching your tweet'
redirect(action: 'index')
return
}

[tweet: tweet]
}
}

My tests passes! We have two scenarios left. Rule stays the same: TwitterReaderService returns something and we test against it. So this line is the heart of each test, change only returned values after >>:


1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }

Here is a complete test for three scenarios and controller that passes it.


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {

TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

def setup() {
controller.twitterReaderService = twitterReaderServiceMock
}

def "show should redirect to index if TwitterError is thrown"() {
given:
controller.params.id = '1'
when:
controller.show()
then:
1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }
0 * _._
flash.message == 'There was an error on fetching your tweet'
response.redirectUrl == '/twitter/index'
}

def "show should inform about not found tweet"() {
given:
controller.params.id = '1'
when:
controller.show()
then:
1 * twitterReaderServiceMock.readTweet('1') >> null
0 * _._
flash.message == 'Tweet not found'
response.redirectUrl == '/twitter/index'
}


def "show should show found tweet"() {
given:
controller.params.id = '1'
when:
controller.show()
then:
1 * twitterReaderServiceMock.readTweet('1') >> new Tweet()
0 * _._
flash.message == null
response.status == 200
}
}

class TwitterController {

def twitterReaderService

def index() {
}

def show() {
Tweet tweet

try {
tweet = twitterReaderService.readTweet(params.id)
} catch (TwitterError e) {
log.error(e)
flash.message = 'There was an error on fetching your tweet'
redirect(action: 'index')
return
}

if (tweet == null) {
flash.message = 'Tweet not found'
redirect(action: 'index')
return
}

[tweet: tweet]
}
}

The most important thing here is that we've tested controller-service interaction without logic implementation in service! That's why mock technique is so useful. It decouples your dependencies and let you focus on exactly one subject under test. Happy testing!