Micro services on the JVM part 1 – Clojure

Micro services could be a buzzword of 2014 for me. Few months ago I was curious to try Dropwizard framework as a separate backend, but didn’t get the whole idea yet. But then I watched a mind-blowing “Micro-Services Architecture” talk by Fred George. Also, the 4.0 release notes of Spring covers microservices as an important rising trend as well. After 10 years of having SOA in mind, but still developing monoliths, it’s a really tempting idea to try to decouple systems into a set of independently developed and deployed RESTful services. Micro services could be a buzzword of 2014 for me. Few months ago I was curious to try Dropwizard framework as a separate backend, but didn’t get the whole idea yet. But then I watched a mind-blowing “Micro-Services Architecture” talk by Fred George. Also, the 4.0 release notes of Spring covers microservices as an important rising trend as well. After 10 years of having SOA in mind, but still developing monoliths, it’s a really tempting idea to try to decouple systems into a set of independently developed and deployed RESTful services.

Micro services could be a buzzword of 2014 for me. Few months ago I was curious to try Dropwizard framework as a separate backend, but didn’t get the whole idea yet. But then I watched a mind-blowing “Micro-Services Architecture” talk by Fred George. Also, the 4.0 release notes of Spring covers microservices as an important rising trend as well. After 10 years of having SOA in mind, but still developing monoliths, it’s a really tempting idea to try to decouple systems into a set of independently developed and deployed RESTful services.

So when I decided to write a simple API for my DevRates.com website, instead of adding some code to existing codebase, I wanted to build a separate tiny app. But what’s the best stack for micro-services? In this series of posts I’ll try to compare various JVM technology stacks for this approach.

Here is my list of must-have features for the stack:

  • declarative REST support (no manual URL parsing)
  • native JSON support (bidirectional JSON-object mapping)
  • single “fat” jar packaging, no web container needed
  • fast development feedback loop (eg. runtime code reloading)
  • Swagger and Metrics integration

In this post I’ll try to cover Clojure with Ring and Compojure.

TL;DR

You can find all the covered concepts in the following GitHub examples:

Basic setup

There is an excellent Zaiste’s tutorial showing how to kickstart REST app with Compojure, just follow these few simple steps (the rest of the post assumes compojure-rest as the app name).

My sample route from handler.clj:

(defroutes app-routes (GET "/messages/:name" [name] {:body {:message (str "Hello World" " " name)}}) (route/resources "/") (route/not-found "Not Found"))

Fat jar

In a simple setup, Compojure app is being run through lein ring plugin. To enable running it as a standalone command-line app, you have to write a main method which starts Jetty server.

project.clj

:dependencies ... [ring/ring-jetty-adapter "1.2.0"] .. :main compojure-rest.handler

handler.clj

To build a single “fat” jar just run lein uberjar, and then java -jar target/compojure-rest-0.1.0-SNAPSHOT-standalone.jar runs the app.

(ns compojure-rest.handler ... (:require ... [ring.adapter.jetty :refer (run-jetty)]) (:gen-class)) ... (defn -main [& args] (run-jetty app {:port 3000 :join? false }))

Swagger

The nice thing about Compojure is that you can easy expose Swagger documentation by using swag library. There are some conflicts between swag and ring lein plugin, so just look at the compojure-swag for a working example.

Here is a typical snippet from handler.clj:

(set-base "http://localhost:3000") (defroutes- messages {:path "/messages" :description "Messages management"} (GET- "/messages/:name" [^:string name] {:nickname "getMessages" :summary "Get message"} {:body {:message (str "Hello World" " " name)}}) (route/resources "/") (route/not-found "Not Found"))

So, swag introduces defroutes-, GET-, POST- which take additional metadata as parameters to generate Swagger docs. If you’re little scared with this ^:string fragment – check metadata section from Clojure manual. Swagger-compatible definition should be available at http://localhost:3000/api-docs.json after running the app.

Metrics

To expose basic metrics of your REST API calls just use Ring-compatible metrics-clojure-ring library.

project.clj

:dependencies ... [metrics-clojure-ring "1.0.1"] ...

handler.clj

(ns compojure-rest.handler ... (:require ... [metrics.ring.expose :refer [expose-metrics-as-json]] [metrics.ring.instrument :refer [instrument]])) ... (def app (expose-metrics-as-json (instrument app) "/stats/"))

After generating some load by eg. wrk, you can check the collected stats by visiting http://localhost:3000/stats/.

ring.requests.rate.GET: { type: "meter", rates: { 1: 189.5836593065824, 5: 39.21602480726734, 15: 13.146759983907245 } }

Some random Clojure thoughts

  • The best newbie guide to Clojure is Kyle Kingsbury’s “Clojure from the ground up” series.
  • Leiningen is probably the best build tool for the JVM. Easy to install, fast, simple, no XML – just doing it right. And the “new” project templates is what’s Maven been missing from ages (anyone using archetypes?).
  • Lighttable is great! I’m really impressed with the fast feedback loop by just ctrl+entering the expressions.
  • Also, live reloading with ring server works fine. Just change the change code and see the changes immediately. Rapid!
  • Unlike other recently popular languages, Clojure has no killer-framework. Rails, Play/Akka, Grails/Gradle – all of these are key parts of Ruby, Scala and Groovy ecosystems. What about Clojure? A collection of small (micro?) libraries doing one thing well and working great together – just like Unix commands.
  • It may be true that Clojure is not good for large projects. With all the complex contructs (meta or ) and no control of the visibility, it could be hard to maintain large codebase. But it’s not a first-class problem in a micro-services world..

Resources

You May Also Like

mount.ntfs high cpu ubuntu

My computer suffers from sudden and continous hard drive load strokes. Sometimes it lasts for a few minutes and hence work is impossible because everything goes very slow.I'm trying to locate the cause because it makes me nervous :)Today I found one of...

Atom Feeds with Spring MVC

How to add feeds (Atom) to your web application with just two classes?
How about Spring MVC?

Here are my assumptions:
  • you are using Spring framework
  • you have some entity, say “News”, that you want to publish in your feeds
  • your "News" entity has creationDate, title, and shortDescription
  • you have some repository/dao, say "NewsRepository", that will return the news from your database
  • you want to write as little as possible
  • you don't want to format Atom (xml) by hand
You actually do NOT need to use Spring MVC in your application already. If you do, skip to step 3.


Step 1: add Spring MVC dependency to your application
With maven that will be:
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>3.1.0.RELEASE</version>
</dependency>

Step 2: add Spring MVC DispatcherServlet
With web.xml that would be:
<servlet>
    <servlet-name>dispatcher</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath:spring-mvc.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
    <servlet-name>dispatcher</servlet-name>
    <url-pattern>/feed</url-pattern>
</servlet-mapping>
Notice, I set the url-pattern to “/feed” which means I don't want Spring MVC to handle any other urls in my app (I'm using a different web framework for the rest of the app). I also give it a brand new contextConfigLocation, where only the mvc configuration is kept.

Remember that, when you add a DispatcherServlet to an app that already has Spring (from ContextLoaderListener for example), your context is inherited from the global one, so you should not create beans that exist there again, or include xml that defines them. Watch out for Spring context getting up twice, and refer to spring or servlet documentation to understand what's happaning.

Step 3. add ROME – a library to handle Atom format
With maven that is:
<dependency>
    <groupId>net.java.dev.rome</groupId>
    <artifactId>rome</artifactId>
    <version>1.0.0</version>
</dependency>

Step 4. write your very simple controller
@Controller
public class FeedController {
    static final String LAST_UPDATE_VIEW_KEY = "lastUpdate";
    static final String NEWS_VIEW_KEY = "news";
    private NewsRepository newsRepository;
    private String viewName;

    protected FeedController() {} //required by cglib

    public FeedController(NewsRepository newsRepository, String viewName) {
        notNull(newsRepository); hasText(viewName);
        this.newsRepository = newsRepository;
        this.viewName = viewName;
    }

    @RequestMapping(value = "/feed", method = RequestMethod.GET)        
    @Transactional
    public ModelAndView feed() {
        ModelAndView modelAndView = new ModelAndView();
        modelAndView.setViewName(viewName);
        List<News> news = newsRepository.fetchPublished();
        modelAndView.addObject(NEWS_VIEW_KEY, news);
        modelAndView.addObject(LAST_UPDATE_VIEW_KEY, getCreationDateOfTheLast(news));
        return modelAndView;
    }

    private Date getCreationDateOfTheLast(List<News> news) {
        if(news.size() > 0) {
            return news.get(0).getCreationDate();
        }
        return new Date(0);
    }
}
And here's a test for it, in case you want to copy&paste (who doesn't?):
@RunWith(MockitoJUnitRunner.class)
public class FeedControllerShould {
    @Mock private NewsRepository newsRepository;
    private Date FORMER_ENTRY_CREATION_DATE = new Date(1);
    private Date LATTER_ENTRY_CREATION_DATE = new Date(2);
    private ArrayList<News> newsList;
    private FeedController feedController;

    @Before
    public void prepareNewsList() {
        News news1 = new News().title("title1").creationDate(FORMER_ENTRY_CREATION_DATE);
        News news2 = new News().title("title2").creationDate(LATTER_ENTRY_CREATION_DATE);
        newsList = newArrayList(news2, news1);
    }

    @Before
    public void prepareFeedController() {
        feedController = new FeedController(newsRepository, "viewName");
    }

    @Test
    public void returnViewWithNews() {
        //given
        given(newsRepository.fetchPublished()).willReturn(newsList);
        
        //when
        ModelAndView modelAndView = feedController.feed();
        
        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.NEWS_VIEW_KEY, newsList));
    }

    @Test
    public void returnViewWithLastUpdateTime() {
        //given
        given(newsRepository.fetchPublished()).willReturn(newsList);

        //when
        ModelAndView modelAndView = feedController.feed();

        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.LAST_UPDATE_VIEW_KEY, LATTER_ENTRY_CREATION_DATE));
    }

    @Test
    public void returnTheBeginningOfTimeAsLastUpdateInViewWhenListIsEmpty() {
        //given
        given(newsRepository.fetchPublished()).willReturn(new ArrayList<News>());

        //when
        ModelAndView modelAndView = feedController.feed();

        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.LAST_UPDATE_VIEW_KEY, new Date(0)));
    }
}
Notice: here, I'm using fest-assert and mockito. The dependencies are:
<dependency>
 <groupId>org.easytesting</groupId>
 <artifactId>fest-assert</artifactId>
 <version>1.4</version>
 <scope>test</scope>
</dependency>
<dependency>
 <groupId>org.mockito</groupId>
 <artifactId>mockito-all</artifactId>
 <version>1.8.5</version>
 <scope>test</scope>
</dependency>

Step 5. write your very simple view
Here's where all the magic formatting happens. Be sure to take a look at all the methods of Entry class, as there is quite a lot you may want to use/fill.
import org.springframework.web.servlet.view.feed.AbstractAtomFeedView;
[...]

public class AtomFeedView extends AbstractAtomFeedView {
    private String feedId = "tag:yourFantastiSiteName";
    private String title = "yourFantastiSiteName: news";
    private String newsAbsoluteUrl = "http://yourfanstasticsiteUrl.com/news/"; 

    @Override
    protected void buildFeedMetadata(Map<String, Object> model, Feed feed, HttpServletRequest request) {
        feed.setId(feedId);
        feed.setTitle(title);
        setUpdatedIfNeeded(model, feed);
    }

    private void setUpdatedIfNeeded(Map<String, Object> model, Feed feed) {
        @SuppressWarnings("unchecked")
        Date lastUpdate = (Date)model.get(FeedController.LAST_UPDATE_VIEW_KEY);
        if (feed.getUpdated() == null || lastUpdate != null || lastUpdate.compareTo(feed.getUpdated()) > 0) {
            feed.setUpdated(lastUpdate);
        }
    }

    @Override
    protected List<Entry> buildFeedEntries(Map<String, Object> model, HttpServletRequest request, HttpServletResponse response) throws Exception {
        @SuppressWarnings("unchecked")
        List<News> newsList = (List<News>)model.get(FeedController.NEWS_VIEW_KEY);
        List<Entry> entries = new ArrayList<Entry>();
        for (News news : newsList) {
            addEntry(entries, news);
        }
        return entries;
    }

    private void addEntry(List<Entry> entries, News news) {
        Entry entry = new Entry();
        entry.setId(feedId + ", " + news.getId());
        entry.setTitle(news.getTitle());
        entry.setUpdated(news.getCreationDate());
        entry = setSummary(news, entry);
        entry = setLink(news, entry);
        entries.add(entry);
    }

    private Entry setSummary(News news, Entry entry) {
        Content summary = new Content();
        summary.setValue(news.getShortDescription());
        entry.setSummary(summary);
        return entry;
    }

    private Entry setLink(News news, Entry entry) {
        Link link = new Link();
        link.setType("text/html");
        link.setHref(newsAbsoluteUrl + news.getId()); //because I have a different controller to show news at http://yourfanstasticsiteUrl.com/news/ID
        entry.setAlternateLinks(newArrayList(link));
        return entry;
    }

}

Step 6. add your classes to your Spring context
I'm using xml approach. because I'm old and I love xml. No, seriously, I use xml because I may want to declare FeedController a few times with different views (RSS 1.0, RSS 2.0, etc.).

So this is the forementioned spring-mvc.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
        <property name="mediaTypes">
            <map>
                <entry key="atom" value="application/atom+xml"/>
                <entry key="html" value="text/html"/>
            </map>
        </property>
        <property name="viewResolvers">
            <list>
                <bean class="org.springframework.web.servlet.view.BeanNameViewResolver"/>
            </list>
        </property>
    </bean>

    <bean class="eu.margiel.pages.confitura.feed.FeedController">
        <constructor-arg index="0" ref="newsRepository"/>
        <constructor-arg index="1" value="atomFeedView"/>
    </bean>

    <bean id="atomFeedView" class="eu.margiel.pages.confitura.feed.AtomFeedView"/>
</beans>

And you are done.

I've been asked a few times before to put all the working code in some public repo, so this time it's the other way around. I've describe things that I had already published, and you can grab the commit from the bitbucket.

Hope that helps.