DDevelopment & Design

Spring Boot 2.0 HTTP request metrics with Micrometer

Introduction

Brand new Spring Boot 2.0 has just been released and TouKs couldn’t wait to try it in the production. One of the newly added features that we investigated was metrics system based on Micrometer library (https://micrometer.io/). In this post I will cover some of our experiences with this so far.

The goal was to get basic HTTP request metrics, report them to InfluxDB and draw some fancy graphs in Grafana. In particular we needed:

  • Throughput – total number of requests in given time unit
  • Response status statistics – how many 200-like and 500-like response occurred
  • Response time statistics: mean, median, percentiles

What was wrong with Dropwizard metrics

Nothing that I am aware of. Metrics Spring integration however is a different story….

Last stable release of Metrics Spring (v. 3.1.3) was in late 2015 and it was compatible with Dropwizard Metrics (v. 3.1.2). From this time Dropwizard Metrics moved to version 4 and 5, but Metrics Spring literally died. This causes a couple of rather unpleasant facts:

  • There are some known bugs that will never be solved
  • You can’t benefit from Dropwizard Metrics improvements
  • Sooner or later you will use a library that depends on a different version of Dropwizard Metrics and it will hurt

As an InfluxDB user I was also facing some problems with reporting tags. After a couple of tries we ended up using an obscure Graphite interface that was luckily compatible with Influx.

Let’s turn on the metrics

Adding metrics to your Spring Boot project can be done in three very simple steps. First add a dependency to micrometer-registry-xxx, where xxx is your favourite metrics storage. In our case:

<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-influx</artifactId>
</dependency>

 

Now it is time for just a little bit of configuration in application.yml:

management:
  metrics:
    export:
      influx:
        uri: http://localhost:8086
        db: services
        step: 5s  ### <- (1)

 

And a proper configuration bean:

@Configuration public class MetricsConfig {
    private static final Duration HISTOGRAM_EXPIRY = Duration.ofMinutes(10);
    
    private static final Duration STEP = Duration.ofSeconds(5);
    
    @Value
    ("${host_id}") private String hostId;
    
    @Value
    ("${service_id}") private String serviceId;
    
    @Bean 
    public MeterRegistryCustomizer < MeterRegistry > metricsCommonTags() { // (2)
        return registry - > registry.config()
        .commonTags("host", hostId, "service", serviceId) // (3)
        .meterFilter(MeterFilter.deny(id - > { // (4)
                String uri = id.getTag("uri");
                return uri != null && uri.startsWith("/swagger");
            }))
            .meterFilter(new MeterFilter() {
                @Override 
                public DistributionStatisticConfig configure(Meter.Id id, DistributionStatisticConfig config) {
                    return config.merge(DistributionStatisticConfig.builder().percentilesHistogram(true).percentiles(0.5, 0.75, 0.95) // (5)
                    .expiry(HISTOGRAM_EXPIRY) // (6)
                    .bufferLength((int)(HISTOGRAM_EXPIRY.toMillis() / STEP.toMillis())) // (7)
                    .build());
                }
            });
    }
}

 

Simple as that. For sure it is not the minimal working example, but I believe some of our ideas are worth mentioning.

Dive into configuration

Config is rather self-explanatory, but let’s take a look at couple of interesting features.

(1) Step defines how often data is sent by reporter. This value should be related to your expected traffic, because you don’t want to see 90% of zeros.

(2) Be aware that there can be many reporters sharing the same config. Customising each behaviour can be done by using more specific type parameter e.g. InfluxMeterRegistry.

(3) Tags that will be added to every metric. As you can see it’s very handy for identifying hosts in a cluster.

(4) Skipping not important endpoints will limit unwanted data.

(5) A list of percentiles you would like to track

(6)(7) Histograms are calculated for some defined time window where more recent values have bigger impact on final value. The bigger time window you choose, the more accurate statistics are, but the less sudden will be changes of percentile value in case of very big or very small response time. It is also very important to increase buffer length as you increase expiry time.

Afterthought

We believe that migrating to Micrometer is worth spending time as configuration and reporting becomes simpler. The only thing that surprised us was reporting rate of throughput and status counts rather than cumulative values. But this is another story to be told…

Special thanks to Arek Burdach for support.

Scala / Java developer at TouK since 2015. Passionate West Coast Swing dancer :)

— Previous article

MapStruct mapper injection in OSGi Blueprint

Next article —

Testing Kotlin with Spock Part 1 - Object

You May Also Like
DDevelopment & Design

How to use mocks in controller tests

Even since I started to write tests for my Grails application I couldn't find many articles on using mocks. Everyone is talking about tests and TDD but if you search for it there isn't many articles.

Today I want to share with you a test with mocks for a simple and complete scenario. I have a simple application that can fetch Twitter tweets and present it to user. I use REST service and I use GET to fetch tweets by id like this: http://api.twitter.com/1/statuses/show/236024636775735296.json. You can copy and paste it into your browser to see a result.

My application uses Grails 2.1 with spock-0.6 for tests. I have TwitterReaderService that fetches tweets by id, then I parse a response into my Tweet class.


class TwitterReaderService {
    Tweet readTweet(String id) throws TwitterError {
        try {
            String jsonBody = callTwitter(id)
            Tweet parsedTweet = parseBody(jsonBody)
            return parsedTweet
        } catch (Throwable t) {
            throw new TwitterError(t)
        }
    }

    private String callTwitter(String id) {
        // TODO: implementation
    }

    private Tweet parseBody(String jsonBody) {
        // TODO: implementation
    }
}

class Tweet {
    String id
    String userId
    String username
    String text
    Date createdAt
}

class TwitterError extends RuntimeException {}

TwitterController plays main part here. Users call show action along with id of a tweet. This action is my subject under test. I've implemented some basic functionality. It's easier to focus on it while writing tests.


class TwitterController {
    def twitterReaderService

    def index() {
    }

    def show() {
        Tweet tweet = twitterReaderService.readTweet(params.id)
        if (tweet == null) {
            flash.message = 'Tweet not found'
            redirect(action: 'index')
            return
        }

        [tweet: tweet]
    }
}

Let's start writing a test from scratch. Most important thing here is that I use mock for my TwitterReaderService. I do not construct new TwitterReaderService(), because in this test I test only TwitterController. I am not interested in injected service. I know how this service is supposed to work and I am not interested in internals. So before every test I inject a twitterReaderServiceMock into controller:


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {
    TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

    def setup() {
        controller.twitterReaderService = twitterReaderServiceMock
    }
}

Now it's time to think what scenarios I need to test. This line from TwitterReaderService is the most important:


Tweet readTweet(String id) throws TwitterError

You must think of this method like a black box right now. You know nothing of internals from controller's point of view. You're only interested what can be returned for you:

  • a TwitterError can be thrown
  • null can be returned
  • Tweet instance can be returned

This list is your test blueprint. Now answer a simple question for each element: "What do I want my controller to do in this situation?" and you have plan test:

  • show action should redirect to index if TwitterError is thrown and inform about error
  • show action should redirect to index and inform if tweet is not found
  • show action should show found tweet

That was easy and straightforward! And now is the best part: we use twitterReaderServiceMock to mock each of these three scenarios!

In Spock there is a good documentation about interaction with mocks. You declare what methods are called, how many times, what parameters are given and what should be returned. Remember a black box? Mock is your black box with detailed instruction, e.g.: I expect you that if receive exactly one call to readTweet with parameter '1' then you should throw me a TwitterError. Rephrase this sentence out loud and look at this:


1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }

This is a valid interaction definition on mock! It's that easy! Here is a complete test that fails for now:


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {
    TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

    def setup() {
        controller.twitterReaderService = twitterReaderServiceMock
    }

    def "show should redirect to index if TwitterError is thrown"() {
        given:
            controller.params.id = '1'
        when:
            controller.show()
        then:
            1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }
            0 * _._
            flash.message == 'There was an error on fetching your tweet'
            response.redirectUrl == '/twitter/index'
    }
}

| Failure:  show should redirect to index if TwitterError is thrown(pl.refaktor.twitter.TwitterControllerSpec)
|  pl.refaktor.twitter.TwitterError
 at pl.refaktor.twitter.TwitterControllerSpec.show should redirect to index if TwitterError is thrown_closure1(TwitterControllerSpec.groovy:29)

You may notice 0 * _._ notation. It says: I don't want any other mocks or any other methods called. Fail this test if something is called! It's a good practice to ensure that there are no more interactions than you want.

Ok, now I need to implement controller logic to handle TwitterError.


class TwitterController {

    def twitterReaderService

    def index() {
    }

    def show() {
        Tweet tweet

        try {
            tweet = twitterReaderService.readTweet(params.id)
        } catch (TwitterError e) {
            log.error(e)
            flash.message = 'There was an error on fetching your tweet'
            redirect(action: 'index')
            return
        }

        [tweet: tweet]
    }
}

My tests passes! We have two scenarios left. Rule stays the same: TwitterReaderService returns something and we test against it. So this line is the heart of each test, change only returned values after >>:


1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }

Here is a complete test for three scenarios and controller that passes it.


import grails.test.mixin.TestFor
import spock.lang.Specification

@TestFor(TwitterController)
class TwitterControllerSpec extends Specification {

    TwitterReaderService twitterReaderServiceMock = Mock(TwitterReaderService)

    def setup() {
        controller.twitterReaderService = twitterReaderServiceMock
    }

    def "show should redirect to index if TwitterError is thrown"() {
        given:
            controller.params.id = '1'
        when:
            controller.show()
        then:
            1 * twitterReaderServiceMock.readTweet('1') >> { throw new TwitterError() }
            0 * _._
            flash.message == 'There was an error on fetching your tweet'
            response.redirectUrl == '/twitter/index'
    }

    def "show should inform about not found tweet"() {
        given:
            controller.params.id = '1'
        when:
            controller.show()
        then:
            1 * twitterReaderServiceMock.readTweet('1') >> null
            0 * _._
            flash.message == 'Tweet not found'
            response.redirectUrl == '/twitter/index'
    }


    def "show should show found tweet"() {
        given:
            controller.params.id = '1'
        when:
            controller.show()
        then:
            1 * twitterReaderServiceMock.readTweet('1') >> new Tweet()
            0 * _._
            flash.message == null
            response.status == 200
    }
}

class TwitterController {

    def twitterReaderService

    def index() {
    }

    def show() {
        Tweet tweet

        try {
            tweet = twitterReaderService.readTweet(params.id)
        } catch (TwitterError e) {
            log.error(e)
            flash.message = 'There was an error on fetching your tweet'
            redirect(action: 'index')
            return
        }

        if (tweet == null) {
            flash.message = 'Tweet not found'
            redirect(action: 'index')
            return
        }

        [tweet: tweet]
    }
}

The most important thing here is that we've tested controller-service interaction without logic implementation in service! That's why mock technique is so useful. It decouples your dependencies and let you focus on exactly one subject under test. Happy testing!