Tag: REST

New HTTP Logger Grails plugin

I've wrote a new Grails plugin - httplogger. It logs:

  • request information (url, headers, cookies, method, body),
  • grails dispatch information (controller, action, parameters),
  • response information (elapsed time and body).

It is mostly useful for logging your REST traffic. Full HTTP web pages can be huge to log and generally waste your space. I suggest to map all of your REST controllers with the same path in UrlMappings, e.g. /rest/ and configure this plugin with this path.

Here is some simple output just to give you a taste of it.

17:16:00,331 INFO  filters.LogRawRequestInfoFilter  - << #1 GET http://localhost:8080/riddle/rest/index?username=admin&search=foo
17:16:00,340 INFO  filters.LogRawRequestInfoFilter  - << #1 headers Cookie: 'JSESSIONID=DF4EA5725AC4A4990281BD96963739B0; splashShown1.6=1', Accept-Language: 'en-US,en;q=0.8,pl;q=0.6', X-MyHeader: 'null'
17:16:00,342 INFO  filters.LogGrailsUrlsInfoFilter  - << #1 dispatched to rest/index with parsed params [username:[admin], search:[foo]].
17:16:00,731 INFO  filters.LogOutputResponseFilter  - >> #1 returned 200, took 405 ms.
17:16:00,745 INFO  filters.LogOutputResponseFilter  - >> #1 responded with '{count:0}'
17:18:55,799 INFO  filters.LogRawRequestInfoFilter  - << #2 POST http://localhost:8080/riddle/rest/login
17:18:55,799 INFO  filters.LogRawRequestInfoFilter  - << #2 headers Cookie: 'JSESSIONID=DF4EA5725AC4A4990281BD96963739B0; splashShown1.6=1', Accept-Language: 'en-US,en;q=0.8,pl;q=0.6', X-MyHeader: 'null'
17:18:55,800 INFO  filters.LogRawRequestInfoFilter  - << #2 body: 'username=admin&password=password'
17:18:55,801 INFO  filters.LogOutputResponseFilter  - >> #2 returned 404, took 3 ms.
17:18:55,802 INFO  filters.LogOutputResponseFilter  - >> #2 responded with ''

Official plugin information can be found on Grails plugins website here: http://grails.org/plugins/httplogger or you can browse code on github: TouK/grails-httplogger.

New HTTP Logger Grails plugin

I've wrote a new Grails plugin - httplogger. It logs:

  • request information (url, headers, cookies, method, body),
  • grails dispatch information (controller, action, parameters),
  • response information (elapsed time and body).

It is mostly useful for logging your REST traffic. Full HTTP web pages can be huge to log and generally waste your space. I suggest to map all of your REST controllers with the same path in UrlMappings, e.g. /rest/ and configure this plugin with this path.

Here is some simple output just to give you a taste of it.

17:16:00,331 INFO  filters.LogRawRequestInfoFilter  - << #1 GET http://localhost:8080/riddle/rest/index?username=admin&search=foo
17:16:00,340 INFO  filters.LogRawRequestInfoFilter  - << #1 headers Cookie: 'JSESSIONID=DF4EA5725AC4A4990281BD96963739B0; splashShown1.6=1', Accept-Language: 'en-US,en;q=0.8,pl;q=0.6', X-MyHeader: 'null'
17:16:00,342 INFO  filters.LogGrailsUrlsInfoFilter  - << #1 dispatched to rest/index with parsed params [username:[admin], search:[foo]].
17:16:00,731 INFO  filters.LogOutputResponseFilter  - >> #1 returned 200, took 405 ms.
17:16:00,745 INFO  filters.LogOutputResponseFilter  - >> #1 responded with '{count:0}'
17:18:55,799 INFO  filters.LogRawRequestInfoFilter  - << #2 POST http://localhost:8080/riddle/rest/login
17:18:55,799 INFO  filters.LogRawRequestInfoFilter  - << #2 headers Cookie: 'JSESSIONID=DF4EA5725AC4A4990281BD96963739B0; splashShown1.6=1', Accept-Language: 'en-US,en;q=0.8,pl;q=0.6', X-MyHeader: 'null'
17:18:55,800 INFO  filters.LogRawRequestInfoFilter  - << #2 body: 'username=admin&password=password'
17:18:55,801 INFO  filters.LogOutputResponseFilter  - >> #2 returned 404, took 3 ms.
17:18:55,802 INFO  filters.LogOutputResponseFilter  - >> #2 responded with ''

Official plugin information can be found on Grails plugins website here: http://grails.org/plugins/httplogger or you can browse code on github: TouK/grails-httplogger.

How to keep session in HttpBuilder with cookies

In my real-world scenario I have a REST service for AJAX purposes. It renders data series for graphs. I want to test it with groovy's excellent HttpBuilder. There is a problem though - these requests are only available for already logged in users.

In this post I present a complete solution to maintain a session state between HttpBuilder's requests.

Session in HttpBuilder

First of all a quick reminder about session. Session is a simulation of state for HTTP requests, which are stateless by its nature. Once you log in you receive a unique cookie (one or more) that identifies you for sequential requests. Every time you send request you send this cookie along. This way server recognizes you and matches you to your session, which is kept on server. Cookie gets invlid once you log out or it times out, for example after 20 minutes of inactivity. Next time you visit a page you get a new, unique cookie.

In order to keep session alive in HttpBuilder I need to:

  1. log in to my Grails application
  2. receive a JSESSIONID cookie in response
  3. store that cookie and send it along with every subsenquential request

I've created RestConnector class that wraps up HttpBuilder. It's main improvement is that it keeps received cookie in a list.

package eu.spoonman.connectors.RestConnector

import groovyx.net.http.Method
import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.HttpResponseDecorator

class RestConnector {
    private String baseUrl
    private HTTPBuilder httpBuilder
    private List<String> cookies

    RestConnector(String url) {
        this.baseUrl = url
        this.httpBuilder = initializeHttpBuilder()
        this.cookies = []
    }

    public def request(Method method, ContentType contentType, String url, Map<String, Serializable> params) {
        debug("Send $method request to ${this.baseUrl}$url: $params")
        httpBuilder.request(method, contentType) { request ->
            uri.path = url
            uri.query = params
            headers['Cookie'] = cookies.join(';')
        }
    }

    private HTTPBuilder initializeHttpBuilder() {
        def httpBuilder = new HTTPBuilder(baseUrl)

        httpBuilder.handler.success = { HttpResponseDecorator resp, reader ->
            resp.getHeaders('Set-Cookie').each {
                //[Set-Cookie: JSESSIONID=E68D4799D4D6282F0348FDB7E8B88AE9; Path=/frontoffice/; HttpOnly]
                String cookie = it.value.split(';')[0]
                debug("Adding cookie to collection: $cookie")
                cookies.add(cookie)
            }
            debug("Response: ${reader}")
            return reader
        }
        return httpBuilder
    }


    private debug(String message) {
        System.out.println(message) //for Gradle
    }
}

A few things to notice in a class above. Constructor sets base URL and creates HttpBuilder instance that can be reused. Next, there is a handler on successful request that checks if I receive any cookie. It adds received cookies to list. Finally, there is a request method that calls HttpBuilder#request but it adds cookies to HTTP headers so server can recognize me as a logged in user.

Sending cookies with every request is a core component in here. It simulates browser's behavior and maintains session.

How to use it?

I will show you how to use this utility class it in Spock test below. It is fairly simple.

First I login to my application and I ensure that I receive a cookie in return, which is equivalent to being logged in. Then I send a request with that cookie sent in HTTP header. This is a Spock test that implements it:

package eu.spoonman.specs.rest

import eu.spoonman.connectors.RestConnector.RestConnector
import groovyx.net.http.ContentType
import groovyx.net.http.Method
import spock.lang.Shared
import spock.lang.Specification
import spock.lang.Stepwise

@Stepwise
class RestChartSpec extends Specification {
    @Shared
    RestConnector restConnector

    def setupSpec() {
        restConnector = new RestConnector('http://localhost:8080')
    }

    def "should login as test"() {
        given:
            Map params = [j_username: 'test', j_password: 'test']
        when:
            restConnector.request(Method.POST, ContentType.ANY, '/frontoffice/j_spring_security_check', params)
        then:
            !(restConnector.cookies.empty)
    }

    def "should allow access to chart data series"() {
        given:
            Map params = [days: 14]
        when:
            Map result = restConnector.request(Method.POST, ContentType.JSON, "frontoffice/chart/series", params)
        then:
            result != null
            result.series.size() > 0
    }
}

I create a new RestConnector instance in setupSpec with my application's base URL. Please notice that it has @Shared annotation so it's shared between tests.

@Stepwise is crucial annotation for this specification. It means that Spock executes tests exactly in order they're defined. I need to ensure that login is executed first. I also need to assert that I receive a cookie and list is not empty. I could move this step into setupSpec method too, but I prefer it to be a first test in a specification.

Second test is always executed after login thus it sends cookies within request headers. This is exactly what I wanted to achieve.

How to keep session in HttpBuilder with cookies

In my real-world scenario I have a REST service for AJAX purposes. It renders data series for graphs. I want to test it with groovy's excellent HttpBuilder. There is a problem though - these requests are only available for already logged in users.

In this post I present a complete solution to maintain a session state between HttpBuilder's requests.

Session in HttpBuilder

First of all a quick reminder about session. Session is a simulation of state for HTTP requests, which are stateless by its nature. Once you log in you receive a unique cookie (one or more) that identifies you for sequential requests. Every time you send request you send this cookie along. This way server recognizes you and matches you to your session, which is kept on server. Cookie gets invlid once you log out or it times out, for example after 20 minutes of inactivity. Next time you visit a page you get a new, unique cookie.

In order to keep session alive in HttpBuilder I need to:

  1. log in to my Grails application
  2. receive a JSESSIONID cookie in response
  3. store that cookie and send it along with every subsenquential request

I've created RestConnector class that wraps up HttpBuilder. It's main improvement is that it keeps received cookie in a list.

package eu.spoonman.connectors.RestConnector

import groovyx.net.http.Method
import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.HttpResponseDecorator

class RestConnector {
    private String baseUrl
    private HTTPBuilder httpBuilder
    private List<String> cookies

    RestConnector(String url) {
        this.baseUrl = url
        this.httpBuilder = initializeHttpBuilder()
        this.cookies = []
    }

    public def request(Method method, ContentType contentType, String url, Map<String, Serializable> params) {
        debug("Send $method request to ${this.baseUrl}$url: $params")
        httpBuilder.request(method, contentType) { request ->
            uri.path = url
            uri.query = params
            headers['Cookie'] = cookies.join(';')
        }
    }

    private HTTPBuilder initializeHttpBuilder() {
        def httpBuilder = new HTTPBuilder(baseUrl)

        httpBuilder.handler.success = { HttpResponseDecorator resp, reader ->
            resp.getHeaders('Set-Cookie').each {
                //[Set-Cookie: JSESSIONID=E68D4799D4D6282F0348FDB7E8B88AE9; Path=/frontoffice/; HttpOnly]
                String cookie = it.value.split(';')[0]
                debug("Adding cookie to collection: $cookie")
                cookies.add(cookie)
            }
            debug("Response: ${reader}")
            return reader
        }
        return httpBuilder
    }


    private debug(String message) {
        System.out.println(message) //for Gradle
    }
}

A few things to notice in a class above. Constructor sets base URL and creates HttpBuilder instance that can be reused. Next, there is a handler on successful request that checks if I receive any cookie. It adds received cookies to list. Finally, there is a request method that calls HttpBuilder#request but it adds cookies to HTTP headers so server can recognize me as a logged in user.

Sending cookies with every request is a core component in here. It simulates browser's behavior and maintains session.

How to use it?

I will show you how to use this utility class it in Spock test below. It is fairly simple.

First I login to my application and I ensure that I receive a cookie in return, which is equivalent to being logged in. Then I send a request with that cookie sent in HTTP header. This is a Spock test that implements it:

package eu.spoonman.specs.rest

import eu.spoonman.connectors.RestConnector.RestConnector
import groovyx.net.http.ContentType
import groovyx.net.http.Method
import spock.lang.Shared
import spock.lang.Specification
import spock.lang.Stepwise

@Stepwise
class RestChartSpec extends Specification {
    @Shared
    RestConnector restConnector

    def setupSpec() {
        restConnector = new RestConnector('http://localhost:8080')
    }

    def "should login as test"() {
        given:
            Map params = [j_username: 'test', j_password: 'test']
        when:
            restConnector.request(Method.POST, ContentType.ANY, '/frontoffice/j_spring_security_check', params)
        then:
            !(restConnector.cookies.empty)
    }

    def "should allow access to chart data series"() {
        given:
            Map params = [days: 14]
        when:
            Map result = restConnector.request(Method.POST, ContentType.JSON, "frontoffice/chart/series", params)
        then:
            result != null
            result.series.size() > 0
    }
}

I create a new RestConnector instance in setupSpec with my application's base URL. Please notice that it has @Shared annotation so it's shared between tests.

@Stepwise is crucial annotation for this specification. It means that Spock executes tests exactly in order they're defined. I need to ensure that login is executed first. I also need to assert that I receive a cookie and list is not empty. I could move this step into setupSpec method too, but I prefer it to be a first test in a specification.

Second test is always executed after login thus it sends cookies within request headers. This is exactly what I wanted to achieve.

Grails render as JSON catch

One of a reasons your controller doesn't render a proper response in JSON format might be wrong package name that you use. It is easy to overlook. Import are on top of a file, you look at your code and everything seems to be fine. Except response is still not in JSON format.

Consider this simple controller:

class RestJsonCatchController {
    def grailsJson() {
        render([first: 'foo', second: 5] as grails.converters.JSON)
    }

    def netSfJson() {
        render([first: 'foo', second: 5] as net.sf.json.JSON)
    }
}

And now, with finger crossed... We have a winner!

$ curl localhost:8080/example/restJsonCatch/grailsJson
{"first":"foo","second":5}
$ curl localhost:8080/example/restJsonCatch/netSfJson
{first=foo, second=5}

As you can see only grails.converters.JSON converts your response to JSON format. There is no such converter for net.sf.json.JSON, so Grails has no converter to apply and it renders Map normally.

Conclusion: always carefully look at your imports if you're working with JSON in Grails!

Edit: Burt suggested that this is a bug. I've submitted JIRA issue here: GRAILS-9622 render as class that is not a codec should throw exception