DDevelopment & Design

TouK Hackathon – April 2021

The last time we wanted to organize a Hackathon our plans were thwarted by “you know what”. This state of affairs has lasted so long that we just couldn’t stand it anymore and launched the next edition of The Hackathon – remotely. This time we had to cope with new conditions – not all together in an open space, but everyone at home. We launched the communicator with a separate room per each project and went into action.
Here we present a brief summary from each team.

RCB Alert

tcb sms screenshot Every now and then we have a problem in our company – we need to inform everyone that the next day some loud redecorating is to take place or our air conditioning is to be cleaned – and we had better work remotely that day.
Unfortunately, sometimes our office managers gain this knowledge after our working hours, so there is no channel to notify everyone (hopefully – no one reads emails at home).

We found a solution to this problem during our last Hackathon.
We wrote a piece of software that orders an SMS to be sent to each person subscribed to TCB Alerts (it’s a pun on our government alerts – RCB).
This was the easy part – as we already had an SMS sending service.
However, we wanted this solution to be as easy as possible, so our office managers don’t have to open their laptops, connect to a VPN, search for special forms etc.
We decided to implement it as a hook to our communicator – RocketChat.
Now, when our office managers need to inform everyone in the evening, they need only to open the RocketChat app on their phone and type a message on a special channel – and that’s it!

We hope that this solution will help us to stay at home and work remotely during the days when it is inconvenient to work from the office.

TouK AboutMe

about me screenshot Currently at TouK we have several selfcare services providing information about TouKs (people working or cooperating with TouK).
The aim of our project was to join these services into one. Of course we know the rule that if you have three separate services then the worst approach is to add a new one. For this reason we enriched the most modern of them with new features such as TouK’s search, adding new information about people, teams information management and much more.
During the hackathon our team of four (+ the business owner) learnt a lot about frontend technologies such as CORS, MongoDB and LDAP.
We all hope that our work will prove to be useful for both current and future TouKs. Our team strove to provide a user experience so seamless that users would wonder at how easy selfcare management could be. We trust that with this hackathon we are a step closer to achieving that goal.

Business Config Manager

During our deployments of Nussknacker we often have the situation that the flow of development of scenarios was done by two teams. One of them is a team called “Configurators” – people who are close to business requirements but also with quite a high level of technical skills. Those people are responsible for the development of scenarios on the Nussknacker side. On the other hand, the second team is made up of Business members, those with lower technical skills but with a good knowledge of customer needs.
Configurators want to outsource some steps of development to business – so some changes can be made faster, without involving Configurators in the process. We found out that we should make a tool that can give Configurators the ability to create definitions of some configurations and after that, Business can fill in the values of those configurations. In the end, this configuration will be used in some steps of the scenario in Nussknacker.

After the deployment of the proof of concept, we realized that it is necessary to handle some important things:

  • Both definitions of configurations and values should be versioned and have some audit information, such as the author and time of the change.
  • Migrations of changes in definitions should be painless.
  • We should support many types of properties: from raw strings to some date time pickers and so on.
  • Configurations should have a lifecycle, so new ideas can be deployed on a lower environment and after some tests can be promoted to higher env.

After brainstorming before the Hackathon, we decided to use modern stack, but with some solid, battle-tested components:

We also designed the domain level of application.

On the first day of the Hackathon, we started with pair programming. We tried to go through all layers of the application to make sure that everyone in the team has a common vision of what the architecture will look like. After that we split the work into four separate parts:

  • Management of definitions
  • Management of values
  • Nussknacker integration with Business Config Service
  • Web application Finally, we integrated all the features together and completed the MVP step.

The project ended successfully. We proved that our design was correct, prepared a solid foundation for future development, and had a lot of fun designing the architecture and testing new tools.

Nussknacker Serverless

Most of you probably know Nussknacker – a powerful platform which allows non-technical users to author and deploy streaming scenarios on Apache Flink.
But Nussknacker Designer can also describe more business rules-oriented scenarios – used e.g. in recommendations or NBA domains. In this case, the scenario is deployed as a REST microservice. During the Hackathon we decided to make this setup more Kubernetes/Serverless-oriented. We decided that a Nussknacker scenario is a good candidate for K8 CRD and that KNative will provide us with a serverless deployment platform.
In two days we reached most of our goals:

  • Nussknacker creating scenario ConfigMap during deployment (in the future it will be CRD)
  • Custom Kubernetes operator/controller which transforms scenario ConfigMap into KNative service (scaling down to zero if needed :))
  • Simple REST microservice image, which serves the scenario
  • Everything deployed via Helm/GitlabCI to our DigitalOcean K8 cluster
    We are also pretty excited about the next steps – scenario observability (metrics, statuses), CRDs and making our serving image serverless ready – by using GraalVM native images.
    Hopefully, in the short to medium term, all of this will be accessible with our Nussknacker offering.

MusicBox

music box screenshot We have created a minimalistic web app for generating music in a loop based on text input.
The idea is to make writing music as easy as possible. E.g. |k h s h | is the most basic percussive beat, while |Am|C| gives a basic chord progression. Furthermore, there’s a collaborative mode (think jam sessions ;).

Frontend:

Backend:

Summary

In our opinion, The Hackathon was successful and fulfilled its task – that is, it allowed us to take a short break from our more important work and experiment with various fun technologies in good company. The fact that during the Hackathon team members were in a voice chat with each other certainly played a big role, which to some extent allowed us to build an atmosphere of cooperation in the fight against challenges.
We are already looking forward to the next hackathon – hopefully, this time on site.

If you want, here you can see what we did during the previous edition.

— Previous article

How to find all objects that use dblinks

Next article —

Real-life Ansible - chapter one - how to run playbooks on production (more) safely

You May Also Like
DDevelopment & Design

How to automate tests with Groovy 2.0, Spock and Gradle

This is the launch of the 1st blog in my life, so cheers and have a nice reading!

y u no test?

Couple of years ago I wasn't a big fan of unit testing. It was obvious to me that well prepared unit tests are crucial though. I didn't known why exactly crucial yet then. I just felt they are important. My disliking to write automation tests was mostly related to the effort necessary to prepare them. Also a spaghetti code was easily spotted in test sources.

Some goodies at hand

Now I know! Test are crucial to get a better design and a confidence. Confidence to improve without a hesitation. Moreover, now I have the tool to make test automation easy as Sunday morning... I'm talking about the Spock Framework. If you got here probably already know what the Spock is, so I won't introduce it. Enough to say that Spock is an awesome unit testing tool which, thanks to Groovy AST Transformation, simplifies creation of tests greatly.

An obstacle

The point is, since a new major version of Groovy has been released (2.0), there is no matching version of Spock available yet.

What now?

Well, in a matter of fact there is such a version. It's still under development though. It can be obtained from this Maven repository. We can of course use the Maven to build a project and run tests. But why not to go even more "groovy" way? XML is not for humans, is it? Lets use Gradle.

The build file

Update: at the end of the post is updated version of the build file.
apply plugin: 'groovy'

apply plugin: 'idea'


def langLevel = 1.7


sourceCompatibility = langLevel

targetCompatibility = langLevel


group = 'com.tamashumi.example.testwithspock'

version = '0.1'


repositories {

    mavenLocal()

    mavenCentral()

    maven { url 'http://oss.sonatype.org/content/repositories/snapshots/' }

}


dependencies {

    groovy 'org.codehaus.groovy:groovy-all:2.0.1'

    testCompile 'org.spockframework:spock-core:0.7-groovy-2.0-SNAPSHOT'

}


idea {

    project {

        jdkName = langLevel

        languageLevel = langLevel

    }

}
As you can see the build.gradle file is almost self-explanatory. Groovy plugin is applied to compile groovy code. It needs groovy-all.jar - declared in version 2.0 at dependencies block just next to Spock in version 0.7. What's most important, mentioned Maven repository URL is added at repositories block.

Project structure and execution

Gradle's default project directory structure is similar to Maven's one. Unfortunately there is no 'create project' task and you have to create it by hand. It's not a big obstacle though. The structure you will create will more or less look as follows: 
<project root>

│

├── build.gradle

└── src

    ├── main

    │   ├── groovy

    └── test

        └── groovy
To build a project now you can type command gradle build or gradle test to only run tests.

How about Java?

You can test native Java code with Spock. Just add src/main/java directory and a following line to the build.gradle: 
apply plugin: 'java'
This way if you don't want or just can't deploy Groovy compiled stuff into your production JVM for any reason, still whole goodness of testing with Spock and Groovy is at your hand.

A silly-simple example

Just to show that it works, here you go with a basic example.

Java simple example class:

public class SimpleJavaClass {


    public int sumAll(int... args) {


        int sum = 0;


        for (int arg : args){

            sum += arg;

        }


        return sum;

    }

}

Groovy simple example class:

class SimpleGroovyClass {


    String concatenateAll(char separator, String... args) {


        args.join(separator as String)

    }

}

The test, uhm... I mean the Specification:

class JustASpecification extends Specification {


    @Unroll('Sums integers #integers into: #expectedResult')

    def "Can sum different amount of integers"() {


        given:

            def instance = new SimpleJavaClass()


        when:

            def result = instance.sumAll(* integers)


        then:

            result == expectedResult


        where:

            expectedResult | integers

            11             | [3, 3, 5]

            8              | [3, 5]

            254            | [2, 4, 8, 16, 32, 64, 128]

            22             | [7, 5, 6, 2, 2]

    }


    @Unroll('Concatenates strings #strings with separator "#separator" into: #expectedResult')

    def "Can concatenate different amount of integers with a specified separator"() {


        given:

            def instance = new SimpleGroovyClass()


        when:

            def result = instance.concatenateAll(separator, * strings)


        then:

            result == expectedResult


        where:

            expectedResult     | separator   | strings

            'Whasup dude?'     | ' ' as char | ['Whasup', 'dude?']

            '2012/09/15'       | '/' as char | ['2012', '09', '15']

            'nice-to-meet-you' | '-' as char | ['nice', 'to', 'meet', 'you']

    }

}
To run tests with Gradle simply execute command gradle test. Test reports can be found at <project root>/build/reports/tests/index.html and look kind a like this.


Please note that, thanks to @Unroll annotation, test is executed once per each parameters row in the 'table' at specification's where: block. This isn't a Java label, but a AST transformation magic.

IDE integration

Gradle's plugin for Iintellij Idea

I've added also Intellij Idea plugin for IDE project generation and some configuration for it (IDE's JDK name). To generate Idea's project files just run command: gradle idea There are available Eclipse and Netbeans plugins too, however I haven't tested them. Idea's one works well.

Intellij Idea's plugins for Gradle

Idea itself has a light Gradle support built-in on its own. To not get confused: Gradle has plugin for Idea and Idea has plugin for Gradle. To get even more 'pluginated', there is also JetGradle plugin within Idea. However I haven't found good reason for it's existence - well, maybe excluding one. It shows dependency tree. There is a bug though - JetGradle work's fine only for lang level 1.6. Strangely all the plugins together do not conflict each other. They even give complementary, quite useful tool set.

Running tests under IDE

Jest to add something sweet this is how Specification looks when run with jUnit  runner under Intellij Idea (right mouse button on JustASpecification class or whole folder of specification extending classes and select "Run ...". You'll see a nice view like this.

Building web application

If you need to build Java web application and bundle it as war archive just add plugin by typing the line 
apply plugin: 'war'
in the build.gradle file and create a directory src/main/webapp.

Want to know more?

If you haven't heard about Spock or Gradle before or just curious, check the following links:

What next?

The last thing left is to write the real production code you are about to test. No matter will it be Groovy or Java, I leave this to your need and invention. Of course, you are welcome to post a comments here. I'll answer or even write some more posts about the subject.

Important update

Spock version 0.7 has been released, so the above build file doesn't work anymore. It's easy to fix it though. Just remove last dash and a word SNAPSHOT from Spock dependency declaration. Other important thing is that now spock-core depends on groovy-all-2.0.5, so to avoid dependency conflict groovy dependency should be changed from version 2.0.1 to 2.0.5.
Besides oss.sonata.org snapshots maven repository can be removed. No obstacles any more and the build file now looks as follows: 
apply plugin: 'groovy'

apply plugin: 'idea'


def langLevel = 1.7


sourceCompatibility = langLevel

targetCompatibility = langLevel


group = 'com.tamashumi.example.testwithspock'

version = '0.1'


repositories {

    mavenLocal()

    mavenCentral()

}


dependencies {

    groovy 'org.codehaus.groovy:groovy-all:2.0.5'

    testCompile 'org.spockframework:spock-core:0.7-groovy-2.0'

}


idea {

    project {

        jdkName = langLevel

        languageLevel = langLevel

    }

}
DDevelopment & Design

Spock, Java and Maven

Few months ago I've came across Groovy - powerful language for JVM platform which combines the power of Java with abilities typical for scripting languages (dynamic typing, metaprogramming).

Together with Groovy I've discovered spock framework (https://code.google.com/p/spock/) - specification framework for Groovy (of course you can test Java classes too!). But spock is not only test/specification framework - it also contains powerful mocking tools.

Even though spock is dedicated for Groovy there is no problem with using it for Java classes tests. In this post I'm going to describe how to configure Maven project to build and run spock specifications together with traditional JUnit tests.


Firstly, we need to prepare pom.xml and add necessary dependencies and plugins.

Two obligatory libraries are:
<dependency>
    <groupid>org.spockframework</groupId>
    <artifactid>spock-core</artifactId>
    <version>0.7-groovy-2.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupid>org.codehaus.groovy</groupId>
    <artifactid>groovy-all</artifactId>
    <version>${groovy.version}</version>
    <scope>test</scope>
</dependency>
Where groovy.version is property defined in pom.xml for more convenient use and easy version change, just like this: 
<properties>
    <gmaven-plugin.version>1.4</gmaven-plugin.version>
    <groovy.version>2.1.5</groovy.version>
</properties>

I've added property for gmaven-plugin version for the same reason ;)

Besides these two dependencies, we can use few additional ones providing extra functionality:
  • cglib - for class mocking
  • objenesis - enables mocking classes without default constructor
To add them to the project put these lines in <dependencies> section of pom.xml:
<dependency>
    <groupid>cglib</groupId>
    <artifactid>cglib-nodep</artifactId>
    <version>3.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupid>org.objenesis</groupId>
    <artifactid>objenesis</artifactId>
    <version>1.3</version>
    <scope>test</scope>
</dependency>

And that's all for dependencies section. Now we will focus on plugins necessary to compile Groovy classes. We need to add gmaven-plugin with gmaven-runtime-2.0 dependency in plugins section:
<plugin>
    <groupid>org.codehaus.gmaven</groupId>
    <artifactid>gmaven-plugin</artifactId>
    <version>${gmaven-plugin.version}</version>
    <configuration>
        <providerselection>2.0</providerSelection>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
                <goal>testCompile</goal>
            </goals>
        </execution>
    </executions>
    <dependencies>
        <dependency>
            <groupid>org.codehaus.gmaven.runtime</groupId>
            <artifactid>gmaven-runtime-2.0</artifactId>
            <version>${gmaven-plugin.version}</version>
            <exclusions>
                <exclusion>
                    <groupid>org.codehaus.groovy</groupId>
                    <artifactid>groovy-all</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupid>org.codehaus.groovy</groupId>
            <artifactid>groovy-all</artifactId>
            <version>${groovy.version}</version>
        </dependency>
    </dependencies>
</plugin>

With these configuration we can use spock and write our first specifications. But there is one issue: default settings for maven-surefire plugin demand that test classes must end with "..Test" postfix, which is ok when we want to use such naming scheme for our spock tests. But if we want to name them like CommentSpec.groovy or whatever with "..Spec" ending (what in my opinion is much more readable) we need to make little change in surefire plugin configuration:
<plugin>
    <groupid>org.apache.maven.plugins</groupId>
    <artifactid>maven-surefire-plugin</artifactId>
    <version>2.15</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
            <include>**/*Spec.java</include>
        </includes>
    </configuration>
</plugin>

As you can see there is a little trick ;) We add include directive for standard Java JUnit test ending with "..Test" postfix, but there is also an entry for spock test ending with "..Spec". And there is a trick: we must write "**/*Spec.java", not "**/*Spec.groovy", otherwise Maven will not run spock tests (which is strange and I've spent some time to figure out why Maven can't run my specs).

Little update: instead of "*.java" postfix for both types of tests we can write "*.class" what is in my opinion more readable and clean:
<include>**/*Test.class</include>
<include>**/*Spec.class</include>
(thanks to Tomek Pęksa for pointing this out!)

With such configuration, we can write either traditional JUnit test and put them in src/test/java directory or groovy spock specifications and place them in src/test/groovy. And both will work together just fine :) In one of my next posts I'll write something about using spock and its mocking abilities in practice, so stay in tune.