DDevelopment & Design

TouK Hackathon – April 2021

The last time we wanted to organize a Hackathon our plans were thwarted by “you know what”. This state of affairs has lasted so long that we just couldn’t stand it anymore and launched the next edition of The Hackathon – remotely. This time we had to cope with new conditions – not all together in an open space, but everyone at home. We launched the communicator with a separate room per each project and went into action.
Here we present a brief summary from each team.

RCB Alert

tcb sms screenshot Every now and then we have a problem in our company – we need to inform everyone that the next day some loud redecorating is to take place or our air conditioning is to be cleaned – and we had better work remotely that day.
Unfortunately, sometimes our office managers gain this knowledge after our working hours, so there is no channel to notify everyone (hopefully – no one reads emails at home).

We found a solution to this problem during our last Hackathon.
We wrote a piece of software that orders an SMS to be sent to each person subscribed to TCB Alerts (it’s a pun on our government alerts – RCB).
This was the easy part – as we already had an SMS sending service.
However, we wanted this solution to be as easy as possible, so our office managers don’t have to open their laptops, connect to a VPN, search for special forms etc.
We decided to implement it as a hook to our communicator – RocketChat.
Now, when our office managers need to inform everyone in the evening, they need only to open the RocketChat app on their phone and type a message on a special channel – and that’s it!

We hope that this solution will help us to stay at home and work remotely during the days when it is inconvenient to work from the office.

TouK AboutMe

about me screenshot Currently at TouK we have several selfcare services providing information about TouKs (people working or cooperating with TouK).
The aim of our project was to join these services into one. Of course we know the rule that if you have three separate services then the worst approach is to add a new one. For this reason we enriched the most modern of them with new features such as TouK’s search, adding new information about people, teams information management and much more.
During the hackathon our team of four (+ the business owner) learnt a lot about frontend technologies such as CORS, MongoDB and LDAP.
We all hope that our work will prove to be useful for both current and future TouKs. Our team strove to provide a user experience so seamless that users would wonder at how easy selfcare management could be. We trust that with this hackathon we are a step closer to achieving that goal.

Business Config Manager

During our deployments of Nussknacker we often have the situation that the flow of development of scenarios was done by two teams. One of them is a team called “Configurators” – people who are close to business requirements but also with quite a high level of technical skills. Those people are responsible for the development of scenarios on the Nussknacker side. On the other hand, the second team is made up of Business members, those with lower technical skills but with a good knowledge of customer needs.
Configurators want to outsource some steps of development to business – so some changes can be made faster, without involving Configurators in the process. We found out that we should make a tool that can give Configurators the ability to create definitions of some configurations and after that, Business can fill in the values of those configurations. In the end, this configuration will be used in some steps of the scenario in Nussknacker.

After the deployment of the proof of concept, we realized that it is necessary to handle some important things:

  • Both definitions of configurations and values should be versioned and have some audit information, such as the author and time of the change.
  • Migrations of changes in definitions should be painless.
  • We should support many types of properties: from raw strings to some date time pickers and so on.
  • Configurations should have a lifecycle, so new ideas can be deployed on a lower environment and after some tests can be promoted to higher env.

After brainstorming before the Hackathon, we decided to use modern stack, but with some solid, battle-tested components:

We also designed the domain level of application.

On the first day of the Hackathon, we started with pair programming. We tried to go through all layers of the application to make sure that everyone in the team has a common vision of what the architecture will look like. After that we split the work into four separate parts:

  • Management of definitions
  • Management of values
  • Nussknacker integration with Business Config Service
  • Web application Finally, we integrated all the features together and completed the MVP step.

The project ended successfully. We proved that our design was correct, prepared a solid foundation for future development, and had a lot of fun designing the architecture and testing new tools.

Nussknacker Serverless

Most of you probably know Nussknacker – a powerful platform which allows non-technical users to author and deploy streaming scenarios on Apache Flink.
But Nussknacker Designer can also describe more business rules-oriented scenarios – used e.g. in recommendations or NBA domains. In this case, the scenario is deployed as a REST microservice. During the Hackathon we decided to make this setup more Kubernetes/Serverless-oriented. We decided that a Nussknacker scenario is a good candidate for K8 CRD and that KNative will provide us with a serverless deployment platform.
In two days we reached most of our goals:

  • Nussknacker creating scenario ConfigMap during deployment (in the future it will be CRD)
  • Custom Kubernetes operator/controller which transforms scenario ConfigMap into KNative service (scaling down to zero if needed :))
  • Simple REST microservice image, which serves the scenario
  • Everything deployed via Helm/GitlabCI to our DigitalOcean K8 cluster
    We are also pretty excited about the next steps – scenario observability (metrics, statuses), CRDs and making our serving image serverless ready – by using GraalVM native images.
    Hopefully, in the short to medium term, all of this will be accessible with our Nussknacker offering.

MusicBox

music box screenshot We have created a minimalistic web app for generating music in a loop based on text input.
The idea is to make writing music as easy as possible. E.g. |k h s h | is the most basic percussive beat, while |Am|C| gives a basic chord progression. Furthermore, there’s a collaborative mode (think jam sessions ;).

Frontend:

Backend:

Summary

In our opinion, The Hackathon was successful and fulfilled its task – that is, it allowed us to take a short break from our more important work and experiment with various fun technologies in good company. The fact that during the Hackathon team members were in a voice chat with each other certainly played a big role, which to some extent allowed us to build an atmosphere of cooperation in the fight against challenges.
We are already looking forward to the next hackathon – hopefully, this time on site.

If you want, here you can see what we did during the previous edition.

— Previous article

How to find all objects that use dblinks

Next article —

Real-life Ansible - chapter one - how to run playbooks on production (more) safely

You May Also Like
DDevelopment & Design

Custom SonarQube rules for Unit Tests

It's a tutorial about creating new rules for SonarQube analysis to be applied to Unit Tests. It is not trivial and involves a few tricky database steps, so I want to share my tutorial about it.It's a tutorial about creating new rules for SonarQube analysis to be applied to Unit Tests. It is not trivial and involves a few tricky database steps, so I want to share my tutorial about it.
DDevelopment & Design

Clojure web development – state of the art

It’s now more than a year that I’m getting familiar with Clojure and the more I dive into it, the more it becomes the language. Once you defeat the “parentheses fear”, everything else just makes the difference: tooling, community, good engineering practices. So it’s now time for me to convince others. In this post I’ll try to walktrough a simple web application from scratch to show key tools and libraries used to develop with Clojure in late 2015.

Note for Clojurians: This material is rather elementary and may be useful for you if you already know Clojure a bit but never did anything bigger than hello world application.

Note for Java developers: This material shows how to replace Spring, Angular, grunt, live-reload with a bunch of Clojure tools and libraries and a bit of code.

The repo with final code and individual steps is here.

Bootstrap

I think all agreed that component is the industry standard for managing lifecycle of Clojure applications. If you are a Java developer you may think of it as a Spring (DI) replacement - you declare dependencies between “components” which are resolved on “system” startup. So you just say “my component needs a repository/database pool” and component library “injects” it for you.

To keep things simple I like to start with duct web app template. It’s a nice starter component application following the 12-factor philosophy. So let’s start with it:

lein new duct clojure-web-app +example

The +example parameter tells duct to create an example endpoint with HTTP routes - this would be helpful. To finish bootstraping run lein setup inside clojure-web-app directory.

Ok, let’s dive into the code. Component and injection related code should be in system.clj file:

(defn new-system [config]
  (let [config (meta-merge base-config config)]
    (-> (component/system-map
         :app  (handler-component (:app config))
         :http (jetty-server (:http config))
         :example (endpoint-component example-endpoint))
        (component/system-using
         {:http [:app]
          :app  [:example]
          :example []}))))

In the first section you instantiate components without dependencies, which are resolved in the second section. So in this example, “http” component (server) requires “app” (application abstraction), which in turn is injected with “example” (actual routes). If your component needs others, you just can get then by names (precisely: by Clojure keywords).

To start the system you must fire a REPL - interactive environment running within context of your application:

lein repl

After seeing prompt type (go). Application should start, you can visit http://localhost:3000 to see some example page.

A huge benefit of using component approach is that you get fully reloadable application. When you change literally anything - configuration, endpoints, implementation, you can just type (reset) in REPL and your application is up-to-date with the code. It’s a feature of the language, no JRebel, Spring-reloaded needed.

Adding REST endpoint

Ok, in the next step let’s add some basic REST endpoint returning JSON. We need to add 2 dependencies in project.clj file:

:dependencies
 ...
  [ring/ring-json "0.3.1"]
  [cheshire "5.1.1"]

Ring-json adds support for JSON for your routes (in ring it’s called middleware) and cheshire is Clojure JSON parser (like Jackson in Java). Modifying project dependencies if one of the few tasks that require restarting the REPL, so hit CTRL-C and type lein repl again.

To configure JSON middleware we have to add wrap-json-body and wrap-json-response just before wrap-defaults in system.clj:

(:require 
 ...
 [ring.middleware.json :refer [wrap-json-body wrap-json-response]])

(def base-config
   {:app {:middleware [[wrap-not-found :not-found]
                      [wrap-json-body {:keywords? true}]
                      [wrap-json-response]
                      [wrap-defaults :defaults]]

And finally, in endpoint/example.clj we must add some route with JSON response:

(:require 
 ...
 [ring.util.response :refer [response]]))

(defn example-endpoint [config]
  (routes
    (GET "/hello" [] (response {:hello "world"}))
    ...

Reload app with (reset) in REPL and test new route with curl:

curl -v http://localhost:3000/hello

< HTTP/1.1 200 OK
< Date: Tue, 15 Sep 2015 21:17:37 GMT
< Content-Type: application/json; charset=utf-8
< Set-Cookie: ring-session=37c337fb-6bbc-4e65-a060-1997718d03e0;Path=/;HttpOnly
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Content-Length: 151
* Server Jetty(9.2.10.v20150310) is not blacklisted
< Server: Jetty(9.2.10.v20150310)
<
* Connection #0 to host localhost left intact
{"hello": "world"}

It works! In case of any problems you can find working version in this commit.

Adding frontend with figwheel

Coding backend in Clojure is great, but what about the frontend? As you may already know, Clojure could be compiled not only to JVM bytecode, but also to Javascript. This may sound familiar if you used e.g. Coffescript. But ClojureScript philosophy is not only to provide some syntax sugar, but improve your development cycle with great tooling and fully interactive development. Let’s see how to achieve it.

The best way to introduce ClojureScript to a project is figweel. First let’s add fighweel plugin and configuration to project.clj:

:plugins
   ...
   [lein-figwheel "0.3.9"]

And cljsbuild configuration:

:cljsbuild
    {:builds [{:id "dev"
               :source-paths ["src-cljs"]
               :figwheel true
               :compiler {:main       "clojure-web-app.core"
                          :asset-path "js/out"
                          :output-to  "resources/public/js/clojure-web-app.js"
                          :output-dir "resources/public/js/out"}}]}

In short this tells ClojureScript compiler to take sources from src-cljs with figweel support and but resulting JavaScript into resources/public/js/clojure-web-app.js file. So we need to include this file in a simple HTML page:

<!DOCTYPE html>
<head>
</head>
<body>
  <div id="main">
  </div>
  <script src="js/clojure-web-app.js" type="text/javascript"></script>
</body>
</html>

To serve this static file we need to change some defaults and add corresponding route. In system.clj change api-defaults to site-defaults both in require section and base-config function. In example.clj add following route:

(GET "/" [] (io/resource "public/index.html")

Again (reset) in REPL window should reload everything.

But where is our ClojureScript source file? Let’s create file core.cljs in src-cljs/clojure-web-app directory:

(ns ^:figwheel-always clojure-web-app.core)

(enable-console-print!)

(println "hello from clojurescript")

Open another terminal and run lein fighweel. It should compile ClojureScript and print ‘Prompt will show when figwheel connects to your application’. Open http://localhost:3000. Fighweel window should prompt:

To quit, type: :cljs/quit
cljs.user=>

Type (js/alert "hello"). Boom! If everything worked you should see and alert in your browser. Open developers console in your browser. You should see hello from clojurescript printed on the console. Change it in core.cljs to (println "fighweel rocks") and save the file. Without reloading the page your should see updated message. Figweel rocks! Again, in case of any problems, refer to this commit.

In the next post I’ll show how to fetch data from MongoDB, serve it with REST to the broser and write ReactJs/Om components to render it. Stay tuned!