{"id":13142,"date":"2017-09-04T08:50:00","date_gmt":"2017-09-04T06:50:00","guid":{"rendered":"https:\/\/touk.pl\/blog\/?guid=23988ccd6e3abac5260ab4b8d22a505f"},"modified":"2023-04-27T11:29:51","modified_gmt":"2023-04-27T09:29:51","slug":"touk-nussknacker-using-apache-flink-made-easier-for-analysts-and-business","status":"publish","type":"post","link":"https:\/\/touk.pl\/blog\/2017\/09\/04\/touk-nussknacker-using-apache-flink-made-easier-for-analysts-and-business\/","title":{"rendered":"TouK Nussknacker – using Apache Flink made easier for analysts and business"},"content":{"rendered":"

Few weeks ago we (TouK) revealed on Github our latest open source project \u2013 Nussknacker. What is it and why should you care?<\/p>\n

Why?<\/h3>\n

First, some history: more than year ago one of our clients decided it\u2019s high time for Real Time Marketing. <\/span>They had pretty large data streams and lots of ideas for marketing campaigns, so one of the key success factors was the ability to process the data fast. We prepared a POC based on Apache Flink and it turned out that it\u2019s really great piece of technology \u2013 fast and accurate.<\/p>\n

There was just one problem \u2013 how to write and customize processes? Flink, as many modern stream processing engines, provides rich and friendly DSL, both in Java and Scala. But for our client that was not really enough. See, they are enterprise that do not employ developers \u2013 they have many decent, competent analysts who know the business and SQL \u2013 but not Java or (Heaven forbid\u2026) Scala.<\/p>\n

Nussknacker to the rescue!<\/h3>\n

So we decided to create a simple process designer for them. It looks more or less like this:
\n<\/p>\n

You draw a diagram, fill in the details (like filter expressions \u2013 \u201c#input.usageData > 0<\/code>\u201d or SMS\/mail content), then you press the Deploy button and voil\u00e0 \u2013 your brand new process in running on Flink cluster!<\/p>\n

Of course first somebody (that is, developer) has to prepare data sources (most probably Kafka topics), design data model (POJOs or case classes) and implement actions \u2013 like sending emails or sending some events to another Kafka topic. But once a model of data and external services are defined, an analyst can define and deploy processes all by him\/herself.<\/p>\n

More features<\/h3>\n

Sounds a bit scary to let your users run stream processes with GUI? Bad filter conditions can have serious performance implications if you deal with streams of tens of thousands of events per second\u2026 That\u2019s why we let user test their diagrams first \u2013 each test case can be generated by sample of real data coming from e.g. Kafka and then run in Flink sandbox mini-cluster.<\/p>\n

We have also many more features that make working with Nussknacker and Flink easier: subprocesses, versioning, generating PDF documentation, migration between environments and last but certainly not least \u2013 integration with InfluxDB\/Grafana to provide detailed insight into how process is doing:<\/p>\n

<\/p>\n

Where can I use it?<\/h3>\n

What are Nussknacker use cases? Our main deployments deal with RTM (Real Time Marketing). One of our clients started with RTM and then found out that Nussknacker is also great choice for fraud detection in real time. Industries we are working with include telcos, banks and media companies. We are also thinking about other possibilities \u2013 for example IoT.<\/p>\n

Sounds cool?<\/h3>\n

If you are interested in easy access for semi-technical analysts to streaming data \u2013 give Nussknacker a try!<\/p>\n

You can find the code at Github: https:\/\/github.com\/touk\/nussknacker<\/a>, we also have a nice, Docker-based quickstart: https:\/\/touk.github.io\/nussknacker\/Quickstart.html<\/a>.<\/p>\n

And if you are coming to <\/span>Flink Forward<\/a> next week in Berlin \u2013 join me on Wednesday afternoon to hear more \u2013 <\/span>.<\/p>\n

In next days\/weeks we\u2019ll post more information <\/span>on TouK blog<\/a> <\/span>both on Nussknacker architecture and internals and on interesting use cases \u2013 stay tuned :)<\/p>\n","protected":false},"excerpt":{"rendered":"Few weeks ago we (TouK) revealed on Github our latest open source project \u2013 Nussknacker. What is it…\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[252,621,622],"class_list":{"0":"post-13142","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-development-design","7":"tag-big-data","8":"tag-kafka","9":"tag-nussknacker"},"_links":{"self":[{"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/posts\/13142","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/comments?post=13142"}],"version-history":[{"count":7,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/posts\/13142\/revisions"}],"predecessor-version":[{"id":15666,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/posts\/13142\/revisions\/15666"}],"wp:attachment":[{"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/media?parent=13142"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/categories?post=13142"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/touk.pl\/blog\/wp-json\/wp\/v2\/tags?post=13142"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}