Multi phased processing in scala

Last time in our project we had to add progress bar for visualization of long time running process. Process was made of a few phases and we had to print in which phase we currently are. In first step we conclude that we need to create a class of Progress which will be passed as an implicit parameter to our service. Then we will wrap method calls be inProgress method which will notify some e.g. akka actor about phase begin and phase end.



But this approach has some disadvantages. Firstly before we start service's operation we need to init progress with count of all phases to get know ratio of progress finish. With this approach we had to add some extra counting before operation start.





If we want to keep real progress notifications the numbers of phases had to fit count of inPhase blocks. Some of phases were dynamically computed and some where omitted in case of failure validations results. This code become to be unmaintained.

We found that we need to join computation of phases with real phase processing. In this case we need to change approach from building process to building chain of phases that will run the process. Each phase will take the result of previous phase and transform it to new output. So example process will look like this:



Code giving this chain functionality looks like this:



We've used right associative operator :: for building chain of phases. "Body" of phases is piped by andThen: processPrevWrapped andThen processNext. For nil-tail we need to have a factory creating empty chain with identity "body" function.

Also if we have this kind of tool, we can modify piping code according to nature of our flow. For example if we are using scalaz.Validation we can do validating chain which will extract a success from n-step output and pass it to input of next step (like flatMap). In the other hand if n-step will return Failure, we will skip all remaining phases of validating chain.



To make building of chain more production-ready we add some extra features:
  • Chaining of chains (sth like ::: in scala Lists)
  • Transforming of input/output - for adding some "glue" code for simpler phases chaining
  • Wrapping of chains - also some "glue" code doing both input and output transformations
  • Sequencing of chains - sequenced processing of multiple phases with the same input

If you are interested in using similar approach, take a look at my github project: scala-phases-chain. If you want to integrate this tool with akka actors, simply change MultiPhasedProgress.notifyAboutStatus method to look like this:

Sonar Gerrit Plugin Release

Initial release

I am happy to announce a first release of my Sonar Gerrit plugin. This plugin reports Sonar violations on your patchsets to your Gerrit server. Sonar analyses full project, but only files included in patchset are commented on Gerrit. Please forward to project page for installation instructions.

This plugin is intended to use with Gerrit Trigger plugin for Jenkins CI server. Together they provide a great tool for automatic static code analysis.

How does it work?

At the moment you push a patchset to Gerrit, Jenkins is notified with a ssh event. It fetches a code with a patchset and it builds your changes. It quits when build or tests fail.

But if it succeeds, Sonar analase your project in a post-build action. This is a place where my Sonar Gerrit plugin shines. It asks Gerrit for changed files before analysis and after Sonar analysis is finished, plugin reports comments on these files as a Gerrit reviewer. Currently plugin always reports +1 for Code Review, as it's still in development. However, you should always treat these comments as hints to improve, not as direct errors.

Extras

I've released also a second plugin: Sonar File Alerts plugin. This plugin raises alerts on file level in Sonar. It extends default behaviour, which raises alerts only at root project level. It is useful when you create alert rules in Sonar like "Code Coverage < 60". Each file is checked against this rule!

If you use Sonar File Alerts plugin and an alert will be generated on some file, then a comment will be published on this file on Gerrit.

Feedback

Please provide a feedback on these plugins. Feel free to submit issues on github or comment. It's still an early stage so your input is very welcome!

Sonar Gerrit Plugin Release

Initial release

I am happy to announce a first release of my Sonar Gerrit plugin. This plugin reports Sonar violations on your patchsets to your Gerrit server. Sonar analyses full project, but only files included in patchset are commented on Gerrit. Please forward to project page for installation instructions.

This plugin is intended to use with Gerrit Trigger plugin for Jenkins CI server. Together they provide a great tool for automatic static code analysis.

How does it work?

At the moment you push a patchset to Gerrit, Jenkins is notified with a ssh event. It fetches a code with a patchset and it builds your changes. It quits when build or tests fail.

But if it succeeds, Sonar analase your project in a post-build action. This is a place where my Sonar Gerrit plugin shines. It asks Gerrit for changed files before analysis and after Sonar analysis is finished, plugin reports comments on these files as a Gerrit reviewer. Currently plugin always reports +1 for Code Review, as it's still in development. However, you should always treat these comments as hints to improve, not as direct errors.

Extras

I've released also a second plugin: Sonar File Alerts plugin. This plugin raises alerts on file level in Sonar. It extends default behaviour, which raises alerts only at root project level. It is useful when you create alert rules in Sonar like "Code Coverage < 60". Each file is checked against this rule!

If you use Sonar File Alerts plugin and an alert will be generated on some file, then a comment will be published on this file on Gerrit.

Feedback

Please provide a feedback on these plugins. Feel free to submit issues on github or comment. It's still an early stage so your input is very welcome!