An error occured [oracle/jdbc/Driver], see error log for details java.lang.NoClassDefFoundError: oracle/jdbc/DriverIf so, try registering your drivers with registerJdbcDriver function, like I did in this snippet of code: What a crappy thing!
- buffer the huge amount of those records - so as not to overhoul other services with large traffic, and not to flood the frontend
- convert then to my representation
- display them - have presentation layer in a browser - since browser-based frontend was the easiest and fastest to develop
Toys usedFor the first step I chose to use RabbitMQ broker. A queue on the broker would receive messages - one message per database table's row. Then I'd use some simple groovy middle ware to convert the data to appropriate format and put it onto another db - this time db specific to my app. You may ask why incorporate another database. It would be good for separating environments - assuming the original data contains some vulnerable content that should be anatomised, or we just don't feel comfortable exposing the whole database of some XYZ-system just to have access to its one table. Since for my presentation layer I chose HTML+JS without any application server-based back-end I've decided on CouchDB . This seemed like a perfect match for this scenario. Why? - ease of use, REST API, with JSON responses - just great for interacting with my simple front-end. The flow of things was as shown on the image below:
Avro - for the beginningAs you can see, I've chosen JSON as my data-format. I've been considering Apache Avro in the first place but using it was a real pain in the ass. Avro itself is used in Apache Hadoop as a serialization layer, so it would seem OK, but it has virtually no documentation. But once you tear through the unintuitive interface and manage to handle all those unthinkable exceptions you get a few pros for this library. It's great in that it does not require code generation - I like it being made on the fly. It also offers sending data in binary format, which was not necessary, but never the less is a nice feature. What I certainly didn't like about it was its orientation on the files rather than chunks of data - so it was not so obvious how should I send data through the wire. Than I found out it can produce JSON output, which would work for me, except the output could not have been parsed by other JSON libraries :) (I've asked on stackoverflow about that, but with no luck). If my whining haven't put you back and still would like to see how to use Avro, try this unit test in project's GitHub repo: AvroSimpleTest.groovy
SvensonI've dropped Avro in favour of a simple JSON lib called (Svenson and that was painless. The only thing I was forced to do was create my model class in Java - the rest of the project is written in Groovy. I've no idea why was that necessary, and didn't want to look into it.
RabbitMQFurther on the way is RabbitMQ, to which records are filled by a feeding middle-ware written in Groovy. Since I use ActiveMQ on a day-to-day basis, I've decided to try something new. This broker is a really nice piece of software. Being written in Erlang makes it really fast. What's more it has some extensive capabilities and is easy to approach for anyone similar with messaging (JMS and friends). For such a lightweight product it is really powerful - implements AMQP!
CouchDBFrom the broker's queue messages are again fetched by a middle-ware just to be put into CouchDB view. This database is also written in Erlang. It's very reliable, however the way it handles refreshing view isn't the most pleasant one - performance-wise. Word of advice - if you're on Debian derivative, be cautious with apt-repository version. It's rather _ancient_. Also remember to add allow_jsonp = true to you config file /opt/couchbase/etc/couchdb/local.ini. It's not enabled by default, and not having this set would result with empty responses from the CouchDB server. The problem here is, that the browser doesn't allow quering a web server with hostname other than the one the script originates. More on this case here. Seems like my problem could be overcame by changing url in index.html and hostname couchdb listens on to the same address. I've also created a view, that would expose an event by key: view code
Presenting the dotsAs a back-end I've done some JQuery based AJAX calls - nothing too fancy. All things necessary for presentation layer are in this file.
Things to considerPlease bear in mind that this whole application is rather a playground, not a full-fledged project!! After creating all the parts I have some doubts about some architectural decisions I made. I don't think the security have been taken into account seriously enough. Also scalability was never an issue ;-) If you have some thoughts about any of the aspects mentioned in this post, please feel free to comment or contact me directly :) And also you may try the application by yourself - it's on the GitHub.
- person's first_name and last_name should be of appropriate length - between 2 and 20, and additionally one could pass a zero-length string just to remove the previous value
- state field should consist only defined values - as in dictionary value - this one would be completable with XSD's enumerations, but would require often changing schema files and redistributing them to interested parties :(
- when file is set - read dictionary content from the file into map
- upon check request just lookup value in dictionary parsed from the input file
And now for part 2 of my visit to Geecon 2011!
1. Jim Webber "Revisiting SOA for the 21st century"
Now this was awesome! Jim Webber, a former ThoughtWorks employee, now Neo4j evangelist (in Neotechnology) described his views on how SOA should look - according to him. This was presented previously, on other occasions as his "Guerilla SOA" talk - generally he advocated for REST based services, loose contracts (stating that WSDLs are too verbose and code generation is evil).
Jim mentioned Martin Fowler's article on integration databases but I couldn't find it anywhere - thou the topic looks interesting. He also recommended BDD and exposing tests on the web for the end user to use them as early as possible.
One big point he made his case with was not relying on enterprise software. Simple tools can do much better job. He compared implementing Web Services security (Secured SOAP over HTTP over TCP IP) to REST based service accessed through HTTPS - basic and easily testable with tools like curl.
Great talk. One of the best!
2. Staffan Noteberg "Regex - the future programming"
I must confess, that this did not go too well. The whole talk was well prepared and laid out but it lacked depth. It was pretty basic introduction to regex. From the presentation's subject I was rather prepared for some novel uses of regex - like for example: showing how to filter big volume of data with simple regex or sth.
But the talk was fun, Staffan is a good speaker. He is also an author of pomodoro technique book - I intend to read sth abut this technique and this may be a nice start
3. Bartosz Kowalewski "Is OSGI ready for wide adoption?"
If it comes to titles I tend to rely on them pretty heavily, however strange it may seem. This time I also did - and the whole talk did not give me a definitive answer to the stated question.
Sure, the presentation was informative, but it described some OSGI specific, quite low level stuff. Of course, if you want to use OSGI - even by leveraging application server with OSGI under the hood - you should know a fair bit about the technology itself. Even thou the AS does a good job of hiding OSGI container specifics from the developer, in case of problems it's better to be well informed. All in all - the talk gave too little information for me.
4. Vaclav Pech "Pick low hanging fruit"
"Parallelism is not hard, multithreading is" - this was the key sentence of the presentation. The speaker showed how to introduce concurrency into normal java/groovy code by sprinkling it with concurrency powder. Easy enough! With GPars library he showed:
- running processing tasks with thread pools
- testing concurrent code
- Fork/join Thread Pool - multiple thread queues (note to self: fork/join is good for hierarchical problems)
- low-hanging fruits:
- async calculations
- parallel collection processing
- Actors are great - use GPars or Akka, is sufficient to use @ActiveMethod and @ActiveObject annotations and Actors are usable in OO-world
Good talk, well received!
5. Anton Arhipov "Bytecode for discriminating developers"
Technical introduction to the world of bytecode, jvm specification details. I've drifted away to some other topics - really - can't recall what this was all about.
6. Andreas Almiray "Polyglot Programming"
This was a nice talk covering Groovy, Scala and Closure. The whole point of it was to show how cool it is to play with emerging JVM languages. They are not only fun but also useful. What's more, they bring freshness to java world, injecting it with some new paradigms and methodologies. It is easier to incorporate new ideas into younger JVM languages than to the mature Java.
7. Jim Webber "A pragmatic introduction to Neo4j"
And Jim Webber again, this time with some Neo4j evangelism. First came some taxonomy information on NoSQL databases (Not Only SQL) as a whole - than some specific examples of problems solvable with graph databases - and Neo4j is a graph database.
Main points of Jim's talk were:
- sharding a database is important for scalability
- series data - should be OK to use Neo4j as their storage
These were all the sessions I attended. On Saturday there was a Hacker-garden, but neither I had time nor will to stay - the topics were very interesting and I'd definitely like to experience such an event, but after 2 days of continuous talks I was rather tired.
To sum up, 2011's Geecon was a great experience, with lots of interesting talks and lots of new inspirations. Keep up the good work guys!
Last week's Java conference - Geecon was very interesting. It was well prepared, and gave an insight into the current Java related trends - concurrency, DSLs, polyglot programming. But not only that - there were also some pretty different talks from excellent speakers.
The whole event took 4 days:
- University day (wednesday)
- 2 regular conference days (Thursday + Friday)
- hacker garden (Sunday)
I decided to attend only on Thursday and Friday - no time for more. Here is what interesting happened during those days.
The morning got me unprepared. After hard enough, after work travel to Krakow on Wednesday, I wasn't in the best shape. However after arriving at the venue, being greeted with breakfast and refreshments I looked at the rest of the day with real hope.
Since the schedule was tight - three parallel tracks of lecture, I had to choose, so bare in mind, that is my account of what I've seen and heard. Others may, of course, differ.
1. Danny Coward "Java SE: The Road Ahead"
Danny, being on Oracle (considering being also former Sun's employee a plus) payroll, gave an insightful talk on new things to came in Java 7. He drew rather serious plans for Java 8. According to Danny, the main trends in today's Java ecosystem are:
- parallel programming
- language dynamics
and he probably is right :) The great things to come with new versions of Java are:
- closures in Java (finally!)
- extending interfaces
- map, filter - functions for collections
- lambda expressions - thou in Java 8
The talk itself was a nice keynote, but I doubt the road map for Java will be met in its full extent - the goals aren't that small.
2. Juergen Hoeller "Enterprise Java in 2011"
Spring Source as one of the sponsors sent Juergen to evangelize about the world of enterprise and Java's place in it :) He emphasized different kinds of deployment: WAR, cloud deployment - and the latter's rise of importance.
He pointed out how outdated current application servers are - the usually lag ~3 years behind the main trends and developers' needs - good point! He proposed looking under the hood of now-popular cloud environments: Google App Engine or Amazon Elastic Cloud to look for schemas in them, etc - I intend to listen to his advice.
All in all this guy gave a great talk covering wide spectrum of technologies and not focusing on technical stuff too much.
3. Heinz Kabutz "Reflection madness"
Despite living on a Greek island, this guy showed also how to whack ones mind with Java Reflection API. Pure magic! Some highlights of his talk were:
- how to get 42 + 1 = 44
- get the size of an object
- get method caller's id
- add enum values dynamically
With all this examples he pointed that using SecurityManager will prevent such nasty coding practices.
Since he is an editor of Javaspecialists.eu newsletter, all the answers to problems presented in his talk (and many many more) can be found there.
Well done, not to useful for me, nevertheless - interesting.
4. Michael Figuiere, Cyrille Le Clere "NoSQL & Datagrid from developer perspective"
I don't know what to think of this talk. It consisted an introduction to NoSQL databases but also a bit of problem's description that can be encountered when dealing with them. Notable thoughts were on:
- creating a sharding ready data structure
- denormalization as a useful process for NoSQL DBs
- NoSQL usually means no transactions
5. Hamlet D'Arcy "New Ideas for old code"
Since a lot of developers (all?) have to deal with legacy code - one way or another, this talk was a must!
. The speaker shared some ideas on how to work with such code and remain sane. The talk was vivid, interesting and entertaining, well, and the notable thought? Here they are:
- 51% rule - if you're not committing 51% of your time/your tasks into fixing your situation than the whole battle is already lost,
- read some good stuff!:
- use static analysis - Find Bugz, pmc
- query-command - a method should be a query or a command:
- query - returns sth
- command - change the state of an object
- he also proposed scratch refactoring
- set a timer
- tag your code
- refactor without tests
- step back and analyze
- is it better? if not, revert
This was nice! - it assumed arriving at a project with no ( or little) tests.
6. Aslan Knutsen "Arquillian"
The last talk of that day was about some new library from JBoss that would allow to test your components with unit tests - test them in a destination container. The whole point of this library is to run the specific fragment of code as if it was build and deployed to some application server (let's say JBoss AS ;-) ). To be honest, I can't find much application for that - thou I'm not doing any serious work in JEE world.
And the day ended. But there was sth else to do after the official part - party time! It took place at Klub Pauza on Floriańska street. It was a rather nice social event.
... to be continued - stay tuned for part 2
Possible problemsIn my search for this problem's solution I've encountered this kind of exception: A little googling led me to this StackOverflow question. It seems that you cannot have multiple keys with different passwords in the same keystore and use KeyManagerFactory class. Oh well... .
EndingTo sum up, the solution given works, but in my opinion using the InstallCert.java app is rather dirty. I've been wondering, do you know other ways of doing that thing?
This weekend I've been fiddling with QEMU. I've installed OpenBSD on a single image and wanted to have two instances of it communicating via network. Installing the system was easy, but the networking setup was quite a pain. See how I did that... To make QEMU instances communicate with each other I needed to plug them to a "network". That's why I've created a bridge to which Virtual Instances would connect to. I've used the following script: Then I just needed to start Qemu with this command line: Since I've set up bridge for Qemu instances, I've plugged TAP interfaces into it. That's why I've needed to specify this in my qemu exec line. I've also added macaddress setting since both my instances were getting the same one. And that's all! It works like a charm. Now on to some harder things!