Blog

Xebia Software Development builds Posterous clone in a day!

20 Sep, 2010

Sharing knowledge is one of our core values and as lot’s of research confirms knowledge transfer is best done between peers. We have a great knowledge sharing platform at Xebia through bi-weekly evening sessions, where we do some experimental coding and some presentations. Once in a while we take it to the max and organise a tech rally. One of those happened last Friday and it was a total blast. I’ll give you some of the highlights. More detailed posts on the technical details will follow and I’ll update the list below as they do:


We started out with a picture and a maven skeleton and starting code for some tricky parts. Graciously prepared by Albert Sikkema.
This image was the only visual aid in the presentation of the case, causing much skepticism and questions about lack of preparation. These early moments of uncertainty were casually ignored during development.
From there we split up in 4 teams and chose common sense and utter anarchy as a methodology.

Javascript flamewars

Prior to the tech rally we had a good old flamewar between javascript zealots and heretics. One of the best moments was when Gerbrand, who isn’t a big fan of javascript turned up as the first volunteer to work on the javascript needed for the front end. The end result was cuddly like a baby bear, I’m really curious to hear if Gerbrand has been converted to the dark arts. I couldn’t find the JSUnit testcases, but that’s a topic for another day.

Spring REST

To render JSON to a client it is tempting to try out a REST framework. To keep things simple we went for Spring REST and the result was as expected. There were a few hick-ups related to setting the content type or facing the 406, but luckily no fairies got killed. The details of this little trick have been described already on the Spring blog, but if you need a working sample you can also have a look at our web project.

Reading email with Spring Integration

Through no influence of yours truly (boy scouts honor) we ended up using Spring Integration to read mails from a gmail account. I avoided joining that team, but I had the pleasure of joining in one particularly interesting discussion on preferred granularity of pipes and filters. The answer I came up with could fill another blog, but the question is more interesting anyway, so I’ll just give you that.

When should we use a synchronous channel and when is it better to just call the method directly?

In other words: how big should endpoints become before you split them up? Think on that and let me know if you have figured it out.
Dealing with javax.mail, even through a framework is a daunting exercise. Andrew Snare (who recently joined us) found a really tricky one, Albert Sikkema found a couple more during the preparations. Because of this, intelligent folder management was not one of our achievements during the day. We did however read the mail successfully and store it in our database. Good enough for a first prototype.

Storing the posts in MongoDB

At Xebia we’re involved in a number of initiatives around NoSQL, most notably the NoSQL Meetups organized by Wilfred. Obviously we cannot store data in a traditional database for fun anymore. Even though the MongoDB part of the application might be the coolest bit on paper the work there was minimal. MongoDB is just too easy to use.
Being in the MongoDB team was pretty cool nonetheless, because being the main integration point of a 20 developer 4 team commit frenzy is the closest thing to being in a warzone I ever hope to experience.
The coolest thing was that at the end, the application actually worked. Nobody got hurt and we could show an end to end demo of the whole thing at five in the afternoon.
This rally was exactly the kind of experience I was looking for when I joined Xebia, so I was totally thrilled. The only thing left to do is find a customer that allows us to really go crazy like that. Any takers?

Iwein is an engineer with Xebia and a member of the Spring Integration team. He's an expert on Spring and Test Driven Development. He specializes in Messaging, OSGi, Virtualization.
guest
2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Peter Veentjer
11 years ago

Hi Iwein,
one of the things that concerns me when people are using NoSQL databases is at how well they understand the relaxed guarantees most NoSQL databases provide (especially in the area of consistency and atomicity). Depending on the implementation used you don’t always get distributed failure atomicity, so a change could partially enter the system (the older versions of Cassandra are suffering from this problem.. haven’t checked out the newest releases).
Another problem I see is with consistency; consistency is a very hard topic and I think I can say I know the bare basics since I deal with it on a daily basis with STM. Dealing with all these relaxed consistency models that form the heart of most NoSQL databases sounds like a gift from heaven from a scalability point of view, but what it essentially means is that the burden is now placed on the developer and my experience is that most developers don’t have the time or understanding to get it right.
I find relaxed consistency models very hard to reason about, so how should most developers do it. I see a similar problem when multi-threading is used; a lot of developers write code that looks alright in their eyes, but if you look closer, you can see all kinds of problems ranging from traditional problems like: liveness, deadlocks and data-races to all kinds of less traditional problems like instruction reorderings or visibility problems.
My bet is that most applications can be designed quickly and scale reasonable well (much better than a central databases because that just doesn’t scale), but under contention or when the system or parts of the system crash, that the integrity of the data could be violated.
Peter Veentjer
Multiverse: Software Transactional Memory for Java
http://multiverse.codehaus.org

Explore related posts