Trends of 2010

The year is almost over and it’s hard to refrain from nostalgic retrospective of major trends than emerged during the past 12 month. None of these trends started in 2010 but each of them seems to reach an important inflection point.

Clouds are now pretty much at the pinnacle of their hype. Everybody is doing something cloudy now. There is a good reason for it – they do seem to provide a real value in certain situations. Their sweet spot is powering development and testing environments, offloading temporary load spikes and low-investment test-bed for startup ideas. The key to their success is their on-demand elasticity and provisioning. So far it seems that Amazon is by far the clear leader in cloud providers. Other PaaS and SaaS providers like Google App Engine and SalesForce are much more nichy and restrictive.

There are still big unanswered questions with the clouds which include security, regulatory compliance, vendor lock-in, etc. Probably next few years will see rapid evolution of cloud computing that would address some of those problems.

Mobile Development is capturing huge attention and interest. Recent success of iPhone Apps and Smart Mobile devices in general look like an inflection point of a broader shift in computation devices away from the traditional desktops. It is true that many Smart Phones today pack in more processing power than desktop computers 10-15 years ago. If this trend continues then new tools and methods will need to be developed and adjusted for mobile applications that are still in their infancy.

NoSQL databases are becoming ever more widespread. Their ability to scale for today’s information and communication needs often outweighs compromises that are made when moving away from traditional RDBMS. I wrote a short blog about various popular NoSQL options. An interesting and very recent trend is support that major frameworks like Spring begin to provide for NoSQL DBs. Grails GORM now carries support for a number of NoSQL databases.

While many people read NoSQL as “NO SQL”, a better approach would probably be “Not Only SQL” because relational database are not going to go away any time soon and still provide very valuable services.

Functional Programming is getting more and more publicity. It seems that Functional Renaissance started from necessity to write programs targeted at multi-core computers. Functional Programming paradigms lend to better distribution of computations among the CPU “heads”. A proliferation of functional languages is now emerging: from grand-daddy of all functional languages Lisp, to new kids on the block - Scala, Closure, Erlang, Haskell and F#.  Some languages such as Scala attempt to bridge Object Oriented and Functional programming by allowing the developer to choose the most suitable tools within the language.

There are many more interesting developments in such areas as Social programming and Agile methodologies but they are somewhat less pronounced than the trends that I have mentioned.

Certainly year 2010 feels like an important stage in development of computing and software in many important areas. Major paradigm shifts are occurring in the whole information processing universe. It will be interesting to see how these trends evolve and what new trends are in store for 2011 and beyond.

Vertical silos vs. Horizontal affinity

It is safe to say that Agile methodologies are considered mainstream this day and age. One of the main precepts of the Agile methodologies is using of cross-functional teams and delivering applications in vertical slices for rapid feedback and early exercise of integration.

There can be no debate that the feedback loop and early integration is critical for the success of the project. However, what happens in the systems where infrastructure is significant, perhaps biggest, part of the system? It is hard to create slices of the infrastructure based on vertical slices of functionality being developed for the current iteration/sprint/user-story.

Such a low-level and application specific infrastructure has much higher affinity within itself than with any of the vertical silos where it might be used. It makes sense therefore to develop this infrastructure separately from the vertical slices and leverage it later with the usual Agile iterations.

An important point in this is that by infrastructure I don’t mean DAO layer or service layer. In typical web-apps such layers are nothing but an amalgamation of parts of vertical pieces of functionality. By infrastructure here I mean something like low-level networking, threading models and abstractions, core messaging abstractions, or low-level application-specific services.

Sometimes parts of an infrastructure can be harvested through an evolutionary architecture and emergent design. But in any big and especially non-plain web-systems there is a significant amount of upfront infrastructure that needs to be in place in order to start developing the “end-user” functionality.

Focusing on a single level of abstraction (SLAP), in this case the low-level infrastructural services increases productivity and helps creating a consistent system.

It is therefore important to carefully examine which parts of the system exhibit highest levels of affinity and develop them together as a fundamental block upon which rest of the system will be built later.

Is Java suitable for today’s web-development?

I know it’s a provocative subject and I might get flamed for it but I have to share my thoughts on the suitability of Java for today’s web development.

As always, the applications are developed to solve business or other human problems. Therefore the closer the application implementation is to the domain of the problem – the faster and better the application development is. If you’re programming bank application you want to operate with bank accounts, not object arrays. If you’re developing gaming application you want to develop players profiles and game rules, not low-level 3d rendering.

If you’re developing a web-application, for example facebook app, you want to deal with RESTful Facebook Graph API, with social features, and at least with somewhat interactive pages. What you don’t want to do is to deal with low-level plumbing like mechanics of reading the URL content. Such details are nothing but an accidencial complexity within the system.

A simple example:

Let’s say you need to read content of RESTful URL something like https://graph.facebook.com/me. All you really want is to get a string with the URL's content.

In Groovy it would look like this: “https://graph.facebook.com/me”.toURL().getText()

After spending few hours trying to find something similar in Java I have reached a conclusion that the best I can do is either:

1. Use apache HttpClient and deal with hierarchy of its objects. This is a bit more than few lines of code.
2. Try to find another obscure library.
3. Write custom code like this:

public static String getText(String url) throws Exception {
        URL website = new URL(url);
        URLConnection connection = website.openConnection();
        BufferedReader in = new BufferedReader(
                                new InputStreamReader(
                                    connection.getInputStream()));

        StringBuilder response = new StringBuilder();
        String inputLine;
        while ((inputLine = in.readLine()) != null) 
            response.append(inputLine);
        in.close();
        return response.toString();
    }

Now, of course, I can write it and encapsulate it in some helper utility object etc.

But the problem is that this code has nothing to do with the problem that I’m trying to address in my application. It represents a much lower level of abstraction. At yet I’m forced to resort to spending time on researching, implementing, testing, maintaining such a basic functionality.

For comparison, Ruby/Groovy/Python developer does not need to know anything about BufferedReaders, InputStreams, using StringBuilder for loops etc. All he really needs to know is the URL, and the string where to put the content.

It might seem that I’m blowing this issue out of proportion but I think it exemplifies a much deeper problem. Java was invented and evolved in a different era. Today’s web applications use different and higher abstractions. This is normal cycle of development of computer languages. Each new generation abstracts low-level details of the previous generation allowing developers to focus more on their problem domain. 15 years ago java simplified development by allowing programmers not to care about pointer arithmetic, destructors and other C++ plumbing.

I know that Java is a generic language and it will continue to be used for a very long time. But guess what: C++, and shockingly even COBOL are still in use, but in only very narrow domains (embedded or low-level programming for C++ and ancient legacy systems for COBOL).

I suspect that Java is going in the same direction. Today I would be hard pressed to start development of a decent web-application in pure Java.

By the way – I had exact same argument about C++ and Java about 10 years ago with my former colleagues.