Running aspects under OSGi 4.2 with Karaf

24 Nov
2011

One of bigest benefits of Java is byte code manipulation. You can change everything you want in your application without touching source code. That’s usefull for many cases, starting from legacy code, where we can’t simply modify and recompile library up to modern applications where aspects can be used to handle runtime exceptions. The most popular project is AspectJ which is part of Eclipse ecosystem, in this post I going to show you how to use AspectJ with Karaf.
Read the rest of this entry »

Preserving message order in ActiveMQ

18 Nov
2011

Few hours ago I’ve found an usefull post about preserving message order with ActiveMQ written by Marcelo Jabali from FUSE Source.

In his example Marcelo used broker feature called Exclusive Consumers. It lets send messages only to one consumer and if it fails then second consumer gets all messages. I think it is not the best idea if we have many messages to process. Why we wouldn’t use few consumers with preserved message order? Well, I was sure it is not possible, but during last training I’ve found solution.

Broker configuration


So how to force ActiveMQ to preserve message order? It’s really simple, we just need to change dispatch policy for destination. We can do this for all queues or only for selected.

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:amq="http://activemq.apache.org/schema/core">

    <broker xmlns="http://activemq.apache.org/schema/core">
        <destinationPolicy>
            <policyMap>
                <policyEntries>
                    <policyEntry queue=">"><!-- Please refer 2nd part of this post -->
                        <dispatchPolicy>
                            <strictOrderDispatchPolicy />
                        </dispatchPolicy>
                    </policyEntry>
                </policyEntries>
            </policyMap>
        </destinationPolicy>
    </broker>
</beans>

After this consumers should receive messages in same order like they were sent from producer. You can find example code on github: example-activemq-ordered. You can run all from maven:

cd broker1; mvn activemq:run
cd broker2; mvn activemq:run
cd consumer; mvn camel:run
cd consumer; mvn camel:run
cd producer; mvn camel:run

Upgrade

After posting update about this blog post to Twitter Dejan Bosanac send me fewupdates. He is co-author of ActiveMQ in Action so his knowledge is much more deeper than mine. 🙂
First of all I mixed XML syntax. strictOrderDispatchPolicy is handled by topics, not queues. For second destination type strict order is turned on by strictOrderDispatch attribute set to true for policyEntry element. This preserves order but, as Dejan wrote, it will broke round robin and all messages will go to only one consumer, as in previous example given by Marcelo.

Also, Marcelo published second post about Message Groups which allows to preserve order and have multiple concurrent consumers on queue.

Introduction to Karaf WebConsole

17 Nov
2011

Management of OSGi – let’s face it – is not very hard. The OSGi environment is clearly defined and that gives programmers many mechanisms to create administrative tools. The problem begins when we would like to use only one tool to manage few projects or artifacts of different types. I know this from personal experience because when I run Camel, ActiveMQ and CXF every from them provides own administration console. Every of them requires own security configuration, looks differently, have own dependencies and so on.

This stands a little bit in contradiction with the OSGi specification idea, which tries to unify management of different things, not only dependencies (Core) but aspects like configuration (ConfigAdmin), deployment (DeploymentAdmin), meta data (MetaType), preferences (PreferencesService), users (UserAdmin) or permissions (PermissionAdmin). Naturally, creation of standard for management tools is too hard to be closed in any specification, even so good like OSGi.

Felix WebConsole

As response for problems with lack of common tool, available from browser a project Felix WebConsole was created. This is a subproject of Felix’a. In its assumptions Felix WebConsole should let easily extend itself through the use of mechanisms known from the core OSGi – that is, services. It should also let change look and feel and localize the tool. All these assumptions was covered, but number of extensions for Felix WebConsole do not grow like mushrooms after the rain.
The question is, why? Now, in its simplicity Felix WebConsole make difficult creation of more complex extensions such as JMX. The problem is that our extension is only a servlet. From one hand it’s too much to put a link in list of bundles, on the other hand it is too little to make a few pages. If we’ll try we’ll begin to implement the second Struts framework based on servlets.
Project team do not make things easier because it puts on the “lightness” console. It is expected to deploy console on mobile devices too.

Karaf WebConsol as alternative

Apache Karaf WebConsole is brand new project, which was made few months ago. After dynamic incubation phase, which maybe was too short, it was moved to sub-project of Karaf. Similar as precursor it points to lightness but also points to collaboration with other web frameworks, in this case it is another ASF project – the Apache Wicket. Through its use we obtained far-reaching component model. It means that you may add link to menu and style it with fragment CSS (without fragment bundle). You may add new tab with content or simply put another widget to dashboard. All these things you may see right after logging into console.
All these extension won’t be possible without Wicket and Pax-Wicket. Thanks to huge amount of work made by Andreas Pieber on second project. Everything is stable and works as fast as Felix WebConsole.

Architecture of Karaf WebConsole

As I meintoned before, WebConsole uses Pax-Wicket and Wicket as presentation layer (we don’t count Java Script libraries). Most of extensions uses also blueprint to register services. There is no problem, similary like in case of Felix WebConsole, to use Spring DM or declarative services. Everything works with Pax-Web, but it should be possible to run it with any HttpService.
Whats’s more, an experimental branch of pax-wicket which I worked on previously named jsr330 let to use same components in a traditional container like is Tomcat. This means that the administrative tool went beyond the OSGi framework and will allow creation of multi-modular consoles, also in a traditional environment.

Extensions

New elements who are added to Karaf WebConsole are typically Wicket components or they are converted to them. Let see how to add new element to navigation:

package org.apache.karaf.webconsole.blueprint.internal.navigation;

import java.util.ArrayList;
import java.util.List;

import org.apache.karaf.webconsole.blueprint.internal.BlueprintPage;
import org.apache.karaf.webconsole.core.navigation.NavigationProvider;
import org.apache.wicket.Page;
import org.apache.wicket.markup.html.basic.Label;
import org.apache.wicket.markup.html.link.BookmarkablePageLink;
import org.apache.wicket.markup.html.link.Link;

public class BlueprintNavigationProvider implements NavigationProvider {

    public List<Link<Page>> getItems(String componentId, String labelId) {
        List<Link<Page>> items = new ArrayList<Link<Page>>();

        Link<Page> link = new BookmarkablePageLink<Page>(componentId, BlueprintPage.class);
        link.add(new Label(labelId, "Blueprint"));
        items.add(link);

        return items;
    }

}


Interface org.apache.karaf.webconsole.core.navigation.NavigationProvider is universal supplier of navigation elements. In web application it’s mostly about links. Now, when we have implementation we need to submit it into registry to let use it. In this particular example we going to use blueprint, but it might be a standard activator or any other declarative way as well.

<?xml version="1.0" encoding="utf-8" ?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

    <service ref="provider" interface="org.apache.karaf.webconsole.core.navigation.NavigationProvider">
        <service-properties>
            <entry key="extends" value="osgi" />
        </service-properties>

    </service>

    <bean id="provider" class="org.apache.karaf.webconsole.blueprint.internal.navigation.BlueprintNavigationProvider" />

</blueprint>

Fragment above will cause addition of BlueprintPage as child of OSGi menu, because we set extends property. Result of execution you may see on attached picture.

Navigation is only example, remember – you have much more possibilities:

  • org.apache.karaf.webconsole.core.brand.BrandProvider – lets to change design (without fragment bundles)
  • org.apache.karaf.webconsole.core.navigation.ConsoleTabProvider – causes addition of new tab in navigation
  • org.apache.karaf.webconsole.core.navigation.SidebarProvider – adds new elements on left side
  • org.apache.karaf.webconsole.core.widget.WidgetProvider – lets to publish new panels with content
  • org.apache.karaf.webconsole.osgi.bundle.IActionProvider – adds specific link to bundle list
  • org.apache.karaf.webconsole.osgi.bundle.IColumnProvider – adds column to bundle list
  • org.apache.karaf.webconsole.osgi.bundle.IDecorationProvider – adds icon bundle list

Building sample middleware – producer

18 Jun
2011

Last two days I’ve spent hacking Swing code. I decided to run standalone producer application to show real interaction with broker. You may treat this Swing app like entry point for people to our middleware system. Users simply do “transfers” from this application and don’t know anything about technical details. I added text area to main window to show structure of message sent to broker.

Producing messages with JMS

Most of communication systems, whanever you will go have two different kinds of values, first – main and mainly used is body, second is typical metadata named headers or properties or parameters. JMS is not different, you can create different kinds of messages and set headers to them (in JMS world they’re named property, but I preffer header).
Let check messaging code we have:

// import declarations ...
public class JmsMessageSender implements MessageSender, InitializingBean {

	private ConnectionFactory connectionFactory;
	private Session session;
	private final String destination;
	private MessageProducer producer;

	public JmsMessageSender(String destination) {
		this.destination = destination;
	}

	public void sendMessage(String message, Map<String, Object> headers)
		throws Exception {

		if (session == null || producer == null) {
			afterPropertiesSet();
		}

		TextMessage jmsMsg = session.createTextMessage(message);
		for (String header : headers.keySet()) {
			jmsMsg.setObjectProperty(header, headers.get(header));
		}
		producer.send(jmsMsg);
	}

	public void afterPropertiesSet() throws Exception {
		Connection connection = connectionFactory.createConnection();
		session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
		Queue queue = session.createQueue(destination);
		producer = session.createProducer(queue);
	}

	// not important
}

First of all we inject connectionFactory using Spring configuration file. In line 20 we create text message, because we sending messages with String as content. Line 22 sets headers for message and finally sends it in line 24 using message producer created in lines 28-31. If we’ll look closer these lines we’ll see standard initalization code automatically called during bean creation by Spring. In line 29 we create session without transaction support and create producer to send messages. Connection factory is configured in XML with properties file to extract informations like broker url username and password.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="view" class="org.code_house.mom.producer.ProducerWindow">
        <property name="title" value="Bank Agent APP" />
    </bean>

    <bean id="controller" class="org.code_house.mom.producer.ProducerController">
        <constructor-arg ref="view" />
        <property name="messageSender">
            <bean class="org.code_house.mom.producer.JmsMessageSender">
                <constructor-arg value="MOM.Incoming" />
                <property name="connectionFactory" ref="connectionFactory" />
            </bean>
        </property>
        <property name="mapper" ref="mapper" />
    </bean>

    <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="${url}" />
        <property name="userName" value="${user}" />
        <property name="password" value="${pass}" />
    </bean>

    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location" value="classpath:activemq.properties" />
    </bean>

    <bean id="mapper" class="org.codehaus.jackson.map.ObjectMapper" />

</beans>

I decided to use JSON over XML. Regarding last post about data structures it is not the best choice but it is very simple. So don’t treat it as reference implementation. I just wish to don’t use JAXB annotations. 🙂

Headers

Headers are common thing for routing. This is main reason why I allow to provide headers as parameter to sendMessage method. In many cases it is better to use headers or even extract headers from body of message in one endpoint to reduce number of content reads. Example how to use headers for routing we’ll see in next part about Apache Camel.

Just as a note types of headers supported by JMS spec:

  • Boolean
  • Byte
  • Short
  • Int
  • Long
  • Float
  • Double
  • String
  • Object

We can get or set headers using getTypeProperty(String name) or setTypeProperty(String name, Type value). Remember that headers of type “object” set by setObjectProperty must be serializable, otherwise they will be dropped before sending.

Producer application

As I said at begining of the post I spent two days on Swing hacks. 🙂 I injected two additional depentencies to this module – mig layout and better beans binding. First is responsible for frame layout, second for interactions between model and Swing controls. Whole application is simple form. All informations are displayed in status bar at bottom of the window, rest is taken by message area and combo boxes.

I have say, that binding framework is cool and reduced number of code lines I had to write without it. If you are interested in exact code structure, please go to producer module in mom-sample github repository. Because it is not in area of this post I will not write more about desktop implementation details.

For these who wish run sample (I belive you would do that) – after executing

mvn clean install

You can simply execute target/producer.jar by doulbe click. This is fat-jar with all dependencies needed by producer application.

Building sample middleware – data structures

13 Jun
2011

Data with middleware

The data structures are very important in middleware world. Because we often connect multiple systems we need to define an “domain model” for integration. The domain model means that objects we share between all systems are well known, well defined, well understand in multiple teams often provided by multiple vendors. Let see what does it means in practice.

Well known

If you talk with business guys you have to use same terms with programmers. You might be mediator but not the translator. As integration team member mediate correct shape of solution but don’t became a translator between business divisions and developers (unless you are business analitic). All people should use same terms to minimalise problems with number of definitions. That’s first step of multiple projects, not only integration. If you look for deeper knowledge of domain definitions check a “Domain Driven Design” book written by Eric Evans and published in 2003. Ok, maybe business is not always part of integration project, but you communicate systems used by people, isn’t?

Well defined

You have names and scopes for your classes and objects. Probably you have 90% of fields that will be defined in them. Remember to don’t put too much informations from one system to domain model. Domain model is not list of entities from one system. Make sure that you don’t share database identifier from one system to second. Identifier should be business, for example most of clients might be identified by personal document, most of companies can be identified by tax id, not by database sequence. For future reasons try to extract atomic values, like for databases. With atomic values you will have less text processing to extract specific data and less code to maintain.

Well understand

That’s case for distributed teams. Usually every team have own parts of software to do and every team plays different game. Someone else produces messages, someone consumes them, this is place when you’ll have small wars. Make sure that data structures you all have to share match common needs, that defined identifiers are fine for different systems too. I met multiple situations when system analytics from two companies spent lots of hours on phone defining field constraints. Don’t follow this path.

Whole integration project is about to fail in 95% cases – if you will not have data structures. Ok, we know that proper structure is key for integration. Second important point in data is representation. I don’t going to talk about REST at all, but rather about format of serialization. We have number of options here, just look table below

Serialization Format Pros Cons
Java Object Serialization
  • Very easy to maintain
  • Easy to use with JMS / ActiveMQ – ObjectMessages
  • Built in Java
  • Available only for systems written in Java
  • Both sides must use same classes and versions (if we use serialization id)
  • As every binary format – it is hard to process
XML (with XML Schema)
  • Easy to read and write, both for computer and people (I don’t belive YAML)
  • Easy to use with JMS / ActiveMQ – TextMessages
  • Very easy validation, with XML Schema
  • Well supported in many languages (including validation with XML Schema), DOM api provides complex operations
  • Requires additional libraries, some languages don’t have any binding from XML to Objects and vice versa
  • Synchronisation between Java and XML Schema (code generation)
  • Without proper XML Schema might be too lax
  • Not the best for big portions of data because overflow
JSON
  • Easy to read and write, both for computer and people, even easier than XML
  • Easy to use with JMS / ActiveMQ – TextMessages
  • Smaller than XML, easier to parse, even without any API
  • Supported in number of languages
  • Lack of well supported official structure control tool (like XML Schema)
  • Lack of official API for handling JSON
  • Requires additional libraries, some languages don’t have any binding from JSON to Objects and vice versa

There is number of different wire formats not listed here – for example Protobuf, EDI which match this scenario too. I don’t count them (even if I should) because first is not well supported in many languages, second is rather legacy format not well for new systems.

Structures used in MOM

Like in most cases, also with middleware we have some data structures. Because we going to process simple cash transactions we’ll have following classes:

  • org.code_house.mom.domain.Transaction
  • org.code_house.mom.domain.Client
  • org.code_house.mom.domain.Money
  • org.code_house.mom.domain.Money.Currency

Transaction

Every transaction will have unique identifier generated by UUID. Transaction will point to some Client and have assigned Money object.

Client

In our case client is very simple POJO which contains fields ID and name. For middleware purposes we going to use only ID. Name might be usefull for user interface operations.

Money

Because we going to transfer some money to client account from producer we have to use proper data structure for a “cash”. Few years ago Martin Fowler in his book “Patterns of Enterprise Application Architecture” defined a Money pattern which is suitable to handle money values. We have to remember that client account might be handled in different currency and we cannot send only amount information. We need a currency too.

That’s all for data structures. It was more about philosophy than about code, but don’t worry – in next part we going to write more. All needed classes you’ll find in git repository. You may download zip archive|tar archive if you don’t have git client.

Building sample middleware – broker

10 Jun
2011

At end of the May I had great time in United Kingdom providing consulting. After this I started thinking about sharing an idea of middleware application based on ActiveMQ and Camel features. I’ve spent few days to create sufficient example.

What middleware is?

Middleware is a general term, about some software which is some kind of proxy between other systems. What for? – you could ask. Generally because communication from point to point is not the best to build bigger applications, and some components in middle of communication allows us to inject new logic without changing source systems. Could you imagine situation where you have a number of system producing messages and an number of consumers of these messages written in different languages? That’s typical case where middleware is going to be usefull.
There is number of middleware tools from various categories, in this post we’ll learn how to use ActiveMQ to build message oriented middleware. What does the MOM means? Generally that we have asynchronous communication without direct method invocations. Producer don’t know anything about consumer and vice versa. If you are interested in MOM – pick up “ActiveMQ in Action” book (written by commiters of ActiveMQ) from Manning Publications Co. where this term is described in greater detail.

Read the rest of this entry »

Apache Karaf commands rediscovered

15 Dec
2010

I work with Apache Karaf almost every day. There is a lot of commands provided by default and most of them are a bit anonymous. In this post I would like introduce these commands.
Read the rest of this entry »

Newer Entries
top