Preserving message order in ActiveMQ

18 Nov
2011

Few hours ago I’ve found an usefull post about preserving message order with ActiveMQ written by Marcelo Jabali from FUSE Source.

In his example Marcelo used broker feature called Exclusive Consumers. It lets send messages only to one consumer and if it fails then second consumer gets all messages. I think it is not the best idea if we have many messages to process. Why we wouldn’t use few consumers with preserved message order? Well, I was sure it is not possible, but during last training I’ve found solution.

Broker configuration


So how to force ActiveMQ to preserve message order? It’s really simple, we just need to change dispatch policy for destination. We can do this for all queues or only for selected.

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:amq="http://activemq.apache.org/schema/core">

    <broker xmlns="http://activemq.apache.org/schema/core">
        <destinationPolicy>
            <policyMap>
                <policyEntries>
                    <policyEntry queue=">"><!-- Please refer 2nd part of this post -->
                        <dispatchPolicy>
                            <strictOrderDispatchPolicy />
                        </dispatchPolicy>
                    </policyEntry>
                </policyEntries>
            </policyMap>
        </destinationPolicy>
    </broker>
</beans>

After this consumers should receive messages in same order like they were sent from producer. You can find example code on github: example-activemq-ordered. You can run all from maven:

cd broker1; mvn activemq:run
cd broker2; mvn activemq:run
cd consumer; mvn camel:run
cd consumer; mvn camel:run
cd producer; mvn camel:run

Upgrade

After posting update about this blog post to Twitter Dejan Bosanac send me fewupdates. He is co-author of ActiveMQ in Action so his knowledge is much more deeper than mine. 🙂
First of all I mixed XML syntax. strictOrderDispatchPolicy is handled by topics, not queues. For second destination type strict order is turned on by strictOrderDispatch attribute set to true for policyEntry element. This preserves order but, as Dejan wrote, it will broke round robin and all messages will go to only one consumer, as in previous example given by Marcelo.

Also, Marcelo published second post about Message Groups which allows to preserve order and have multiple concurrent consumers on queue.

1 Response to Preserving message order in ActiveMQ

Avatar

todd

November 21, 2011 at 3:02 pm

Have you looked at using Message Groups within ActiveMQ, http://activemq.apache.org/message-groups.html to achieve what your looking to accomplish. Per the documentation:

Message Groups are ehancements to the Exclusive Consumer feature to provide:

  • guarranteed ordering of the processing of related messages across a single queue
  • load balancing of the processing of messages across multiple consumers
  • high availability / auto-failover to other consumers if a JVM goes down

So logically Message Groups are kinda like a parallel Exclusive Consumer. Rather than all messages going to a single consumer, the standard JMS header JMSXGroupID is used to define which message group the message belongs to. The Message Group feature then ensures that all messages for the same message group will be sent to the same JMS consumer – while that consumer stays alive. As soon as the consumer dies another will be chosen.

Another way of explaining Message Groups is that it provides sticky load balancing of messages across consumers; where the JMSXGroupID is kinda like a HTTP session ID or cookie value and the message broker is acting like a HTTP load balancer.

Comment Form

top