Month: May 2016

My Raspberry Pi runs the Qpid Dispatch Router

This morning my “embedded” soul had an idea for supporting my “messaging” soul and spending some time in a productive way … trying to compile the Qpid Dispatch Router on a different machine with a different architecture : it’s C/C++ code … so it’s portable by definition (even if related to Linux for now) … it uses Python but today this language is available on a lot of different platforms … so it seems to be no problems.

Searching for some embedded stuff in my closet and discarding Cortex-Mx based boards (for obvious reasons regarding Linux lack on them) I got my Raspberry Pi … the first version (ARM11 based … not Cortex-Ax) 🙂

embedded_stuff

I have the Pi2 as well (I have to by the third version) but I preferred to stop my research for starting immediately. I followed all the steps needed (explained in my previous article) using the Pi as a normal PC (connected via SSH) and after a while for compiling directly on the board, the router was up and running !

router_pi

The only tweak needed was to force cmake to use the Python 2.7 library with following command line :

cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DPYTHON_LIBRARY=/usr/lib/arm-linux-gnueabihf/libpython2.7.so -DPYTHON_INCLUDE_DIR=/usr/include/python2.7 -DPYTHON_EXECUTABLE=/usr/bin/python

because the 3.x version is installed on the Pi as well but we can’t use it for the router.

I know .. it’s not the right way to compile source code for embedded devices and cross-compilation from my PC is the better way to do that but I preferred it in order to go fast and avoid to setup a complete ARM toolchain on the laptop; of course be patient … building the Qpid Proton took me about one hour ! I can suggest you to have a good coffee …

Before starting the router, another simple tweak was needed in order to make persistent the value of the PYTHONPATH environment variable writing the following line to the .bashrc file :

export PYTHONPATH=/usr/lib/python2.7/site-packages

Now I have an idea … Pi with its Linux version is SSL/TLS capable but there are a lot of resources constrained devices which are able to “speak” AMQP but can’t support SSL/TLS connections. Why don’t use the Pi as a “shadow” IoT gateway and it’s security capabilities in order to bring above constrained devices to reach SSL/TLS and AMQP based cloud platforms even if they can’t “speak” SSL/TLS ?

Routing mechanisms for AMQP protocol

In the previous article, we installed the Qpid Dispatch Router and had a quick overview about the available tools inside the installation package for both router management and monitoring.

Now it’s time to start using the router from simple to complex configurations and network topologies with some examples which will involve AMQP senders, receivers and/or brokers. The broker part won’t be always necessary because the AMQP protocol is a “peer to peer” protocol and it works great connecting two clients directly without the “store and forward” mechanism provided by a broker in the middle. For more information about that you can read the first article of this series.

In this article, I’ll use the router with the default configuration showing how a sender and a receiver can connect to it and exchange messages through the router itself.

Routing mechanisms

First of all it’s interesting to say that the router supports two different types of routing mechanisms.

Message routing

When the router receives a message on a link, it uses the address specified in the target terminus when the sender attached the link itself to the route; if this address wasn’t specified, the destination address is get from the “To” property of the message. Based on such information, the router inspects its routing table to determine the route for delivering the message : it could be a link attached by a direct receiver to the router or another router inside the network that will be the next hop for reaching the destination. Of course, the message could be sent to different receivers all interested in the same address. The main point here is that the routing decision is made for each received message and there is always a communication between internal router nodes and external clients.

message_routing

As you can see in the above picture, a link is established between sender and router and between router and receiver. They are two completely distinct links that the router uses for messages exchange between sender and receiver through the routing mechanism on message basis.

For example, it means that there is a different flow control between router (with its internal receiver) and sender and between router (with its internal sender) and receiver : of course, it’s true that if the receiver grants few credits (i.e. 10) but the router (the internal receiver) grants more credits to the sender (i.e. 250 by default), it takes in account this difference. If for any reason the receiver closes the connection (after receiving 10 messages) and sender has already sent more than 10 messages (acknowledged by an “accepted” disposition), the router will reply with a “released” disposition for the next 40 messages because they can’t be delivered to the closed receiver.

Another interesting point is related to the message “settlement” : the router always propagates the delivery and its settlement along the network. On receiving a “pre-settled” message, its nature is propagated to the destination. The same is for “unsettled” messages : in that case, the router needs to track the incoming delivery and send the unsettled message to the destination; when it will receive the disposition (settlement) from the final receiver, it will reply in the same way to the original sender.

The last interesting aspect of message routing is the available routing patterns which define the paths followed by messages across the network :

  • closest : even if there are more receivers for the same address, the message is sent on the shortest path to reach the destination. It means that only one receiver will get the message;
  • balanced : when more receivers are attached to the same address, the messages sent to that address are spread across receivers. It means that each receiver will receive a different message at time in a sort of “competing consumers” way;
  • multicast : it’s something like a “publish/subscribe” pattern where all the receivers will receive the same message published on the address they are attached;

When the receivers for a specific address are all connected to the same router, we could think that “closest” and “balanced” have the same behavior because all the paths have same length and receivers are closed at same level to the router. It’s not so true because with “closest” the router uses a strict round-robin distribution across receivers while with “balanced” it takes into account the number of unsettled deliveries for each receiver and favors the “faster” of them (who settled messages faster than others).

To be more clear, suppose to have a sender S, two receivers R1 and R2 and a routers network with R1 connected to the same router as S and R2 connected to a different router (connected to the previous). We can say that R1 is “one level closed” to S and R2 is “two level closed” to S.

In the following scenario, with the “closest” distribution all the messages sent by S will be always delivered to R1.

closest_routing

“closest” message routing

Using the “balanced” distribution, the messages are spread across both receivers and there is nor relation with the path length.

balanced_routing

“balanced” message routing

Finally, with “multicast” distribution all messages are sent to all receivers.

multicast_routing

“multicast” message routing

Link routing

When the router receives an attach request, it’s propagated through the network in order to reach the destination node and establish the real link. When the sender starts to send messages to the router, it propagates that message through the established link to the destination without making any decision at message level. You can think of it as a sort of virtual connection or a tunnel between sender and receiver through a routers network.

link_routing

As you can see in the above picture, the link is established directly between the two peers and all performatives go through it.

From a flow control point of view, it’s directly handled between sender and receiver; any credits granted by the receiver arrives to the sender with a flow performative without any interference by the router in the middle. The link through the router is like a “tunnel” and it seems that the two peers are directly connected.

The same is true for disposition about settlement for “unsettled” messages that the sender receives directly from the receiver.

The concept of different routing patterns doesn’t make sense because in this case there is a direct link between sender and receiver so the router doesn’t make any decision on single message basis but it has only to propagate the frame along the link.

Let’s start … simple !

Now it’s time for a very simple example with a router started using the default configuration and only one sender and one receiver connected to it and attached to the “/my_address” address.

I used the “simple_recv” and “simple_send” C++ client examples for that and you can find them inside the Qpid Proton installation folder.

First of all let’s start the receiver specifying the address and the number of messages it wants to receive (it will grant link credits for that), i.e. 10 messages.

simple_recv_start

Using the qdstat management tool we can see that an endpoint for the “my_address” address is defined with “out” direction (from router to the receiver) with no messages delivered yet.

qdstat_01

After that let’s start the sender in order to send some auto generated messages, i.e. 5 messages.

simple_send_start

As you can see, the messages sent are all settled and confirmed. What’s happened at receiver side ?

simple_recv_messages

All messages sent by sender are now received and the simple application doesn’t close the connection because it’s waiting for the other 5 messages in relation to the 10 credits granted (of course it’s only an application behavior and not related to the router mechanisms).

Inspecting the router with the qdstat management tool we can see that on the output endpoint for the “my_address” address there are 5 delivered messages. What we can’t see is the endpoint on the same address with the opposite “in” direction (from sender to router) because after sending the messages, the sender closed the connection to the router and the endpoint is deleted. You can trust me … it was alive for all the time the sender was sending messages !

qdstat_02

As you can see we have directly connected sender and receiver without the need for a broker in the middle with its “store and forward” mechanism. In the above scenario, when the messages are settled and confirmed to the sender it means that they are really received by the receiver.

Conclusion

With this article I introduced the different mechanisms for messages routing that the Qpid Dispatch Router provides. For every scenarios we can choose the better way and what we need in order to distribute messages in a useful way for our distributed application. We saw a simple example on connecting sender and receiver through the router without the need for a broker in the middle. In the next articles, I’ll increase the complexity starting to use non default configuration files and exploring different way to connect routers with clients and brokers.

M2Mqtt : another moving … now under the official Eclipse Paho GitHub repo !

eclipse_github_repo

As you know they’re about 2 years that my M2Mqtt library is under the Eclipse Paho project as the official and supported C#, .Net (desktop, compact and micro) and WinRT client for MQTT protocol.

It’s source code was first released on CodePlex and then moved on GitHub inside my personal account in the last year but today it’s official part of the Eclipse Paho GitHub account.

It means that in the next few days I’m going to delete it as my personal GitHub repository and you will find all updates and fixes in its new “home”. Of course, you have to use this new repository in order to open new issues.

His brother, the GnatMQ broker, will stay on my personal GitHub account.

I’ll remain the principal committer and contributor but as open source project, the real power will come from the community !