Month: April 2016

Qpid Dispatch Router installation on your Linux machine

In the previous article I introduced the Dispatch Router from the Apache Qpid project, its main features, capabilities and the scenarios where it could be useful in order to develop high scalable AMQP based messaging solutions.

The first step for starting to use the router is the installation step and I’ll explain how to do that in this short post. I’ll use some personal Docker files in order to build fully functional images but the you can find the official ones in the router GitHub account here.

Qpid Proton : the dispatch router foundation

This router is based on the Qpid Proton project, a messaging library developed in C, C++ and Java (pure ProtonJ implementation) with bindings for other different languages like Python, PHP and Ruby; in order to work properly, the router needs only the base ProtonC implementation and the Python binding.

First of all, we need the main compiler tools like gcc, cmake and make. The UUID library is needed for unique identifier generation (i.e. container name, message id, …) and the OpenSSL library for encryption and for handling SSL/TLS connections. Furthermore, the last library version leverages on the Cyrus library for the SASL protocol used for supporting different authentication mechanisms on the AMQP protocol.

To simplify the installation process, I wrote two Docker files available here on GitHub both for Fedora and Ubuntu you can use as reference for reproducing the steps on your machine or to generate a Docker image.

Now it’s router time

The Qpid Dispatch Router uses the same tools (like gcc, cmake, make, …) needed for Qpid Proton; if you can compile the messaging library without problems then you are ready to compile and install the router as well.

At time I’m writing this article the official released version is the 0.5 with the 0.6 version under beta. We can use both this versions : in the former case we can download the released package, in the latter we can clone the official GitHub repository and recompile the bits from there having in this way the latest updated code and features under development. The new 0.6 version has gone through major architectural changes to make it highly scalable; furthermore, the router configuration is now more intuitive and easy to understand than the previous versions.

Even in that case I wrote both Docker files for the official released version and for compiling the bits (on both Fedora and Ubuntu of course) in order to have a Docker image and the router up and running in a related container.

Let’s start the router

To check the installation process we can start the router with the simple “qdrouterd” command; it is launched automatically in the case of starting the Docker image.

qdrouterd

Typing that command, the router starts with the default configuration file. We will dig into in the next articles to understand the meaning of all main available configuration options for tuning the router behavior.

In the above output, the main points are :

  • The router instance is named Qpid.Dispatch.Router.A;
  • The router operates in “standalone” mode which means that it doesn’t cooperate with other routers and won’t be used in a routers network;
  • It exposes a management endpoint we can use in order to interact with the router itself and changing its internal configuration. It’s a pure AMQP endpoint on which the available operations are defined by the AMQP management specification (which is in draft, here). For all developers who don’t know it well, you can think that as a sort of RESTful interface with CRUD operations for managing resources but instead of having HTTP as transport protocol, it used AMQP and its semantics;
  • A listener is started on all network available interfaces and listening for connections on the standard AMQP port (5672, so not encrypted);
  • The instance is using 4 threads for handling messages traffic and all other internal operations;

The package tools

Other then the router itself, the project provides a couple of tools useful for showing main running information and for interacting with it.

The “qdstat” tool shows all statistics information, endpoints and router traffic.

qdstat

As we can see from the above picture (-l option is for showing router AMQP links), the router is exposing the $management endpoint and a local temporary endpoint that is used for communicating with the qdstat tool.

Indeed, when we start the tool, it opens an AMQP connection to the router and attaches a link on the management endpoint for sending requests as you can see in the following picture with traffic capture (using Wireshark).

qdstat_management

At same time another link is attached for a temporary local endpoint used for receiving information to show on the console.

qdstat_temp_endpoint

When the tool sends a request to the router through a transfer performative, it specifies the local reply address using the “Reply-To” AMQP property.

The “qdmanage” tool is a general purpose AMQP management tool that we can use to interact with a running router and changing its internal configuration. It’s very useful because even if our router starts with a static configuration written in the related file, we can change it dynamically at runtime to adapt the router behavior to our ongoing use case.

For example we can show all the active connections on the router.

qdmanage

In this case, there are no connections other than the one related to the tool itself which connects to the router via AMQP in order to query and receive the related information.

Conclusion

With this post now we have installed the dispatch router stuff on our machine (or on a Docker image) and checked that it runs properly with the default configuration at least. We peeked the tools available with the installation and all related operations and information provided in order to interact with the router at runtime. Starting from the next article we’ll see how to use the router in different scenarios with an increasing complexity, from a standalone router to a routers network.

 

IoT developer survey : my point of view

Few days ago, the Eclipse Foundation published the report of the last IoT developer survey sponsored by the foundation itself with IEEE IoT and Agile IoT. This survey has as main objective to understand what are the preferred technologies used by developers in terms of languages, standards and operating systems; furthermore, it shows what are the main concerns about IoT and how companies are shipping IoT solutions today.

Great content about this report was published by Ian Skerrett (Vice President of Marketing at Eclipse Foundation) on his blog and on slideshare with a summary of all main information about it.

I’d like just to add my 2 cents and doing some absolutely personal considerations about the results …

Companies are investing …

Regarding how companies are delivering IoT solutions, it’s clear that the IoT market is growing. A lot of companies already have IoT products in the fields and the others are planning to develop them in the coming months. It’s not a surprise, other than a buzzword, the IoT is a real business opportunity for all companies strictly related to the embedded devices (silicon vendors, OEMs, ..) or software companies (for the cloud and application side) which are rapidly change how their business is made.

Security and interoperability : the big concerns

The result related to the main concerns about IoT is very clear : people and companies are worried about the security. All data flowing from our personal life or owned by companies to the cloud need to be protected in order to avoid someone can steal them. The concern about security is strictly related to software protocols (i.e. SSL/TLS, …) and hardware stuff (i.e. cryptochip, …) and today it seems that a very good solution isn’t available. The same is for interoperability : having a lot of IoT standard protocols means having NO standard protocols. A lot of consortiums are trying to define some standard specifications and frameworks in order to define a standard but … they are too much; all big companies are divided in different consortiums and some of them are part of more then one : this is a big deal, as for protocols … it means NO standard.

Developer prefer Java and C … what about JavaScript ?

It’s not a surprise the first place of Java as preferred language and C as second one : Java is used in a lot of cloud solutions which are based on open source products and C is the better language for developing on devices side with great performance at low cost (at least from an hardware point of view). First strange position is about JavaScript as third most used language : I hope this position is related to its huge usage with NodeJS on server side and not as “embedded” language on devices … I’m scared about that.

Protocols : the current know-how is leading

Now, the protocols …

Having HTTP/1.1 as first used protocol is real because today it’s the only well known protocol in the developers world; in order to develop and deliver an IoT solution with a quick time to market, companies leverage on internal know-how and sometimes they don’t invest to figure out how other protocols work and if they have other advantages. It explains to me this position, thanks to HTTP/1.1 simplicity and its ASCII/text based nature : a lot of developers don’t like binary format so much. Last point is that the REST architecture is a very good solution in a lot of scenarios and HTTP/1.1 is the most used protocol (the only one ?) for that.

MQTT and CoAP are used a lot thanks to the available open source projects and their simplicity; MQTT is very lightweight and works great on tiny embedded devices, CoAP tries to overcome some HTTP/1.1 disadvantages (i.e. server push, observer, …) with new features and its binary nature.

A lot of developers are scared about AMQP because I have to admit it’s not so simple like the previous ones but it’s powerful and everyone should give it a try. If you want to start with it, you can find a lot of links and resources here.

I’m surprised by the fourth position of HTTP/2.0 ! I mean … how many developers know, love and use HTTP/2 today ? I was surprised by this high position … I expected it behind “in-house, proprietary”, AMQP and XMPP. I suppose that companies are prototyping solutions using this protocol because they think that thanks to the HTTP/1.1 knowledge it’s quite simple to move to the next version : I think it’s totally wrong, because HTTP/2.0 is completely different from HTTP/1.1. I love it … I’ll invest in it.

OS : Linux and RTOS on bare metal

Regarding operating systems, the first position for Linux isn’t a surprise but we have to consider it both on server side and devices side (even if embedded devices based on Linux are a lot). The other OS are only for embedded devices (low constrained devices) so the percentages don’t have any help from cloud side. Finally Linux is useful for IoT gateway too (as we know with Kura) even if Microsoft, for example, is investing in its Windows IoT Core and will release an IoT Gateway SDK in the next months.

All the services in the cloud

Not a surprise Amazon AWS with its first position as Cloud services provider but I don’t think about their relatively new AWS IoT platform but all the IoT open source stuff that developers prefer to run on Amazon VMs than Azure VMs.

Conclusion

Here the great news is that IoT market is growing and developers/companies are investing in it to try to be on the market as soon as possible. The “bad” news is that too much different protocols and frameworks are used and the way to interoperability and interconnection is quite long or … infinite ?