M2M

Internet of Things : reactive and asynchronous with Vert.x !

vertx_iot

I have to admit … before joining Red Hat I didn’t know about the Eclipse Vert.x project but it took me few days to fall in love with it !

For the other developers who don’t know what Vert.x is, the best definition is …

… a toolkit to build distributed and reactive systems on top of the JVM using an asynchronous non blocking development model

The first big thing is related to develop a reactive system using Vert.x which means :

  • Responsive : the system responds in an acceptable time;
  • Elastic : the system can scale up and scale down;
  • Resilient : the system is designed to handle failures gracefully;
  • Asynchronous : the interaction with the system is achieved using asynchronous messages;

The other big thing is related to use an asynchronous non blocking development model which doesn’t mean to be multi-threading but thanks to the non blocking I/O (i.e. for handling network, file system, …) and callbacks system, it’s possible to handle a huge numbers of events per second using a single thread (aka “event loop”).

You can find a lot of material on the official web site in order to better understand what Vert.x is and all its main features; it’s not my objective to explain it in this very short article that is mostly … you guess … messaging and IoT oriented  🙂

In my opinion, all the above features make Vert.x a great toolkit for building Internet of Things applications where being reactive and asynchronous is a “must” in order to handle millions of connections from devices and all the messages ingested from them.

Vert.x and the Internet of Things

As a toolkit, so made of different components, what are the ones provided by Vert.x and useful to IoT ?

Starting from the Vert.x Core component, there is support for both versions of HTTP protocol so 1.1 and 2.0 in order to develop an HTTP server which can expose a RESTful API to the devices. Today , a lot of web and mobile developers prefer to use this protocol for building their IoT solution leveraging on the deep knowledge they have about the HTTP protocol.

Regarding more IoT oriented protocols, there is the Vert.x MQTT server component which doesn’t provide a full broker but exposes an API that a developer can use in order to handle incoming connections and messages from remote MQTT clients and then building the business logic on top of it, so for example developing a real broker or executing protocol translation (i.e. to/from plain TCP,to/from the Vert.x Event Bus,to/from HTTP,to/from AMQP and so on). The API raises all events related to the connection request from a remote MQTT client and all subsequent incoming messages; at same time, the API provides the way to reply to the remote endpoint. The developer doesn’t need to know how MQTT works on the wire in terms of encoding/decoding messages.

Related to the AMQP 1.0 protocol there are the Vert.x Proton and the AMQP bridge components. The first one provides a thin wrapper around the Apache Qpid Proton engine and can be used for interacting with AMQP based messaging systems as clients (sender and receiver) but even developing a server. The last one provides a bridge between the protocol and the Vert.x Event Bus mostly used for communication between deployed Vert.x verticles. Thanks to this bridge, verticles can interact with AMQP components in a simple way.

Last but not least, the Vert.x Kafka client component which provides access to Apache Kafka for sending and consuming messages from topics and related partitions. A lot of IoT scenarios leverage on Apache Kafka in order to have an ingestion system capable of handling million messages per second.

Conclusion

The current Vert.x code base provides quite interesting components for developing IoT solutions which are already available in the current 3.3.3 version (see Vert.x Proton and AMQP bridge) and that will be available soon in the future 3.4.0 version (see MQTT server and Kafka client). Of course, you don’t need to wait for their official release because, even if under development, you can already adopt these components and provide your feedback to the community.

This ecosystem will grow in the future and Vert.x will be a leading actor in the IoT applications world based on a microservices architecture !

Advertisements

M2Mqtt and GnatMQ are dead ? Long life to them !

In the last months a lot of people asked me if the M2Mqtt and GnatMQ projects are dead because there were no commits for a long time.

I can say … no ! They are not absolutely dead !

I was very busy in the last months, starting my new exciting Red Hat career in the messaging and IoT team, but I want to continue to support the projects as lead and main committer. Of course, I’ll be still busy with my new job but my commitment is to continue to improve the libraries, fixing bugs and adding new features. Just remember that I’ll do it in my spare time so I could not be so quick to reply your suggestions; of course your help with useful pull requests will be great !

As you now, today the M2Mqtt client library is under the official Paho GitHub repo and I have finally deleted the repo on my personal GitHub account.

The GnatMQ, based on M2Mqtt, will continue to live where it’s now here.

I’ll do my best to check the issues and the pull requests provided by the community, happy to know that my projects have a lot of followers and there are a lot of professional solutions using them out there !

As always I invite you to share with me your experience using my libraries in order to mention your solution in the “case studies” section of the official web site.

Thanks !

M2Mqtt : another moving … now under the official Eclipse Paho GitHub repo !

eclipse_github_repo

As you know they’re about 2 years that my M2Mqtt library is under the Eclipse Paho project as the official and supported C#, .Net (desktop, compact and micro) and WinRT client for MQTT protocol.

It’s source code was first released on CodePlex and then moved on GitHub inside my personal account in the last year but today it’s official part of the Eclipse Paho GitHub account.

It means that in the next few days I’m going to delete it as my personal GitHub repository and you will find all updates and fixes in its new “home”. Of course, you have to use this new repository in order to open new issues.

His brother, the GnatMQ broker, will stay on my personal GitHub account.

I’ll remain the principal committer and contributor but as open source project, the real power will come from the community !

M2Mqtt and Amazon AWS IoT

Few weeks ago I wrote about the new M2Mqtt feature : TLS client authentication support !

One of the M2Mqtt community friends, Nick Payne, has already used this new feature to connect to Amazon IoT platform. He wrote an article for us to show how to use the M2Mqtt client to connect AWS IoT !

The article is hosted on the official M2Mqtt web site and you can read it here. It’s a well written walk through all the needed steps to setup AWS IoT stuff and source code for connecting to it using my MQTT client.

A must read !

TLS client authentication for M2Mqtt !

Today, the new M2Mqtt 4.3.0.0 version was released !

It has “only one” great improvement … the TLS client authentication support !

Now, you can use a client certificate to authenticate the client itself during the SSL/TLS handshake with the broker. A big IoT platform like the Amazon AWS IoT uses it and for this reason, we support this authentication now !

You can connect your IoT device MQTT based to AWS IoT thanks to the new TLS client authentication feature.

The release is available as source code under the Eclipse Paho project, on GitHub and as binaries on Nuget.

WPC 2015 Milan : Azure IoT Hub and IoT Suite

wpc2015

Organized by Overnet, in collaboration with Microsoft, WPC is the most important italian conference focused on Microsoft technologies. This year it will be covered in two full immersion days on December 1st and 2nd with 70 sessions in 8 tracks.

I’m honoured to be part of the speakers team this year as Microsoft MVP on Windows Embedded and IoT; on December 2nd, I’ll have a session about Microsoft Azure IoT Hub with an overview of the new Azure cloud gateway and the related Azure IoT Suite.

For sure, the conference will be great for contents and networking with all experts about Microsoft technologies. Don’t forget the “Ask The Expert” corner with a “bunch” of Microsoft MVPs ready to answer your questions.

All information and details about the registration and the conference on the official web site.

AMQP isn’t so scary … if you know how to start !

One of the first class citizens in the “IoT protocols city” is the AMQP (Advanced Message Queue Protocol) protocol that is so popular even if it seems that there are few good resources to start understanding how it works under the cover (after a few searching on Google and Bing). Of course, for each protocol the most complete resource is the official specification but more people consider it very difficult to read (if they don’t need to implement it).

To simplify your understanding, I’d like to wrap up some resources useful to you to start studying AMQP protocol.

A good starting point is an old but very useful article by Kelly Sommers (aka “kellabyte” on Twitter) who explains how the AMQP 1.0 specification is completely different from the old AMQP 0.9.1 specification : Clarifying AMQP.

For an overview at high level, I found the articles posted by Chuck Rolke (from Red Hat) on his blog very interesting :

To deep into the protocol about message framing and type system you can read my articles :

The great thing is that all examples related to the above blog posts use the AMQP .Net Lite library (an AMQP C# implementation) from Microsoft as client.

Few articles … but very useful.

Of course, I read the official specification because in my humble opinion it’s always the best resource and I’d like to understand how the things work under the cover (in my case it’s all related to Azure Service Bus used in the IoT space that support AMQP protocol).

Last but not least, there is a great discussion on Channel9 about AMQP 1.0 and its general availability in Microsoft Azure Service Bus (it was 2013) between Clemens Vasters and David Ingham that we can consider the AMQP gurus without any doubts.

If you have other resources to enrich the above list, please don’t hesitate to contact me !

[Update October 2nd 2015]

Very nice and interesting article titled AMQP as network protocol by Ted Ross (from RedHat) with a brief overview on the protocol, multiplexing, flow control and delivery features.

Clemens Vasters (from the Microsoft Azure Messaging team) is a guru on AMQP protocol and today he published the great presentation Introduction to AMQP 1.0 with an in depth analysis about the architecture and all the related features. It’s a must read for understanding how the protocol works under the hood on the wire.

[Update October 5th 2015]

After its great slide deck about AMQP 1.0 protocol, today Clemens Vasters published a great video series on his Subscribe! blog on Channel9. The video series has the following 6 parts :

If you prefer, the videos are also available as playlist on YouTube here.

What can I say ? No words … it’s Clemens’ stuff 😉

[Update January 18th 2016]

Today my “AMQP Essentials” cheat sheet was published on DZone web site as refcard. It’s a lightweight introduction to the AMQP protocol with all main features in a single short PDF file (6 pages).