MQTT

Vert.x and IoT in Rome : what a meetup !

Yesterday I had a great day in Rome for a meetup hosted by Meet{cast} (powered by dotnetpodcast community) and Codemotion, speaking about Vert.x and how we can use it for developing “end to end” Internet of Things solutions.

17352445_10208955590111131_6229030843024604532_n

17352567_10208955588791098_766816304298598626_n

I started with an high level introduction on Vert.x and how it works, its internals and its main usage then I moved to dig into some specific components useful for developing IoT applications like the MQTT server, AMQP Proton and Kafka client.

17342690_10208955588751097_8818320599257580571_n

17352571_10208955588951102_2851165399929439718_n

It was interesting to know that even in Italy a lot of developers and companies are moving to use Vert.x for developing microservices based solutions. A lot of interesting questions came out … people seem to like it !

Finally, in order to prove the Vert.x usage in enterprise applications I showed two real use cases that today work thanks to the above components : Eclipse Hono and EnMasse. I had few time to explain better how EnMasse works in details, the Qpid Dispatch Router component in particular and for this reason I hope to have a future meetup on that, the AMQP router concept is quite new today ! In any case, knowing that such a scalable platform is based (even) on Vert.x was a great news for the attendees.

17264802_10208955590191133_8923182437405273553_n

If you are interested to know more about that, you can take a look to the slides and the demo. Following the link to the video of the meetup but only in Italian (my apologies for my English friends :-)). Hope you’ll enjoy the content !

Of course, I had some networking with attendees after the meetup and … with some beer 🙂

17310150_1421561734583219_8414988688301135801_o

“Reactive Internet of Things : the Vert.x way” … meetup in Rome !

vertx_iot

On March 16th I’ll be guest of the Meet{cast} and Codemotion community for a meetup in Rome speaking about … “Reactive Internet of Things : the Vert.x way”.

meetcastcodemotion

It’s a pleasure for me showing how the Vert.x toolkit can be used for developing Internet of Things solutions leveraging on the pillars of the reactive manifesto (responsive, elastic, resilient and asynchronous).

Starting from an introduction on what Vert.x is, what it provides and its main features, I’ll move to the messaging and IoT focused components that the toolkit offers. So we’ll see the new MQTT server and Kafka client (officially in the latest 3.4.0 Beta 1 release) and the well know AMQP Proton and Bridge components. Of course …. demos around them !

Finally, I’ll show how these components are already used today for enterprise IoT solutions introducing the Eclipse Hono project, for handling IoT connectivity, and the EnMasse which provides a Message as a Service platform. The great thing is that we’ll have the chance to see the code because … they are open source of course !

So … what are you waiting for … register for the meetup here ! See you in Rome 😉

Designing MQTT over AMQP

mqtt_over_amqp

Let’s think about what we can consider some weaknesses of the MQTT protocol …

  • it provides native publish/subscribe pattern only; having request/reply is quite cumbersome with “correlation injection” inside topics and payload messages;
  • it doesn’t provide flow control;
  • it doesn’t have metadata information inside messages (i.e. content type);
  • it’s fully broker centric;

but there are some unique features that we have to consider as strengths for this protocol as well …

  • retain message also known as “last well known” message;
  • last will testament (LWT);
  • session handling;

Now, let’s think about the main features that fill the gap for the MQTT protocol but provided by AMQP …

  • native support for both publish/subscribe and request/reply patterns (correlation for free);
  • flow control at different levels (with max window size and message credits);
  • a full type system and metadata information on each message;
  • peer-to-peer model (the broker is just a peer);

but it lacks of the above three MQTT features !

So how greater could be AMQP protocol having such features on top of it ?

Under the open source EnMasse project, I have been working on having a design (so a kind of “specification”) for having retain message, last will testament and session handling over AMQP. At same time I have been developing an MQTT “gateway” in order to allow remote MQTT clients to connect to an AMQP based “Message as a Service” like EnMasse.

Having such a design means that not only an MQTT client can leverage on receiving retain message after subscribing to a topic, sending its LWT on connection or receiving messages for a topic when it was offline; it means having the above features for native AMQP clients as well.

Each feature is made by a well defined AMQP message using specific subject, annotations and payload in order to bring all MQTT related information like retain flag, will QoS, will topic and so on but using AMQP semantic.

This sort of “specification” doesn’t force to use a specific implementation; the EnMasse project leverages on the Qpid Dispatch Router for connections and Apache Artemis brokers where state is needed (but other implementations could use something different like a simple file system or a database). Of course, some additional services are needed in order to handle LWT and subscriptions (we called them just “Will Service” and “Subscription Service”).

If you are so curious and want to give some feedback on that, you can find all the open source stuff on GitHub in the MQTT over AMQP documentation section.

Feel free to enjoy the related EnMasse implementation ! 😉

M2Mqtt and GnatMQ are dead ? Long life to them !

In the last months a lot of people asked me if the M2Mqtt and GnatMQ projects are dead because there were no commits for a long time.

I can say … no ! They are not absolutely dead !

I was very busy in the last months, starting my new exciting Red Hat career in the messaging and IoT team, but I want to continue to support the projects as lead and main committer. Of course, I’ll be still busy with my new job but my commitment is to continue to improve the libraries, fixing bugs and adding new features. Just remember that I’ll do it in my spare time so I could not be so quick to reply your suggestions; of course your help with useful pull requests will be great !

As you now, today the M2Mqtt client library is under the official Paho GitHub repo and I have finally deleted the repo on my personal GitHub account.

The GnatMQ, based on M2Mqtt, will continue to live where it’s now here.

I’ll do my best to check the issues and the pull requests provided by the community, happy to know that my projects have a lot of followers and there are a lot of professional solutions using them out there !

As always I invite you to share with me your experience using my libraries in order to mention your solution in the “case studies” section of the official web site.

Thanks !

M2Mqtt : another moving … now under the official Eclipse Paho GitHub repo !

eclipse_github_repo

As you know they’re about 2 years that my M2Mqtt library is under the Eclipse Paho project as the official and supported C#, .Net (desktop, compact and micro) and WinRT client for MQTT protocol.

It’s source code was first released on CodePlex and then moved on GitHub inside my personal account in the last year but today it’s official part of the Eclipse Paho GitHub account.

It means that in the next few days I’m going to delete it as my personal GitHub repository and you will find all updates and fixes in its new “home”. Of course, you have to use this new repository in order to open new issues.

His brother, the GnatMQ broker, will stay on my personal GitHub account.

I’ll remain the principal committer and contributor but as open source project, the real power will come from the community !

IoT developer survey : my point of view

Few days ago, the Eclipse Foundation published the report of the last IoT developer survey sponsored by the foundation itself with IEEE IoT and Agile IoT. This survey has as main objective to understand what are the preferred technologies used by developers in terms of languages, standards and operating systems; furthermore, it shows what are the main concerns about IoT and how companies are shipping IoT solutions today.

Great content about this report was published by Ian Skerrett (Vice President of Marketing at Eclipse Foundation) on his blog and on slideshare with a summary of all main information about it.

I’d like just to add my 2 cents and doing some absolutely personal considerations about the results …

Companies are investing …

Regarding how companies are delivering IoT solutions, it’s clear that the IoT market is growing. A lot of companies already have IoT products in the fields and the others are planning to develop them in the coming months. It’s not a surprise, other than a buzzword, the IoT is a real business opportunity for all companies strictly related to the embedded devices (silicon vendors, OEMs, ..) or software companies (for the cloud and application side) which are rapidly change how their business is made.

Security and interoperability : the big concerns

The result related to the main concerns about IoT is very clear : people and companies are worried about the security. All data flowing from our personal life or owned by companies to the cloud need to be protected in order to avoid someone can steal them. The concern about security is strictly related to software protocols (i.e. SSL/TLS, …) and hardware stuff (i.e. cryptochip, …) and today it seems that a very good solution isn’t available. The same is for interoperability : having a lot of IoT standard protocols means having NO standard protocols. A lot of consortiums are trying to define some standard specifications and frameworks in order to define a standard but … they are too much; all big companies are divided in different consortiums and some of them are part of more then one : this is a big deal, as for protocols … it means NO standard.

Developer prefer Java and C … what about JavaScript ?

It’s not a surprise the first place of Java as preferred language and C as second one : Java is used in a lot of cloud solutions which are based on open source products and C is the better language for developing on devices side with great performance at low cost (at least from an hardware point of view). First strange position is about JavaScript as third most used language : I hope this position is related to its huge usage with NodeJS on server side and not as “embedded” language on devices … I’m scared about that.

Protocols : the current know-how is leading

Now, the protocols …

Having HTTP/1.1 as first used protocol is real because today it’s the only well known protocol in the developers world; in order to develop and deliver an IoT solution with a quick time to market, companies leverage on internal know-how and sometimes they don’t invest to figure out how other protocols work and if they have other advantages. It explains to me this position, thanks to HTTP/1.1 simplicity and its ASCII/text based nature : a lot of developers don’t like binary format so much. Last point is that the REST architecture is a very good solution in a lot of scenarios and HTTP/1.1 is the most used protocol (the only one ?) for that.

MQTT and CoAP are used a lot thanks to the available open source projects and their simplicity; MQTT is very lightweight and works great on tiny embedded devices, CoAP tries to overcome some HTTP/1.1 disadvantages (i.e. server push, observer, …) with new features and its binary nature.

A lot of developers are scared about AMQP because I have to admit it’s not so simple like the previous ones but it’s powerful and everyone should give it a try. If you want to start with it, you can find a lot of links and resources here.

I’m surprised by the fourth position of HTTP/2.0 ! I mean … how many developers know, love and use HTTP/2 today ? I was surprised by this high position … I expected it behind “in-house, proprietary”, AMQP and XMPP. I suppose that companies are prototyping solutions using this protocol because they think that thanks to the HTTP/1.1 knowledge it’s quite simple to move to the next version : I think it’s totally wrong, because HTTP/2.0 is completely different from HTTP/1.1. I love it … I’ll invest in it.

OS : Linux and RTOS on bare metal

Regarding operating systems, the first position for Linux isn’t a surprise but we have to consider it both on server side and devices side (even if embedded devices based on Linux are a lot). The other OS are only for embedded devices (low constrained devices) so the percentages don’t have any help from cloud side. Finally Linux is useful for IoT gateway too (as we know with Kura) even if Microsoft, for example, is investing in its Windows IoT Core and will release an IoT Gateway SDK in the next months.

All the services in the cloud

Not a surprise Amazon AWS with its first position as Cloud services provider but I don’t think about their relatively new AWS IoT platform but all the IoT open source stuff that developers prefer to run on Amazon VMs than Azure VMs.

Conclusion

Here the great news is that IoT market is growing and developers/companies are investing in it to try to be on the market as soon as possible. The “bad” news is that too much different protocols and frameworks are used and the way to interoperability and interconnection is quite long or … infinite ?

Azure IoT Hub is GA : the news !

Yesterday, the Microsoft Azure IoT Hub was released in GA !

The public preview had a good success with a lot of people (makers) and companies (professional) try to use it for developing their IoT end to end solutions.

In a previous blog post, I have already discussed about its mean features with a comparison with AWS IoT, the Internet of Things platform by Amazon.

Relating to that article, there are the following differences it’s important to focus on :

  • Azure IoT Hub now supports MQTT 3.1.1 natively ! There is no need to use a field gateway for translating MQTT to AMQP (or HTTP) to communicate with the Hub. Now, your MQTT enabled devices can connect directly to the Cloud and you can use the SDK provided by Microsoft (with an API abstraction layer on top of MQTT) or any MQTT library (and M2Mqtt is a good choice for C# applications). Of course, the connection must be always encrypted with SSL/TLS protocol. More information at official documentation page here.
  • The pricing is changed : first of all, the pricing isn’t related to the number of devices (as the public preview) but only to the total number of messages/day. The bad news is that starting from April 1st the S1 and S2 plans will have a doubled price. Of course, the Free plan … will be still free !
  • AMQP over WebSockets : the AMQP protocol is supported on WebSockets too (like Event Hubs for example).

With the above two major news, the Azure IoT Hub offer is closer to AWS IoT offer : it supports MQTT and removed the devices limit on pricing.

News are not only on the Cloud side but on devices side too !

In the last months, a lot of OEMs and hardware companies worked hard to support Windows 10 for IoT Core and Azure IoT Hub connection on their platforms. Today the number of Azure Certified IoT Partners is literally increased !

 new_iothub_partners

It’s great to see that the Hub ecosystem is growing … now we have to wait for real IoT solutions based on it !

To start learning about Azure IoT Hub, I advice you the link to the Azure IoT Hub Learning Path which will guide you through all the steps needed to use the Hub in the best way.