Speaking about the AMQP (Advanced Message Queuing Protocol) protocol, a lot of people think that it’s only about messaging between clients and brokers. There are a bunch of MOMs (Message Oriented Middleware) which support this protocol to enable clients to send and receive messages so to exchange data each other. We can think about open source projects like the Apache ActiveMQ and Apache ActiveMQ Artemis brokers or commercial products from big companies like JBoss A-MQ (Red Hat), Azure Service Bus (Microsoft) and MQ Light (IBM).
Not all people know that AMQP is also a peer to peer protocol which enables clients to communicate each other without a messaging broker in the middle. Furthermore, we can use this protocol in order to connect clients to a server : more generally speaking, an AMQP endpoint can be a “pure” client (which starts a connection) or a server (which listens for connections).
The typical MOM features
An AMQP server seems to be exactly what a broker is but it’s not so true. Of course, a broker listens for connection requests from clients so acts as a server but it’s a MOM and adds a bunch of concepts. First of all, it provides a “store and forward” capability because every message received from producers is stored in a queue or a topic/subscription entity and after that it’s ready to be delivered to consumers which get messages at their own pace. It provides asynchronous communication between them decoupling senders and receivers and the messages can be stored in memory or as usual in a permanent storage (like a database or a log file).
What does it mean in terms of message acknowledgement having a queue or topic/subscription between a producer and a consumer ? Basically, it means that the producer knows when the message is arrived to the broker but not when it’s consumed by the consumer (it could be offline or it’s consuming messages at a different pace). In order to have this information, it’s needed to setup a “reply to” queue where the consumer can put an acknowledge message for the producer : if you think about that, also in that case the consumer knows that the acknowledge message is arrived to the broker but not yet to the producer.
Of course, it’s the typical scenario in the “integration business” where different enterprise systems exchange data in such a way. It’s also true that in the new “Internet of Things business”, the devices in the field and the services in the cloud communicate in that way for telemetry and command/control purposes.
Sometimes this kind of communication isn’t acceptable and the producer wants to know exactly when its message is arrived to destination (consumer) without another entity in the middle, the broker, which takes responsibility on messages delivery. Furthermore, there are other patterns that aren’t possible using brokers only.
Qpid Dispatch Router : what is this ?
In order to add more and different features in an integration and/or IoT scenario made of clients, servers and brokers, the Apache Foundation developed a new interesting project under the Qpid umbrella : the Qpid Dispatch Router.
As the official documentation says, “The router is an intermediary for messages but it is not a broker. It does not take responsibility for messages”. When a producer sends a message to a broker, it takes responsibility for it and will deliver the message itself when a consumer will ask for it. The router simply propagates the AMQP traffic between nodes and it means that if producer and consumer are directly connected (in a peer to peer fashion) through the router, they are the only ones responsible for message acknowledgement.
Other than in “stand-alone” mode, the best configuration for using router is in a “network of routers”; let’s consider more routers connected each other in a specific topology : clients can connect on that network asking to send/receive on specific AMQP addresses and communicate through routers that can choose the best path for message delivery (algorithms similar to OSPF or IS-IS from the networking world are used). It’s possible thanks to their internal “routing tables” which are constantly updated using a specific router protocol. All the routers exchanges updating/hello messages about a new router connected (or about a router went offline) and new clients attached/detached to specific addresses. In that way, the network can change dynamically in terms of routers connections and paths between clients for sending/receiving messages on different/new addresses.
Of course, a network of routers is also useful for connecting to one or more brokers/servers : thanks to routers, a broker/server could handle a few TCP connections in order to communicate with a huge number of clients. It’s possible because we can use few routers directly connected to the broker/server and having a lot of clients connected to such routers (that could be part of a huge network). These clients aren’t directly connected to the broker/server but can communicate with it through the routers; it determines a reduction of connections from clients to broker/server.
Another great advantage is to expose a server/broker outside of a private network. Without routers, in the above scenario we need to enable port traversal from outside our NAT firewall to allow external clients to connect to server/broker inside the LAN. It means enabling incoming connection that is not so good due to security concerns. Using a router inside the private network, it’s able to connect to the server/broker (they are in the same LAN) and to connect to an external router (outside the LAN) : in that way we don’t need to open any port on NAT firewall, we have an outgoing connection from the router inside the LAN. All clients can connect to the external router and having their traffic routed to the server/broker into the private network leveraging on connection between the routers.
Conclusion
Using AMQP routing could be considered as “different” way to do messaging or as a new possibility for adding features and patterns to the typical MOM scenarios. In the next articles we’ll dig into it with some examples starting from a simple stand-alone router to a network of routers; we’ll see the available configurations for enabling clients/servers/brokers communication through them.
DEVEXPERIENCE 