Integrating PHP Applications with Ecotone and RabbitMQ

Integration between PHP Applications (Services) can be really challenging, as we enter area where a lot of things make break and fail. It does not make it easier that often different Applications are owned by different Teams. And to create such integration we need to agree on way we communicate, so both side can understand each other.

The two most common ways of integrating are: “HTTP” or “Message Broker”. There are potential problems to consider when doing HTTP integration but discussing them is out of the scope of this Article. In this Article we will be focusing on the integration using Message Broker and more precisely RabbitMQ.

Integration using Message Broker may require a lot of knowledge about Messaging and Routing Patterns and can easily get complicated. Due to that it often happens that integration between Services becomes non-trivial task, which includes a lot of discussions, failed tries and changes, that often span over several days, or in worse scenarios weeks.

The problem with Service to Service integration using Message Brokers is, they can quickly become burden to maintain. As they easily get complicated and potential bugs may impact multiple Applications and Teams, therefore people become afraid of changing them anyhow.

The solution to this lies in working from higher level abstraction. Abstraction which is high enough, so we don’t need to deal with low level Routing Patterns directly in the Message Broker. The aim is to lower the entry barrier and make the integration easily understood and done. So the integration can be done within hours or even minutes. And this is the aim of the article, to provide you with knowledge and tooling to do this in PHP using Ecotone with RabbitMQ.

Yet to understand where Ecotone’s solution comes from, we need first to understand one fundamental difference — the difference between logical and physical part of the System.

Logical and Physical part of the System

While we integrate Services together, we either put focus on the Logical part or the Physical part of the system:

• The Logical part is the Business side of things. In here we discuss things using Business concepts like: “When Payment Was Processed in Payment Service then Shipping Service will deliver the Order”.
• The Physical part is about technical details. In here we discuss things using Message Broker specific concepts like: “We need to create Topic based Exchange named ‘software.public.payment.order’ and then publish Message with routing key ‘payment.ordered’ so the ‘order_shipping’ Queue can bind to this”.

The lower level abstraction we work in, the more focus we will put in the Physical part instead of the Logical one. This means we will invest more time into writing, maintaining and understanding code and configuration, than in understanding business side of the things.
This as a result can make us easily lose track of why we do given things, as our focus will go into details, not the higher level picture.

Ecotone provides higher level abstractions so we don’t need to deal with low level Message Broker concepts. It aims to push the focus on the Logical side of the code, which takes us from “how” to the to the “why” side of things. This a result help us spot business misconceptions and deliver business features much quicker.

Ecotone’s Distributed Bus

Each Service in Ecotone connects to Distribution Mechanism under given name: “shipping_service” or “payment_service”.
This where the logical part comes in, as we actually define naming for the business boundaries (Applications).

We will be using this name to communicate between Services, using Ecotone’s Distributed Bus. Distributed Bus provides higher level abstraction, which allow us to focus on logical part of the system, instead of low level Message Broker concepts.

With Distributed Bus we will be working with two different types of Messages: Commands and Events. The difference between them is important, and we will see why soon.

With Ecotone’s Distributed Bus, we won’t be in need to declare Exchanges, Queues and bindings. We will be working on business level and Ecotone will take care of those low level concepts for us.

Let’s kick off now with sending Command Messages via Distributed Bus.

Enable Ecotone’s Distribution Mechanism

To start sending Messages (Command and Events) via Distributed Bus, we first need to enable it. We will enable it for RabbitMQ using Ecotone’s ServiceContext configuration:

This as a result will register DistributedBus in our Dependency Container, which we can start using right away.

To receive Messages from Distributed Bus, we want to enable Distributed Consumer:

This as a result will register new Message Consumer (Worker process), that we can run using “ecotone:run” Console Command. Name of the Message Consumer will be equal to our Service Name defined earlier:

Symfony:

bin/console ecotone:run payment_service

Laravel:

php artisan ecotone:run payment_service

By enabling given Service as Message Publisher, Message Consumer or both, we state how given Service will interact with the rest of the System. This make it clear for everyone what is the role of given Service.

Sending an Command

Suppose we are working in “order_service” that process new Orders. After Order is received we want to take a Payment. To take payment we will be separate Service “payment_service”.
When we want to trigger action on given Service, we will using Commands.

Commands carry intention of triggering given business action in specific Service.

If you’re unfamilar with concepts of Commands, you may get more details in this article.

Let’s stop for a moment and give a thought to the above diagram.
On the diagram we’ve two logical points which helps us answer following questions:

1. Where do we want to send Command to? — We send the Command to Payment Service
2. What action do we want to trigger there? — We want to trigger taking an Payment

This is crucial information, as it states how and with what boundaries do we interact.

When using higher level diagrams we talk in Business terms like Boundaries (Context/Service) and Business Actions (Commands). Yet it’s important that we don’t need to carry diagrams with us to understand the code, the code itself should tell the story.

Let’s actually make it it happen now using Ecotone’s Distributed Bus:

The code answers the same questions as above diagram, therefore we don’t need it to understand higher level perspective. We can easily understand that we are sending an Command to Payment Service in order to Take the Payment.
After Distributed Bus is triggered, Command Message will be sent to “payment_service”.

Diagrams like to get outdated, yet the code always represents how the System works at this very moment. So having code that clearly states how we interact with other Services is just pure gold.

Receiving an Command

We can now receive the Command in Payment Service, let’s register Distributed Command Handler then:

We are using CommandHandler and Distributed attributes here, which does following things:

• By marking given method with CommandHandler attribute, we enable it to be triggered by local CommandBus
• By adding Distributed attribute we state that this Command Handler should be available for DistributedBus too.

From now on this Command Handler will be available for distributed communication using “payment.take” routing key.
This is all we need to do to communicate between Services using Ecotone.

When we mark given Message Handler as Distributed, we connect it to Distribution Mechanism. Yet what is also important is that we make the code explicit, that given Handler can be executed by other Services. Therefore it becomes clear for everyone in the Team that this is an Distributed Command.

How does Commands works another the hood

We won’t be writing integration with RabbitMQ directly, as we are working from higher level code. However it’s still worth to see how does the physical part work under the hood, so it’s clear for us what’s happening.

When we send an Command, we are actually sending Message with target Service Name as routing key:

Command always targets single Command Handler, which on higher level means single Service. Therefore Ecotone always routes Commands based on the Service Name, which ensures only single Service will receive it.

When Service Connects to Ecotone’s Distributed Mechanism as Consumer, it will automatically create an Queue which will be bound by the Service Name:

This means that whenever Command will be sent to Payment Service, it will be delivered to this Service’s Queue.
Then when Message is consumed from Payment Service Queue, it will trigger our Distributed Command Handler.

Publishing Event Messages

So far we’ve discussed Commands, yet there is second type of Message, which is Event Message. Events instead of being sent to specific Service are published, and whoever is interested may subscribe to it. Therefore Event Message can be delivered to multiple Services.

If you’re unfamilar with concepts of Events, you may get more details in this article.

Suppose as a result of Successful payment, we want to deliver the Order to the Customer. Payments are handled by “payment_service” and delivery is done by “shipping_service”

Under the hood when we publish Event Message, we are sending an Message to the Ecotone’s Distributed Exchange with provided routing key:

Subscribing to Event Messages

Subscribing to Distributed Event is pretty straightforward. We provide EventHandler with routing key name and Distributed attribute.

Under the hood, Ecotone binds our Service’s Queue with given routing key:

That would be all to communicate using Command and Events, using Ecotone is pretty straight forward and this is how it should be, if we want to focus on the logical part of the sytem. How to install Ecotone Distributed Module, you can read at documentation page.

Now, we can take a look on few different scenarios which are often discussed when we start to do distributed communication.

Keeping the Events Private

Often Systems do not make distributed Communication explicit. In those situations external Services simply bind to our Events, sometimes without any control from our side, if given Event is meant to be exposed or not.

As a result we can easily lose track what and how is being consumed by External Services and this brings explicit Service Boundaries down:

1. Accidencial breaking other System — If other Service can bind to our internal Events directly, this means it becomes Consumer of our Events. If we change our Event structure, we may now accidentally break external Service.
2. Lack of modernisation — Our Events became public Events, which means we are no longer in full ownership of those. As a result we now need to discuss and consult what we can actually change. Which often result in people not willing to make changes to the Events, as it’s basically takes too much time.
3. Logical part is lost — Discussing boundaries, Service to Service communication using Business language is often lost or hard to follow.
   We’ve dived into low level programming where business concepts do not take place.

With Distributed Bus on other hand, things are made explicit and the Service boundaries are being respected. We explicitly state what we want to publish outside and what we want to keep private. This make it clear for everyone in the Team what lives on the edges of the boundary and what is kept within.

With Ecotone distinction between Public and Private Events is clear. We will be explicitly stating what is leaving our Service boundary using DistributedBus.

Distributing all the Events

There are many systems that publish all Events outside by default. This is not recommended as in inherit problems described above, yet it may be needed if our System is already working like that.

When migrating from legacy System to Ecotone we may want to preserve this behaviour to avoid bigger changes. In those situations, we most likely use some kind of Event Bus. In those situations we can replace our current Event Bus with Ecotone’s Event Bus, that publish Events internally:

And then simply subscribe to “object” which means subscribe to all Events to distribute them:

Instead of object, we could pass here an Interface that given set of Events are implementing:

or union type of Event Classes:

Distributing all the Events is far from ideal as Events may fly over Message Broker without any meaning, as nobody really subscribe to them. Besides that any changes to Event structure may result in failure in other Service. Therefore it’s better to keep distinction on private and public Events.

Private vs Public Events

In general it’s good to follow distinction between Distributed Events (Public) and ones meant to he handled on the single Service level (Private).
This way will know which Events are safe to change as they are used internally only and which need special attention before changing.

With Ecotone Public Events are explicit, as those are the one that goes via Distributed Bus.

During Event Distribution we can do one more step and provide custom structure for our Public Event. This way our internal Event structure will be fully decoupled from external Services:

Decoupled Message Classes

In some Frameworks we are bound to use the same Class on the publishing and consuming side. This means in order to deserialize Event or Command, we need to have same Class with the same name and namespace in each participating Service.
This is far from ideal as it creates hard coupling between Services, which can be easily broken if class name is changed.

Ecotone promotes decoupled communcation between Services. It delivers Messages based on routing keys, not Classes.

In Ecotone, Message is delivered based on the routing, not the Class. The class to which we should deserialize is determined just before Message Handler execution, based method’s parameters.
Therefore even if the Classes are named differently in each Service, we will be able to deserialize it. This go further, as we don’t even need to use Classes as we can deserialize to Arrays instead:

The same way Publishing side is decoupled, and we can use whatever type we want, for example, for example array:

If you want find out more about Decoupling your Services, you can read one of my previous article on the matter.

Ecotone decouples Services from each other by delivering Messages based on routing keys instead of Classes. This way we avoid hard dependency as Services can deserialize given Events to the form that actually make sense in given Context.

Subscribing to more Events at once

If given Service is meant to subscribe to larger portion of Events, we can use star (*) for that:

This way we subscribe to all events having prefix “billing.”.
It will include: “billing.order_charged”, “billing.refund.made”.

Failure mode

In case of problems with processing given Message, Ecotone provides Error Handling. The Error Handling works exactly the same, as Service level asynchronous processing.
In case of Exception, depending on the configuration it will either block processing or use delayed retries:

After delay retries are exceeded we can either drop the Message or store it in Dead Letter Database:

If you want to read more about Retries and Dead Letter, you can check documentation page.

Providing Custom Error Mechanism

If we want we can fully take over the process of Error Handling by defining our custom Error Channel:

and then we can hook in using Service Activator:

Missing Command Handler

It may happen that Command routing key have actually been changed, or given Command Handler was simply dropped. We want to ensure that in those situation Message won’t be simply dropped or ignored, as this is potential bug which need to be fixed and Data preserved.

As Commands are routed by Service Name, Message will be delivered to target Service even if routing have been changed on target Service.
In those situations Ecotone will kick off failure mode, which store given Command in Dead Letter.

Distributing Events Safely with Outbox Pattern

When we send Messages to RabbitMQ and storing changes in the database within single action we may end up in inconsistent state. This is because we are doing changes on two storages at the same time, where one of them mail fail:

In the code it would like this:

To solve this we can use Ecotone’s inbuilt feature to send Messages over Database:

To do this we will first publish the Event internally using Ecotone’s Event Bus.

As each Command Handler is wrapped in Database Transaction by default, Order and Message will be committed together:
And then we subscribe to internal Event and distribute it away:

In order for this Event Handler to store the Message in Database, we need to define “orders” as Database Message Channel:

If you want to read more about Resilient Communication, you may one of the previous articles.

Subscribing to Private Events and then republishing them as Public is not only about safe Publishing, as it also creates space for decoupling. In this place we can remap private Event into Public Event with different structure.

Sending Metadata

There may a cases where together with Command or Event we would like to send Metadata. Those may be details like Executor Id, Timestamp or event HTTP Domain from which the request for action came.

Those details mostly do not matter from Message Handling perspective, yet may be crucial for auditing, debugging or side effect which are triggered later in the flow. Putting them directly in Command or Event may blur their purpose and be cumbersome to be passed around.

Ecotone solves that by making Metadata first class citizen, which is passed together with the Command and Event:

Then we can access it directly in Distributed Event Handlers:

Metadata and propagation are crucial in Message-Driven Systems and Ecotone supports much more than it’s shown here. If you want to explore the topic in depth, I recommend you to read Multi-Tenant in Laravel or Multi-Tenant in Symfony where this topic was described in much more details.

Separate Queues and Processing

So far we have been discussing handling Distributed Messages in context of single Message Queue. However in large scale systems we may actually want to process some of the Messages separately or with higher priority than the others.

Suppose we want to scale separately Message Consumers related to Taken Payments and Failed Payments. We could then separate those into different Message Channels (Queues):

In the code on the Service Level it will looks like this:

And then we define “taken_payments” and “failed_payments” Message Channels using Service Context configuration:

This would allows us to scale Message Consumers for taken payment independently from failed payments and treat Distributed Queue just as proxy.

Message Channels can have multiple implementations, therefore we could use RabbitMQ for Message Distributed, yet use Amazon SQS or Redis internally. It’s up to us what works best in given context.

Summary

The lower level code we work in, the higher trade off we will have to put on the logical part of the system. As focusing on the technical details moves as away from focusing on the business part.

Ecotone on other hand gives us tools to work from higher level abstraction, so we can deliver quicker, with less configuration and higher business focus. Business focused do not need to be forced, as if we spend much less time on the integration, we will naturally shift our focus on the business parts. This way we create environment where people produce high quality software that is aligned with business needs.

The example implementation of Distributed communication in this repository and example of full Application written in Symfony and Laravel integrated via Distributed Bus can be found here.