Symfony Multi-Tenant Applications with Ecotone

How multi-tenancy is implemented depends on the business domain we work in. We may require shared database or a separate database for full isolation. We may have few Tenants or hundreds of them, we may need to throttle or speed up performance for given Tenant. All of this creates unique environment, in which Multi-Tenancy is not only a technical consideration, but also a Business one.

In my previous article I was describing how to build Multi-Tenant systems in Laravel with Ecotone with the least possible effort. And in this article we will do the same, yet for the Symfony Framework.

Scenarios in this Article will have Demos linked at the end of each section. This way we will not only discuss the example, but we will also be able to refer to executable demo.

This will be practical guide, after which you will know why and how you can apply Multi-Tenancy for different scenarios in your project. If you want to explore theory behind Multi-Tenancy, I highly encourage reading Michał Kurzeja Article first.

Sending Messages with Database per Tenant

Suppose we are in E-Commerce Domain and we’ve two Tenants where each has it’s own separate Database (DB per Tenant strategy).
First thing which need to happen in E-Commerce system is of course registration of new Customer, and this what we will focus on now.

The process of registering new Customer will go as follows:

We will be sending Register Customer Command Message to our Command Handler

We will send an Register Customer Command using Command Bus, to the Command Handler which will store new Customer in the database. The tricky part is that, we want to store the Customer in database related to given Tenant.

Let’s kick off by installing Ecotone for Symfony:

composer require ecotone/symfony-starter

This will provide us with Ecotone’s Symfony integration and Database supporting tooling.

Mapping Connection to Tenants

We will be using Doctrine ORM for our example. As each Tenant will have it’s own database connection, we will need to define Doctrine configuration for each Tenant first (doctrine.yaml).

When connections are defined, we can now setup how will they map to Tenant names. We do it using Ecotone’s configuration method marked with ServiceContext attribute.

This is basically it. Ecotone will now know how given Tenant name maps to given Connection. So whenever we will send any kind of Message (Command/Query/Event) it will know which connection should be used.

Multi-Tenant Command Handler

We will be using Ecotone’s CQRS for our Multi-Tenant System. This will provide us with great amount of inbuilt features, that we can use out of the box in Multi-Tenant Systems.

Let’s define our Register Customer Command Handler:

As you can see Command Handler is nothing special. It’s just an method which perform business logic marked with PHP Attribute. Our Command Handler takes an Command Class and stores the Customer using Doctrine ORM. This code would work in single Tenant environment just fine.

The tricky part however is that we need to use ObjectManager/EntityManager for specific Tenant, as each has it’s own Database Connection.
By adding #[MultiTenantObjectManager] Attribute we are telling Ecotone to inject ObjectManager for currently activated Tenant. This way we can store our Customer in correct Tenant’s Database, and we keep our code agnostic to Multi-Tenancy.

Ecotone make use of Attributes to provide Declarative Configuration, which keep our business code agnostic of Multi Tenant environment.
This way we can develop like there would be a single Tenant, yet deliver System that will work with Multi-Tenancy by default.

Let’s define RegisterCustomer Command Class:

Command Class is simple POPO (Plain Old PHP Object), it does not extend or implement any framework specific classes. Command contains all the data needed for Customer registration.

Ecotone will manage flushing and clearing our Object/Entity Managers by default after Command Handler is executed. This way our code is simplified as all we need to do it to persist given Entity and we are good to go.

Multi-Tenant Message Bus

After introducing Command Handler in our code base, we can now send Command to it for given Tenant.
We will be executing given Command in context of specific Tenant:

In here we are sending Command over Command Bus and passing Tenant name using metadata (Message Headers). This way Ecotone will understand that we performing given Command Handler in context of given Tenant’s database. Typically we would resolve Tenant name here, based on the HTTP Domain or User Session.

We’ve defined Command Handler for Multi-Tenancy, but we can do the same for Query Handlers (Responsible for fetching data) and Events. We will take a deeper look on Event Handlers in later part of the article.

This is basically all we need to store Customer in Multi-Tenant environment. Basically our code would work either for single or multiple Tenants, as it’s fully agnostic to Multi-Tenancy. Let’s now check more scenarios, which we may need in our Multi-Tenant Systems.

The demo implementation can be found under this link.

Shared and Multi Database Tenants

We may have business model where by default we put every Tenant in the same Database, yet if Customer will buy premium he will receive separate Database instance.

To handle such cases, Ecotone provides the default connection. This way, if there is no mapping for given Tenant name, default will be used:

Accessing Current Tenant in Message Handler

For specific scenarios we may need to be aware of Tenant’s context in which execution is done. For example given Tenant may have luxury Shop where delivery should happen right away after order is made, when for other Tenants time does not matter.

In case of Ecotone, whatever we send via Message Headers (Metadata) is accessible for us on the Message Handler level. This way depending on the need we can ignore or access given metadata. And as we send Tenant name via Message Headers, we can access it in case of need:

Header attribute states what Message Header we want to access. In our case we want to access tenant header, which we sent earlier via Command Bus.

We can access any Message Header in our Message Handlers. This means, whatever Metadata we will pass with Command/Query/Event (e.g. User Id, User Role, HTTP Domain from which request is made etc), we can then access it when needed.

Hooking into Tenant Switch

If we already have Multi-Tenant application running, most likely we are using some custom libraries or integration. In such cases, it may be required to trigger some code when given Tenant is activated or deactivated.

Ecotone opens possibility to hook into the process of Tenant switch, where it can provide Connection that is going to be activated and the Tenant name.

To hook in all we have to do it to mark given method with OnTenantActivation or OnTenantDeactivation, given methods will be triggered following actions will happen. This way by simply marking given method with Attribute, we can actually hook into the flow and perform needed logic.

The demo implementation can be found under this link.

Ecotone follows declarative configuration. This means that we mostly going to state what we want to achieve by marking methods with Attributes. This way we can focus on business part of the system, instead of configuration and setups.

Events and Tenant Propagation

When Customer is registered we may want to trigger side effects, like sending an Email with Welcome Message. For those situation we can define Events and Event Handlers.

When Customer is registered, we publish CustomerWasRegistered Event Message using Event Bus. Then all methods marked with Event Handler that subscribe to it (First parameter indicates Event we subscribe too) will be executed as a result.

As you can Ecotone, we could access Tenant Message Header in our Event Handler, this happens thanks to Ecotone’s metadata propagation capabilities.

The demo implementation can be found under this link.

Context and Metadata Propagation

Ecotone by default propagate all Message Headers automatically. This as a result preserve context Tenant. In our case sending Notification will happen in context of the same Tenant, as Customer Registration was done:

Metadata is automatically propagated from Command to published Event

This way we can of course access Tenant name in our Event Handlers too:

Whatever metadata we send at the beginning of the flow (e.g. Register Customer Command), we will be able to access in any synchronous or asynchronous sub-flows (e.g. Customer was Registered Event Handlers).
This means we can easily pass things that are not directly related to Customer Registration Command and access them, in context which make sense. For example we could pass HTTP Domain, IP Address in Metadata, and access it in Event Handler that stores those for auditing.

Asynchronous Events

We can run our Event Handler synchronously which is default way, but we can execute Event Handlers Asynchronously. Ecotone provides set of integrations for Asynchronous handling, like RabbitMQ, Redis, Database Channels and we can also use Symfony Messenger Transport.

We want to use Database Channel, this means that we expect Messages for given Tenant, to be stored in given Tenant’s Database. For this we will use Ecotone’s Database Message Channel, as it provides support for Multi-Tenancy.

Let’s mark our Event Handler as Asynchronous.

This Event Handler will be now understood to be handled asynchronously (in the background) and Event Message will be sent to “notifications” Message Channel. So let’s define this Channel now as Database Queue:

This is all we need to do to configure given Event Handler as asynchronous. Now whenever our Event Handler will be executed, Event Message will first go to Database Queue for given Tenant and then will be consumed asynchronously.

All we need to do, is to place Asynchronous Attribute on top of the Event Handler, and Ecotone will now that this Handler should be executed asynchronously. This will work exactly the same for Command Handlers.

Running Asynchronous Message Consumer

When we publish Message to Asynchronous Message Channel (in our case Database Queue), we need then to consume it.
To run Message Consumer we will be using inbuilt Console Command “ecotone:run”:

bin/console ecotone:run notifications

This will run separate Message Consuming process which will be fetching and executing our Messages coming to “notifications” Channel.

As we are in Multi-Tenant environment and our “notifications” is Database Queue, this actually means that for each Tenant there may be a separate Database having it’s own Queue. And this need to be considered during consumption.

Depending on Business Domain we work in, we may have hundreds of Tenants, so running hundreds of Message Consumers may be far from ideal. For those situations, Ecotone by default use Round-Robin strategy to consume using single process. This means that we will be fetching from each Tenant in order:

Ecotone using Round Robin Strategy to consume Messages from multiple Tenants

This way of consuming works out of the box, we don’t need to do any customer configuration to make it happen. If we would like to speed up message consumption we could run multiple of those processes.
We could actually take over the whole process and throttle given Tenant, when he produces too many Messages, or speed up consumption for specific Premium Tenants. However this will be explored in separate article.

Round-Robin consumption strategy is great as it allows having single process which can manage multiple Tenants. However Ecotone allows us for much more here, as it allows for defining our own consumption strategies to throttle or speed up consumption for given Tenants. This allows for full customization accordingly to our Business needs.

The demo implementation can be found under this link.

Database Transactions and Outbox Pattern

We may want to enable Database Transactions to make the system more resilient to failures. Of course in our case we want Transaction to start for given Tenant’s Database.

Database transaction will be started automatically when Command Bus is executed

Ecotone will start Database transaction for correct Tenant database automatically, when we execute Command. This comes out of the box with Dbal Module, which installed with Symfony Starter. Therefore no extra configuration is needed. You can read more about configuring Transactions in the documentation.

When we publish Events Asynchronously to Database Queue this will be also covered with Transaction. This way in case of exception, we can be sure that everything will be rolled back together.

This works as Outbox pattern that we get out of the box in Multi-Tenant system. Together with that Ecotone provides so called combined Message Channels, where Messages could be moved automatically from the Database to the Message Broker (e.g. RabbitMQ, Redis, SQS). This way actual handling of the Messages would be done for Message Broker Consumers (and we would scale those), not Database ones.

Dbal Business Methods

Dbal Module provides Business Interface — an easy way to write database queries hidden behind abstraction.

We define interface of what we want to achieve and Ecotone take care of how. This means that all we need to do is to write Interface and implementation will be delivered and registered it in our Dependency Container.
Business Interfaces when called from our Message Handlers (Command/Query/Event Handlers) will automatically inherit Tenat’s connection.

If you want to find out more about using Dbal based Business Interfaces, read this article.

Sending Commands straight to the Model

Ecotone provides support for sending Command straight to our Doctrine ORM Entity. This way there is no need to write any delegation level code.
This of course works with Multi-Tenancy too:

As we can see on the example above we’ve created static factory method, this way we tell Ecotone, that this factory method “register” will create new Customer. After this method is executed, Ecotone will call use EntityManager for given Tenant to store it in the correct Database.
This means we don’t need to write such code anymore:

From the Controller side, nothing changes we still send it just as before:

What is important it also work for Action based Methods, which in some scenarios allows us to drop Command Classes completely:

And then we can execute Command Bus like below:

It’s enough to pass aggregate.id in metadata to state which Customer instance we want to execute method at. If you want to explore more on the topic, you can read about using Doctrine ORM as Aggregates in this article.

The demo implementation can be found under this link.

Event Sourcing

When we need to build different Views or audit changes in our system, we may want to use Event Sourcing for that.

Ecotone comes with full Event Sourcing support, which allows us roll out production ready Event Sourced Application in no time for Multi-Tenant systems.

The flow works the same as Doctrine ORM Aggregates, which we explored earlier. The difference is that Event Sourced Aggregates return Event classes instead of changing internal state.

Auto-Setup

Of course we need a place where Events will be stored for given Tenant, and for this we use Event Store in Tenant’s Database.

Ecotone will take care of serializing and deserializing Events, setting up Event Store in given Tenant database (inbuilt support for PostgreSQL, MySQL, MariaDB), and will also support us with setting up Read Model Projections.

Read Model Projections

Projections are used to build different views from Events. Each Projection can be a separate table or set of tables in database, which are dynamically created:

Whenever Event will be published, related Projection will be triggered. Ecotone based on Metadata will understand which Tenant it’s related to and initialize Projection first (if this did not happen before).
After initialization our Projection’s Event Handler will be triggered:

By default this will all happen synchronously, this make it super easy to start working with Event Sourcing. In case of need we can switch our Projections to run Asynchronously however.

You may read documentation, if you want to explore more on the topic of Event Sourcing.
The demo implementation can be found under this link.

Summary

In this article we’ve enabled way to build Symfony Applications which are Multi-Tenant friendly, using code that is not coupled to Multi-Tenant configuration. This way of creating applications make it easy to build and maintain applications, as the code we write can work in single and multi-tenant environments without any changes.
Ecotone will take care of context propagation. This way it does not matter if the code is synchronous or asynchronous, as context of Tenant in which action is done will be preserved for us.

If we enter asynchronous processing and background tasks however, we may face a need for more advanced Queuing based solutions. This may happen because we would like to throttle given Tenant because it produces too many Messages, speed “premium” Tenant and handle failures and retrying in easy to work way. Ecotone provides that, however this topic deserves a separate article.