Supporting multiple event stores in Commanded using an adapter based approach for Elixir
Commanded v0.10 includes support for Greg Young’s Event Store, in addition to the existing PostgreSQL-based Elixir EventStore. This article describes how an Elixir behaviour and adapter approach was used to support multiple event stores.
Elixir CQRS/ES Commanded
Announcing the release of Commanded v0.10 with support for using Greg Young’s Event Store.
Commanded is an open-source library you can use to build Elixir applications following the Command Query Responsibility Segregation and event sourcing (CQRS/ES) pattern. The project’s README contains a useful getting started guide that covers its features and I’ve written an article on building a CQRS/ES web application in Elixir using the library.
In previous versions, Commanded used the PostgreSQL-based Elixir EventStore that I built in tandem with it. The EventStore library provided the event persistence, event handler subscriptions, and snapshot storage. Commanded was heavily dependent upon the EventStore library.
In the past few months I had received a few requests to allow Commanded to use Greg Young’s Event Store. This was a great feature request that I happily added to the development roadmap. Unfortunately, it wasn’t a priority for me to build out straight away since I was satisfied with using the PostgreSQL event store.
Two months later a comment was left on the GitHub issue by a user named sharksdontfly. He had forked Commanded and added support for multiple event stores. Demonstrating one of the real benefits of open source projects. I took on the pull request containing his changes to merge into Commanded.
This article describes how an Elixir behaviour and adapter approach was used to support multiple event stores.
- Adding support for multiple event stores
Adding support for multiple event stores
The high level approach used was an adapter strategy:
- Create an Elixir behaviour to define the contract required for an event store.
- Implement a set of unit/integration tests against the behaviour, used to verify a given storage provider adheres to the contract.
- Build an adapter module that implements the behaviour for each supported event store as a separate package.
- Extend the Commanded configuration to specify the chosen event store adapter.
- Replace all existing event store usage with the adapter specified in config.
The end goal was to allow the consumer of Commanded to choose which event store they wanted to use. They would install the adapter package in their Elixir application, configure any specific event store settings, and use it.
For development purposes I wanted each event store adapter to be a separate package. This allows mix dependencies to be cleanly defined and there would be no need to specify
optional: true. Having an individual Elixir application, Hex package, Git repository, and GitHub project per adapter allows releases to be made separately and any issues to be tracked against particular event stores.
Create an Elixir behaviour
In Elixir, behaviours provide a way to:
- define a set of functions that have to be implemented by a module;
- ensure that a module implements all the functions in that set.
They are similar to an interface in an object oriented language. This is exactly what we need to state, in code, the set of function signatures that an event store adapter module must implement.
I identified all occurrences of the event store used in Commanded. The features were: appending events to a stream; reading a stream forward; subscribing to events from all streams; and recording snapshots.
This determined the public API of the event store that needed to be replicated across additional adapters:
Additional event stores can be supported by Commanded by writing an adapter that confirms to this behaviour.
Implement unit/integration tests against the behaviour
With the behaviour module defined, I wanted to write a set of unit and integration tests. These would be used to verify each event store adapter meets the expected behaviour. By defining the tests in Commanded – and including the files in the published Hex package – any adapter can reference them within its own tests.
Here’s an example of a test to append events:
In the Commanded
mix.exs file I included the test/event_store_adapter folder containing the tests:
For an example adapter, the
test/test_helper.exs file configures itself to be used by Commanded:
The adapter test files from Commanded, located in the
deps/commanded folder, are loaded to be executed when the tests are run:
Tests are run using the standard mix command:
This allows the same tests to be used for all adapters, pertinent to the version of Commanded they support.
Build an adapter module
I decided to implement an in-memory
GenServer adapter within Commanded. This provides ephemeral storage, so is only used for testing purposes. It is used by both the adapter tests and when testing the Commanded library itself. An in-memory process event store allowed very fast test runs – resetting the event store merely required restarting the process – and that there were no external dependencies required.
PostgreSQL EventStore adapter
The existing Elixir EventStore library, built to support Commanded, had an API that only required a very lightweight adapter (commanded-eventstore-adapter). Mapping data between the structs defined by the EventStore and Commanded libraries. An example of appending events to a stream is shown below:
This adapter includes Commanded as a dependency in
mix.exs so that it can use the
Commanded.EventStore behaviour. I had to specify
runtime: false due to the change in Elixir 1.4 where dependencies are included as runtime applications by default. This allows access to the dependency modules, but doesn’t start the dependency at runtime.
Extreme event store adapter
The Extreme Elixir TCP client library is used to connect to Greg Young’s Event Store.
The adapter (commanded-extreme-adapter) required additional work as Extreme did not have support for persistent subscriptions. This is a feature of the Event Store where the server remembers the position of all subscribers to an event stream. Commanded requires a event store subscription with this facility. Event handlers acknowledge receipt and successful processing of each received event so that on restart they resume from the next event following the last one acknowledged.
I submitted a pull request to the Extreme library to add support for persistent subscriptions (#38).
For development and test purposes, Docker is used to host the Event Store using their official image (eventstore/eventstore). In between test runs the Docker image is recreated, using the module below:
This ensures clean test runs where no individual test can impact another.
Docker is also supported by Travis CI.
.travis.yml config file you include the
docker service and pull the image before the build and tests:
Extend the Commanded configuration
I extended the Commanded configuration to support specifying the event store adapter to use:
For a consumer of Commanded, switching between event stores only requires changing this config setting.
Use the event store via an adapter
Finally, an Elixir macro was written to support compile-time lookup of the configured adapter module and provide this as a module attribute constant.
The macro is
used in a consumer module and
@event_store provides access to the configured store.
The final step to allow Commanded to be storage agnostic was to replace all usages of
EventStore with the macro and
The pull request containing these changes in Commanded is “Support multiple event stores”(#55).
Commanded v0.10 now supports two event stores:
PostgreSQL-based EventStore using the commanded_eventstore_adapter package.
Greg Young’s Event Store using the commanded_extreme_adapter package.
Additional storage can be supported by writing an adapter following these implementations.
This article has demonstrated how you can add support to your Elixir library for drop-in replacements to third party dependencies.
Please get in touch with any feedback, corrections, suggestions, and improvements you have.