Skip to content

Separating Concerns with Pipes & Filters

Sponsor: Do you build complex software systems? See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling. Get started for free.

Learn more about Software Architecture & Design.
Join thousands of developers getting weekly updates to increase your understanding of software architecture and design concepts.


How do you separate concerns when processing a request? Typical concerns such as Logging, Validation, Exception Handling, Retries, and many more. One way is to build a request pipeline for separating concerns by using the Pipes and Filters pattern. You can also build a pipeline using the Russian Doll model that allows you to short circuit at any point throughout the pipeline.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Pipes & Filters

Often time when processing a request you need to handle various concerns. The Pipes & Filters pattern allows you to break up these various concerns into a series of steps that create a pipeline. A request can be an HTTP request but can also be a message that’s being processed. If you’re using ASP.NET Core, you’re already using the Pipes & Filters pattern!

Pipes & Filters

In the diagram above, the sender is sending a request to the receiver, however, there are filters in between. These filters in between are what can handle various cross-cutting concerns.

Pipes & Filters

The request will pass through the various filters until it finally reaches the Receiver. Both Sender and Receiver are unaware that the request passed through the filters.

Pipes & Filters

As mentioned, the filters can be cross-cutting concerns such as logging, validation, caching, etc.

Pipes & Filters

One important note is that filters are independent. They should be composable with various types of requests. As you can assume a logging filter could be used in many different types of requests.

You should be able to decide per request type, which filters you want to use. They should be plug-and-play.

This allows the receiver of the request to really focus on the behavior that it needs to perform and not other concerns.

A good example of where you may have used this is with ASP.NET Core Action Filters.

Russian Doll

Another way of implementing Pipes & Filters is with the Russian Doll Model. The concept is the same that you have a Sender and a Receiver with Filters in between. The difference however is that each filter calls the next filter in the pipeline.

Each filter is still independent but because the pipeline is using a uniform interface, each filter can be provided the next filter to be invoked.

Russian Doll

Since each filter is calling the next filter, this means that you can short circuit the pipeline at any point.

Russian Doll

In the diagram above, the Validation filter may choose to short circuit the request if validation fails. Because it’s not calling the next filter in the pipeline the Receiver will never get executed to handle the request.

A good example of where you maybe have used this is with ASP.NET Core Middleware.

Messaging

Processing a message is often very similar to processing any other type of request, except it often doesn’t have an in-process response. There are still various concerns when processing a message that creating a pipeline helps with.

As an example using the Brighter messaging library, which supports Pipes & Filters using the Russian Doll model using attributes. This feels very similar to action filters in ASP.NET Core.

In the example, the PlaceOrderHandler has 3 different filters that will be executed before it is executed. Logging, Retry, Validation. I’ve implemented logging as a generic filter that can be used in any type of request. While Retry and Validation are specific to the PlaceOrderCommand.

Pipes & Filters

Creating a pipeline can be a powerful way to separate various concerns when processing a request. It allows you to focus on each individual task as a task (filter) within the pipeline.

You can create generic filters that can be used with any type of request or create very specific filters that are only used for a very specific request Also leverage the Russian Doll model to short circuit your request pipeline.

Source Code

Developer-level members of my CodeOpinion YouTube channel get access to the full source for any working demo application that I post on my blog or YouTube. Check out the membership for more info.

Related Posts

Learn more about Software Architecture & Design.
Join thousands of developers getting weekly updates to increase your understanding of software architecture and design concepts.


Leave a Reply

Your email address will not be published. Required fields are marked *