If you have ever worked on Linux, you are probably familiar with the concept of pipes. The whole concept of pipes within Linux is a based on the architectural style pipe and filter. This blog will try to simply give an overview of the pattern. To start of this blog about the Pipe and Filter architectural pattern, it is important to first realize what the pattern can be used for in order to solve a certain problem. Even though, this pattern has certain advantages, it definitely has disadvantages and is therefore no applicable for every situation.

Problem

The Pipe and Filter pattern is best applicable when a system is required to execute multiple manipulations on an ordered dataset. Generally, the performance of the ordered but independent computations is a feature of interest for the application.

Solution

As stated in the problem section the Pipe and Filter architectural style is effective in usage, when multiple independent computations need to be executed on an ordered dataset. The Pipe and Filter pattern realizes these different computations into different components that are called the filters, the connectors between the filters are called the pipes. Every filter component has an certain amount of inputs and outputs. A filter receives streams of data on its inputs and produces streams of data that are an instance of the result in a standard order on its outputs. This is achieved by performing transformations locally on the input stream and computing incrementally. The connectors serve a conduit between filters that transport the output of a filter to the input of another. The data flows in one direction and that is downstream. The whole chain of filters and pipes is called the pipeline. Every filter must be an independent entity that should not share its state with other filters. Filters are not aware of the identity of the upstream and downstream filters. The correctness of the output of a pipeline should not depend on the order of the filters in which the incremental processing is performed.

Figure 1. Unix Pipeline. A Unix pipeline that is used for spelling checking.

The architectural pattern has many applications in which it is used, such as text-based utilities in the UNIX operating system [1] [2] [3]. Figure 1 shows such an application that is used within Unix.

There are common specializations on the pipe and filter architecture [4]. These are as followed:

• Pipelines: The network of pipes and filters is restricted to a linear topology. There are no branches possible with this specialization of the pipe and filter architecture.
• Bounded pipes: With this specialization of the pipe and filter architecture, the amount of data within a pipe is restricted and thus can not surpass a certain amount.
• Type pipes: This specialization requires for the data that is transported between filters to be a well-defined type.

The pipe and filter architecture should be considered to be used when multiple manipulations on datasets should be performed [1].

Advantages of the pipe and filter architecture

The pipe and filter architecture has several advantages because of the independent components that can be used in any order. Having independent filters ensures that the software architecture is loosely and flexibly coupled which allows for filters to be changed without modifying any of the other filters. It also supports re-usability, since the filters can be called numerous times. A system that is set up according to the pipe and filter architecture can be easily maintained, extended and enhanced over time.

Not only does the architecture support maintainability and extensibility, the pipe and filter style is naturally concurrent in its execution. Since each filter can be implemented as a separate task, the filters can be possibly executed in parallel with one another [3] [4].

Disadvantages of the pipe and filter architecture

It is obvious that this architecture is not a fit for every situation, since it has its advantages, but also its disadvantages. The pipe and filter would be a good choice for the design of compiler or an application that does different manipulations on seismic data, but the application of the pattern would not be suitable for an interactive system.  An interactive system expects user feedback at any point of the process. The pipe and filter pattern does not allow for interactive actions when processing has started. Also, the output pattern for incremental updates is entirely different from the pattern for filter output [1] [4].

Even though the architectural style allows for concurrency, pipes may have a negative effect on the overall performance. Since there is no collaboration between the different filters, the input data needs to be copied from the pipe buffer into a local filter buffer. Also, if a common denominator is set for the transmission of the data then the input needs to be unparsed and processed before any of the data manipulations can be executed. After the completion of the manipulations, the data is parsed and copied back into the next pipe buffer. Having the overhead of both the parsing of data before manipulations and the different copying actions, will not make this architectural pattern suitable for critical applications, like autonomous vehicles and medical devices [2] [5].

Conclusion

The pipe and filter architectural pattern is a solution that can be used in the design of a system when there are multiple manipulations that need to be performed on datasets. There are several specializations of the architectural pattern, like pipelines, bounded pipes and type pipes.

The pattern has several advantages since the network of pipe and filters consists of filters that are not dependable on each other. Modifications that are made in one filter will not affect the other. This results in a maintainable and extensible system that can be improved and evolved over a period of time. Since all the filters are implemented as a separate task, the pipe and filter architectural pattern is concurrent in its nature.

Even though the architectural style has its advantages for certain problems, it is not suitable for interactive or time critical systems. As mentioned before, an interactive system expects users’ actions at any time and acts accordingly upon it. The pipe and filter architectural pattern does not allow for such interruptions during the processing and manipulation of the data. When the process has started, it does not act upon interactive actions until the completion of the process. A system that has the pipe and filter pattern as its structure, might experience loss of performance since there are going to be multiple copies between the filters and the pipes. If a common denominator is set for the transmission of the data over the pipe, then the overhead of parsing the data will also negatively affect the performance the system. This makes the pipe and filter architectural pattern not suitable for time-critical applications.

References