Pipes and Filters

One of this weeks readings was a pattern from the POSA book on pipes and filters. The chapter didn't really contain much new information for me, but I had not thought of it in terms of active/passive filters before. Other than this I thought the chapter mixed the general idea with Unix concept too much, which would make it a bit confusing for readers not familiar with this. Perhaps it would have been better to first discuss the generalized pattern and then have an example describing its use Unix?

Pipes and filters, often called pipelines in the parallel world, is a very common pattern that is used many places for different reasons. In Unix shells its primary usage is to provide a facility for communication between small applications that do one thing and one thing well. The intent there is chiefly flexibility, modularity and reusability - the applications can be used together with other applications to solve problems the original authors may not have thought of.

In the parallel world pipelines are used as a means to expose parallelism in a problem. A task is broken into different stages and if many tasks have to be processed or if one stage can start processing a task before the previous stage is finished then we can do this concurrently. A third place where pipes and filters and the general form of often called data networks is extremely common is in media processing systems. In a previous blog I mentioned gstreamer which is a popular pipes and filters framework for all sorts of media processing in the Linux world. Other media APIs that employ pipelines are OpenGL and OpenAL.

Typical systems implementing the latter two demonstrate yet another benefit of pipelines: If we have broken a task into specialized pipeline stages then we can process some stages more efficiently using specialized HW. In the mobile world there are typically many different specialized processors on a SoC (System On Chip) such as graphics, audio, video, wireless and bluetooth and the most common form of using them is through pipelining. As the number of transistors on a chip, at least for the time being, still increases exponentially (even though the speed doesn't) I suspect that more heterogeneity will start to become the norm on PC microprocessors as well. As such we in the computer world will also have the specialized machinery that Eric G. mentions.