Yesterday I read some excerpts from Christopher Alexander's "A Timeless Way of Building" and "A Pattern Language". I first heard of Christopher Alexander during my undergrad and have been meaning to read some of his works so I am glad I finally got an "excuse" to do it.
Alexander's writings was pretty much what I expected based on his reputation with a zen-like prose and insightful observations. In the flower and the seed he talks about the quality of living things - the quality without a name. To an engineer the concept seems very vague yet at the same time it feels so familiar. The QWAN can not be made, but must flow out of our work on its own. Since the QWAN can't be made we can not bring it into life through some monumental task. We can only generate things with life through an incremental process where each part is shaped individually to be in perfect harmony with its surroundings. The way I read it is that we must therefore not try to make things with QWAN, but to try to shape our process so as to let QWAN emerge on its own.
It seems clear indeed how the founders of XP was influenced by Alexander to distrust big design up front and instead favor small iterations where the system and its architecture is allowed to emerge on its own.
With "Our Patterns Language" Alexander tries to document the patterns that already exist in our traditional towns and buildings. These are not the patterns created from the minds of a few architects, but the ones that have emerged on their own wherever buildings have been built by their users in harmony with their surroundings and their use. Alexander's patterns are not strict replicas, but recurring structures that are similar, but yet slightly different in each manifestation. He describes how each house in the Alps is similar yet different, each one being perfectly adapted to its particular location and use. In this I believe some of the QWAN lies. Beauty lies not in perfect geometric shapes or in sameness, but in the small variations on a common theme that would only make a structure beautiful in its particular location with its particular use.
For me a pattern have always been a solution to a problem that many people have faced before and solved with the same general of idea. A pattern does not need to follow the common rules of thumb of a field. It must only have been useful in solving a problem for people in the past. Indeed I remember thinking when reading design patterns that I thought some of them were a breach of certain design principles as I understood them. But that is OK because they, although the probably not the best, are good solutions to the problems they address.
Alexander thinks design patterns are important because they are the natural rules for creating beautiful structures that have emerged on their own. They are not only "... a pattern which one might or might not use ...", but they are ".. desirable pattern[s] ..." that one must create "... in order to maintain a stable and healthy world." As such Alexander regards the patterns that naturally arise when the people who are most familiar with a structure's use and location and who are familiar with how similar structures have been created build something. These builders does not sketch out every detail, but brick by brick build a structure from the mental images in their mind. It is these shared images in "a farmers mind" that are what Alexander calls patterns.
In the last excerpt a few of Alexander's patterns are presented. I must admit that I had a hard time reading them while focusing on their application in CS as my mind constantly wandered to thoughts of the places I have lived and worked an how they fit in. Even so I want to attempt to one of them to our field.
Site repair states that one should always build a structure on the parts of a land that are in worst condition. The good parts are already healthy and do not need our help. It is on the rest that we should apply ourself to improve our surroundings. In terms of software engineering I can not help thinking about the concept of technical debt. In a system there is often one component that is in a worse state than the others. Perhaps it had to be rushed in order to reach an important deadline. Or perhaps it has gradually deteriorated to a point where people dread touching it and apply great ingenuity to avoid changing its internals. Because why risk breaking something that after all does work. And why not implement that new feature which may belong in this component in another one that is cleaner and that won't break down when changed. If we were to apply site repair to this problem we should fix the part of our system that are in the worst shape. The good parts are already healthy and do not need our help. It is in the unhealthy part, and everyone will have an idea of where this is, where we can increase the healthiness of our system.
The intimacy gradient is also quite easy to apply in terms of layers, but I am curious to hear suggestions for analogies for light on two sides of every room. I suspect that there aren't really any which is fine. We can still learn from Alexanders thoughts and his description on how he uncovered the pattern, not by analytical studies, but by casually observing it wherever he went.
Friday, September 11, 2009
Wednesday, September 9, 2009
Beautiful Architecture - Resource-Oriented Architectures
Chapter 5 of the book Beautiful Architecture dealt with a architectural "style" I was not familiar with which they called "resource-oriented architecture". The chapter describes the current popular SOA approaches (which I am not very familiar with either) to the architecture of business system and how these fail to solve the problem of reducing complexity for the developers. The chapter also claims that SOA approaches to a degree also fail to reach their goal of allowing heavy reuse of business services in different contexts. They then present an alternative modeled on the world wide web which is what they call resource-oriented architecture or ROA.
My understanding of ROA, which will likely reveal my ignorance as I have only had limited exposure to the business system development world, is that it moves the primary focus from the services (behavior/verb/action) to the the data (content/noun/thing). In a sense I see this as weakly analog to the move from structural programming to object oriented programming. The first emphasized functions that perform some action while the latter emphasize the things that are being modeled (I believe the original goal of Simula was to model things in the real world in order to simulate them more intuitively). Stretching this analogy too far a SOA is similar to providing function pointers that implements services while a ROA would give you a pointer to an interface of an object which you could then use (and reuse) to query/set different properties independent of the underlying implementation.
Of course ROA resource links, like objects, provide services allowing manipulation of and access to different representations of the data (GET/POST/PUT/DELETE - what is wrong with good ol' CRUD?), but the first class concept is the statical dimension (the content pointed to by the URLs) as opposed to the time dimension (the action to perform on that content). As E. W. Dijkstra observed in his letter arguing why goto sentences are harmful "My second remark is that our intellectual powers are rather geared to master static relations and that our power to visualize process evolving in time are relatively poorly developed" (yes I'm a bit of a sucker for good quotes). I think this is an insightful and rather pragmatic observation and one we should make use of.
The chapter also discuss more practical considerations such as security being more easily handled by design in a ROA system as it passes links around instead of data. The clients could therefore request and get a link to a resource. This link could then be used several times by the client and could potentially even be shared with other clients without worrying too much about authentication, authorization and encryption. The access control is then be performed when one of the clients try to use this link, which in many ways is how it works on the internet. Another benefit they discussed was that ROA made it easier to handle caching, which seemed to be grounded in the observation that it is easier to cache a resource than a "service".
Finally, Will Leinweber put forth the question of whether always putting data up front is a good idea. It was an interesting question and I thought about it a bit. My instinct is to not believe in silver bullets, but for an information handling system I would be inclined to say that I think it is a good idea as it is easier to conceptualize and aligns the architecture with the goal being solved (I.e. accessing information).
My understanding of ROA, which will likely reveal my ignorance as I have only had limited exposure to the business system development world, is that it moves the primary focus from the services (behavior/verb/action) to the the data (content/noun/thing). In a sense I see this as weakly analog to the move from structural programming to object oriented programming. The first emphasized functions that perform some action while the latter emphasize the things that are being modeled (I believe the original goal of Simula was to model things in the real world in order to simulate them more intuitively). Stretching this analogy too far a SOA is similar to providing function pointers that implements services while a ROA would give you a pointer to an interface of an object which you could then use (and reuse) to query/set different properties independent of the underlying implementation.
Of course ROA resource links, like objects, provide services allowing manipulation of and access to different representations of the data (GET/POST/PUT/DELETE - what is wrong with good ol' CRUD?), but the first class concept is the statical dimension (the content pointed to by the URLs) as opposed to the time dimension (the action to perform on that content). As E. W. Dijkstra observed in his letter arguing why goto sentences are harmful "My second remark is that our intellectual powers are rather geared to master static relations and that our power to visualize process evolving in time are relatively poorly developed" (yes I'm a bit of a sucker for good quotes). I think this is an insightful and rather pragmatic observation and one we should make use of.
The chapter also discuss more practical considerations such as security being more easily handled by design in a ROA system as it passes links around instead of data. The clients could therefore request and get a link to a resource. This link could then be used several times by the client and could potentially even be shared with other clients without worrying too much about authentication, authorization and encryption. The access control is then be performed when one of the clients try to use this link, which in many ways is how it works on the internet. Another benefit they discussed was that ROA made it easier to handle caching, which seemed to be grounded in the observation that it is easier to cache a resource than a "service".
Finally, Will Leinweber put forth the question of whether always putting data up front is a good idea. It was an interesting question and I thought about it a bit. My instinct is to not believe in silver bullets, but for an information handling system I would be inclined to say that I think it is a good idea as it is easier to conceptualize and aligns the architecture with the goal being solved (I.e. accessing information).
Tuesday, September 8, 2009
ArchJava
The paper "ArchJava: Connecting Software Architecture to Implementation" addresses the problem of architecture and implementation happening independently of each other. A problem with this duplicity is that the implementation usually ends up diverging from the architecture with backdoor communication paths and "hacks" that are not captured in the architectural diagrams. In fact in many cases I believe such backdoors are intentionally kept out of diagrams and descriptions to prevent them from messing up the otherwise beautiful illusion. To fix this the developers either have to update the architecture diagrams (which would probably complicate them) or change the implementation to conform to the architecture description.
The paper present ArchJava which is a set of extensions to the Java programming language that captures the architecture through components and communication paths. As our previous readings have tried to tell us (BA Ch. 1, 4+1, Boxology) these are far from all the aspects of architecture, but they are probably among the most important ones. We could see this quite clearly in the wreck that was the messy metropolis (BA Ch. 2). Components in ArchJava are a special type of classes that can contain subcomponents, thus forming a component hierarchy. Communication paths are "ports" that can be connected to other components. These ports consist of required and provided methods that forms a contract that all components we want to connect to them must conform to. In fact they rather remind me of two-way Qt/boost signal-slots, which allows developers to connect conforming signals and slots from any two classes together. Signal/slots in these C++ libraries are really useful, but have been around for a long time. Furthermore ArchJava disallows all non-port method calls to components that are not a sub-component of the caller and guarantee that this restriction is enforced.
An important feature of ArchJava is that it is implemented directly in the Java language as a set of extensions. The developers are thus relieved from having to maintain two duplicate architectural descriptions; the explicit description on paper (or in a separate ADL) and the implicit one in the source code. In addition to less maintenance is the practical consideration that everything that is duplicated will diverge and cause confusion, misunderstandings and sometimes havoc (I've always wanted to use that word in an academic setting!).
In addition to avoiding duplications having the architecture description explicit in the source code allows the ArchJava compiler to guarantee that there are no hidden back doors (with the rather big exception of shared data). This guarantee is rather useful as one can be sure that the architecture one discusses is the one that actually exist, even though it is probably not the same as one started out with. I think it could also be useful in allowing developers to refactor the architecture with more confidence the same same way that unit tests and interface contracts do.
The authors of the paper spent considerable paper-space on a case study of a small to medium sized application. Their intent was on demonstrating how ArchJava could be added to an existing application to force the architecture to be explicit, but I got the feeling most of the paragraphs were about refactorings that were based more on the knowledge of more experienced Java developers. That being said they did manage to convince me of the benefits in being sure the architecture you think you have is the one you actually have, especially as the size of the project increases into the hundreds of thousands of lines.
Referring to the questions from Professor Johnson, like Jason Danielson (I really liked the video blog by the way), I do not think ArchJava would have helped the team making memories much in the initial development. The system was created by a team of seemingly skilled developers that appeared to be highly motivated and set on making a great system with a great architecture. In a sense they had solved this problem through process, skills, gelling and caring and this is far more potent than any set of language extensions.
However, the part of the making memories saga that we were told about was only the initial development which is a fraction of the whole story. The system they made will need to be maintained for many more years and it is my experience that architectural rot has a higher tendency to set in in this phase. The discipline that ArchJava enforces would probably be far more useful ten years down the line when all the original developers and all the excitement of creating a new great system are long gone.
The paper present ArchJava which is a set of extensions to the Java programming language that captures the architecture through components and communication paths. As our previous readings have tried to tell us (BA Ch. 1, 4+1, Boxology) these are far from all the aspects of architecture, but they are probably among the most important ones. We could see this quite clearly in the wreck that was the messy metropolis (BA Ch. 2). Components in ArchJava are a special type of classes that can contain subcomponents, thus forming a component hierarchy. Communication paths are "ports" that can be connected to other components. These ports consist of required and provided methods that forms a contract that all components we want to connect to them must conform to. In fact they rather remind me of two-way Qt/boost signal-slots, which allows developers to connect conforming signals and slots from any two classes together. Signal/slots in these C++ libraries are really useful, but have been around for a long time. Furthermore ArchJava disallows all non-port method calls to components that are not a sub-component of the caller and guarantee that this restriction is enforced.
An important feature of ArchJava is that it is implemented directly in the Java language as a set of extensions. The developers are thus relieved from having to maintain two duplicate architectural descriptions; the explicit description on paper (or in a separate ADL) and the implicit one in the source code. In addition to less maintenance is the practical consideration that everything that is duplicated will diverge and cause confusion, misunderstandings and sometimes havoc (I've always wanted to use that word in an academic setting!).
In addition to avoiding duplications having the architecture description explicit in the source code allows the ArchJava compiler to guarantee that there are no hidden back doors (with the rather big exception of shared data). This guarantee is rather useful as one can be sure that the architecture one discusses is the one that actually exist, even though it is probably not the same as one started out with. I think it could also be useful in allowing developers to refactor the architecture with more confidence the same same way that unit tests and interface contracts do.
The authors of the paper spent considerable paper-space on a case study of a small to medium sized application. Their intent was on demonstrating how ArchJava could be added to an existing application to force the architecture to be explicit, but I got the feeling most of the paragraphs were about refactorings that were based more on the knowledge of more experienced Java developers. That being said they did manage to convince me of the benefits in being sure the architecture you think you have is the one you actually have, especially as the size of the project increases into the hundreds of thousands of lines.
Referring to the questions from Professor Johnson, like Jason Danielson (I really liked the video blog by the way), I do not think ArchJava would have helped the team making memories much in the initial development. The system was created by a team of seemingly skilled developers that appeared to be highly motivated and set on making a great system with a great architecture. In a sense they had solved this problem through process, skills, gelling and caring and this is far more potent than any set of language extensions.
However, the part of the making memories saga that we were told about was only the initial development which is a fraction of the whole story. The system they made will need to be maintained for many more years and it is my experience that architectural rot has a higher tendency to set in in this phase. The discipline that ArchJava enforces would probably be far more useful ten years down the line when all the original developers and all the excitement of creating a new great system are long gone.
Wednesday, September 2, 2009
Beautiful architecture - Making Memories
The system in this story, a photo storage, manipulation and processing system, was a bit strange to read about for me since so many of their architecture choices and considerations were so similar to the ones we made in an open-source application I co-wrote years ago during my undergrad. (The application was called Stopmotion and can now be found in both the Debian and Mandriva official repositories.)
However, this is not all that strange since this is really, as far as I can see, just a variant of the well known MVC architecture pattern. Their domain is the MVC model, their forms are controllers and their properties are the subjects in the observer relationship between the model and the view/presentation layer, which allows the model to indirectly and dynamically inform the presentation layer of changes and thus keep it in sync. In fact I find it strange that they can even describe all of this without even mentioning that pattern. Also, when reading the article I do wonder whether this system is a bit over-engineered and perhaps even an example of the second system effect (I wonder about the same thing with Stopmotion by the way). The Kiosks seemed fairly basic with the GUI operating in a screen-by-screen workflow, which limits the amount of interacting components on the screen at one time. However, I do believe it is better to err on the side of a bit too much infrastructure than the opposite and based on this story they do seem to have been successful at keeping entropy in check so that they could work efficiently. This impression is strengthened by the author's closing comment about remembering the different classes and their interaction fondly, which is one I share from my experience with Stopmotion.
One thing I would like to point out though is the usefulness of domain/application facades in implementing things like undo (or logging, etc.). These classes work as gateways to the domain/model which means they are ideal places to log commands or take snapshots of the model state so that these can later be run backwards/restored in order to undo an action. In our application we used a variant of the Command pattern to implement this functionality.
However, this is not all that strange since this is really, as far as I can see, just a variant of the well known MVC architecture pattern. Their domain is the MVC model, their forms are controllers and their properties are the subjects in the observer relationship between the model and the view/presentation layer, which allows the model to indirectly and dynamically inform the presentation layer of changes and thus keep it in sync. In fact I find it strange that they can even describe all of this without even mentioning that pattern. Also, when reading the article I do wonder whether this system is a bit over-engineered and perhaps even an example of the second system effect (I wonder about the same thing with Stopmotion by the way). The Kiosks seemed fairly basic with the GUI operating in a screen-by-screen workflow, which limits the amount of interacting components on the screen at one time. However, I do believe it is better to err on the side of a bit too much infrastructure than the opposite and based on this story they do seem to have been successful at keeping entropy in check so that they could work efficiently. This impression is strengthened by the author's closing comment about remembering the different classes and their interaction fondly, which is one I share from my experience with Stopmotion.
One thing I would like to point out though is the usefulness of domain/application facades in implementing things like undo (or logging, etc.). These classes work as gateways to the domain/model which means they are ideal places to log commands or take snapshots of the model state so that these can later be run backwards/restored in order to undo an action. In our application we used a variant of the Command pattern to implement this functionality.
Tuesday, September 1, 2009
A Field Guide to Boxology
The paper on boxology was to me the most interesting paper/chapter of the first two weeks of this course. The paper attempts to establish a taxonomy and framework for describing different architectural styles and to place different well known styles in a two dimensional topology. The paper appears to be from before architecture styles started to be documented as patterns (i.e. from before the POSA book) and treats styles in a somewhat more formal manner than patterns do. However, since the paper is about creating a framework for classifying different architectures it is equally applicable as a classifications of architectural patterns. In addition the papers treatment of architectural styles is observational as opposed to a priori or analytical and as such it is very similar to the pattern approach.
After the introduction the main part of the paper has one section describing their framework for classification followed by two example architectural (dataflow networks/pipes and filters and message passing processors) that are elaborated to demonstrate and attempt to validate the classification.
To me the section on their classification strategy was by far the most interesting. They defined components and connectors as primary classification criteria in addition to the secondary criteria control, data organization and the interaction between these. One of the things I like about this section is the treatment of connectors as an aspect that are as important as components since there are many more ways to do this than simple static function calls (observers, function pointers, virtual functions, message passing, etc.). I also found their examples of different characteristics that can be associated with connectors such as format conversion (Adapters...) and even additional functionality such as logging and performance monitoring to be interesting.
The architectural styles that I am most familiar with from the their classification in the appendix ("Table 1") are layered architectures, dataflow networks (mostly through pipelines), different call-and-return styles and to some degree message passing processors. The architectures I am the least familiar with are the data-centered repository architectures and I don't actually know how a blackboard works so I guess I have some reading to do.
I have encountered pipelines mainly through my work on graphics drivers. A graphic driver, along with the GPU it drives, is primarily a pipeline for producing images. In this particular instance of a pipeline the user will provide stream data in the form of vertices and attributes as well as auxiliary data such as textures and matrices. In addition to this the user can typically configure the operation of different pipeline stages (lighting, texture stages) and/or, in more modern systems, even provide whole stages to be executed on the stream data in the form of shader programs. Such a system can consist of many stages such as draw-call setup, vertex processing, fragment processing and each stage can typically operate concurrently on different sets of data as one would expect from a pipeline. Some stages are performed on a CPU while other stages are more typically offloaded to specialized hardware such as a GPU.
An example of directed acyclic data network would be the gstreamer architecture for media processing. This pipe-and-filter architecture allows the user to define different filters to process or transform data as well as sources of data and sinks (for example a file or a surface to be blitted to a computer screen). These components can then be stitched together into a DAG that can be used to process a flow of data (video/audio/images). This provides for an incredible powerful architecture (although at the time I was trying to use it several years ago gstreamer was quite immature and caused me to loose some of my hair) where a relatively small set of filters can be composed into many different interesting components of a larger application.
One architecture that is interesting to try to place in the authors table is model-view-controller. This architecture (often, but certainly not exclusively, found as a "sub-architecture" inside the server of a client/server architecture) can in some sense be seen as variation of main program/subroutines specialization of call-and-return. Flow of control, for the most part, moves hierarchically from the presentation layer downwards, but that also has a feedback loop that is dynamically bound at runtime in the form of an observer that synchronizes different parts of the presentation layer with the current model state. However, a typical MVC can also be seen as a data-centered repository style architecture as the model will usually represent a set of data to be manipulated and presented.
After the introduction the main part of the paper has one section describing their framework for classification followed by two example architectural (dataflow networks/pipes and filters and message passing processors) that are elaborated to demonstrate and attempt to validate the classification.
To me the section on their classification strategy was by far the most interesting. They defined components and connectors as primary classification criteria in addition to the secondary criteria control, data organization and the interaction between these. One of the things I like about this section is the treatment of connectors as an aspect that are as important as components since there are many more ways to do this than simple static function calls (observers, function pointers, virtual functions, message passing, etc.). I also found their examples of different characteristics that can be associated with connectors such as format conversion (Adapters...) and even additional functionality such as logging and performance monitoring to be interesting.
The architectural styles that I am most familiar with from the their classification in the appendix ("Table 1") are layered architectures, dataflow networks (mostly through pipelines), different call-and-return styles and to some degree message passing processors. The architectures I am the least familiar with are the data-centered repository architectures and I don't actually know how a blackboard works so I guess I have some reading to do.
I have encountered pipelines mainly through my work on graphics drivers. A graphic driver, along with the GPU it drives, is primarily a pipeline for producing images. In this particular instance of a pipeline the user will provide stream data in the form of vertices and attributes as well as auxiliary data such as textures and matrices. In addition to this the user can typically configure the operation of different pipeline stages (lighting, texture stages) and/or, in more modern systems, even provide whole stages to be executed on the stream data in the form of shader programs. Such a system can consist of many stages such as draw-call setup, vertex processing, fragment processing and each stage can typically operate concurrently on different sets of data as one would expect from a pipeline. Some stages are performed on a CPU while other stages are more typically offloaded to specialized hardware such as a GPU.
An example of directed acyclic data network would be the gstreamer architecture for media processing. This pipe-and-filter architecture allows the user to define different filters to process or transform data as well as sources of data and sinks (for example a file or a surface to be blitted to a computer screen). These components can then be stitched together into a DAG that can be used to process a flow of data (video/audio/images). This provides for an incredible powerful architecture (although at the time I was trying to use it several years ago gstreamer was quite immature and caused me to loose some of my hair) where a relatively small set of filters can be composed into many different interesting components of a larger application.
One architecture that is interesting to try to place in the authors table is model-view-controller. This architecture (often, but certainly not exclusively, found as a "sub-architecture" inside the server of a client/server architecture) can in some sense be seen as variation of main program/subroutines specialization of call-and-return. Flow of control, for the most part, moves hierarchically from the presentation layer downwards, but that also has a feedback loop that is dynamically bound at runtime in the form of an observer that synchronizes different parts of the presentation layer with the current model state. However, a typical MVC can also be seen as a data-centered repository style architecture as the model will usually represent a set of data to be manipulated and presented.
Subscribe to:
Posts (Atom)