How to analyse architectures

The very point of a documented architecture is that it will tell you important properties of the system even if the system is still not constructed. While creating architectures, we make decisions that will have an impact on other systems with which our system will interact. The architecture created, as a result of these decisions, needs to be analyzed to make sure it achieves what it intended to achieve.
Since the architecture creation itself might is an iterative process, an analysis of the decisions made can be carried out at any step to make sure quality evaluation is done as early as possible in the lifecycle.
The benefits of architecture evaluation are multiple.

• Financial There are significant financial savings to be had by evaluating the architecture early in the life cycle my reducing rework at later phases.
• Forced Preparation During architecture construction, team members are engaged in discussions about multiple alternatives. Many of these discussions are, unfortunately, never documented. A formal architecture evaluation or forces the formal documentation of these discussions and makes sure nothing falls through the cracks.
• Documenting Rationale An architecture evaluation also forces team members to document the rationale of their choices which is otherwise not documented in many cases.
• Early detection of problems An architecture evaluation with different perspectives will provide early visibility in product capabilities and limitations.

There are multiple systematic methods for architecture evaluation.  In this post, we will talk about two of the most commonly used methods, Architecture Tradeoff Analysis Method (ATAM) and Cost Benefit Analysis Method(CBAM).

Architecture Tradeoff Analysis Method

This method requires the participation of the following stakeholders.

1. The Evaluation Team This groups is external to the project team. The teams are 3-5 members and each member is assigned a specific perspective(s). For software systems with a long life cycle, one may keep a standing team for all their evaluations.
2. Project Decision Makers These people represent the project team. Project Managers, Customer, and Architect are part of this group.
3. Stakeholders These are stakeholders whose ability to do the job hinges on the quality of architecture. These would include developers, testers, integrators, maintainers, etc.

An architecture evaluation using the above method will produce the following artifacts.

• Concise presentation of architecture
• Articulation of business goals
• Quality requirements in terms of collection of scenario 
• Mapping of architectural decisions to quality requirements
• Set of identified sensitivity and tradeoff points
• Set of risks and non-risks
• Set of risk themes.

The evaluation itself is conducted in the following phases.

• Partnership and preparation -- Everybody meets informally and works out the details of the exercise
• Evaluation -- This phase involves information gathering followed by the analysis of information gathered. The information gathering is performed by the evaluation team and the analysis is performed by all the people including decision makers and stakeholders. This phase consists of the following activity that is required to be completed as part of this phase.
• Present and explain the process
• Present business drivers
• Present architecture
• Identify architectural approaches
• Generate quality attribute utility tree
• Analyze architectural approaches
• Brainstorm and prioritize scenarios
• Analyze architectural approaches
• Compile and present results
• Follow-up -- This phase involves the generation of the evaluation report by the evaluation team. The essence of this phase is self-examination and improvement. It is followed by a post-mortem which articulates what worked well and what didn't.

Following outputs are generated at the end of the evaluation process.

• Prioritized statement of quality attribute requirements
• Catalog of architectural approaches used
• Approach and quality attribute specific analysis questions
• Mapping of architectural approaches to quality attributes
• Risks and Non-Risks
• Sensitivity and tradeoff points

Cost-Benefit Analysis Method

This method of architectural analysis provides for economic decision making. This approach is not necessarily an independent approach. It builds on the previous approach.

We begin by considering the collection of scenarios generated as part of ATAM or specifically for this exercise. Each of these scenarios is examined to find how they differ in their utility. The utility of the scenarios is determined by looking at variation in scenarios, priorities of scenarios and architectural strategies. The benefit function of an architectural strategy can be computed by the sum of products of all the utility functions and importance of various scenarios.

The ROI can be computed for each architectural strategy by the following method.

As we see, both the methods provide us mechanisms that we can use to evaluate any architecture for its quality. The benefit of a formal architecture evaluation during a project life cycle is always beneficial for the overall life of the project.