Why software quality attributes matter
What exactly is a software quality attribute? Do all software systems share the same qualities? And is there any point in defining quality attributes at all?
Software architecture is difficult. It is an endless series of decisions and tradeoffs.
Every change that comes in affects the software. The effect may be enormous or tiny, but it cannot be avoided. Netflix, for example, chose to migrate from a monolith to microservices, which shook its entire infrastructure. At the other end of the scale, every new line of code has a small effect of its own. Over time, though, without proper safeguards, those accumulated lines turn into chaos whose impact nobody can predict.
Plenty of good practices help to reduce this risk. Among the most common are tests, CI/CD, canary releases and ADR, to name just a few. Together they provide a certain level of confidence when changes are made to a system.
Still, when architecture is central to an organization’s success, the system has to be guided by a set of quality goals. Suppose, for instance, a system offers behavior-based security to new customers and exposes its functions and support through websites, APIs and so on, with an emphasis on efficiency so that tasks are completed quickly and resources are used optimally. To succeed, it also has to operate within tight limits on performance, availability, security, reliability and supportability. If the system and its architecture can’t meet them, the business will most likely collapse.
Reasoning about software quality attributes is where it gets more complicated. What is a software quality attribute in the first place? Do all software systems share the same qualities? Is there any point in defining quality attributes at all? What happens if none are defined? And how does that affect quality assurance?
This article attempts to answer these questions.
Software quality attributes
A quality attribute is a characteristic of a software system that decides how useful and successful it is from the perspective of the client and the business. A ledger system built with technical excellence that is nonetheless difficult to use, lacks support and needs a long time to bring new features to market will most likely fail in the long term.
A quality attribute serves as a driver for a software system. It sets the constraints and goals that the system architecture, the development process and the entire Software Development Life Cycle (SDLC) have to meet; without them, the team in charge of an application may lose its way. Quality attributes are frequently called non-functional requirements.
Do software systems without defined QA exist, though? In short – no. The longer answer is that systems fall into two categories. The first has clearly defined, known quality attributes that drive informed technical decisions; the second covers all remaining systems. Later sections of this article show examples of each.
Classification
The classification of Software Quality Attributes (QA) is laid down in the ISO/IEC 25010 standard. Wikipedia also offers a long list of QA. Describing every quality attribute in detail is beyond the scope of this article; interested readers can consult the standard mentioned above.
What is the value added from defining software quality attributes?
The QA classification suggests that quality attributes drive a system by pointing to the direction in which it should evolve. They also put limits on the changes that the system will inevitably undergo. Clearly defined characteristics let developers reason about every change. If, for example, security is a critical part of the system and it states that ‘all connections between services must use OAuth2 authentication’, every service has to meet this quality attribute, or else it won’t be deployed to production.
Quality attributes should normally stem from business needs. Identifying and defining them is the difficult part, since business needs can shift over time – and that is exactly why the effort of defining them pays off. As business needs change, we as developers can use QA to challenge or confirm whether a particular change makes sense. When a system has a set of quality attributes backed by metrics, it becomes far easier to estimate the potential cost of a change, its impact and risk, and how to deal with it. And when money enters the conversation, trade-offs become easier to reason about as well.
Evaluate software quality attributes
Ideally, each quality attribute should be expressed as a discrete, measurable value, with automated measurement tests that are effortless to run. In practice, a quality attribute can rarely be captured by one value. Usually it consists of several values, and how complex it is depends on the business needs. So how do you distill quality attributes, quantify them and apply them to a system’s architecture?
Picture a fictional company called WorkoutPlans that is about to launch a business that relies heavily on software. The plan is to build a service that helps athletes follow their workout plans so they can prepare effectively for competitions.
The service should be available as a gamified application with 3D visuals for cyclists, runners and kayakers who use training equipment connected to a PC or laptop. It has to support smart equipment whose resistance can be controlled according to the virtual environment.
Down the line, it should be possible to add support for any new equipment that appears on the market.
The service has to calculate precisely all the values athletes need to assess their fitness progress. It has to be available 24/7.
It has to handle at least 40000 active users. It has to run on a range of devices – PC, Apple MacBook, Apple Desktop, IPad and Android tablets. Lastly, users must find it easy to set up a workout plan, build their own workout, train without a structure, move around the settings, see their progress, connect with other athletes and link up with external fitness services.
From this brief business description, several quality attributes can be derived.
- Reliability – users must be able to reach the system at any moment
- Performance – the system must handle at least 40000 active users
- Security – the data of many users must be protected with authentication and authorization
- Portability – the system must run on numerous platforms
- Maintainability – the system must stay open to updates
- Usability – the system must be easy to use for both casual and advanced users
- Functional Suitability – the system must precisely calculate fitness metrics from the data gathered by training equipment
For the business, this set should offer insight into the system’s characteristics and what can be expected from it. Discussing things with stakeholders at this level should be fairly straightforward. The system architect can, for example, agree with the person concerned whether Portability is essential for success given the effort and time it demands. Depending on the needs, Portability may be dropped or reduced. Conversely, it may turn out to be critical once the target market is considered.
“Observability means being able to measure a system’s internal states by looking at what it outputs. A system counts as “observable” when its current state can be estimated solely from the information in its outputs, i.e. sensor data.”
Observability of the system is essential for establishing quality attributes. Without it, nobody can reason about the system’s current state, and in particular about its quality attributes. If the system exposes no sensors, it is difficult to tell whether there are issues with traffic, performance, communication, dropped connections, database failures and so on.
At the same time, a system architect has to know what to observe before settling on an architecture. Otherwise, the solution ends up shaped by a best guess and whatever was known at the time.
Introducing quality attributes makes it possible to plan observability from the very beginning. It lets architects concentrate on the goals a particular system should pursue. In civil engineering, for instance, it would be very hard to build a bridge that can carry the expected loads without knowing what those loads are. That knowledge profoundly affects the final design and construction. Software engineering works the same way, only with far less dramatic consequences.
With the quality attributes derived from the WorkoutPlans example, measurable values can be defined that specify exactly what should be observed. The table lists the identified quality attributes along with descriptions and value definitions in the ‘Invariants’ column. These values should guide the architecture design. They were picked arbitrarily for the sake of the example.
This simplified process of distilling and defining quality attributes illustrates how to bring architects nearer to informed decisions. Decisions an architect takes early on matter most, since the starting point of the architecture strongly shapes all the decisions that follow. The more closely the architecture reflects business needs, the lower the chance of friction between IT and the business, and the fewer significant architecture-level decisions get overlooked.
Based on these characteristics, an architect can plan disaster recovery strategies, scalability strategies, security policies and more. Much less is left to guesswork about what has to be done.
Architecture with unknown QA
A company operating in the loans domain relies heavily on its software. The system calculates loans, assesses money laundering risk, keeps track of existing loans, manages customer accounts and much more. Put simply, without this system the company would have to do all of this manually or in excel. The system is therefore at the heart of the business.
Nevertheless, it was decided to place the entire IT department, together with the system, in the cost budget rather than the investment budget. This heavily influenced every decision and action concerning the system.
The one true driving force for the whole company is earning more money at the lowest possible cost. And there’s nothing wrong with that. At bottom, that is the reason every company exists.
You could say the system has exactly one quality attribute defined: cost minimization. That single characteristic leads to a string of architectural decisions:
- Observability of the system is restricted to logs
- No security policies at all
- One monolith deployment for a system serving multiple countries
- One shared database serving multiple countries
- A shared library that computes all use cases for the various countries
- A release train introduced for all teams building the country components
- One code base for all teams
- One software testing department serving all teams
What a mess
These decisions are taken without much thought, as a by-product of how the company is managed, one by one as time passes and complexity increases.
As a result, the system turns unreliable and difficult to maintain, change and diagnose. On top of that, there is neither room nor time for substantial changes that would repair the system, because such changes are hard to fit into the cost budget. Unsurprisingly, the actual cost of maintaining the system keeps rising.
These are the main factors driving the overall cost:
- Fixing bugs is slow because observability is missing
- Delivering bug fixes depends on how often the release train runs
- Delivering features depends on how often the release train runs
- One code base increases the likelihood of unintended side effects
- Having one software testing team test every aspect of the system requires huge amounts of time and effort, and it is error-prone as well.
- Fixing the database is slow because of the shared database and missing observability
- The shared library for processing loans across all countries is difficult to maintain because of the many different scenarios, and it is error-prone as well.
On top of all that, the teams working on the system grow increasingly frustrated and stressed, particularly around releases, because a team that doesn’t finish on time misses the release window. These problems make the business lose money and customers, which brings us back to where it all began – a system whose foundations rest on cutting the IT department’s costs.
Summary
Software engineering is a highly dynamic field. Requirements shift, problems crop up and business needs develop. Everything is fragile. The only constant is change.
Uncertainty and complexity are inherent to software systems. Software engineers have to cope with and find their way through this ever-changing world while constantly evaluating software performance. Decisions are typically made on a best guess or gut feeling. At first this seems good enough, since the system’s complexity is still fairly easy to understand. But unavoidable changes arriving from all sides strain the system until it becomes unstable and difficult to reason about and maintain. The business loses out, and people grow more frustrated as time goes on.
Then again, making decision-making more analytical and predictable is hard and can look like too great an investment of time and money. Software system quality attributes are one way to tackle this challenge and make it more manageable. They help set the system’s goals and, as a starting point, steer the system architecture in an informed way, which lowers costs in the long term. The right attributes can result in higher-quality applications and successful systems.
