What is Quality

Quality means different things for different people. For some people, the brand Apple is the synonym of quality. For some people it is difficult to define quality, but surprisingly easy to recognize it based on their experience .

A quality product or service is the one that satisfies the needs of a customer, that meets their expectations or even exceeds them. When you receive quality, in whatever form, you are eager to get more. You want to come back and make another purchase, you refer the product to your friends and connections, you have a tendency to talk about it in public. Quality is what we should aim for if we want to have returning customers and a strong brand as a company.

According to IEEE quality is “The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.”

Who is responsible for Software Quality?

Everyone involved in a software project for e.g Product Owner, Scrum Master, Developer, Tester, and other stakeholders including Business Team, Security Team, & Architecture Team can contribute to Software Quality in several ways.

Quality should be everyone’s responsibility and all should work together to assure quality, doing things right from the beginning. Quality can only be achieved if everybody cares about achieving it.

Three aspects of Software Quality

The three aspects of software quality are functional quality, structural quality, and process quality.

Functional quality means that the software correctly performs the tasks it’s intended to do for its users. Among the attributes of functional quality are:

• Meeting the specified requirements. Whether they come from the project’s sponsors or the software’s intended users. In some cases, this might even include compliance with applicable laws and regulations. And since requirements commonly change throughout the development process, achieving this goal requires the development team to understand and implement the correct requirements throughout, not just those initially defined for the project.
• Creating software that has few defects. Among these are bugs that reduce the software’s reliability, compromise its security, or limit its functionality.
• Good enough performance. From a user’s point of view, there’s no such thing as a good, slow application.
• Ease of learning and ease of use. To its users, the software’s user interface is the application, and so these attributes of functional quality are most commonly provided by an effective interface and a well-thought-out user workflow. The aesthetics of the interface — how beautiful it is — can also be important, especially in consumer applications.

The second aspect of software quality, structural quality, means that the code itself is well structured. Unlike functional quality, structural quality is hard to test. The attributes of this type of quality include:

• Code testability. Is the code organized in a way that makes testing easy?
• Code maintainability. How easy is it to add new code or change existing code without introducing bugs?
• Code understandability. Is the code readable? Is it more complex than it needs to be? These have a large impact on how quickly new developers can begin working with an existing code base.
• Code efficiency. Especially in resource-constrained situations, writing efficient code can be critically important.
• Code security. Does the software allow common attacks such as buffer overruns and SQL injection? Is it insecure in other ways?

Both functional quality and structural quality are important, and they usually get the lion’s share of attention in discussions of software quality. Yet the third aspect, process quality, is also critically important. The quality of the development process significantly affects the value received by users, development teams, and sponsors, and so all three groups have a stake in improving this aspect of software quality.

The most obvious attributes of process quality include these:

• Meeting delivery dates. Was the software delivered on time?
• Meeting budgets. Was the software delivered for the expected amount of money?
• A repeatable development process that reliably delivers quality software. If a process has the first two attributes — software delivered on time and on budget — but so stresses the development team that its best members quit, it isn’t a quality process. True process quality means being consistent from one project to the next.

Software Quality Metrics

Software Testing Metrics are pointers or numbers which help you understand the attributes of a product, (like its complexity, its size, it’s quality, etc.), the attributes of the process (which can be used to improve the quality and speed of development) and the attributes of the project (which includes the number of resources, costs, productivity and timeline among others), popularly known as the three P’s.

Software quality metrics can be divided further into three categories −

• Product quality metrics
• In-process quality metrics
• Maintenance quality metrics

Product Quality Metrics

This metrics include the following −

• Mean Time to Failure
• Defect Density
• Customer Problems
• Customer Satisfaction

Mean Time to Failure

It is the time between failures. This metric is mostly used with safety critical systems such as the airline traffic control systems, avionics, and weapons.

Defect Density

It measures the defects relative to the software size expressed as lines of code or function point, etc. i.e., it measures code quality per unit. This metric is used in many commercial software systems.

Customer Problems

It measures the problems that customers encounter when using the product. It contains the customer’s perspective towards the problem space of the software, which includes the non-defect oriented problems together with the defect problems.

Customer satisfaction

Customer satisfaction is often measured by customer survey data through the five-point scale −

• Very satisfied
• Satisfied
• Neutral
• Dissatisfied
• Very dissatisfied

Satisfaction with the overall quality of the product and its specific dimensions is usually obtained through various methods of customer surveys. Based on the five-point-scale data, several metrics with slight variations can be constructed and used, depending on the purpose of analysis. For example −

• Percent of completely satisfied customers
• Percent of satisfied customers
• Percent of dis-satisfied customers
• Percent of non-satisfied customers

Usually, this percent satisfaction is used.

In-process Quality Metrics

In-process quality metrics deals with the tracking of defect arrival during formal machine testing for some organizations. This metric includes −

• Defect density during machine testing
• Defect arrival pattern during machine testing
• Phase-based defect removal pattern
• Defect removal effectiveness

Defect density during machine testing

Defect Density is the number of defects confirmed in software/module during a specific period of operation or development divided by the size of the software/module. It enables one to decide if a piece of software is ready to be released.

Defect density is counted per thousand lines of code also known as KLOC.

Defect Density = Defect count/size of the release

Defect arrival pattern during machine testing

The overall defect density during testing will provide only the summary of the defects. The pattern of defect arrivals gives more information about different quality levels in the field. It can be calculated in Mean Time to Repair (MTTR), Mean time Between Failure (MTBF), Mean Time to Failure (MTTF) etc

Phase-based defect removal pattern

This is an extension of the defect density metric during testing. It summarizes the relationships among three metrics

• Defect injection
• Defect removal
• Effectiveness

Defect removal effectiveness

Defect Removal Effectiveness relates to the ability to remove defects introduced to a system by a project during the project life cycle.

It can be calculated or expressed as

DRE = (total defects found during the project/total defects introduced by the project)x 100

Maintenance Quality Metrics

Although much cannot be done to alter the quality of the product during this phase, following are the fixes that can be carried out to eliminate the defects as soon as possible with excellent fix quality.

• Fix backlog and backlog management index
• Fix response time and fix responsiveness
• Percent delinquent fixes
• Fix quality

Fix backlog and backlog management index

The fix backlog metric is a count of the number of defects in the product following its release that require a repair. It is a simple count of reported problems that remain at the end of each month or each week. It is usually measured at regular intervals of time and plotted for trend analysis. A more useful way to represent the fix backlog is by defect severity. By calculating the fix backlog by severity level, you can begin to draw useful conclusions from your measurements

Backlog Management Index-The backlog management index (BMI) is calculated by dividing the number of defects closed during some period of time by the number of new defects that arrived during that same period of time.

BMI =Number of problems closed during the month/Number of problem arrivals during the month ×100%

Fix response time and fix responsiveness

The fix response time metric is usually calculated as the mean time of all problems from open to close. Short fix response time leads to customer satisfaction.

The important elements of fix responsiveness are customer expectations, the agreed-to fix time, and the ability to meet one’s commitment to the customer.

Fix Quality

Fix quality or the number of defective fixes is another important quality metric for the maintenance phase. A fix is defective if it did not fix the reported problem, or if it fixed the original problem but injected a new defect. For mission-critical software, defective fixes are detrimental to customer satisfaction. The metric of percent defective fixes is the percentage of all fixes in a time interval that is defective.

A defective fix can be recorded in two ways: Record it in the month it was discovered or record it in the month the fix was delivered. The first is a customer measure; the second is a process measure. The difference between the two dates is the latent period of the defective fix.

Usually the longer the latency, the more will be the customers that get affected. If the number of defects is large, then the small value of the percentage metric will show an optimistic picture. The quality goal for the maintenance process, of course, is zero defective fixes without delinquency.

Software testing attributes

Software Quality Attributes (aka non-functional requirements) help software architects to evaluate the performance of a software application. These quality attributes decide whether the software is of good quality or not.

These quality attributes are also sometimes called “ilities” after the suffix most of the words related to system capability share such as availability, reliability, scalability, testability, etc.,

Functional requirements are just the tip of the iceberg as shown in the above image. The consequences of leaving quality attributes lead to increased technical debt and quality problems.

Reliability

Measure if the product is reliable enough to sustain in any condition. Should give consistently correct results.

Product reliability is measured in terms of working of the project under different working environments and different conditions.

Maintainability

Different versions of the product should be easy to maintain. For development it should be easy to add code to the existing system, should be easy to upgrade for new features and new technologies from time to time.

Maintenance should be cost-effective and easy. The system is easy to maintain and correcting defects or making a change in the software.

Usability

This can be measured in terms of ease of use. The application should be user-friendly. Should be easy to learn. Navigation should be simple..

Portability

This can be measured in terms of Costing issues related to porting, Technical issues related to porting, Behavioral issues related to porting.

Correctness

The application should be correct in terms of its functionality, calculations used internally and the navigation should be correct. This means the application should adhere to functional requirements.

Efficiency

Major system quality attribute. Measured in terms of time required to complete any task given to the system. For example, the system should utilize processor capacity, disk space, and memory efficiently.

If the system is using all the available resources then the user will get degraded performance failing the system for efficiency. If the system is not efficient then it can not be used in real-time applications.

Integrity or Security

Integrity comes with security. System integrity or security should be sufficient to prevent unauthorized access to system functions, preventing information loss, ensure that the software is protected from virus infection, and protecting the privacy of data entered into the system.

Testability

The system should be easy to test and find defects. If required should be easy to divide into different modules for testing.

Flexibility

Should be flexible enough to modify. Adaptable to other products with which it needs interaction. Should be easy to interface with other standard 3rd party components.

Reusability

Software reuse is a good cost-efficient and time-saving development way. Different code library classes should be generic enough to use easily in different application modules. Dividing the application into different modules so that modules can be reused across the application.