Software Usability Testing Standards: ISO/IEC 12119 And AS/NZS ISO/IEC 25062

Select and research one Australian and one international standard relating to software testing to ascertain the answers to the following questions. Be sure to check appendices in these documents as they can be very helpful. Your answers should illustrate a good understanding of the standard documents you select.

What is the standard name?

Who holds the copyright for the standard?
Amongst the acknowledged contributors to the document, which universities were involved (if any)?
What is the scope or intent of the standard?
What are the key terms and understandings needed for the standard to be understood and applied?
In your own words, what does the application of the standard result in? Or in other words, what does the standard do?

What are ISO/IEC 12119 and AS/NZS ISO/IEC 25062?

Usability of any software is one its significant non-functional requirements that shall be included to make sure that the user experience is higher.

The scope of this software testing standard is to report the results obtained after carrying out the usability test cases so that there are no gaps in the final release of the software. It has been defined to be used by the following people and entities.

• Usability resources and professionals belonging to the supplier organizations to come with the results and reports that may be used by the customer organizations.
• Customer organizations and end-users to verify and validate the conformance of application/system usability with the international standards.
• Human resources in customer organizations to understand the technical aspects of the usability (Iso, 2018).
• Technical professionals in the organizations to make decisions as per the results of the usability tests.

• Usability: It is defined as the extent to which a specific product or system may be used by the user or a group of users to meet the expectations and gain enhanced satisfaction.
• Effectiveness: It is defined as the accuracy which the users shall have to achieve the specific objectives in the application or the system.
• Efficiency: The utilization of the resources in association with the effectiveness of the application/system.  
• Satisfaction: Absence of any loopholes or discomfort in the application to make sure that a positive user attitude is maintained.
• Context of Use: The application or the system environments in which it is utilized by the user.
• User/Groups: The person or groups that access the product

The standard is applied in the area of software engineering and software testing to determine the usability of a specific system/application/software/product.

The standard is extremely relevant to software testing as it includes the concept of usability which is one of the primary non-functional requirements to be included. The user experience and user satisfaction can be obtained from the enhanced usability. Also, software cannot be successful unless it is usable and hence, the standard attempts to keep the usability of the software intact (Infostore, 2016).

ISO/IEC 12119: 1994

International Organization for Standardizations (ISO) and International Electrotechncial Commission (IEC)

Not Applicable

The scope of the standard is limited to software packages and it includes the definition of quality requirements for the software packages. It includes the detailed set of instructions that shall be followed for testing the software package against the quality requirements. The standard does not include the production phase of the software package to test or confirm its quality during this phase.

The standard has been defined and is intended to be used by the following set of entities:

• Suppliers: At the time of producing the specifications for the requirements of software package or while assessing their own product.
• Certification authorities and bodies while trying to define and establish standards for the third-parties.
• Testing agencies and laboratories to test the software package while testing the same for certification (Bcc, 2015).
• Accreditation bodies
• Auditors and senior managers of the testing laboratories
• Buyers of the software packages looking for certified products or while comparing the requirements between the two products

• Function: A specific algorithm that shall be implemented to provide functionality to the user in the end-product.
• Requirements Documentation: A descriptive, detailed, and formal document covering the requirements of the system or the product belonging to any of the requirements category, such as functional, non-functional, user, security, performance, and likewise.
• Product Description: A detailed, descriptive, and formal document stating the properties of a specific product that shall be achieved and met before the software package is purchased.
• User Documentation: It includes the printed or non-printed documents that are developed for the usage by the end-users of the system or the product.
• Test Case: It is the detailed information covering the test objective, functions that shall be tested, test data to be used, steps to be followed, expected and actual behaviour of the system of the functionality.
• Maintenance: The activities associated with the maintenance of the software package that may include modifications or alterations to the existing set of features and processes.

The standard has been developed and defined by the international organizations and bodies as ISO and IEC to make sure that the quality requirements of the software packages are included and are met in the end products.

The quality requirements that are included with a software package state that there must be product description and user documentation present with each of the software package. It also states that all of the features and functionalities of a product shall be testable and as per the requirements. The requirements shall be included in the software package in terms of the requirements categories as functional, non-functional, user, security, performance, and likewise.  

The standard is relevant to software testing as it ensures that the quality of the end-product is maintained and is achieved. There are several non-functional requirements associated with a software package, such as usability, maintainability, quality, security, performance, availability, accessibility, and likewise.

Scope and Relevance of Standards

The standard provides the mechanisms to make sure that the quality of the software packages is maintained by including the quality requirements in the same.

The two standards that have been described above have certain common set of features while there are also several differences between the two.

The common features between the two standards are their area of application. Both of the standards belong to the area of software testing and are applied for the software packages to make sure that the non-functional requirements of these packages and products are achieved. However, the scope and intent of the two standards varies from one another. AS/NZS ISO/IEC 25062: 2006 is a software testing standard that has been defined by Standards Australia and Standards Council of New Zealand. While, ISO/IEC 12119: 1994 is an international standard that has been defined by International Organization for Standardizations (ISO) and International Electrotechncial Commission (IEC).

The scope of the two standards also varies from each other. The scope of the former standard is to report the results obtained after carrying out the usability test cases so that there are no gaps in the final release of the software. It is defined to be intended to be used by usability resources and professionals. While, ISO/IEC 12119: 1994 is limited to software packages and it includes the definition of quality requirements for the software packages. It includes the detailed set of instructions that shall be followed for testing the software package against the quality requirements.

TestRail is a modern test case management tool that has been developed for the QA and development teams. It is a web-based tool that comes with the following set of features:

• Manage all of the test processes and activities in a single and integrated manner.
• Web-based tool to make sure that the QA and development team resources are able to access the tool on dedicated or hosted servers.
• Use of productive and modern interface to make sure that the productivity is enhanced.
• Can be used for any of the workflow or approach, such as agile or traditional approach.
• The tool can be used by an individual resource or an entire team without any performance (TestRail, 2018).
• Manage the test cases and results from various types of test process as manual, automated, and exploratory testing.
• The test case results can be tracked using the automated graphs, charts, and percentages.
• Testing reports and metrics can be acquired from the tool.
• Seamless integration can be achieved using the tool.

The project organization is flexible and tool allows the integration of all the testing activities on a single platform. There are various test teams that can parallel access the tool so that the project progress of different projects can be accessed all at once. The test case management of the tool is rich and powerful. The functionality of the tool can be used to develop the rich test cases that shall be understood by all. All of the testing activities can be managed in a structured manner.

Test Collab is a testing tool that comes along with the following set of features.

• The test cases may be planned and fed in to the system at the start of the project or the start of the sprint. The document requirements and features shall be installed in the tool with ease.
• Security and privacy is one of the primary features of the tool and is secured with the 256-bit encryption.
• Platform independence is one of the features of the tool and for self-hosted applications; the tool allows the use of virtual machines and middleware.
• The tool allows the integration with the any of the test automation tools and applications and several machines can be executed in parallel.
• The interface that comes along with the tool is easy to use and understand and can be utilized by the resources from all of the teams, such as testers, developers, designers, and likewise.
• There is an option to define the reusable steps and these can be defined for the common set of test cases.
• Requirements management is one of the features that come along with the tool and the requirements can be defined and set up as per the acceptance criteria.
• Time tracker is one of the features of the tool to understand the time being taken by the testers in every test activity (Testcollab, 2018).
• The tool allows setting up of the milestones with a clear start and finish date for each of the project activity.
• Team collaboration is one of the features of the tool to connect the managers, testers, and other resources with each other.

The two test management tools that have been described above come with the specific set of features and functionalities.

The common features present in these tools include that both of these have been designed for the purpose of software testing and the management of the testing activities. The common features that have been defined have been listed below.

• Both of these tools come with the feature of management of all of the test processes and activities in a single and integrated manner.
• Different forms and types of testing activities can be carried out in both of these tools. These allow for the management of the test cases and results from various types of test process as manual, automated, and exploratory testing.
• Both the tools allow the integration with the any of the test automation tools and applications and several machines can be executed in parallel.
• Requirements management is one of the features that come along with the tool and the requirements can be defined and set up as per the acceptance criteria.
• The test case results can be tracked using the automated graphs, charts, and percentages in both the tools.
• The tool allows the integration with the any of the test automation tools and applications and several machines can be executed in parallel in both the tools.

There is difference in the pricing model of the tools.

• TestRail comes with two options in terms of pricing and purchase. The first is TestRail Cloud that includes a fee of $30 on a per month basis. TestRail Server comes with the charges as $290 for one year. The annual license comes for a minimum of five users.
• Test Collab comes with three types of models as $0, $25, and $10 on a per month basis. The start-ups shall utilize the $25 model for a per month basis for the growing teams. The enterprises with 50+ users can make use of $10 per user model. The start-up plan comes up with a plan for 14 days.

References

Bcc. (2015). Information technology - Software packages - Quality requirements and testing. Retrieved from https://bcc.portal.gov.bd/sites/default/files/files/bcc.portal.gov.bd/page/adeaf3e5_cc55_4222_8767_f26bcaec3f70/ISO_IEC_12119.PDF

Infostore. (2016). Software engineering—Software product Quality Requirements and Evaluation (SQuaRE)—Common Industry Format (CIF) for usability test reports. Retrieved from https://infostore.saiglobal.com/preview/as/as20000/25000/25062-2006_r2016.pdf?sku=394870

Iso. (2018). AS/NZ ISO/IEC 25062:2006. Retrieved from https://www.iso.org/obp/ui/#iso:std:iso-iec:25062:ed-1:v2:en

Testcollab. (2018). Free Test Management Software & Test Case Management Tool. Retrieved from https://testcollab.com/

TestRail. (2018). TestRail Pricing & Licensing. Retrieved from https://www.gurock.com/testrail/pricing