Australian And International Standards For Software Testing

Select and research one Australian and one international standard relating to software testing to ascertain the answers to the following questions. Be sure to check appendices in these documents as they can be very helpful.  Your answers should illustrate a good understanding of the standard documents you select.

• What is the standard name?
• Who holds the copyright for the standard?
• Amongst the acknowledged contributors to the document, which universities were involved (if any)?
• What is the scope or intent of the standard?
• What are key terms and understandings needed for the standard to be understood and applied?
• In your own words, what does application of the standard result in? Or in other words, what does the standard do?
• Finally, what specific relevance to software testing is the standard?

Australian and International Standard Overview

Standards are guideline documentations which reflect agreements to various practices, products and operations procedures nationally and globally recognized industrial, professional, trade associations or government bodies (Chinosi & Trombetta, 2012). The national standard for Australia researched on NATA (National Association of Testing Authorities, Australia 2013). This standard was established in 1947 but in 1988 its roles were affirmed in the country. The standard borrows from the ISO 29119 standard for software testing.

Internationally ISO/IEC/IEEE 29119 is the standard used in software testing. It is applicable throughout the development and maintenance of software products and software-intensive systems. This standard covers a wide range of subtopics which include software testing concept that is testing in all stages of the life cycle model, testing processes which include  test process for organizations, the process of management as well as dynamic test (the advanced level of the test management) process.

The test management process basically covers the planning, monitoring and controlling stages of the project n the project cycle whereas dynamic test processes covers the design and implementation, environment set-up, execution and incident reporting.

Software testing is critical to achieving software quality and reliable software.  The standards help the consumer be confident in the product. They portray the authors' expertise on the standard, the manufacturers conform to the market requirement by using the standards and also it provides the safety from liability (Stallings, Brown, Bauer, & Bhattacharjee, 2012).

NATA

The NATA standard copyright is under the Commonwealth of Australia act 1968.

From the research of the two standards, there are no indicated universities that were involved in the contribution of the development of the standard document.  

The NATA standards scope mainly covers software acceptance testing, the functionality testing, the stability tests, compatibility, healthcare tests and technical embedded system test. This test is aligned to ISO 14598.5 (Suprem, Mahalik, & Kim, 2013). The testing involves writing down the tests of a software version to be released of the developed software against the software requirement, there are review and update tests, also a dry test on the software (Mell & Grance, 2011).

Test verdict: The declaration after the test. They can be no, yes or inconclusive, this is declared that the test case.

Test tools: the hardware and software are used to carry out the testing requirement.

Test cases: The complete test case sets that help in achieving the test objective.

Test software: The software used to carry out the testing.

NATA Standard

Test methods: The technical procedures for performing test services.

Test case:  sets of inputs, preconditions, executions and the expected outcome.

Evaluator: The individual who performs the software testing.

Evaluation requirement: description of the objective of the evaluation. Generally relating to the intended use and the risks associated.

ISO/IEC/IEEE 29119

 The copyright of this standard was made in Switzerland.Tthe copyright is held in the ISO office in Switzerland (Myklebust, Sta alhane, Hanssen, Wien, & Haugset, 2014)

 The intention of ISO/IEC/IEEE 29119 is to give an introduction to keyword driven testing. It provides a reference approach to do the keyword-driven testing implementation. The standard defines the framework requirements of the Keyword-Driven Testing that facilitates sharing of work among software testers (Kawaguchi, 2014). The scope defines the requirements of the tools which will aid in Keyword-Driven Testing. The specifications would apply to tools that would support the Keyword-Driven approach.

It helps define an interface and a common platform for data exchange ensuring that tools of different vendors exchange their data  

The standard helps to define levels and use of hierarchical keywords (Garcia, Dávila, & Pessoa, 2014). The standard provides a list of low-level keywords, which include input data and check values. These keywords are specify the technical level test cases and also can be joined to form the required keywords in business level (Ruy, Falbo, Barcellos, Guizzardi, & Quirino, 2015).

Since the main aim of the standard is to understanding and use of other standards within the series of 29119.  This standard covers a wide range of topics which need to be understood and also applied to. The areas span from the time software begun to be developed to the end and implementation of the software.

 Introduction to software testing- this helps understand the role played by testing in validation and verification, exhaustive testing and also testing as a heuristic.

Also understand the processes of genetic testing in a life cycle of the software (ISO/IEC, 2013).

 The testing is done to mitigate risks. The objective of the test and the item being tested are among the sub-process testing (Felderer, Wendland, & Schieferdecker, 2014).

 Once the standards are well followed they provide harmony in how things are done internationally. In this case, IS0/IEC/IEEE 29119 shows the minimum guideline on how software testing should be done globally. This helps to avoid development and production of software with diverse test- methods. Therefore, it is advisable to use the standard in all types of software and throughout the development and maintenance of software.  

ISO/IEC/IEEE 29119 Standard

Because the standard serves as a guideline to software testing from the time of creation of the software to deployment, maintenance and changes, the standard is beneficial to businesses, buyers and the government in verifying the quality of the product in the market on well-studied and measured basis (Hamlen & Thuraisingham, 2013). The standard can be applied in businesses that require specifications that are keyword driven, create frameworks that are corresponding, or create keywords based test automations.

Both standards fulfil standards of the design, safety and product, code of practices ,the methods of testing, standards of management systems. Alongside that they are both performance based and prescriptive standards.  The standards have the same format as which is covered in three main divisions, preliminary, body and the supplementary part.

Both the standards are uniquely identified by a letter or number combinations which are called designations. The designations may consist of prefix, main numbers, part numbers, interim, year or suffix.

The two standards support the approach and the inclusion of the standards in the system development.  They offer global recognition and offer a competing opportunity with other markets around the world. The two also offer an opportunity to create quality and reliable software. They provide the needed components and the requirements for developing a high-quality product.  

Both NATA and IS0/IEC/IEEE 29119 make it possible to test, access, and develop the software without stopping the development process.

Both the standards make use of the white box and black box test case techniques.  

Standardization from the two standard documents saves time by preventing possible failures and errors on the software.  

 From a security aspect, the two standard documents offer standardization and continuous quality evaluation which is very necessary since the unsafe system can cause lot problems to users and also the developers, which may include sharing of personal data.

The standards at the analysis stage help in declaring that the standard document was very useful and also advice the developers whether they should continue using the standards.

Both the national and the international standards require a lot of documentation- the standards are aligned to the agile development, so the users of the standards take a lean approach with documentation so as a company or user can comply with the standard.

Both the standards require that the users adopt them and use them, since these are guidelines that reflect agreement on practices and operations of a nation or a world so as to recognize the professional bodies and many more.

Benefits of Standardization

The standards are the baseline for accreditation for all software that will be in the market globally and nationally. The national standards should advice the software with a nation as well as the international standards advices both the international and national software to allow harmony of the software.

The standards cover the context-driven testing; whose mandates is aligned with the risk-based approach, the standards key premise is the idea of them is to performing the optimal testing which are within the given constraint and context using the risk-based approach.

Both the ISO/IEC/IEEE29119 and the NATA standards are published as a single document. This document is subject to revising over a period of time as it is stated in the standard document. Mostly the revision is done after a specified number of years. This is done to incorporate the new trends in technology.

The two standards borrow ideas from each other. That is the two standards have a mutual relationship, in that a national standard borrows from the international to incorporate some of the international guideline as well as the international borrow from the national standards so as to relate the national requirement for the international standard.

 The biggest difference in developing the two standards is that NATA is widely accepted within Australia as a nation and may not be used internationally as the ISO/ IEC/IEEE standard could be used. As for the ISO/IEC/IEEE internationally and it serves as a reference framework for other nations to come up with a software testing standard.

The national standards are not globally accepted; therefore they do not allow system communication around the world. They make system communication easy within a nation and not like the international standards which allow communication. The international standards improve communication via a common terminology that is based on the accepted guideline.

The international software testing standards it provides a wider coverage of the testing discipline while providing a pragmatic solution that helps the testers and organizations.

The international standard ISO/IEC/IEEE 29119 gives a baseline for the discipline of testing internationally. This is in terms of the test techniques, and also provides an ideal baseline to measure out the effectiveness and the design technique.

A test management tool is software that that enables the management and documentation of software. They enable management of test cases, tracking of requirements, reporting and tracking. When selecting a test management tool, support on mobile devices, cost effectiveness, flexibility, compatibility with the system, essential features, usability and productivity are some factors considered.  In this report two test management tools are discussed, test rail and zephyr.

Requirements and Documentation

Zephyr has management of requirements while test rail does not. Zephyr has a flexible mechanism for issue management while test rail does not. Zephyr has an external dashboard. Test rail has a highly productive user interface. The interface includes a one-click test result and a three pane layout that eases testing.

Both zephyr and test rail can be integrated in JIRA. They both have report feature, and test case management. The test cases are not parameterized. Batch processing, integration of API and automations are featured. They both have support although test rail has a better support compared to zephyr. They include agile and waterfall methodologies, user and administrators’ training and consultations.  They can neither prevent and detect duplicate issues nor use email to directly report issues ("2018 Test Management Tool Comparison: TestRail Vs. Zephyr", 2018).

Conclusion

Advancements in research on software testing has made acquisition of  knowledge on testing techniques contribute to software quality.  The standards offer global recognition of a system and also offer a competitive advantage in the market. Well tested software as per the standards will help avoid errors and other issues to do with the software. Inability to use the standards as a guideline to software testing can lead to duplication of software which can reduce the project quality.

Standards can remove the creative part of the project, at the same time the certificate does not guarantee that the company will deliver quality software. Therefore there is need to develop standards that will ensure the company is able to deliver a quality product.

Selecting the appropriate testing tool for project management is important. The selection should be based on the requirements of the business and the test tool features.

References

Chinosi, M., & Trombetta, A. (2012). BPMN: An introduction to the standard. Computer Standards & Interfaces, 34(1), 124–134.

Felderer, M., Wendland, M.-F., & Schieferdecker, I. (2014). Risk-based testing. In International Symposium On Leveraging Applications of Formal Methods, Verification and Validation (pp. 274–276). Springer.

Garcia, C., Dávila, A., & Pessoa, M. (2014). Test process models: Systematic literature review. In International Conference on Software Process Improvement and Capability Determination (pp. 84–93). Springer.

Hamlen, K. W., & Thuraisingham, B. (2013). Data security services, solutions, and standards for outsourcing. Computer Standards & Interfaces, 35(1), 1–5.

ISO/IEC. (2013). ISO/IEC/IEEE 29119-1: 2013 Software and systems engineering-Software testing-Part 1: Concepts and definitions. SwitzerlandInternational Organization for Standardization Geneva.

Kawaguchi, S. (2014). Trial of organizing software test strategy via software test perspectives. In Software Testing, Verification and Validation Workshops (ICSTW), 2014 IEEE Seventh International Conference on (pp. 360–360). IEEE.

Mell, P., & Grance, T. (2011). The NIST definition of cloud computing.

Myklebust, T., Sta alhane, T., Hanssen, G. K., Wien, T., & Haugset, B. (2014). Scrum, documentation and the IEC 61508-3: 2010 software standard. In International Conference on Probabilistic Safety Assesment and Management (PSAM). PSAM, Hawaii.

Ruy, F. B., Falbo, R. A., Barcellos, M. P., Guizzardi, G., & Quirino, G. K. (2015). An ISO-based software process ontology pattern language and its application for harmonizing standards. ACM SIGAPP Applied Computing Review, 15(2), 27–40.

Stallings, W., Brown, L., Bauer, M. D., & Bhattacharjee, A. K. (2012).Computer security: principles and practice. Pearson Education.

Suprem, A., Mahalik, N., & Kim, K. (2013). A review on application of technology systems, standards and interfaces for agriculture and food sector. Computer Standards & Interfaces, 35(4), 355–364.

2018 Test Management Tool Comparison: TestRail Vs. Zephyr. (2018). Retrieved from https://www.practitest.com/testrail_vs_zephyr/