SIT707 Software Quality And Testing

This report gives an analysis and evaluation of Testing tools like Selenium and JUnit used in Web Application Testing. An in depth study of of Web Applications and the necessity to deliver them with the least number of errors is made. A sample web application is used to do the analysis and measure the efficacy of automating the test scripts.  Selenium is a group of software tools which has distinct methods and approaches for automating the test script. This report analyzes the use of Selenium in different operations like locating UI elements and comparing the expected results to the actual results while running a script. The ability of Selenium to execute the same test scripts in different browsers is also analyzed here. The document elucidates the fact that Selenium is not just a tool but a combination of short concise applications which together constitute a Suite of Software’s. This suite is designed to cater to the needs of different requirements and customers depending upon the Testing strategy they adopt.

A study of how JUnit could be used for Unit Testing Web Applications is made here. JUnit is a unit testing framework. The use of JUnit in writing the test scripts that are intended for repeatable execution is studied here. When the sample application is checked unit wise using JUnit, the bugs could be identified early in the Testing process. The Graphical User Interface of JUnit helps in makng the Testing procedure more easy by denoting the progress of testing in a bar that turns red if a test is failed. Testing the sample application with JUnit ensures that the software can be easily rectified of errors as soon as they are found.

The intent of Software Testing is to find the errors in a Software Application while it is being executed. These errors are called by different terminologies such as errors, bugs, deviations or defects (Bertolino 2007). A complete and comprehensive testing strategy checks each and every requirement that the software application intends to satisfy. Both Verification and validation are two processes that ensures that software meets the business requirements of the client as well as it works exactly as it is expected to.

Software systems are not error free. The reasons and chances of program errors are plentiful. Thus companies around the world are spending a big portion of their resources to make the software systems robust, reliable, scalable, safe, secure and defect free. This is where the significance of an economical and decisive software testing methodology becomes important. In spite of the best effort in the design, development and coding of software programs, some errors may persist in the software after its completion.  Ever increasing faults in the software applications at various stages of development has invoked interest in Software Testing. As a result, the importance of software testing as an integral part of the software development process has been steadily growing and it has been regarded as the standard technique in the Software Industry to ensure quality and verify the correctness of the software.

Formally, Software Testing is defined as “the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements and actual results”. Software testing plays a vital role in the software development process. Nearly one-third of the loss caused due to software errors can be easily eliminated by an improved testing infrastructure. A well designed testing process ensures that the software program performs as per the requirement under all conditions. The process of testing a software system is a humongous task which is not only time consuming but also very costly. It was estimated that software testing and debugging alone consume almost 50% of software system development resources. The quality of software can be enhanced by rigorously testing the program at the various stages of its development.

The types of software testing has been evolving continuously to match with the rapidity of increase in the complexity of software design, frameworks and programming languages.

Manual testing is the process of  testing the application manually by a person for detecting the  software. It requires a tester to play the role of an end user where they randomly check the application by keying in values and see if the application responds properly. To guarantee completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

Automation Testing:

Automation testing is a technique that automates the test cases using specific tools which can detect the bugs in less time. In Automation testing, the tester writes scripts by own and uses suitable software to test the software. It is basically an automation of a manual process intended to reduce the time and cost overhead in repeated testing.

A comparison of three basic software testing techniques is done in (Kuhn et al 2004). The techniques used are analysing the software structurally by complete statement coverage, testing the code line by line after abstracting it and equivalence partitioning and boundary value analysis, which are used as methods in functional testing. The fault detection capabilities of these three methods are analyzed in this paper. It was found that all three methods fared differently based on the professional capacity of the testers. Comparatively, code reading had better results when used by professional programmers, which indicates its slight efficacy over the other methods.

A neglected aspect of Software Testing is that as the use of the application increases, the resources which hosts the application can behave erroneously over time. This problem is analyzed and addressed in (Kuhn et al 2009). The paper evaluates the aspect of Combinatorial Testing, where the different combinations of the resources that host the application are determined and verified up to a pre specified level. This can help in detecting the possible errors early at the testing phase, as it ensures there would not be a problem due to the different combinations of resources used.

Regression Testing becomes mandatory for an application that has to undergo frequent changes. These changes may be due to the new requirements or new modifications to the existing application. It becomes mandatory to check the whole application each time a modification is made to ensure that the current application is not affected adversely by new changes. (Wong et al 1997) elucidates the different methods by which Regression Testing could be made simpler by reducing the number of test cases required to be run each time a modification happens. This is done by prioritizing and minimizing the test cases and using only a subset of them that recognizes the deviations specifically.

The significance of using Web Applications and methods to improve their quality is analyzed in (Ricca and Tonella 2001). A UML representation of the web application is proposed first. The UML representation helps in deriving the static site structure and also other high level details of the web application. These details could be used to generate a framework for white box testing and automate the test cases. Testing by this method increases the accuracy and reduces the effort in testing a Web Application. The effectiveness of using automated testing tools and a comparative study of these tools like Selenium, QTP and Test Complete is made in (Kaur and Gupta 2013). The study evaluates the performance of these tools specific to requirements such as Black Box Testing, Functional Testing and Regression Testing. Each tool has its advantages based on the requirements and the type of application.

In (Holmes and Kellog 2006) explains the use of Selenium as a tool of choice for testing Web Applications. The paper elaborates certain features of Selenium. Selenium as a tool has the ability to do Testing initially before the design and can determine when to start or stop User Acceptance Testing.  It also has the capacity to provide automated test suites for Regression Testing. An analysis is also made on the duration required to write the scripts in development environment and how far the functional requirements could be covered completely by the tool. Apart from its use as an automation tool for Web Applications, (Kongsli 2007) discusses the use of Selenium in Security Testing. Unmasking the susceptibilities of an application by creating test scripts that simulate anomalous behaviour is called Security Testing, which is discussed in length in the paper. Test scenarios and Test scripts are designed meticulously which demonstrate the use of the tool in creating such test scripts.  

The use of a Keyword Driven Framework is explained in (Singla and Kaur 2014) to automate the test scripts. A testing tool called ‘Selenium Webdriver’ is used to abstract the detailed functions necessary to automate the testing process. These concise functions are then populated into an excel file. The description as to how the program should run is also given in the excel sheet and the automation process is carried out more efficiently.

The different methods by which Unit Tests can be written in an improved way is investigated in (Cheon and Leavens 2002). The paper elucidates the methods by which a runtime assertion checker is used to determine the correctness of the test scripts. The use of specifications which abstract the elaborate test scripts is also considered here. Specifications are used instead of descriptive test code which makes the the testing process more concise and readable. This specification enabled Unit Testing with JML and JUnit is examined in depth here. How test case prioritization can increase the fault detection capability of a test suite is analyzed in (Rothermel and Kinneer 2004). An experimental setup where unit test cases are executed after test case prioritization is implemented here. The paper reiterates the fact that unit testing using JUnit after test case prioritization proves to be more cost effective. Prioritization ensures that the Test Cases are executed in a particular order which could reduce the overhead in Regression Testing. By using JUnit for Java programs, the fault detection capabilities of the Unit tests significantly improve.

Selenium

Selenium is an open source tool that gives a framework for testing Web Applications (Singh and Tarika 2014). The tool can be easily accessed, downloaded and used. Essentially, the Selenium tool is an add-on feature for the Firefox browser, and can be used to automate the test scripts written for a Web Application. The sample program selected here is the Mercury Tours Web Application that is provided along with QTP installation to learn the initial concepts of Web Application Testing.

Since it is a Web Application that is being tested here, a cost effective, simple and compact technique for testing should be adopted. Selenium meets the criterion of being cost effective, since it is an open source tool. Selenium also supports a variety of platforms, languages and browsers which makes it very handy for the testers to work with. As with other modern tools, Selenium also has an active community based support for its users. It is a package of different tools intended to address different testing issues and hence referred to as a Test Suite. It is relatively easier to use and since the scripts run directly on the browsers, the results can be seen by the users (Bruns et al 2009). The reporting feature of Selenium is also user friendly and allows the testers to analyze the results and take necessary steps to rectify them.

The basis of Selenium lies in a client – server architecture. The client component comprises of a WebDriver API and a RemoteWebDriverClass. The WebDriver API is used to interact with the elements of the application while the RemoteWebDriver class acts as an interface and corresponds with the Selenium server. Similarly the server comprises of a server component that receive requests, a WebDriver API that executes the test scripts on the server and a SeleniumGrid which ensures the cross browser and cross platform capabilities of the tool.

While testing the Mercury Tours Web Application, simple scripts can be used to run the tests directly within the browser. Thus the same test scripts could be used to test the same functionality in different browsers. For testing the login page of the sample application, the Selenium IDE provides a number of context menu’s and within the context the number of available commands could also been seen. There are certain assertion statements which ensure that the action performed is correct. The test scripts could be abstracted to HTML tables which make the testing process more concise and efficient. The dynamism of Selenium is illustrated by the Selenium IDE, also called the recorder which helps in the user to navigate and simulate the tests which may be done manually (Choudhary et al 2010). The server in Selenium also gives the option to write the test with any program which enables different functionality like conditional logic to be integrated into the automated testing process (Gundecha 2012).

There is a sequential order by which a test script is designed and executed in Selenium. First a WebDriver instance is created for the sample Mercury tours Web Application. Then an HTML element like an input box for the username is selected after navigating to the page. Once the HTML element is selected an action could be carried out on that particular element. Then the tests are run on the elements and the results are recorded. By creating test scripts for the sample application using Selenium, it can be determined that this tool is one of the best in the industry that can perform even miniscule user events. Because Selenium allows a great deal of freedom for the Tester to write their own logic, it provides the chance to do complex operations (Pajunen et al 2011).

Selenium proves to be the best among all the other tools that captures almost all the activities that a user performs in a browser. Since Selenium has the capability to add logic by using programming concepts, it is the tool of choice for many Testers in the industry. In addition to that Selenium and JUnit are open source tools which make it economical and viable for using them at will. Advantages like being portable, cross browser and cross platform friendly makes it the tool of choice for automating the test scripts in Regression Testing. The fault detection capabilities of the tool is also analyzed and found to be better than most other tools in the industry now.

• Kuhn, D. R., Wallace, D. R., & Gallo, A. M. (2004). Software fault interactions and implications for software testing. IEEE transactions on software engineering, 30(6), 418-421.
• Kuhn, R., Kacker, R., Lei, Y., & Hunter, J. (2009). Combinatorial software testing. Computer, 42(8).
• Wong, W. E., Horgan, J. R., London, S., & Agrawal, H. (1997, November). A study of effective regression testing in practice. In Software Reliability Engineering, 1997. Proceedings., The Eighth International Symposium on(pp. 264-274). IEEE.
• Ricca, F., & Tonella, P. (2001, July). Analysis and testing of web applications. In Proceedings of the 23rd international conference on Software engineering(pp. 25-34). IEEE Computer Society.
• Kaur, H., & Gupta, G. (2013). Comparative study of automated testing tools: Selenium, quick test professional and testcomplete. International Journal of Engineering Research and Applications, 3(5), 1739-43.
• Holmes, A., & Kellogg, M. (2006, July). Automating functional tests using selenium. In Agile Conference, 2006(pp. 6-pp). IEEE.
• Kongsli, V. (2007, October). Security testing with Selenium. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion(pp. 862-863). ACM.
• Singla, S., & Kaur, H. (2014). Selenium Keyword Driven Automation Testing Framework. IJARCSSE ISSN, 2277.
• Cheon, Y., & Leavens, G. T. (2002, June). A simple and practical approach to unit testing: The JML and JUnit way. In ECOOP(Vol. 2374, pp. 231-255).
• Do, H., Rothermel, G., & Kinneer, A. (2004, November). Empirical studies of test case prioritization in a JUnit testing environment. In Software Reliability Engineering, 2004. ISSRE 2004. 15th International Symposium on(pp. 113-124). IEEE.
• Singh, I., & Tarika, B. (2014). Comparative analysis of open source automated software testing tools: Selenium, sikuli and watir. International Journal of Information & Computation Technology, 4(15), 1507-1518.
• Bruns, A., Kornstadt, A., & Wichmann, D. (2009). Web application tests with selenium. IEEE software, 26(5).
• Choudhary, S. R., Versee, H., & Orso, A. (2010, September). A cross-browser web application testing tool. In Software Maintenance (ICSM), 2010 IEEE International Conference on(pp. 1-6). IEEE.
• Gundecha, U. (2012). Selenium Testing Tools Cookbook. Packt Publishing Ltd.
• Pajunen, T., Takala, T., & Katara, M. (2011, March). Model-based testing with a general purpose keyword-driven test automation framework. In Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on(pp. 242-251). IEEE.