Evaluation Methodology And Procedure For Usability Of Interactive System

1. Identifying the use cases or aspects of the functioning of the system to be considered, and briefly describing them in your report. (These don’t need to be a complete set of use cases; for very complicated systems focusing on one part of what they do is just fine. However you should give a clear indication of what subset of the functionality of the system you are considering, and what you are not considering. If in doubt, cover less functionality in more detail.)

2. Choosing an evaluation methodology. You should apply a standard evaluation methodology such as user testing, cognitive walkthrough, or heuristic evaluation. (You need to show that you know what the evaluation methodology is; not knowing what evaluation methodology you have been asked to use or are using is a common cause of assignment failure. If you want to do something non-standard, ask advice from your tutor.)

3. Defining an evaluation procedure. This will include stating one or several user tasks to be tested or considered with exact descriptions of the scenario and the goal the user is trying to achieve, as well as what the evaluator will do to collect results and produce an evaluation. The evaluation procedure needs to be described in full, separately from the description of the results. For a user trial, this involves describing the exact wording of the instructions given to the subjects. For a heuristic evaluation this involves being clear about the heuristics being considered; using a set that is more detailed and concrete than Nielsen’s set of ten principles is a good idea.

4. Carrying out the evaluation. This will involve applying the procedure and documenting what happens, and what the procedure finds. (If applying your procedure looks like an excessive amount of work, or is producing an excessively large volume of documentation, ask advice; we would prefer an evaluation giving detailed insight into part of the functionality to an evaluation with broad coverage but a thinner or more superficial analysis.) Aim to be as detailed and exact as possible about what the problems are and when and where they turn up.

5. Deriving findingsabout the usability of the interactive system from the results of the usability evaluation. This should include consideration of how strong and how general the conclusions are.

The interactive system is defined as the computer system that have a significant interaction between the computers and the humans. As an example of graphical interactive system are the operation system that are installed in the computers. The application software such as CAD, data entry systems, simulation software requires high level interaction of the user with the system. The IDEs and the web browsers can also be considered as an example of the interaction system (Preece, Rogers and Sharp 2015). About 90 percent of the effort in development of the technology is given on enhancing the interface and improve the interaction of the user with the developed system.

The preference of the people and the feature that can attract more number of people should be understood for the development of the interface of the system. There is a need to analyse the needs of the user for the development of the interface and align the interface with the developed system. The construction of the unambiguous icons needs to understand and the common errors that are the part of the usability of the system should also be identified for the development of an interactive system (Card 2017). Previously command line interface was used as an interactive system that have a controlled interaction between the computer and the user. In the command line interface the user using the system needs efficient knowledge about the commands or the arguments for getting the desired output. The data is needed to be inputted in the system in a sequential manner. There was a limitation for getting the output but currently there is a development with the generation of agents and the virtual system and different types of system such as direct manipulation system and embedded systems has been developed for the improvement of the interaction and get the maximum output from the system (Lazar, Feng and Hochheiser 2017).

For the evaluation of the usability of a system information should be gathered using different types of methodology for gathering user feedback about the website created or the site that is planned to create. There are different usability evaluation methodology that can be applied and are given below:

Usability test result reporting – The results of the usability testing should be prepared by focusing on the recommendation and findings that can be differentiated with the severity level (Grenha Teixeira et al. 2017). The pertinent information obtained from the test planning should be enough for methodology.

Test participant recruitment – Participants should be recruited those are expected to use the website or have precious experience of using similar type of sites. The participants should be dependent on the product or the site and there can be a multiple potential groups or users (Roedl, Odom and Blevis 2018). Each of the group should have representatives for optimizing the test performance.  

Usability test planning – It is one of the essential factor and a test plan is needed to be created for proceeding with the testing.

First click test – In this test the components of the interface that is clicked first by the user for the completion of the intended task. For proceeding with the testing a prototype of the website, wireframe or the functional website can be used.

Usability Evaluation Methodology

System usability scale – It is considered as a reliable tool that measures the usability and it mainly consists of a set of 10 questions in the questionnaires. Five response is taken from the users with the different options that ranges from strongly agree to strongly disagree.   

Expert review and heuristic evaluation – In this method the site is reviewed by the expert and compared with the principles of acceptance of the usability (Woods 2018). The result of this type of analysis is identification of the potential issues in the system.  

The main focus for the evaluation of the usability is the identification of the satisfaction level of the user using the system. Information is gathered for the identification of the level of satisfaction using different methods such as gathering feedback from the users about the existing sites and the plan site. The usability is also termed as the identification of the quality of the experience that the user gains while interacting with the system or the product i.e. application, software or websites (Fidas et al. 2015). The effectiveness, satisfaction and the efficiency is the main components that are evaluated for the identification of the usability of the developed information system. For the evaluation of the usability it should be noted that the usability is not a single property but it is a combination of different factors i.e. intuitive design, efficiency of usage, frequency of error, learning ease, memorability and subjective satisfaction (Abdelnour-Nocera et al. 2015). There are different evaluation methodology that can be used for the analysis of the usability for the user centred design and with the use of the methodology an assistant can be used for developing the visual design, architecture, user satisfaction and interaction design.

There are different opportunities for performing the test and it includes the usability testing for the baseline of an existing site, focusing on a targeted group of people for interviewing them for the establishment of the goals of the users. The application of the card slot testing can help the IA development. Navigation is used for the development of the wireframe and it is tested for analysing the satisfaction level of the users (Bunch et al. 2015). The control design and the work design should be created for facilitating the design and spatial awareness should be used for solving different problems faced during the development of the interface design of the system. The evaluation of the interface design should be based on the two different types of data such as qualitative and quantitative (Glas, Kanda and Ishiguro 2016). On gathering the data from the system and the user the usability of the website can be evaluated and recommendation regarding the improvement of the system can be provided. The recommendations are implemented and tested for measuring the effectiveness of the changes made for aligning the requirement of the user with the developed interface.   

A quality framework can be used for the evaluation of the usability and it mainly focuses on the needs of the usability practitioners for accepting and adapting the methodology. The information system is evaluated for the measurement of the attributes of user acceptance, usability, functionality, efficiency and reliability of the developed system (Endsley 2016). The usability evaluation is used for the identification of the error rate, efficiency, satisfaction and learnability of the system. International standards should be used that complies with the different rules of usability and the communication should be evaluated with the application of different standardization. It has been identified that the following are the five standards of ISO that are used for addressing the usability are listed below:

Definition and description of evaluation procedure that is needed to be applied

ISO /IEC 9126 (2000) – It is used for the analysis of the product quality and act as a quality model.

ISO / IEC 4598 (1999) – It provides a general overview for the evaluation of the software product.

ISO 9241 (1998) – It provide a guide for the usability and manages the ergonomic requirement for the management of the office work with the application of visual display terminal.

ISO 13407 (1999) It is used for the management of the human centred interactive system design

ISO 18529 (2000) – It is used for providing a description of the human centred life cycle and identification of the Ergonomics.

The scopes of the application of the international standards should also be analysed for the identification of the elements and such as;

• The entity under the evaluation
• Stakeholders for addressing the standard
• Different phases of the life cycle.

Different approaches can be used for examining the details and the guidelines and by comparing the different approaches the following observation is made.

ISO /IEC 14958 is used for providing a general process for the management of the quality of the software product. The different phases involved for the development of the project should be well defined and a project plan should be created for the identification of the input and output of each of the activity (Höök et al. 2015). Different quality models should be used and it should be specified for reaching to the final stage of the product.

ISO 9241 is used for distinguishing the interface during the evaluation and the design phase of project.

ISO 13407 is used for addressing the issues in the design and maintain a long term monitoring of the design comparing it with the human centred design (Mast et al. 2015). The standard provides a guidance for the development of the framework for identifying the different situation and objectives for addressing the usability issues.

ISO 18529 is used for the evaluation of the design objectives by comparing the requirement as a human centred design process. All the phases involved in the development process are evaluated and the standard is used as a guide for performing different activity at the different stages of evaluation.

CH MiLE methodology is used for the web applications and this methodology is a result of a joint research group at Politecnico di Milano. It is developed from the quality framework evaluation and is integrated with the concept of novel framework for the detection of the problems in the usability of a system (Baecker 2014). The MilE provides a conceptual framework for supporting structured and systematic evaluation procedure. Two perspective is sued for the evaluation of the usability of the interface that is the technical, application independent, neutral and the user experience or application dependent perspective. The technical inspection is done considering the design elements that can be used typically and it exploits the technical heuristic in built library that are bundled with a set of operation guidelines for the suggestion of the inspected task elements (Cooper et al. 2014). Navigation. Content, Performance and interface design are considered as a design dimension for the evaluation of the website usability. The user experience evaluation is done focusing on the user aspect and focusing on the expected outcome of the system. Different indicators are used for the evaluation of the usability such as Content experience indicator, cognitive and navigation experience indicator and operational flow experience (Vallgårda et al. 2015). The processing capability of the human needs to be understood by the designers and the programmers because the sufficient knowledge in the programming knowledge is not enough for the improvement of the effectiveness and efficiency of the computer software.

The usability of the ecommerce system should be evaluated for the management of the status of visibility and creating a match between the real world and the system. The user should be provided freedom for controlling the different operation of the ecommerce website and avoid the unwanted state to reach the main objectives. The website should be opened in different device and platform for the identification of the errors and flexibility in the design (Rapoport 2016). The application of the hybrid approach helps in understanding the different issues faced by the user using the website and a group of friendly user should be involved in the testing for the collection of evidence about the usability of the system. The goals and action of the user should be created and a walkthrough should be made for the identification of the issues in the usability. The issues should be consolidated and it should be rated for analysing the issues for the identification of the standards of the consistency. A scenario should be developed for performing the test and the application of the DEPTH approach helps in evaluation of conceptual framework (Pan et al. 2015). The following activity should be involved for the evaluation such as gathering the requirement of the design pattern that is related with the domain. A list consisting of the user centred task should be generated for the management of the design pattern aligning the pattern of functionality. The criteria of usability should be applied for the developed scenario and a toolkit such as set of questionnaire should be used for quantitative and qualitative analysis of the website.

The toolkits should be used for gathering information about the user requirement and the performance of the website. The results obtained from the toolkit should be gathered and the global and partial preference of the user regarding the quality should also be evaluated for the identification of the site preference using the LSP approach (Kim 2015). The data gathered from the analysis should be evaluated for presenting the results and supporting the test of usability comparison for the development of the e commerce website. From the analysis a series of design pattern has been found for example:

Shopping cart – It used by the website to add the products the user want to purchase and allow the user to add more product such that the user can review the product and make payment for all the products at a time.

Advisory /Recommendations – The search pattern of the user is tracked and stored in the database for helping the user to search the specific product. It also helps the user to get assistant from the website and increases the usability of the website.

Linking Opportunities – The user should be attracted to use the website and navigate through the different webpages that the user is looking. New technology should be used and innovative ideas should be applied for improvement of the look and feel of the website and meet all the needs of the user.

Explicit process – It should be used for helping the user to assist for purchasing the product and complete the payment. The shopping cart is used for the management of the orders and generate the details of the order for verification of the product and maintain a transparency in the current business.

A web application and similar requirement is used for creating a navigation structure and the interface is designed following the consistency standards. Similar font size of the text are maintained such that the user does not face any difficulty while reading the content of the website. Similar header and footer should be kept for each of the webpage linked with the website such that the user does not feel redirected while navigating from one page to another while finding their product and completion of the purchase of the product (Patterson et al. 2016). The use of text in the website should be minimum and more number of relevant images should be used such that the all the contents of the website is visible to the user at a glance. The images of the product should be clear and the dimension of the product should be mentioned such that the user have all the data before they purchase the product. More than one image should be used taken from all the angles such that the details of the product can be viewed by the user.

The search option and listing the product by application of different types of filters should be available such that the user can narrow the search for finding the appropriate product. The preference of the user should be given and the feedback regarding the usability of the website should be taken for the improvement of the system and integrate with the different functionality for improvement of the current business process (Alavi et al. 2016). Third party application can be integrated with the system for increasing the effectiveness and efficiency such as learning management system, AI for automating some of the options by analysing the behaviour of the user and reduce the effort of the user.

The interface of the ecommerce website is designed following the eight golden rules that is the website is consistent with the use of familiar icons and hierarchy of menus that are used for getting an expected sequence of action. The way of information that is conveyed to the user is standardized and the information are consistent for helping the user to reach their goals easily (Oppermann 2017). Shortcut methodology is applied that would help the user to complete their intended task in less time i.e. the user can reach the goal in tree steps with les effort. Proper error handling methods are used for providing the user a readable feedback and about the wrong action taken by the user. Meaningful message are returned that would help the user to avoid the action and it increases the usability of the website. Design dialogue are created that would help the user informed about the usability and they do not have to guess about the action they would take and their result. The application of simple error handling in the website helps in reduction of the errors the users are guided step by step for solving their problem and management of the input from the user (García-Peñalvo and Durán-Escudero 2017). The action taken by the user should be reversible and the designer have aimed to create different point of failure for the management of sequence of action. The reduction of the memory load for the short time helps to reduce the response time and increase the efficiency of the design interface (Yi, Jiang and Benbasat 2015). The user using the website have the full control and their trust can be earned by behaving as the expected outcome and meting all the needs of the heuristic design of interface.

With the comparison of the standards with the scopes of the usability of an interface the following interpretation are discussed:

• ISO /IEC 9126 and ISO /IEC 14598 (ISO /IEC 25000) – Its main focus is on evaluation and definition of the quality of the different king of the products that may include application software or the data in the firmware. The software product is focused and thus the scope is narrow when compared with the other standards because the other standards take into consideration both the hardware and the software for the evaluation of the usability of the interface (Candello and Pinhanez 2018). On the other hand it covers all the quality needs required for the development of software product including its usability.
• ISO 9241 – It mainly focuses on the interface usability of the component that is needed to be a part of an interactive system irrespective of the type of its nature that is software, service or hardware. It finds its application in the virtual display terminals for eliminating the errors and increase its efficiency.  
• ISO /IEC 9126, ISO 9241 and ISO /IEC 14598 focuses on the needs of the stakeholders who are involved in the different phases of the software development life cycle model for the development or procurement of the software product. It helps in improvement of the interactivity of the stakeholders and aligns their needs for addressing the different needs and develop the interface according to it (Li et al.2016). The ISO /IEC 14598 and the ISO /IEC 9126 is used for the creating a guideline during the maintenance phase of the developed system.
• ISO 13407 – it mainly focuses on the development of computer based design of the interactive system by evaluation the successful system design solutions and following the same software development methodology for eliminating the unwanted risk during the development of the project (Bennett and Hoffman 2015). The project manager and the stake holders are involved in the different phases of software development.
• ISO 18529 – It can be differentiated from the other available standards because it is created for the analysing the design process and there are different other standards that are used for addressing the hardware or the software issues such as how to implement, design and assess the needs of the elements required for the development of the interface of the system.

It should be considered that the usability is the main criteria for the success of a website or other information system irrespective of its type and field. There are different findings and guidelines that can be followed for the improvement of the usability of a system. For finding the usability of the interface designed for the system focus should be given on the requirement of the users and a group of friendly users should be created who have knowledge about the human computer interaction (Kolling et al. 2016). Training should be provide to the user regarding the usability of the information system for performing a professional evaluation. The inspection should be made following the different fields such as technical, user experience, scenario, production, etc. The simulation software can be used for the analysis of the usability and logical assumption are made for the management of the test (Cheng et al. 2016). The main features identified for the website is the usability of the website and the level of difficulty faced by the user to reach their goals and objectives.

The search pattern of the user can be tracked for suggesting new items related with the item searched by the user. There are different types of testing that are performed for finding the usability such as pre testing the homepage, analysis of the navigation, analysis of the accessibility and evaluation of the response time. The usability of the website is analysed by comparing it with the successful websites and identifying the different features that are available in the website used for handling the errors and management of the input from the user (Grudin 2017). There are different methodology that is used for finding the usability and usability evaluation scheme are used for finding the general acceptance with respect to the Shneiderman Eight Golden Rules for the development of the interface design. The similar types of websites are evaluated and the following table is created for the evaluation of the website.

Category	Description
Transparency	The action taken by the user should be predicted and the layout of the website should be created such that it can record all the details that are needed by the user. Minimizing the navigation and creating predictive action after taking input of the user can improve the transparency of the website and thus the efficiency of the website is improved.
Consistency	The consistency of the website is analysed by finding the information available in the website that are relevant with the product and service. The usability of the menu and the hierarchy and grouping of thee content is important for creating a spatial layout. A pre-defined set of colour and proper formatting should be used for the management of the consistency.
Cognition	The text used for the description should be kept short and it should be relevant for increasing the density. A minimalistic design approach should be used and effort should be given on formatting and the contrast colour used such that it uses. A conceptual model should be created for the management of the environment with respect to the context and meeting the visual needs of the users.
Control	The user should be able to run parallel tasks such that they needs less time for completion of the task. The user should also be provided the direct access of the data depending upon the hierarchy level (McNamara and Klein 2016). In case of failure the user should be able to reverse the action and reach the consistent state.
Terminology	The language used for the development of the interface should be meaningful and it should be compatible with the established terms and local variations.
Customization	The user should be able to customize the user interface according to their needs such that the efficiency of the system is improved. This can be done with the application of different filters and management of the persistence for the different session established between the user and the website.
Workflow	The data entry, navigation completion of the intended task is important for the management of the flow if the information between the different pages of the e commerce website (Saariluoma 2015). The user should be able to find all the relevant product and information for purchasing the product from the website.
Error Handling	The invalid input from the user should be manged with the implementation of proper message. The state of the user should be saved for preventing the data loss and management of the errors in the system.

References:

Abdelnour-Nocera, J., Baricelli, B.R., Lopes, A., Campos, P. and Clemmensen, T. eds., 2015. Human Work Interaction Design: Analysis and Interaction Design Methods for Pervasive and Smart Workplaces: 4th IFIP 13.6 Working Conference, HWID 2015, London, UK, June 25-26, 2015, Revised Selected Papers (Vol. 468). Springer.

Alavi, H.S., Churchill, E., Kirk, D., Nembrini, J. and Lalanne, D., 2016. Deconstructing human-building interaction. interactions, 23(6), pp.60-62.

Baecker, R.M. ed., 2014. Readings in Human-Computer Interaction: toward the year 2000. Elsevier.

Bennett, K.B. and Hoffman, R.R., 2015. Principles for interaction design, Part 3: Spanning the creativity gap. IEEE Intelligent Systems, (6), pp.82-91.

Bunch, L., Bradshaw, J.M., Hoffman, R.R. and Johnson, M., 2015. Principles for human-centered interaction design, part 2: Can humans and machines think together?. IEEE Intelligent Systems, 30(3), pp.68-75.

Candello, H. and Pinhanez, C., 2018, July. The Role of Dialogue User Data in the Information Interaction Design of Conversational Systems. In International Conference of Design, User Experience, and Usability (pp. 414-426). Springer, Cham.

Card, S.K., 2017. The psychology of human-computer interaction. CRC Press.

Cheng, Y., He, F., Wu, Y. and Zhang, D., 2016. Meta-operation conflict resolution for human–human interaction in collaborative feature-based CAD systems. Cluster Computing, 19(1), pp.237-253.

Cooper, A., Reimann, R., Cronin, D. and Noessel, C., 2014. About face: the essentials of interaction design. John Wiley & Sons.

Endsley, M.R., 2016. Designing for situation awareness: An approach to user-centered design. CRC press.

Fidas, C., Hussmann, H., Belk, M. and Samaras, G., 2015, April. iHIP: Towards a user centric individual human interaction proof framework. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2235-2240). ACM.

García-Peñalvo, F.J. and Durán-Escudero, J., 2017, July. Interaction design principles in WYRED platform. In International Conference on Learning and Collaboration Technologies (pp. 371-381). Springer, Cham.

Glas, D.F., Kanda, T. and Ishiguro, H., 2016, March. Human-robot interaction design using Interaction Composer eight years of lessons learned. In Human-Robot Interaction (HRI), 2016 11th ACM/IEEE International Conference on (pp. 303-310). IEEE.

Grenha Teixeira, J., Patrício, L., Huang, K.H., Fisk, R.P., Nóbrega, L. and Constantine, L., 2017. The MINDS method: Integrating management and interaction design perspectives for service design. Journal of Service Research, 20(3), pp.240-258.

Grudin, J., 2017. Obstacles to participatory design in large product development organizations. In Participatory Design(pp. 99-119). CRC Press.

Höök, K., Dalsgaard, P., Reeves, S., Bardzell, J., Löwgren, J., Stolterman, E. and Rogers, Y., 2015, April. Knowledge production in interaction design. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2429-2432). ACM.

Kim, G.J., 2015. Human–Computer Interaction: Fundamentals and Practice. Auerbach Publications.

Kolling, A., Walker, P., Chakraborty, N., Sycara, K. and Lewis, M., 2016. Human interaction with robot swarms: A survey. IEEE Transactions on Human-Machine Systems, 46(1), pp.9-26.

Lazar, J., Feng, J.H. and Hochheiser, H., 2017. Research methods in human-computer interaction. Morgan Kaufmann.

Li, K., Tiwari, A., Alcock, J. and Bermell-Garcia, P., 2016. Categorisation of visualisation methods to support the design of Human-Computer Interaction Systems. Applied ergonomics, 55, pp.85-107.

Mast, M., Burmester, M., Graf, B., Weisshardt, F., Arbeiter, G., Špan?l, M., Materna, Z., Smrž, P. and Kronreif, G., 2015. Design of the human-robot interaction for a semi-autonomous service robot to assist elderly people. In Ambient Assisted Living (pp. 15-29). Springer, Cham.

McNamara, L.A. and Klein, L.M., 2016, May. Context-sensitive design and human interaction principles for usable, useful, and adoptable radars. In Radar Sensor Technology XX (Vol. 9829, p. 982906). International Society for Optics and Photonics.

Oppermann, R., 2017. Adaptive user support: ergonomic design of manually and automatically adaptable software. Routledge.

Pan, Y., Roedl, D., Blevis, E. and Thomas, J., 2015. Fashion thinking: Fashion practices and sustainable interaction design. International Journal of Design, 9(1).

Patterson, M., Bond, R.R., Mulvenna, M., Reid, C., McMahon, F., McGowan, P., Cowan, K. and Cormican, H., 2016, September. A Web-based Human Computer Interaction Audit Tool to Support Collaborative Cognitive Ergonomics Within Interaction Design. In Proceedings of the European Conference on Cognitive Ergonomics (p. 40). ACM.

Preece, J., Rogers, Y. and Sharp, H., 2015. Interaction design: beyond human-computer interaction. John Wiley & Sons.

Rapoport, A., 2016. Human aspects of urban form: towards a man—environment approach to urban form and design. Elsevier.

Roedl, D., Odom, W. and Blevis, E., 2018. Three principles of sustainable interaction design, revisited. In Digital Technology and Sustainability (Vol. 17, No. 30, pp. 17-30). ROUTLEDGE in association with GSE Research.

Saariluoma, P., 2015. Four Challenges in Structuring Human-Autonomous Systems Interaction Design Processes. Issues for Defence Policymakers, p.226.

Vallgårda, A., Winther, M., Mørch, N. and Vizer, E.E., 2015. Temporal form in interaction design. International Journal of Design, 9(3), pp.1-15.

Woods, D.D., 2018. Toward a theoretical base for representation design in the computer medium: Ecological perception and aiding human cognition. In Global perspectives on the ecology of human-machine systems (pp. 157-188). CRC Press.

Yi, C., Jiang, Z. and Benbasat, I., 2015. Enticing and engaging consumers via online product presentations: The effects of restricted interaction design. Journal of Management Information Systems, 31(4), pp.213-242.