Usability Evaluation Of FIFA Game Application: Methods And Results

Your firm of interaction design consultants is trying to build up a portfolio of impressive work, to enable it to pitch for business convincingly in the future.

Your task is to produce a usability evaluation of an interactive system by carrying out user testing, plus a presentation of your results. You have a completely free choice of what interactive system you evaluate

What is an interactive system?

Conducting a usability for an interactive system is very essential as this help the developers and owners of that system identify the existing risks as well as the perception of the users towards it. An interactive system therefore is any computer application that allows different users to perform various functions with it. Some of the interactive systems therefore include: computer games, E-commerce websites and other software that provide automated services to different users (Halgren, 2009). During development of any application, a usability test is carried out before the application is commission for use or production. These usability testing includes the learnability of the application, efficiency of the application in performing its tasks , the consistency of its design, existing errors and their capabilities to hinder effective service delivery using the software. All these aspects of usability acceptance test determine whether the application meets the user requirements and whether the developed system satisfies their needs (McCracken, 2016). During usability testing, the developers also have an opportunity to gather the users feedback based on what they see, listen and record. This helps in knowing what ought to be improved in order to produce a quality software.

This research therefore focuses on the usability test of FIFA game application. This is a game application developed by EA Sport and it is meant for entertainment purposes by providing a fiction or simulation of what happens in the real football contest. This interactive game application provides various functionalities related to football matches and different contests hence making it more close to its users. This report therefore, discuss some of the main functionalities of this game application and analyze these functionalities for various usability features. The evaluation methods used during this usability test are also discussed in this report and they include: observations, interviews and the use of questionnaires. Once the evaluation process is done, the report provides the results obtained and analyzing this results by looking at the general view of human interaction with this system based on the results obtained (Mifsud, 2016). In addition, the researcher also highlight some of the recommendations provided by the participants of this usability test for the betterment of the FIFA application.

This game application, has been developed in such a way that it simulates every action that takes place in the soccer world into a computer application.

The application uses the cartoons of the existing real football players such as Messi, Ronaldo among others as well as the names of the existing football teams in the world. With this application, it is possible for a single user to select a team and play against another team controlled by the computer. Also, two users can both select their football teams and customize them by arranging their formation and controlling their players while playing against each other. With this application also the users are able to create careers where they can choose to act as the football clubs managers or create players of their own. FIFA application allows the users to customize various application settings such as control buttons, the screen size and the language to be used among others. All these functionalities provides an entertainment interaction between the users who in most cases are football lovers with the FIFA application hence need for need for usability test. The figure below therefore shows some of the functionalities that FIFA application user can perform as defined in this system.

Importance of Usability Testing for Interactive Systems

Figure 1: FIFA use Case

From the above use case, it can be seen that there exist many functionalities for the FIFA application and therefore it is not possible to evaluate all the functionalities in this research (Schumacher, 2010). Thus the research will only involve evaluation of the main functionalities of this application by finding out how users of this application navigate or interact with the said functionalities when playing simulated soccer using this system.

There are many ways of carrying out software usability testing. These methods include; interviewing the users, use of test scenarios to be performed by the participants and use of questionnaires to gather information (Rogers, 2015). For this research therefore, a selected 10 users participated in this research which took a maximum of 4 hours to complete. This research population consisted of both the real football fans and those who were not fans of any football club but had an idea of what happens in the football pitch. Out of the 10 participants three of them were female of age between 14 to 18 years and the rest were male of the same range of years as the female participants (Salvo, 2018). Each of these participants were assigned each a computer installed with FIFA 14 application and a game pad. During this evaluation process, the participants were assigned a number of tasks to perform while the researchers observe and take note of the experience. Afterwards, an interview was carried out to get the participants feedback that could be captured during observation. Also considering that these participants, were all computer literate and majority of them had in one way or another interacted with gaming software such as PS and GTI, they all had to have knowledge on how to use the game controller (game pad) provided. The following scenarios were therefore given to the participants to perform while the researchers gathers the feedback (Norman, 2014).

The first test scenario, required the participants using the FIFA icon on the desktop to double click on it and launch the FIFA application, once the application has opened the participants were required to select the language of their choice from the dropdown list and following the steps shown in the application proceed to the FIFA application Home Page as shown below by setting up their FIFA setting that they will use to play.

 

Figure 2: Home Page

Using the top menu bar, the participants were required to navigate to the subsequent windows through the application by familiarizing themselves with the information contained in each window and analyze the clarity of the information used.

Evaluation of FIFA Game Application

Using the PLAY button on the application main menu, the participants were required to select the KICK OFF option from the list provided. Here, the participant were to select a team of their choice that will play against an automatic team selected by the computer. On this scenario the half of the participants were to play against the computer team while at home ground while the other half were to play as an away team. The away team players were assigned away kits by the participants. This was supposed to be done by scrolling left until the away kit is found as shown in the screenshot below.

 

Figure 3: Select Kit

Once this was done the participants were to proceed by clicking ENTER on the keyboard until the following window appears.

 

Figure 4: Game Settings

On this window the participants were required to click on the Team Management where the participants were required to change the formation of their team players as well as substitute an player that they needed to. Once this was done using the escape button or the back button on the game controller, the participants were required to go back and click on the game setting, choose Semi Pro for the game level, Summer for the season and Overcast for the weather. Once this setting had been done the participants were required to go back and click PLAY MATCH for the game to start. The participant was only required to play for only 4 minutes before exiting the game.

Here the participants were required to create their own careers by choosing the league of their choice. The research population was divided into two with one group creating careers as team managers while the other group were required to create their own players. Once this was done the groups interchanged the roles by each group again creating their careers using the opposite objects as previously used. For both the player and the manager roles, the participants were required to give unique features to these objects by changing their physique, define the type of clothes they will wear and defining their skin colors and Save their details. A sample screenshot of the career creation window that was to be used by the participants to create both Manager Career and Player Career is as shown below.

 

Figure 5: Career Window

These tasks were carried out for about 3 hours and the remaining time was used to interview the participants based on their experience with the FIFA 17 application.

Usability Test Evaluation Methods

During the interview, the researchers were so keen in capturing the feelings of the participants on various usability features of this application. These features include the ease of navigation using the provided navigation features such as the menu bar and the hyperlinks (Quesenbery, 2006). Another feature analyzed during the interview is the layout of the application and the compatibility of the information used to describe various functionalities of FIFA 17 application. The participants were also asked about some of the challenges they might have encountered during the interaction and what suggestions they had on improving features of this application in order to enhance its interaction with the gamers. The interviews consisted of both the open-ended questions and closed-questions. This ensured that the participants were not limited or hindered from airing their feedback about this application.

All these methods provided a firsthand reactions and feeling of the participants about various interaction issues when using this application and this was helpful in determine the overall user acceptance result for the FIFA 17 software.

During the research, the evaluation process was carried out successfully and all the selected participants were involved fully throughout the process. This therefore, means that the researchers were able to gather many results both through observing the participants performing the evaluation task and analyzing the information gathered during the interview (laitinen, 2005). Therefore, the following shows the results obtained during the evaluation exercise for both the tasks and the interview carried out.

This task was carried out in order to evaluate the ability of the participants to start a FIFA 17 application and navigate through to the home page where they could execute its main function. During this process it was observed that most of the participants that is 9 out of 10 were able to launch the application without facing any challenges while only one got stack during the process and had to seek assistance from one of the participant. The challenged faced by this user was the inability to differentiate SPACE and Enter as in some step the application required the user to use a space button on the keyboard or a corresponding button on the game controller. Apart from this challenge, no other challenges was encountered during the execution of this task. Therefore, from the result 90% of the research population successfully launched the application and proceeded to the homepage section while just 10% needed help.  The screenshots shows the observation made during the execution of the application by the participants.

Test Results and Analysis

 

Figure 6: FIFA Launch

 

Figure 7: Language Selection

 

Figure 8: FIFA File Settings

This test was aimed at finding out if the participants were able to start a game and customize the game and team settings. During this process it was observed that all the participants were able to select their teams both for away and home games. But during the settings, only 7 participants were able to customize the settings as per the requirements successfully while the other three did not succeed to change the game parameters as defined in the scenario. The three participants cited various challenges such as inability to navigate back from the setting window as they could not comprehend which of the buttons could take them back. Also they were unable to change their team’s formation as they did not know how to select the formations provided from the list hence making them unable to complete the task successfully. The other 7 participants, performed this tasks easily and they did not encounter any challenges. All the seven participants that performed this task successfully were all male while those who faced challenges were female participants. The figure below shows a screenshot of the game that was created by one of the participants.

 

Figure 9: FIFA Match

This task was aimed at evaluating the capabilities of the FIFA 17 application users to create a league of their choice by creating their player or acting as the team’s manager. During the execution of these tasks, it was observed that only 6 out of the total population were able to create careers as managers as the rest 4 did not. On the other hand only 5 participants were able to create their league career by creating their own players successfully. For those participants who failed to create their careers, they faced challenges when trying to customize the objects that is the manager and the players and gave up making them not to succeed in achieving the tasks goal. These participants cited difficulties in determine what controls to use when customizing the physical appearance of the players and the managers. Out of the six that created their player careers, two of them were female participants while the other four were their male counterpart. The table below therefore shows a summary of the results obtained for this test scenario.

Results	No. of Participants (Total 10)
Successful	5
Unsuccessful	5

Table 1: Results for Player Career Creation

Evaluation Results Obtained	No. of Participants (Total 10)
Successful	6
Unsuccessful	4

Table 2: Results for Manager Career Creation

Participants Feedback and Recommendations

From this results a population of above average was able to perform the task successfully and this provided a positive feedback from them. Those who were unable suggested that the developers of the application should provide a readily available help section that will enable them to check for assistance when stack in creating their soccer careers.

The interview carried out was based on getting feedback from the sampled population on the learnability of this application, its design layout, functionalities and any available errors they might have encountered. The following are the results obtained during the interview session.

When asked about the learnability of the FIFA game application, 6 out of 10 said that it was easy to understand this application’s functionalities and following the step by step instructions for every task, it was therefore possible to learn and use the application. 3 of these participants felt that the functionalities were a little bit difficult and a help feature was needed for reference purposes. The summary of this results are as shown below.

Learnability	Results
Easy to learn	6
Slightly difficult	4

Table 3: Learnability Results

On the design the all the participants were satisfied with the way the application is designed and none of the participants had an issue with FIFA 17 application design. This 100% design acceptance was a great positive feedback from the evaluation process.

All the functionalities of this application were satisfactory to the participants as noted during the interview. The participants said that the application observed completeness of all the functionalities that were subjected for testing during this process. There were no errors encountered by the participants that hindered them from performing their tasks as all the participants gave a positive feedback during the interview.

Based on the above analysis of the results obtained during the evaluation process of FIFA 17 game application usability test a general finding or overall results for this user acceptance test was derived. From these results it can be seen the number of participants that felt satisfied with different aspect of usability features of this application were above average (Naji, 2016). It is also evidence that 50% of the population found it easy to learn how to use FIFA 17 application. 60% created their managerial careers successfully, 100% did not find any errors related to this application and all the participants were satisfied with the application design and functionalities.  In order to understand the general findings, the following bar graph shows the overall acceptance test result as analyzed.

Conclusion

Figure 10: Overall Evaluation Results for FIFA 17 Game Application

From the above bar graph it is evident that FIFA 17 application has passed various usability features test with no errors recorded and just few response on the ability to perform its entertaining functions. This positive results in general makes FIFA 17 game application a system that enables human interactions swiftly with fewer issues that are easy to address. It therefore serves its interaction entertainment functions well thus making it a better interactive software for use by the lovers of FIFA games.

There were not many recommendations made for this application during the research process since there were no errors or great difficulties encountered by the participants during the evaluation process (Bright, 2005). The only recommendation made for this gaming software was the integration of a help section that is easy to understand and follow when the user is unable to execute a given function. This recommendation was made by the four participants who were unable to complete some of the defined tasks successfully.

Conclusion

It is therefore concluded that for every software application to be accepted for use by the intended users, it has to pass through the usability test process. This process provides an opportunity for the users to interact with it and give out their first hand feedback in the presence of the researchers and their recommendation is considered before the application is released for production. For a good application, it should be easy to learn its functionalities and execute them, the designs should meet the standard and it should not have any errors. Once the software passes these tests then it is acceptable for use by the targeted users

References

Bright, L. (2005). [online] Rachaelgilg.com. Available at: https://rachaelgilg.com/docs/testplan_final.pdf [Accessed 16 May 2018].

Halgren, S. (2009). [online] Sage-research.com. Available at: https://www.sage-research.com/v4/sage_usability_study_report_example.pdf [Accessed 16 May 2018].

Istqbexamcertification.com. (2018). What is Usability testing in software and it’s benefits to end user?. [online] Available at: https://istqbexamcertification.com/what-is-usability-testing-in-software-and-its-benifits-to-end-user/ [Accessed 16 May 2018].

laitinen, s. (2005). Better Games Through Usability Evaluation and Testing. [online] Gamasutra.com. Available at: https://www.gamasutra.com/view/feature/130745/better_games_through_usability_.php [Accessed 16 May 2018].

McCracken, C. (2016). How to Conduct Usability Testing from Start to Finish. [online] UX Mastery. Available at: https://uxmastery.com/beginners-guide-to-usability-testing/ [Accessed 16 May 2018].

Mifsud, J. (2016). [online] Usabilitygeek.com. Available at: https://usabilitygeek.com/usability-testing-mobile-applications/ [Accessed 16 May 2018].

Naji, C. (2016). [online] Usabilitygeek.com. Available at: https://usabilitygeek.com/usability-testing-prototypes/ [Accessed 16 May 2018].

Norman, N. (2014). Task Scenarios for Usability Testing. [online] Nielsen Norman Group. Available at: https://www.nngroup.com/articles/task-scenarios-usability-testing/ [Accessed 16 May 2018].

Quesenbery, W. (2006). [online] Wqusability.com. Available at: https://www.wqusability.com/handouts/notes&analysis.pdf [Accessed 16 May 2018].

Rogers, M. (2015). [online] Ahrq.gov. Available at: https://www.ahrq.gov/downloads/pub/advances/vol2/Rogers.pdf [Accessed 16 May 2018].

Salvo, D. (2018). [online] Owl.english.purdue.edu. Available at: https://owl.english.purdue.edu/research/OWLreport.pdf [Accessed 16 May 2018].

Schumacher, R. (2010). [online] Nist.gov. Available at: https://www.nist.gov/sites/default/files/documents/2017/05/09/LowryNISTIR-7742Customized_CIF_Template_for_EHR_Usability_Testing_Publicationl_Version-doc_0.pdf [Accessed 16 May 2018].

Synergisticparadigms.com. (2011). [online] Available at: https://synergisticparadigms.com/uploads/Sample_Usability_Test_Report.pdf [Accessed 16 May 2018].

Webjunction.org. (2018). [online] Available at: https://webjunction.org/content/dam/WebJunction/Documents/webJunction/Sample-Usability-Report.pdf [Accessed 16 May 2018]