Usability Evaluation And User Testing Of An Interactive E-commerce System

Submission and Marking Procedure

This assignment is designed to provide practical experience of carrying out an analysis of usability requirements and priorities, performing a systematic usability evaluation using a standard method, and producing a presentation reporting the findings. 

Submission and Marking Procedure 

This is an individual assignment. It is worth 50% of the total mark for the module. 

Your report should be submitted electronically as a Word document via the Turnitin link for Assignment Two under Assessment on Blackboard, and should be submitted in hardcopy form at FOTAC.

The submission deadline is 12:30 on Thursday 17 May 2018. The target date for the completion of marking and the return of results is Friday 7 June 2018. 

University Rules 

Where coursework is submitted later than the agreed deadline without an application for an extension or deferral being approved by the appropriate member of staff, then the following tariffs apply: Work which is submitted unauthorised up to 14 calendar days after the original submission date will receive a mark that is capped at 50%.Regardless of how the work is submitted, submission constitutes an assertion by the author that the work is neither plagiarized nor otherwise committing any academic offence.

Task

Your firm of interaction design consultants is trying to build up a portfolio of impressive work, to enable it to pitch for business convincingly in the future.Your task is to produce a usability evaluation of an interactive system by carrying out user testing, plus a presentation of your results. You have a completely free choice of what interactive system you evaluate.

Choice of Interactive System 

Possibilities include software applications such as CASE tools or games or e-commerce websites or photo editing systems; electronic devices such as remote controls for televisions or DVD players, or digital cameras, or car radios; or control panels for appliances such as microwave ovens or home heating systems; or a self-service system such as an automatic train ticket vending machine. You may, if you wish, choose to evaluate two very similar and directly competing products, and assess ways in which one is superior to the other.

It’s perfectly okay to decide to evaluate a part of a big or complicated system, or consider a limited set of use cases.  A reasonable amount of functionality to consider is what you can get one user to work through in one user trial. 

The one piece of advice we can give is to choose something that is complicated or difficult to use, or is used to carry out complicated tasks, and preferably has obvious usability problems. Studying more complicated and less frequently used features of a system is likely to be more fruitful than focusing on the standard functions people use all the time. You may choose to interpret ‘interactive system’ very broadly and present a usability evaluation of a static information display, but this would require a sophisticated and detailed analysis of how people use it for practical tasks, and these tasks would need to be complicated enough to give you something to analyse. Ask advice if you consider this. 

The Usability Evaluation 

Producing the usability evaluation will involve

1. Identifying the use cases or aspects of the functioning of the system to be considered, and briefly describing them in your report. (These don’t need to be a complete set of use cases; for very complicated systems focusing on one part of what they do is just fine. However you should give a clear indication of what subset of the functionality of the system you are considering, and what you are not considering. If in doubt, cover less functionality in more detail.)
2. Defining an evaluation procedure. This will include stating one or several user tasks to be tested with exact descriptions of the scenario and the goal the user is trying to achieve, as well as what the evaluator will do to collect results and produce an evaluation. It requires a description of the procedure to be followed with each test subject from start to finish, including the exact wording of the instructions given to the subjects. The evaluation procedure needs to be described separately from the description of the results. Ideally your procedure should include filming the subjects.
3. Carrying out the evaluation. This will involve following the procedure and documenting what the users do and what problems they have. This should include brief descriptions of your test subjects including what relevant experience they have. Your raw observation notes should be scanned and presented as an appendix. You should test the system with several subjects. You may want to treat your first test or two as a pilot and revise your procedure; if you do this, comment on it.
4. Deriving findings about the usability of the interactive system from the results of the usability evaluation. This should include consideration of how strong and how general the conclusions are. Don’t be afraid of lists. 

Written Submission 

Your report should comprise the following elements:

• Part ONE: The interactive system and its users. A brief statement of what the interactive system is and what it does – sufficient to make the rest of the report comprehensible; plus a description of the user populations and the assumptions it is reasonable to make about the capabilities of the users. The word count should be between 200 and 500 words.
• Part TWO: The use cases. Brief accounts of the use cases considered, plus a statement of what you are not considering, if you are only looking at part of the system. A use case diagram is optional. The word count should be between 100 and 300 words.
• Part THREE: The evaluation methodology. An exact description of the evaluation procedure to be followed, including exact descriptions of the user tasks being considered and the scenarios and data used, and the instructions to be given to users. The word count should be between 600 and 1500 words.
• Part FOUR: The evaluation. The results of applying the evaluation procedure: what you saw test subjects doing, measurements of their performance, answers to debriefing questions, and so on. The word count should be between 1500 and 2500 words.
• Part FIVE: The findings of the evaluation. The findings of your evaluation about the usability of the interactive system. Include comments on how the findings relate to the results of the evaluation procedure, and ideally about how strong the evidence is, as well as judgements of how serious you think the usability problems are. An itemized bullet point structure is likely to be easier to read than long paragraphs of text. This should also include an appraisal of the strengths and weaknesses and successes and failures of the evaluation process. The word count should be between 500 and 1000 words.
• Acknowledgements of any help received, any information we should have when assessing the assignment, etc, in an appendix.
• Your notes made during observations of user trials, while conducting a heuristic evaluation, etc, should be included in an appendix. Handwritten notes should be scanned or photocopied.

Electronic commerce is the art of selling and buying of goods over the internet as well as providing information to the customers or clients (Shaw, Blanning, Strader, & Whinston, 2012). E-Commerce systems such as online shopping systems rely on computer network technologies for functionality (Kenneth & Carol, 2009). These technologies provide services such as electronic transfer of funds, automated data collection programs, managing inventory systems and electronic data interchange between businesses (Jetton, 2013). Millions of people world over have adopted E-Business as their preferred means of transacting business. Nowadays electronic commerce uses technologies such as emails or the World Wide Web to carry out some aspects of transactions (William & Ephraim, 2014).

An online shopping system is a business that offers goods and services in exchange for other goods and services or money (Luo, Ba & Zhang 2012). A customer is the receiver of goods or services from a vendor or supplier herein referred to as a retailer (Armstrong, Kotler , Harker & Brennan, 2015). Online shopping allows a customer to go through the available products or services from different vendors with the intent of making an order of the most appealing product or service (Brunk et al 2018).

The customer is able to see all the available retailers and other intermediaries while on the system. The online retailer may employ the services of a third party in shipping of ordered goods. The third party offers services such as shipping of goods to the customer from the retailer and sometimes linking the customer and the retailer (Kuruzovich et al 2008).The beauty of online shopping and marketing on the side of the customer is that he/she gets to order goods from the comfort of their homes and the commodity is delivered at their doorstep. Retailers too are able to make sales with ease.

Online shopping is informed on a variety of factors;

• Customer Demographics

1. Customer's net income may positively or negatively influence the online shopping tendency of the customer (Richa, 2012).
2. Men are more involved in online shopping than women. This can be attributed to the skeptical nature of women when it comes to shopping online (Forte Consultancy, 2015).

• Education levels of customers may hinder them from shopping online (Saidalavi, 2014).

1. A society exercising a collective culture is likely to have declined levels of online shopping compared to an individualistic cultured one.

• Internet experience influence customer perception on product risk. Optimistic customers are likely to shop more over the internet(Zheng, Zhao, & Stylianou, 2013).
• Motivation to shop and the orientation received on shopping platforms. The satisfaction achieved after shopping online impacts on the future online shopping tendencies by the customer .

USE CASES

A use case is a step to step written description on how activities will be rolled out in a system (Ruparelia, 2016). In this case, we will focus on the user interface from a customer and retailer oriented perspective and performance based on technology deployed by the online shopping system. Our research aims to explore customer/seller versus system interaction. We will not venture into the inner details of system development. The following use cases will be considered;

• The customer bids on products and services. New customers are required to sign up for the system. During sign up, the customer's email address and password are requested for. After customers are signed up and logged into the system, they are able to see the available products, prices, and information about the products (Wallach, 2015).
• After placing an order the customer is redirected to check out. This is where payment is made.
• The customer rates the level of services offered by the online shopping system by dropping comments on a drop box available at the system upon completion of the purchase (Park & Nicolau, 2015).
• The retailer posts products they are offering, their price, products on promotion, an inventory of products stocked, fulfilled orders as well as pending orders (Forman, Ghose, & Goldfarb, 2008).
• The retailer outsources the services of shipping of goods. The shipping company system is then linked and integrated with the online shopping system.
• The retailer takes stock at the warehouse where the actual physical goods are stored. The retailer runs an inventory of goods received from suppliers and prepares them for sale by packaging them in readiness for delivery.

Payment of goods purchased is made through the retailer acceptable modes of payment. Most commonly used mode of payment includes the use of prepaid cards, PayPal, and mobile money cash transfers (Sumanjeet, 2009). The shipping/delivery company gets the goods to the customer. 

EVALUATION METHODOLOGY

The evaluation procedure will involve testing the usability of the system by the customers and retailers through tasks fronted by the evaluator. This evaluation aims at;

1. Testing the visibility of the system status- a good system keeps its user on the know-how on what's happening at realistic time intervals.
2. Compare the similarities between the system and the users - users should be able to identify themselves with the terms and concepts used in the system (Liu, He, Gao, & Xie, 2008).

• Assist users in identifying errors and recover from errors- the users should be able to identify errors and get suggestions on the solution to these errors

The users will be classified into three classes namely, customer class and retailer class. The views of other users of the system as proxies will be factored in through the class of users who link them to the system. In each class, three users will be involved in the evaluation process. Each user will be closely monitored, observed and timed as they perform tasks posed to them by the evaluator. The evaluator will have absolute control of the testing environment and the format to be used. A laboratory set up kind of isolation will be employed. The users will be filmed while performing the task. For the purposes of this evaluation users in each class will be classified as user one, user two and user three.

Task

Data used will be quantitative and statistically validated. All the classes of users will be asked to comment on a list of user level of satisfaction questions in a filmed interview. The performance margin will be quantified by the evaluator at the end of each evaluation and recorded for later reference.

Customer test

New customers will be required to sign up for the system by filling their details on the pop-up notification box that appears on their screen after the customer clicks on the sign-up link. Customers already signed up in the system will be required to log in to the online shopping system, search for goods or services they want and in case they find a variety to choose from they will be required to conduct a personal product analysis by checking on the product review comments from customers who rated the product before them after purchase.  Once the customer has arrived at a conclusive decision of what they want they will then be required to place a bid and request for delivery of the purchased goods. The customer will then be required to proceed in making payment for the purchased goods using the mode of payment preferred.

After the delivery of the purchased product(s), the customer will be required to scrutinize if the products delivered are faulty or if they meet the standards the customer was looking for while placing the order. This would be necessary mostly likely due to the deceptive outlook of the product(s) when viewed at the system. If the product falls short of the customer expectations, a return request will be sent to the retailer customer care. The customer will then wait for the response from the retailer concerning their request. 

The evaluator will observe and record the amount of time taken by a customer to log into the system, go through the steps involved in the purchase of the products, strength of password required in the password field, the easiness of accessing commodities searched on the system, response time of the system on different prompts by the customer, graphical user interface of the system and the systematic arrangement of items on the system. The evaluator will also assess the errors made by the customer when performing the above tasks as a result of faults in the shopping system.

At the end of the customer usability test, a filmed interview will be conducted to document the views of the customer about the usability of the system. It will therefore not be necessary for the customer to rate the system at the customers portal at the end of the task. A couple of randomized set of questions will be posed by the evaluator to the customer and the answers from the customer documented in a recorded film. The questions asked will be as follows;

1. What's the customers' opinion on the time that is taken to log in the system, the strength of the password required to log into the system and the overall time taken to successfully complete a purchase?
2. How did the customer find the product delivered as compared to the product ordered from the system?
3. How long did it take for the purchased product to be delivered to the customer?
4. What's the customer's perception of the time taken by the retailer to acknowledge receipt of customer's order and the processing time?
5. The customer's rating of the graphical user interface of the system.
6. Customer's opinion on the process of making payments of purchased goods.
7. The opinion of the customer on the integrity of the customer reviews on the rating section.
8. Customer's satisfaction or dissatisfaction by the sequence of links in the system and the premium ads. 

Retailer test

The retailers will be required to log into the system, run a close check of the product(s) stocked on the system against those stocked at the physical warehouse by going through the inventory section of the portal. Confirm orders placed by the customers, arrange with the shipping or delivering company for shipment or delivery of the goods to the customer's specified location.

Choice of Interactive System

The retailer will also be required to come up with a list of frequent customers from the system by checking from the orders section, customers who have made orders more than the typical number of times in a period of one month. From the orders section, the retailer will also be required to come up with the list of frequent shoppers and all shoppers who have ever purchased products from the system. The retailer will be asked to post goods he/she wish to put on offer at the system, reward the loyal customers through his/her appraise ship program policy available on the system on the retailer's policy section.

THE EVALUATION

Systems program evaluation aims to help understand the functioning of the system features and ways to help improve the performance and appeal to users.

Customer Evaluation

With our online shopping system customers' evaluated involved new customers trying to sign up for the system and already signed up current customers. For this purpose customers were classified as a customer "one", "two" and "three". The customer was closely monitored when login into the system. New customers were required to sign up in order to access the system. Already signed up customers took a moderately considerable amount of time to log in the system with the fastest customer logging in a matter of seconds and the slowest in a minute.

The time taken to log in the system by all the evaluated customers was not in any way affected by outside factors such as the internet speed, therefore quantification of time taken was based on a presumption that all the customers were evaluated in the same environment. The password field in the login section required the customer to feed the password in form of letters and numeric only.

Once the customer logs in, the system automatically redirects them to the main shopping page. The customer is able to see the products available from different retailers, navigate on the different sections of the system, view price of products posted, and search for products without having to navigate through the list of posted products. While on the system, the customer initiates the process of purchasing products by clicking on the "purchase" link below every product (Li, Liu & Wang, (2018).

The customer is redirected to another page where a form with details of the customer's credentials is provided; this includes customer's physical address, state, and city, desired means of payment and the quantity of product the customer wishes to purchase. The customer completes the form by filling in the empty fields. Once dully field, the form is sent to the retailer who is expected to confirm and process the order upon receiving the form (Hou & Elliott, 2010). The customer receives an acknowledgment message from the seller after the order is received. Once the retailer has fully processed the order, the customer is invoiced the total cost of the purchase order.

The customer proceeds to the check out section. In this section, the customer is allowed to choose their preferred mode of payment from the available modes of payment. Different customers used different modes of payment as indicated below:

1. Prepaid cards processor payment method - this involves payment by use of debit or credit cards. The customer has to make a request to their online payment provider asking them to credit the account of the retailer the cost of the purchased goods.
2. PayPal - it entails controlled bank transfer management services. The customer first deposit cash into their PayPal account from their bank account or at least confirms that they have enough money in their account before making payments.
3. Mobile money transfer- this form of payment involves the use of mobile phones in processing payment using direct carrier services. The cost of purchases is included in the next phone bill (Vaughan, Fengler & Joseph, 2013).

The Usability Evaluation

After choosing the desired mode of payment and keying in the required information, the customer confirms the shipping/delivery address they provided and authorizes payment (Tutorialspoint, 2016). When the customer clicks on the logout link at the system, a drop box appears on the screen prompting the customer to rank the system and give some reviews (Riquelme, Isabel & Sergio, 2014).

 After the goods are delivered, the customer double checks the goods to confirm if they are a true reflection of the commodities purchased from the system. In case the product fails to meet the customer's preference, the customer establishes contact with the retailer customer care in a bit to see if they can change the product delivered. However, the customer is required to cater for the shipping back costs.

At the end of the evaluation, the customer was asked a couple of questions regarding how they found the system responsive to their needs. The following were their answers;

1. Time taken to log into the system was considerably fine from the customer's perspective but the time taken to complete the purchasing process needed some improvement. In the customer's opinion, signing up for the system before making a purchase was not necessary. Customer three and two had no issues with the password strength required. On the other hand, customer one questioned the credibility and security threshold of the password strength prompted at sign up.
2. Customer one found the product delivered to be the exact product they purchased from the system. Customer two and three had been supplied with the wrong product. The size or color and quality of product delivered didn't match the customer's standards when making the purchase.
3. The three customers evaluated had issues with the time taken for the product to be delivered. The product either took so long or it was delivered later than the time they wanted to use it. In one case customer, two who had purchased designer shoes to wear on an event had to opt for alternative shoes as the shoes were not delivered on time.
4. Time used to make payment was comparatively fine with the three customers evaluated. However, customer three expressed concerns about why it was necessary for a customer to be signed up. The retailer's response on receipt of order and processing time was slightly slow (Mallat & Tuunainen, 2008).
5. Customer one rated the graphical user interface of the system as excellent while customer two and three ranked it as slightly above average. This could be attributed to the wrong product being delivered to the two customers other than what they had viewed from the system interface (Mittal, 2017).
6. The integrity of the reviews by previous shoppers was questioned by the three users. The numerous premium ads and the design of the links in the system were found to inform their answer on this(Kim, Ferrin, & Rao, 2008).

Retailer Evaluation

Three different retailers were evaluated. They were named retailer one, retailer two and retailer three. The evaluator observed a set of activities being performed by each retailer on the system. The retailer updated their inventory by close checking on goods stocked at the physical warehouse versus those posted on the system. The retailer was also tasked to post new products and their prices on the system. These include goods stocked for promotion purposes and launching of entirely new commodities.

The retailer was monitored while responding to a customer request for purchase and time taken to acknowledge receipt of an order from the customer. The retailers appraise ship program to loyal customers was tested by noting the number of customers rewarded with free gift shopping vouchers as deduced from the customer relationship management linked software (Meziane & Kasiran, 2008).

The evaluator also monitored the communication between the retailer and the shipping company. The retailer was required to first make a formal application for shipment of the goods to the customer and thereafter negotiating the price charged for the service. The retailer also confirmed receiving money in their bank account from the customer's online payment service provider. To help understand the retailer's opinion on the responsiveness of the system to retailer needs, a set of questions were administered to the three users. The following were their responses;

1. When asked on how they found the system accommodating while posting products and updating the price changes with different products, all retailers decried of the time-consuming nature of the process and the limited space for products catalog.
2. Retailer one raised issues on the ability of the system to handle up surging traffic. With an increased number of customers, retailer one failed to identify and recognize their most loyal customers due to heavy traffic. Retailer two and three had no issues with traffic. It's worth noting that retailer one had no customer relationship management system inter-linked with the shopping system.
3. When asked about the appropriateness of the system in advertising their products, retailer three and one decried of the stiff competition from digital marketing channels. Retailer two had no issues with marketing. Retailer three and one were forced to explore other channels like display ads, shopping engines, and email subscriber lists. The retailers were also asked about their ability to attract new customers while retaining the current ones.
4. The choice of the right partners and technology was also found to influence the growth of their business (Qiu & Benbasat, 2009).The inventory management software being used, customer relationship management systems used and the email software employed by the system had direct impacts on the retailers business as stated unanimously by the retailers, when asked to comment on technology. The choice of company to be contracted in delivering the goods to the customers was found to influence the likelihood of the customer shopping again with the retailer. Companies that took so long discredited the retailer's service delivery (Lin & Lee, 2009).
5. The workmanship used to man the physical warehouses and the system was found to influence the growth of the retailer business. Retailer three proposed hiring of the right talent and purpose-driven individuals as the bedrock of desired growth (Jing & Lewis, 2011). 

FINDINGS OF THE EVALUATION

The evaluation findings were based on empirical facts of the data collected, the opinion of users and answers to questions asked in the interview (Zellman & Kilburn, 2015). Different class of users had a divergent opinion on different functionalities of the system. This can be justified by taking into consideration that different class of users have a different set of expectations on how the system should suit them (Hong, Thong & Tam, 2015). The evaluation findings have been categorized into two sections with each section representing a single class of user evaluated as demonstrated below;

Customer class

1. Overall time taken by the customer to complete shopping was found to be exhausting, this was because of too many products in the shopping catalog (Guided Selling, 2015)
2. The signing up process was not found to hold any relevance. It ended up wasting the customer's time while shopping. This had a negative implication on the customer's intention to shop from the system (Tung, Xu & Tan, 2009).
3. Time taken by the system to deliver messages to the retailer from the customer was compromisingly slow. Some customers had to log out of the system and later come back to check if the sender had written to them. This informed the customer's decision on whether to shop through the same retailer next time (Romain & Cuestas, 2008)
4. The products displayed on the system in some cases were manipulated by the system in a manner either the color, size and other physical attributes deceived the customer (Smith, Johnston & Howard, 2011). The graphical user interface of the system, therefore, needed some improvements (Holbrook, Holbrook, Lehmann & Schmitt, 2016).
5. Time taken to deliver products to the customer was way too long. This, however, was occasioned by the contact time between the retailer and the delivering company (Lin & Lee, 2009). The message response time of the link between the shopping system and the delivery company is inconveniently annoying. There was a need for intermediaries such as delivery companies to be incorporated and directly integrated into the system features.
6. The integrity of reviews was questionable (Neijens, Bronner, & Ridder, 2011). It was possible for one person to review the system twice. Majority of those participating in rating the product had been lured with tokens or promised rewards after rating the system. This, therefore, ended up misleading innocent new shoppers. 

Retailer class

1. It was quite easy for the retailer to post products on the system inventory. However, it wasn't possible for the retailer to take stock of goods at the warehouse from the system. To do this, it required extra features on the system(Goo, Kishore, Rao, & Nam, 2009).
2. Product marketing was found to be an expensive affair as the system had no marketing program features integrated with it (Chintagunta, Chu & Cebollada 2012). The retailer relied on the system interface to attract customers to shop or employed marketing services sourced elsewhere.
3. Complimentary technologies used by retailers to meet customer needs such as the inventory software management, customer relationship management, and email applications were not compatible with the system (Luo, Ba & Zhang 2012). They could not be configured to match the system specifications.
4. The system was also found to lack human resource management abilities. The retailer had to man their labor force using different systems. This greatly impacted on the quality of services offered to customers (Prateek, Dr. Richa & Sibongiseni, 2016).
5. It was also not possible for the retailer to come up with the list of customers who have purchased goods from the online shopping system, they had to refer to the inter-linked customer relationship management system.
6. The limited space in the products catalog limited the number of products that the retailer could stock at the system.
7. Fear by customers as to whether goods purchased will get to them after payment and delays brought up by a long chain of processes limits the number of sales done by the retailer (Bhatia & Dahiya, 2013)
8. The system needed some modifications to restore customer's trust on the retailers and means of payment provided. 

Through the evaluation, the methodology employed it can be deduced that users' opinion on the functionality of the system will form a good base for all future system improvements. The information received from customers was found unbiased, the evaluator could assess the answers given on the interview questions using the results observed while the users went on to perform the tasks issued earlier.

Written Submission

Challenges were notwithstanding in the evaluation process. Some users of the system were not familiar with the concept of system evaluation. This made it a bit difficult for the evaluator to get expected results in some cases. Users also were found not to be so helpful when answering interview questions. In questions requiring a brief explanation, a one-word answer was given. The evaluation process had some setbacks in that it didn't factor in a scenario where a physically challenged person would be the customer (Kaufman & Childers, 2009). It also proved difficult to quantify user's personal opinion. 

Amanor-Boadu, V. (2009). In Search of a Theory of Shopping Value: The Case of Rural Consumers. Review of Agricultural Economics, 31(3), 589-603. Retrieved from 

Armstrong , G , Kotler , P , Harker , M & Brennan , R (2015) , Marketing : An Introduction . 3rd edn , Pearson Education. Retrieved from 

Chintagunta, P., Chu, J., & Cebollada, J. (2012). Quantifying Transaction Costs in Online/Off-line Grocery Channel Choice. Marketing Science, 31(1), 96-114. 

Forman, C., Ghose, A., & Goldfarb, A. (2008). Competition Between Local and Electronic Markets. How the Benefit of Buying Online Depends on Where You Live , 47-57.

Forte Consultancy. (2015, May 5). Marketing, Sales and Analystics Articles. Retrieved May 12, 2018, from E-Commerce Customer Segmentation:

Goo, J., Kishore, R., Rao, H., & Nam, K. (2009, March 1). MIS quarterly. Retrieved May 12, 2018, from The Role of Service level Agreements in Relational Management of Information Technology Outsourcing

Guided Selling. (2015). Customer Behaviour. Retrieved May 6, 2018, from Solve the 5 Biggest Problems of Online Shoppers: 

Holbrook, M., Lehmann, D., & Schmitt, B. (2016). Marketing. In the faculty of Columbia business school (Author) & Thomas B. (Ed.), Columbia Business School: A Century of Ideas (pp. 81-106). New York: Columbia University Press. Retrieved from 

Hou, J., & Elliott, K. (2010). Profiling Online Bidders. Journal of Marketing Theory and Practice, 18(2), 109-126. Retrieved from 

Jetton, K. (2013). E-Commerce. Counterpoints, 391, 221-232. Retrieved from https://www.jstor.org/stable/42981448

Jing, X., & Lewis, M. (2011). Stockouts in Online Retailing. Journal of Marketing Research, 48(2), 342-354. Retrieved from 

Kaufman-Scarborough, C., & Childers, T. (2009). Understanding Markets as Online Public Places: Insights from Consumers with Visual Impairments. Journal of Public Policy & Marketing, 28(1), 16-28. 

Kenneth, Laudon & Carol Traver. (2009). E-commerce Business Technology Society. 5th Edition. 2009. See Chapter 12, “Volkswagen Builds B2B Marketplace.”; wikepedia.com; Ford.com.articleid=209168).

Kim, K., Ferrin, D., & Rao, H. (2008, January 1). Desion Support Systems. Retrieved May 12, 2018, from A trust-based consumer decision-making model in electronic commerce:

Kuruzovich, J., Viswanathan, S., Agarwal, R., Gosain, S., & Weitzman, S. (2008). Marketspace or Marketplace? Online Information Search and Channel Outcomes in Auto Retailing. Information Systems Research, 19(2), 182-201. Retrieved from 

Li J., Liu J., Wang S.J. (2018) In-Store Shopping Experience Enhancement: Designing a Physical Object-Recognition Interactive Renderer. In: Brooks A., Brooks E., Vidakis N. (eds) Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT 2017, DLI 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 229.

Lin, P., & Lee, C. (2009). How Online Vendors Select Parcel Delivery Carriers. Transportation Journal, 48(3), 20-31. Retrieved from 

Liu, X., He, M., Gao, F., & Xie, P. (2008). An empirical study of online shopping customer satisfaction in China. A Holistic Perspective , 919-940.

Luo, J., Ba, S., & Zhang, H. (2012). The Effectiveness of Online Shopping Characteristics and Well-Designed Websites on Satisfaction. MIS Quarterly, 36(4), 1131-1144. Retrieved from 

Mallat, N., & Tuunainen, V. (2008). Exploring Merchant Adoption of Mobile Payment Systems: An Empirical Study. E-Service Journal, 6(2), 24-57. doi:10.2979/esj.2008.6.2.24

Meziane, F., & Kasiran, M. (2008). Evaluating Trust in Electronic Commerce: A Study Based on the Information Provided on Merchants' Web sites. The Journal of the Operational Research Society, 59(4), 464-472. Retrieved from 

Mittal, T. (2017). Your Story. Retrieved May 06, 2018, from Common problems faced by customers while shopping online: 

Neijens, P., Bronner, F., & Ridder, J. (2011). The Content Characteristics and Perceived Usefulness of Online Consumer Reviews. Computer Mediated Communication , 19-38.

Park, S., & Nicolau, J. (2015). Asymmetric effects of online consumer reviews. Annals of Tourism Research , 67-83.

Prateek Kalia, Dr. Richa Arora, & Sibongiseni Kumalo. (2016). E-service quality, consumer satisfaction and future purchase intentions in e-retail. E-Service Journal, 10(1), 24-41. doi:10.2979/eservicej.10.1.02

Qiu, L., & Benbasat, I. (2009). Evaluating Anthropomorphic Product Recommendation Agents: A Social Relationship Perspective to Designing Information Systems. Journal of Management Information Systems, 25(4), 145-181. Retrieved from 

Richa, D. (2012). Impact of Demographic Factors of Consumers on Online Shopping Behaviour. A Study of Consumers in India , 43-52.

Riquelme, Isabel P., and Sergio Romain.(2014). "The Influence of Consumers' Cognitive and Psychographic Traits on Perceived Deception: A Comparison Between Online and Offline Retailing Contexts." Journal of Business Ethics 119, no. 3 (2014): 405-22. 

Romain, S., & Cuestas, P. (2008). The Perceptions of Consumers regarding Online Retailers' Ethics and Their Relationship with Consumers' General Internet Expertise and Word of Mouth: A Preliminary Analysis. Journal of Business Ethics, 83(4), 641-656. 

Ruparelia, N. (2016). use case pattern # 3: INAAS. In Cloud Computing (pp. 169-178). MIT Press. Retrieved from 

Saidalavi, H. (2014). Advances in Management. A Study on Online Shopping Experience and Customer Satisfaction , Vol 7, No. 5.

Shaw, M., Blanning, R., Strader, T., & Whinston, A. (2012, December 6). Revistasice. Retrieved May 12, 2018, from Handbook on Electronics Commerce:

Smith, S., Johnston, R., & Howard, S. (2011). Putting Yourself in the Picture: An Evaluation of Virtual Model Technology as an Online Shopping Tool. Information Systems Research, 22(3), 640-659. Retrieved from 

Sumanjeet, S. (2009). Emergence of Payment Systems in the Age of Electronic Commerce. The State of Art , Vol. 2, No. 2.

Tung, L., Xu, Y., & Tan, F. (2009). Attributes of Web Site Usability: A Study of Web Users with the Repertory Grid Technique. International Journal of Electronic Commerce, 13(4), 97-126. Retrieved from

Vaughan, P., Fengler, W., & Joseph, M. (2013). Scaling Up through Disruptive Business Models: The Inside Story of Mobile Money in Kenya. In Chandy L., Hosono A., Kharas h., & Linn J. (Eds.), Getting to Scale: How to Bring Development Solutions to Millions of Poor People (pp. 189-219). Brookings Institution Press. Retrieved from 

Wallach, B. (2015). Shopping. In A World Made for Money: Economy, Geography, and the Way We Live Today (pp. 1-36). Lincoln; London: University of Nebraska Press. Retrieved from 

William H. DeLone & Ephraim R. McLean (2014) Measuring e-Commerce Success: Applying the DeLone & McLean Information Systems Success Model, International Journal of Electronic Commerce, 9:1, 31-47, DOI:

Zellman, G., & Kilburn, M. (2015). Methods. In Final Report on the HawaiÊ»i P–3 Evaluation (pp. 19-30). RAND Corporation. Retrieved from 

Zheng, Y., Zhao, D. K., & Stylianou, D. A. (2013). The impacts of information quality and system quality on users' continuance intention in information-exchange virtual communities. Decision Support Systems , 513-534.

APPENDIX

User trials observation notes

• The classes of users evaluated were the customer class and the retailer class.
• Customer three took more time to shop, he had to be signed up before proceeding to the products catalog.
• Customer three was also observed to be having sight problems, and he had to use reading glasses while viewing products.
• Customer two was from a remote area while customer three and one were from urban areas. It took more time to get goods purchased by this customer to the destination (Amanor-Boadu, 2009).
• All the three customers failed to review the system and instead raised questions about the credibility of the reviews done before.
• All the customers had the money or their means of payment ready before opting to shop.
• All the evaluated customers were predominantly male, the evaluator could not get a lady who was ready to be evaluated.
• Retailer one stocked electronics and electrical appliances, retailer two clothes and shoes. Retailer three stocked motor vehicles and automobiles.
• The retailer had no control of the system expect to post and update product information.
• Time taken by the delivery company to get goods to the customer was dependent on the geographical location of the customer in comparison to the retailer warehouse. However, it took more time to deliver goods to remote areas (Amanor-Boadu, 2009).