Certifying Off The Shelf Software Components: Evaluation And Selection Strategies

Introduction to Off the Shelf Software and COTS

Discuss About The Certifying Off Shelf Software Components.

In the last 10 years, software development world has evolved tremendously. This evolution has caused CBSD or Component Based Software development to rise and this in turn has caused a massive spike in plug-n-play based software which ultimately lead to the concept of Commercial of the Shelf solutions or COTS (Gashi, Popov & Stankovic, 2009). COTS has increasingly become a commonplace in today’s enterprise world (Rakic & Medvidovic, 2001). This is also because, COTS reduce the total cost of operation and help clients save huge investments upfront (Voas, 1998). As such, a lot of marketplace has come up that helps in aggregating and listing of COTS including but not limited to their specifications, compatibility factors and available configurations. The rise of COTS has provided company with abundant choices and companies also need to come up evaluation strategy in order to choose the right solution. This paper would also provide an example where the same COTS would be used for comparison and selection of a COTS.

• To provide a background research that helps in understanding the importance of COTS.
• To provide a background research that explores the current methods and practices in selection of a COTS.
• To provide a recommended evaluation criteria in a generic template that allows companies in selecting an appropriate solution among the lot.
• To provide an example of how to use the evaluation criteria in deciding between two of the most popular applications in enterprise world.

Internet is the most commonplace among enterprise users in helping them find an OTS solution (Mahmood, Lai & Kim, 2007). The next most important place where off the shelf solution is searched for and researched upon is the OTS marketplace. It’s also the place where interaction happens over OTS among integrators, OTS providers and also the actors that help promote or facilitate such transactions. These actors include marketing channels and intermediaries (Ayala et al., 2009). A study conducted by Ayala et al., in 2011 with regards to finding the systematic approach enterprises around the world take for COTS acquisition. The study revealed that the companies never used any established guidelines or procedures for identification of COTS solution. Most of the companies simply knew what X requirement were being fulfilled by Y product as they had heard about it before and thereby simply went ahead with the product (Maiden & Ncube, 1998). Apart from this, the other major ways through which the company searched for COTS solutions were through internet research, hiring of an expert who would help with the selection of a recommended COT solution and the last method through which COTS were being picked were the higher level executives of the company (Ayala, Hauge, Conradi, Franch & Li, 2011).

After companies searched for the product, they went straight to the evaluation stage wherein they were to evaluate the COTS solutions they had found. All the enterprises being interviewed in the study showed that they did not follow a proper methodology and went about without a proper plan while selecting the OTS. Some of the methods company evaluated the product consists of the following:

Searching and Researching for Off the Shelf Software

People’s experiences: The interviews mentioned they would gather knowledge from employees within the company, conferences, external contacts and diverse professional networks.

Domain specific websites: They continually monitored OTS system websites.

Internet research: The interviewees mentioned they constantly browsed the web for being on the lookout for probably solutions.

Repositories: The interviewees also made use of repositories similar to Sourceforge in order to identify probably OTS (Cooper & Chung, 2005).

Consultancy: Some of the interviews also took the route of hiring external consultancies for the same.

Most of the interviewees mentioned, there were no cored established criteria for evaluating the OTS and hence they went ahead with a strategy of their own:

1. They ensured that the requirements were being met.
2. The checked whether the technological solution was stabled.
3. They also made sure if there were any support provided by the company.
4. Also, the interviewees checked for documentation, help and tutorials.
5. Some interviewees also checked for the OTS solution’s performance and scalability factors.

Company History & Experience: The vendor should be researched for its background as unlike traditional companies a software company are smaller in size and could be beyond national boundaries. Within this segment, the following needs to be evaluated:

Company establishment: Customers need to check how long the company has been operating their line of business i.e., the software industry (Miller & Yeoh, 2006).

Niche: Next step is finding out if the company knows the niche well-enough to be able to develop a commendable solution.

Testimonials: Testimonials help in finding out if their products are being used by other businesses and what is their opinion about it.

Cost: Cost is an extremely important factor in deciding of a COTS, however cheaper solution does not always means a cost-effective one. This is because, a company needs to evaluate whether the features being offered by the company justify its price as customers may not want 100s of features when their usage lies around only 5 of them (Lutters & Seaman, 2007).

Ease of Use and Adoption: The learning and adoption curve of the OTS should be narrow and if a product is too complicated to understand then time and efforts would be wasted in bringing up the staff up to speed on them.

Ease of use: As mentioned, the product should be simple to use with an effective user interface and shorter learning curves (Peaker, 2002).

Adoption: Nearly 20 to 25% of the products an enterprise acquires never get used. This tantamount to a huge waste of resources. As such, the products being acquired should have a current use in the organization.

Support: The products being offered should be supported by the company. There should be multiple means of support such as telephonic, VOIP based and other methods. The product should also have a long-term support by the company.

Evaluation of Off the Shelf Software

Training: If the product being acquired needs training, then the vendor should be able to arrange for that. Additionally, it may add value if 3^rd party service providers exist in the market to provide training for them.

Maintenance: The OTS being acquired should have a high degree of up-time and should be regularly patched and upgraded for bugs and features.

Security: Security is among the top concerns for any piece of hardware or software that a company acquires. Security means that the software is regularly patched for known bugs, does not have any severe vulnerabilities within it and has encryption to ensure safety of information. Apart from these, the application should come with a robust database backup mechanism. Also, another key criteria is that, if the OTS solution is cloud based then the server which it is based on should be company’s own server and if it’s a third party server, then it should have enough security hardening to ensure data and system security (Miller & Yeoh, 2006).

Familiarity: The look, feel and the user experience of the software should be based on something which is known, familiar and accepted as an industry standard (Gashi, Popov & Stankovic, 2009). These specifically includes elements such as the navigational bar, the menu bar, the user dialog box, the dashboard, the page layout and others.

The commercial off the shelf software being used here is Microsoft Project and it’s being evaluated against it’s one of the most popular alternative called as Open Project. Microsoft Project is a full-featured project management platform that consists of relevant tools that helps a project manager in planning, controlling project, scheduling and allocating tasks. A project plan developed under project management can contain several elements including but not limited to task lists, cost data, schedules, calendars, resources etc. On the other hand, OpenProject is a desktop as well as a cloud based solution for project management that is comparable with Microsoft project. In fact, OpenProject also has a similar user experience and even opens various MS Project files such as Gantt Charts, PERT charts and Project.

1. Gantt chart: Gantt chart is like a bar chart which is primarily used for scheduling of project and it allows breaking of several subtasks.
2. Project Network: This shows various intermediate, before and after tasks in a graphical representation
3. Resources Chart: This provides details of all the resources assigned in the gantt chart.
4. RBS: Resource Breakdown Structure is used for categorizing several resources in different functions.
5. Output generation: Both application provide several tracking methods and consolidation of tasks / resources and reports.

1. Industry standard application: MS project is an industry standard application and the components and feature sets used in MS project are commonly accepted in the industry.
2. Help and Documentation: MS Project has an extensive official as well as community support as well as other 3^rd party service providers that dedicatedly provide training and help to end-customers.
3. Features: MS Project is a continually evolving project and has a multiude of features at the moment compared to many other solutions on the market.

1. Free: OpenProject is free to use for users who wish to use a community version of their application with some restricted feature-set.
2. Adaptability: OPenProject can open MS Project files as well.
3. Lightweight: Unlike MS Project, OpenProject is quite lightweight and takes less resources.
4. OS Independent: OpenProject is OS independent to some-degree as there are instances OpenProject available on the cloud and Linux as well as Windows.

Since, there is no actual case for which we’ve to decide between the two, the generic evaluation criteria discussed above would be used. If there was an actual case for which the two COTS were being decided then their feature set and functionalities would have also been taken into considerations. So the evaluation of the two is as follows:

1. Company Age: Microsoft is among the oldest and the largest software corporations in the world today and as such is extremely large and diverse compared to OpenProject Foundation that develops and manages OpenProject.
2. Niche: Microsoft has a diversified business and they cater to nearly ever major segments and industries in the world today. They develop products ranging from operating system, to productivity, to ERP and other highly specialized softwares and systems. As such, MS Project is not their niche. In comparison, OpenProject is the sole product of the OPF and thereby the company is much more focused towards their offering.
3. Dangers: As mentioned previously, Microsoft is among the oldest and at present the largest software company in the world today and is a trusted name globally. In comparison to this, OPF is a relatively new brand and does not have enough market goodwill to compete in this regard with Microsoft.
4. Cost: MS Project comes in multiple pricing variants. MS Project can be acquired as ‘On-premise’ solution or ‘Cloud-based’ solution. The cost for the professional version of On-premise solution is roughly around 1000 USD / user. On the other hand cloud based variants come in three pricing tier starting at 7 USD / user / month to 30 USD / user / month and finally up to 55 USD / User / month. On the other hand, community version of OpenProject is completely free and the cloud hosted variants are priced at 5 and 10 USD / User / Month. It’s quite evident that MS Project is relatively pricier in comparison, however, depending upon the features required by the company and the justification of the cost the company is willing to pay, will ultimately be the deciding factor.
5. Ease of Use: Ease of use, is subjective however, Microsoft has been fine-tuning their application for a long time now and since it’s also based on the other productivity suite of Microsoft, in particular Microsoft Office. In comparison, OpenProject requires a slightly steeper learning curve..
6. Adoption: Adoption is an evaluation criteria that suggests that many times a software is purchased but is never used. This can be avoided if the system being offered in pay-as-you-go pricing model (Umarji & Seaman, 2005) .
7. Support: Support available for MS Project is much greater in comparison to OpenProject because, Microsoft has a world-class enterprise and consumer support channels and also an active community in addition to 3^rd party support providers. In comparison, Open Project does not have much of these available except for limited official support and community support.
8. Training: For training, MS Project certified trainers and institutes are available all over the world and Microsoft also provides extensive, training and self-help documents. In comparison, OpenProject does provide training too officially, but there are no 3^rd party training available.
9. Maintenance: Microsoft’s Project can be on-promise as well as cloud-hosted and Microsoft themselves are into the business of cloud computing services and are a renowned name in the industry. In comparison, the maintenance support provided by Openproject would not be as resilient and dependent as that of Microsoft.
10. Security:
11. Data Backup : Cloud hosted solutions provide data backup in both of the solutions, whereas on premise solutions of both the solutions come with the ability to save their work and even auto-save it from time to time.
12. Track Record: Track record mentions the ability of company to provide security within their system so there’s no instance of data loss from security breaches. Both of the companies rank the same here as none of them have any bad security breaches in this regard.
13. Server System: The server system is owned and managed by Microsoft itself where MS Project is typically hosted which makes it more secure and resilient. On the other hand, Openproject does not own the server.
14. Familiarity of the software: MS Project is the more familiar in comparison as its user interface is primarily based on other productivity and other familiar Microsoft software. In comparison, Openproject is not as familiar in comparison with the MS Project.

Conclusion

Selecting a right COTS is imperative for the success of an organization if the choice of the company comes down to an OTS. The COTS evaluation and selection process is often subjective as they are based on various dynamics and factors and also include groups such as technical experts, stakeholders and the executives of the company. The best method in selecting an off the shelf solution is to make use of an aggregated consensual opinion list by asking the asking the relevant group that the COTS solution would concern. Once this is done, the above mentioned evaluation criteria can be used to come up with a probable choice of OTS.

References

Ayala, C., Hauge, Ø., Conradi, R., Franch, X., & Li, J. (2011). Selection of third party software in Off-The-Shelf-based software development—An interview study with industrial practitioners. Journal Of Systems And Software, 84(4), 620-637. https://dx.doi.org/10.1016/j.jss.2010.10.019

Cooper, K., & Chung, L. (2005). Managing change in an OTS-aware requirements engineering approach. ACM SIGSOFT Software Engineering Notes, 30(4), 1. https://dx.doi.org/10.1145/1082983.1082955

Gashi, I., Popov, P., & Stankovic, V. (2009). Uncertainty explicit assessment of off-the-shelf software: A Bayesian approach. Information And Software Technology, 51(2), 497-511. https://dx.doi.org/10.1016/j.infsof.2008.06.003

Lutters, W., & Seaman, C. (2007). Revealing actual documentation usage in software maintenance through war stories. Information And Software Technology, 49(6), 576-587. https://dx.doi.org/10.1016/j.infsof.2007.02.013

Mahmood, S., Lai, R., & Kim, Y. (2007). Survey of component-based software development. IET Software, 1(2), 57-66. https://dx.doi.org/10.1049/iet-sen:20060045

Maiden, N., & Ncube, C. (1998). Acquiring COTS software selection requirements. IEEE Software, 15(2), 46-56. https://dx.doi.org/10.1109/52.663784

Miller, J., & Yeoh, H. (2006). COTS acquisition process: incorporating business factors into COTS vendor evaluation taxonomies. Software Process: Improvement And Practice, 11(6), 601-626. https://dx.doi.org/10.1002/spip.297

Peaker, B. (2002). Review: Testing IT: An Off-the-Shelf Software Testing Process. The Computer Bulletin, 44(2), 31-31. https://dx.doi.org/10.1093/combul/44.2.31

Rakic, M., & Medvidovic, N. (2001). Increasing the confidence in off-the-shelf components. ACM SIGSOFT Software Engineering Notes, 26(3), 11-18. https://dx.doi.org/10.1145/379377.375228

Umarji, M., & Seaman, C. (2005). Predicting acceptance of Software Process Improvement. ACM SIGSOFT Software Engineering Notes, 30(4), 1. https://dx.doi.org/10.1145/1082983.1083121

Voas, J. (1998). Certifying off-the-shelf software components. Computer, 31(6), 53-59. https://dx.doi.org/10.1109/2.683008

Voas, J. (1998). COTS software: the economical choice?. IEEE Software, 15(2), 16-19. https://dx.doi.org/10.1109/52.663777