Task 2: Demo Code Package And Technical Report For Integration Plan

Overview of Task 2

Task 2 consists of a demo code package and a technical report for the integration plan. Submit your work as a .zip file, which contains the code files (another .zip file) and your report (a .docx file), to Blackboard Task 2. Please follow the submission instructions described in this document. 

The assignment will be marked out of a total of 100 marks and forms 30% of the total assessment for the course. ALL assignments will be checked for plagiarism by SafeAssign system provided by Blackboard automatically.  

Refer to your Course Outline or the Course Web Site for a copy of the “Student Misconduct, Plagiarism and Collusion” guidelines. Assignment submission extensions will only be made using the official Faculty of Arts, Business and Law Guidelines.

Requests for an extension to an assignment MUST be made to the course coordinator prior to the date of submission and requests made on the day of submission or after the submission date will only be considered in exceptional circumstances.

Case Study: Industrial Internet for GE

After evaluating various platforms, GE finally decided to adopt a Service Oriented Architecture (SOA) for its future IT infrastructure. GE was impressed with your Template based, Reference Architecture approach set out in your Task 1 ‘Industrial Internet’ Report and has accepted your recommendations regarding:

? Computing and storage infrastructure design, Application / service integration, and Information integration.

However GE Executives would like to gain a deeper understanding of the Technologies behind your recommendation and have asked for a small specific demonstration of these technologies along with a brief explanation of the concepts and principles of how it works.Because GE have medical clinic related applications in many clinics around Australia, GE Executives would like a demonstration of keeping track of demographic information for each medical clinic location. The demo enables GE employees to browser search the locations of medical clinics in Google Maps using a postcode and retrieve an example of current demographic information along with the map. There are three major components in the demo system:

1. Data integration demo. In this demo, there are two data source files: “clinics.csv” contains the information about fictional clinics, and “locations.xml” contains the information about the location coordinates for each clinic. A Python scrip file with the name

“data_merger.py” should be implemented to read these two data files. Using the Python petl framework you are required to:

1. clean the data in the “clinics.csv”. This involves using python and Regex to standardise email addresses so they are usable as a html link, and
2. output the merged and cleaned data into a CSV file with the name “csv”.

2. RESTful Web service server demo. In this demo, you are required to build a RESTful Web service which supports a ‘getlocation’ query from the client browser similar to

“/getlocation?postcode=xxxx”. The server Web Service will return a JSON object with five attributes (clinicname, latitude, longitude, email and state). The data for this JSON object will be retrieved from the “clinic_locations.csv” file created in component 1. The framework Bottle is required for this implementation. A Python scrip file with the name “clinic_locator.py” will contain this RESTful Web Service Demo.

3. Mashup demo. In this demo, you are required to build a mashup application which accepts a postcode as the input. If a medical clinic in that suburb can be found, then its location is displayed in Google Map. Otherwise, the default clinic (with postcode 2000) is displayed. A HTML file “html” should be implemented. In this file, a text field is provided at the page top to accept user’s input of postcode. In addition, a Submit button will trigger the “getlocation” Web Service to retrieve its corresponding latitude and longitude, and highlight and centre the location in Google map.

Data Merging and Cleaning

In addition to highlighting the positon on the map, a user is able to click on the marker bubble and see an information window containing:

1. suburb specific demographic information or a link to that information, and
2. a link to the email address for that clinic. After clicking on the email link, the user’s email application must pop up and automatically insert the subject line “Clinic Enquiry”.

Important Note: For ease of demonstration to GE executives, your code must be selfcontained. In addition to using Standards based HTML and Python, petl, Bottle and parser are the only additional frameworks you should need. If you use others they must be provided and not require installation.

In addition to the above Demo code (70%) you are required to include a Report (30%) explaining the key concepts around the demo design and implementation. Moreover, you need to explain how to run your demo/code.

Please use the following outline in your report: 

1. Introduction
2. Data merging and cleaning
3. RESTful Web services
4. Mashups
5. Demo running instructions
6. Conclusion

The detailed distribution of marks can be found at the end of this file.The following guidelines will be used in marking each part of the assignment: 

100%	90%	75%	65%	50%	25%	0
Outstanding:	High Distinction:	Distinction:	Credit:	Pass:	Fail:	Not Submitted:
An outstanding attempt – well formatted and professionally presented piece of work.	An excellent piece of work that meets all the specified criteria with very minor omissions or mistakes	More than competently meets the criteria specified with only minor mistakes or omissions.	Competently meets the criteria as specified with few minor mistakes or omissions.	Satisfactorily meets the criteria.	Did not sufficiently meet the criteria to pass.	No attempt made or different from what is acceptable

Code Package Format The code implementation should be with Python 3 and PyCharm.

Your .zip code package should include Python source files, data files, and all supporting packages/modules. Make sure your demo codes are runnable in PyCharm after being unzipped.            

Report Format 

Your report should be no less than 800 words and it would be best to be no longer than 1500 words.The report MUST be formatted using the following guidelines:

• Paragraph text – 12 point Calibri single line spacing
• Headings – Arial in an appropriate type size
• Margins – 2.5cm on all margins ? Header – Report title
• Footer – page number (including the word “Page”)
• Page numbering – roman numerals (i, ii, iii, iv) up to and including the Table of Contents, restart numbering using conventional numerals (1, 2, 3, 4) from the first page after the Table of Contents.
• Title Page – Must not contain headers or footers. Include your name as the report’s author.
• The report is to be created as a single Microsoft Word document (version 2007, 2010, 2013 or Office 365). No other format is acceptable and doing so will result in the deduction of marks.
• No reference is required.

Please follow the conventions detailed in: Summers, J. & Smith, B., 2014, Communication Skills Handbook, 4^th Ed, Wiley, Australia.

Submission 

Submit your work as a zip file to Blackboard Task 2 by the due date of 11:59pm Friday, Week 12. Inside the zip file, a zip package of code and a Word file of report should be included.The assignment will be assessed according to the marking sheet. Late submission will be penalised according to the policy in the course outline. Please note Saturday and Sunday are included in the count of days late. 

Assignment Return and Release of Grades 

Assignment grades will be available on the course website in two weeks after the submission. An electronic assignment marking sheet will be available at this time.Where an assignment is undergoing investigation for alleged plagiarism or collusion the grade for the assignment and the assignment will be withheld until the investigation has concluded.

Assignment Guidelines 

This assignment will take a number of weeks to complete and will require a good understanding of application and information integration technologies for successful completion. It is imperative that students take heed of the following points in relation to doing this assignment: 

1. Ensure that you clearly understand the requirements for the assignment – what has to be done and what are the deliverables.
2. If you do not understand any of the assignment requirements – Please ASK the course coordinator or your tutor.
3. Each time you work on any aspect of the assignment reread the assignment requirements to ensure that what is required is clearly understood.

Appendix A  Marking Sheet for ATMC ICT705 2017.S3

Assignment 2 

Student name: 

Student ID: 

Report (30%):

Items		Maximum Marks	Marks Obtained
Report formatting (font, header and footer, table of content, numbering, correct spelling, grammar)		3
Introduction		2
Data merging and cleaning		7
RESTful Web services		7
Mashups		6
Demo running instructions		3
Conclusion		2
	Total =	30	0.0

 Code implementation (70%):

Items		Maximum Marks	Marks Obtained
1.a - data_merger.py – cleaning		10
1.b - data_merger.py – merging		10
2.a - clinic_locator.py – Web service		10
2.b - clinic_locator.py – input error handling		10
3.a - clinic_map.html – form layout		10
3.b - clinic_map.html – locator		10
3.c - clinic_map.html – information window – email link		5
3.d - clinic_map.html – information window – demographics		5
	Total =	70	0.0

RESTful web services are very useful for the different web applications that depends on the multiple data sources on the internet to provide and meet a specific data requirements of the users. For this the data and service integrations is an important aspect.  For GE it is important to successfully integrate the data available from different sources as well as the applications that uses this data.  In addition to that, use of the latest technological trends such as cloud-based services, data analytics (after cleaning) available from different sources and mobile computing to gain competitive advantage in the market and efficiency in its different processes. 

In this paper we will discuss about the techniques used for cleaning and merging of data sources that are in different formats. After this development of a RESTful web service and its use in a mashup application is discussed in the different sections of this report. 

Data merging and cleaning

  Initially we have two data files which are clinics.csv and location.xml. the first one contained the details of the clinics with the postal code and the second one contained the longitude, latitude values of the location of the clinic along with the postal code.

 In order to merge the two data sets, we first converted the xml file in to csv file using the “xml” package available in python. After this step the next step was to merge and lean the data sets.  Depending on the common column (the “PostalCode” column) in both the csv data sets we used the “pandas” data analytics package to merge them. Following is the screenshot of the code used for the cleaning and merging of the data sets.In the merged dataset there are some emails addresses of the clinics are not complete as they do not contain the @myclinic part.  In order do this we found the use of the “Petl” package as used in the lectures and provided in the notes.

After cleaning and refining the merged data set we got the table in a consistent format that can be used as a consistent data set for the RESTful as well as Mashup application. The redefined dataset in the csv files contains the following data, 

Name	Suburb	State	Postcode	Email	Lat	Lon
22	Ipswich Clinic	Ipswich	QLD	4305	[email protected]	-27.614604	152.760876
23	Cattle Creek Clinic	Cattle Creek	QLD	4407	[email protected]	-27.648300	150.836678
24	Caboolture Clinic	Caboolture	QLD	4510	[email protected]	-27.085007	152.951707
25	Booie Clinic	Booie	QLD	4610	[email protected]	-26.498426	151.935421
26	Rockhampton Clinic	Rockhampton	QLD	4700	[email protected]	-23.378941	150.512323
27	Townsville Clinic	Townsville	QLD	4810	[email protected]	-19.267358	146.806540
28	Cairns Clinic	Cairns	QLD	4870	[email protected]	-16.925397	145.775178
29	Adelaide Centre Clinic	Adelaide	SA	5000	[email protected]	-34.925770	138.599732

 RESTful Web services

The Web services are different types of service delivered through the web or internet. Every service ono internet is considered as an application module. These services communicate with each other and other web applications through the open protocols. The protocols through which the services communicate are WSDL and WSGI. These are mainly based on the XML and HTTP.

Data Merging and Cleaning

The traditional SOAP based web services, which are large in size and consumes a lot of bandwidth in delivering the services.  Compared to the traditional SOAP based web services, the REST architecture based web services are much lightweight and small which can be integrated with the any web applications in order to improve the performance of the applications for user’s data requirements.

 In Rest based web services, the resources are represented as the virtual folders on the internet. The services provide accessibility to these folders identified by the URI (Unique resource Identifiers)

 For this assignment, we have developed a web service server, that will get a postal code to get the details of the clinic using the merged data source which is “Clinic_location.csv” RESTful web services use the data caching technique in order to minimize the response time against a user’s data query through the web service. Rather than generating the same results frequently it stores the same results and delivers from the cached results. 

After the webserver service is executed successfully it will occupy a specific port as shown in the image,Mashups

The mashup applications are used in order to provide a unified view of different kind of data collected from different sources having different attributes of data. By using t mashups, the web applications help the end users to get the required data analysed from different perspectives. 

The mashup application developed for this project integrates the data from the merged dataset in the day merger stage of this project. The mashup application is Integrated with the Google maps application to plot the location of the clinics that are given in the csv file clinic_locatiin.csv. after getting the details from the csv dataset it will find the location on the Google map and plot the marker at the specific longitude and latitude.

 In order to get the detailed information about the clinic on the map the application uses ajax calls. 

Demo running instructions

The mashup application or the HTML page can take the postal code of any region of Australia. After this the web server gets the subtle elements for the postal code to plot the area of the clinics on the map.

The information for a particular postal code is gotten as the Json information question. This information question is then devouring by the application and marker is plotted on the google delineate.

For utilizing the application, at first the web application should be executed utilizing any python IDE. In the following stage the HTML page is to be opened and any postal code to check if there is any centre situated in that postal address then the web service will plot a marker on that locations by retrieving the longitude, latitude and other details.

As the RESTful web services can give pre-assemble extensibility to the web applications and administrations and gauges in this way enables the designers to utilize less measure of code with the implicit blunder taking care of highlight for the applications to which these web services are coordinated.

Conclusion

Being lightweight and low in bandwidth consumption the RESTful web services are the obvious choices for the developers to integrate with other applications and make those applications more efficient and productive.

The REST architecture is favoured by the developers rather than traditional SOAP or Simple Object Access Protocol architecture based web services due to its above mentioned features.