Academic Integrity Statement and Big Data Assignment Aims and Learning Outcomes

Academic Integrity Statement

Academic Integrity Statement: You must adhere to the university regulations on academic conduct. Formal inquiry proceedings will be instigated if there is any suspicion of plagiarism or any other form of misconduct in your work. Refer to the University’s Assessment Regulations for Northumbria Awards if you are unclear as to the meaning of these terms. The latest copy is available on the University website.

· Do NOT submit code from other people or web sources as your own, this is plagiarism.

· Do NOT buy your assignments on the Internet or submit work written for you by others. This is ghosting.

· For the individual element Do NOT work with other students and submit identical code, this is collusion.

· Both plagiarism, ghosting, and collusion are academic misconduct, which is not allowed.

Failure to submit: The University requires all students to submit assessed coursework by the deadline stated in the assessment brief.  Where coursework is submitted without approval after the published hand-in deadline, penalties will be applied as defined in the University Policy on the Late Submission of Work.

The aim of this assignment is to introduce a practical application of Big Data and Cloud Computing using a realistic big data problem. Students will implement a solution using an industry leading Cloud computing provider together with the distributed processing environment Apache Spark. This will involve the selection of problem appropriate Machine Learning algorithms and methods.

LO 1. Apply big data analytic algorithms, including those for visualization and cloud computing techniques to multi-terabyte datasets.

LO 2. Critically assess data analytic and machine learning algorithms to identify those that satisfy given big data problem requirements

LO 3. Critically evaluate and select appropriate big data analytic algorithms to solve a given problem, considering the processing time available and other aspects of the problem.

LO 4. Design and develop advanced big data applications that integrate with third party cloud computing services

LO 5. Critically assess the relationship between knowledge and the ethical and social interpretation of primary research using big data.

Portfolio Assignment: A collection of pieces of work

Individual Work: Work carried out by one person only

Group Work: Work carried out collaboratively seeking to improve each other’s elements

Peer Review: Critical analysis and subsequent grading of a social equal’s work

Semi-Formative: Training tasks assigned course credit to reward and ensure engagement.

The portfolio assignment is divided into components as follows:

Training Tasks

In the television documentary “Ross Kemp and the Armed Police” broadcast 6^th September 2018 by ITV, multiple claims were made regarding violent crime in the UK.

These claims were:

1. Violent Crime is increasing

2. There are more firearms incidents per head in Birmingham than anywhere else in the UK

3. Crimes involving firearms are closely associated with drugs offences

In this assignment you will investigate these claims using real, publicly available data sets that will be made available to you and placed in Amazon S3. These include, but are not limited to:

1. Street Level Crime Data published by the UK Home Office. This dataset contains 19 million data rows giving a crime type, together with their location as a latitude and longitude.

2. Land Registry Price Paid Data: This gives the postcode of a property, the property type from an enumeration of D (Detached), S (Semi-Detached), T (Terraced), F (Flats/Maisonettes) and the price paid.

3. Postcode Data: This data set is based on material provided by the Ordinance Survey. It gives a latitude and longitude to every postcode. This is useful as it relates between the Land Registry Price Paid dataset postcode, and the original crime dataset

Specifics

1. Process the data prepared for you using Apache Spark.

2. Filter the dataset so that crimes refer to relevant events only.

3. Using appropriate visualization methods, statistics, and machine learning, determine whether the claims made by Ross Kemp were true, false, or could not be determined.

4. Explain the reasoning behind your code so that it is clear what each block is intended to achieve, and why.

5. Report critically on the advantages, disadvantages, and limitations of the methods used.