Build a network trace analyser utility using Python 3

Functionality Requirements - read very carefully!

The analysis of network traces (packet captures) is core to any cybersecurity operation.

In this coursework, you will therefore build a network trace (pcap) analyser utility using Python 3 (3.7 or higher).

This will open a packet capture (pcap file) and parse it for specified information such as IP and email addresses.

It will also perform some statistical analysis on the contents and visualise the traffic flows.

Your code must be documented appropriately. It must also be PEP8 compliant and get a pylint score as close to 10/10 as possible.

Use separate functions for each aspect of the functionality, and group these together appropriately in separate modularised scripts.

You will have coded some of the required functionality in lab exercises - and could import those modules rather than copying the code.

The requirements are described in detail below.
Functionality Requirements - read very carefully!

Refer to the separate FAQs for additional explanations.
Points 1-10 below correspond directly to the 10 marking criteria overleaf, while 0 is a general requirement.
1. While your script is running, it should print status information to the standard output (screen) as well as the specific output from the requirements listed in the following points. By status information, we mean, for example, "file xxx read successfully", "results written to file yyyy", etc.

2. Open, parse and close a pcap file. For the demo, this will be evidence-packet-analysis.pcap, but it should be easy to replace this with a different file. Store the parsed packets in Python, in one or more objects, for later re-use. Choose suitable data types for the object(s).

3. Summarise the contents of the capture in a table format, with one row for each traffic type. 
Each row should show the number of packets of that type, the first and last timestamps and the mean packet length for that type.
The mean packet length should be of the entire packet object (i.e. the buffer - not just the eth or the TCP payload). Make use of the in-built len() function instead of the embedded len attribute in the packet.
The example packet capture contains only three types (UDP, TCP, IGMP), but for full marks your script should handle the types adaptively and add other packet types if they occur. To test this, you should use other pcap files, see additional information below for some suggestions.

4.Find the information listed below. For some of these, you need to use regex, others are fields in the request parts of packets or can be extracted using the os module. 
1.Find any email addresses present in To: fields from emails in the packets
2.Find any email addresses present in From: fields from emails included in the packets
3.Find the URIs of all requests for image (e.g., jpg, gif, PNG) files and
4.extract the filenames of the requested image files.
      Print the unique email addresses, full URIs and filenames to screen in a suitable format.
Extract the sender and destination IP address pairs for all packets and count how many packets were sent from/to each.
Return the results in the form of a dictionary.
Consider carefully what you should use as the dictionary key.
Print the results to screen sorted so that most frequent traffic comes first, and nicely formatted.

Open, parse and close a pcap file

5.Find the geolocation information for every unique destination IP address for which this is available, and create a KML file that can be opened e.g., in Google Earth. 
oInclude only the destination IP addresses.
oFor each IP address, you should include city, country, latitude and longitude.
oYou will need to deal with incomplete information – this can be part of your exception handling
oStore the results in a KML file. 
Choose a suitable structure. 
Each point should include the packet count and city (if available) in its information.

6 .Plot the number of packets against time as a line chart. 

Part a. is about the preparation, while part b. is the actual graph. Each part is worth 3 marks. 
1. To start, you need to group packets into equal length time intervals, and store the start time and the number of packets for each interval.
•Experiment with the length of time interval to get a good graph – it needs to be long enough for each interval to contain a good number of packets and short enough to give you a bit of a timeline.
•The length should be a variable that is easy to change.
•Calculate the threshold for exceptionally heavy traffic. A rule of thumb in statistics is to use the mean number of packets per interval plus two standard deviations.
2. Plot the data as a line chart.
•It is expected that you use matplotlib.pyplot, but you can choose different tools if you prefer.
•Make sure the plot is annotated well – using suitable x- and y-axis labels, caption etc.
•Show the threshold visually in your plot. This could be a horizontal line, or represented e.g. by using different colours.
•Display the plot on screen automatically while the program runs and save it as a PNG

7.Include comprehensive exception handling in your scripts.
•For example, you should check for non-existent files and allow for packet captures with different types of traffic to be parsed and processed successfully.
•Any output from Exception handling should be directed to sys.stderr.

•Test your exception handling with several pcap files, such as the ones used in the lab and some of the ones mentioned in the additional information below.

•Consider the overall design, style, readability and efficiency of your code.

8.Each element of the functionality should be achieved by one (or more) functions. The functions should be grouped appropriately into several separate modules.

9. The central script which is run to call the code must be named py.

10. Use variables rather than hardcoding, use return statements to assign results to variables for future use.

11. Do not repeat the same operations or code unnecessarily.

12. Document functions, modules etc with appropriate doc strings, and use function annotations.

13. Use additional comments in key places to explain complex code.

14. Reference/cite any websites and other sources of code from others that you make use of. Give the specific source (e.g. the stackoverflow page, ideally also the author).