Portfolio Assessment on Data Analytics, Computer Vision, and Ethics in AI

The assessment for this module is via a portfolio of work that will be assembled over the course of our three lab sessions. Topics will include (1) a data analytics, interpretation + visualisation task (lab session 1, 4/5 Nov - 18/19 Nov), (2) a computer vision task (lab session 2, 02/03 Dec), and (3) an ethical analysis (based on the lecture and materials in Week 5, w/c 29 Nov). You will receive formative support on all of these items during our lab sessions. Specific topics that should be included in the portfolio are as follows.

This component uses CEFAS’ 2021 data on biotoxins and phytoplantkon (see https:// www.cefas.co.uk/data-and-publications/habs/england-and-wales-biotoxins-andphytoplankton-results-2021/) to find patterns of higher or lower concentration of either (or both) according to features provided. You should read the data into a program (second tab on phytoplankton), clean it and then train a multi-layer feed-forward neural network to predict from a set of input features whether the phytoplankton level detected is above the threshold specified (see end of file). You will need to make a range of decisions in your analysis on data cleaning, network architecture and evaluation setup.

You should answer the following questions:

• Specify the accuracy you achieved across 3 architectural modifications (e.g. different numbers of layers, different hyperparameters, etc.)
• Why do you think your accuracy is not higher / lower?
• What effect does the optimisation function have on network performance?
• What happens if you include more than 4 (hidden) layers?
• What is the effect of the data size on your accuracy?

Generate and include in your your report the most suitable graphical plot of the data.

Download the “vehicles” dataset from here and adapt your CNN from the lab session to recognise the 4 object types in the dataset. Generate a graphical plot of your training and validation accuracy during training. Then answer the following questions:

• How long does the network need to train until reaching an accuracy of 95% (or does it not reach this level at all)?
• What is the tradeoff between using many layers (i.e. having a “deeper” network) and accuracy? And layers and time?
• What is the effect of changing the pooling mechanism, e.g. average vs max?

As a follow-on part, collect your own dataset of images containing the four object categories above. Make sure that they occur in different context, e.g. close-up, far-away, in a busy visual context, in an isolated image, etc. It is up to you how you collect these images- you can either take photos yourself or collect image from the internet. You should collect 20 images and copy these into your report, so I can see them.

• How well does your network do at classifying these images?
• Does fine-tuning make a difference?

Choose one of these three research papers to discuss:

1. Energy and Policy Considerations for Deep Learning in NLP
2. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
3. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 

In 800 words: highlight briefly the ethical challenge described and the researchers’ approach to uncovering and addressing it. Discuss in more detail areas of applied AI where you speculate similar challenges may occur and what incentives can be provided to AI researchers to tread carefully around ethical challenges.

This part of your portfolio should use a formal academic writing style and references in Harvard style, see here for guidance.

Portfolio 100%, with each component being worth 1/3 of the overall mark.

Do not include programming code into the report, i.e. screenshots or similar. If you want to present an algorithm, neural network architecture etc., then use pseudocode, a diagram or some other presentation that is not copy-pasted code.

You will need to submit your code alongside your report. It will not be marked separately but will be checked to ensure that it supports the functionality described in the report and is not plagiarised.

The portfolio is due: 14 December 2021, 2pm

Hand-in will be via Canvas.