Constructive AI: Designing an Autonomous Learning System

In this assignment, you will apply the notions, principles, methods and algorithms we touched on in the Constructive AI lectures to design, program, test, explain and demonstrate the learning system (robots) described below.
There are 25 marks to achieve which each translate to 1% of your overall module grade.
This assignment consists of one main task comprised of various subtasks.
You will implement an autonomous behaviour on a robots in the free Webots Robot Simulator used in the practical sessions. You need to use the same robot as in ”6COM1035 Practical Assignment 1: Constructing Autonomous Systems”.

The work must be your own. You may of course discuss the work with your peers but the work handed in must be distinctly yours. The following 4 sets of documents must be submitted on StudyNet/Canvas:
(a) the completed assignment briefing sheet
(b) a .zip archive containing your commented code (you need to include all the files for the robot and the environments)
(c) a report in PDF format providing the explanations requested in the task
(d) a .zip archive containing the requested short video demonstrating your work.

During this task, you will use the reinforcement learning framework and learn a policy which selects the actions to solve the 2-resource problem you defined in ”6COM1035 Practical Assignment 1”.

Create a model that fits the MDP framework [5 marks]
Adapt your model from ”6COM1035 Practical Assignment 1: Constructing Autonomous Systems” to create an MDP. You will need to:
• create a discrete state space from your sensor space (internal and external ones)
• create a policy selecting a behaviour
• model a reward function which is linked to the physiological variables
• choose a balance between exploration and exploitation

 

In your report, define you model briefly by showing the state space, the action space and the policy as sets or tables. State both your reward function and your model for exploration and exploitation (30 to 120 words, formulas and tables do not count as words but provide much information). 

You will need to test all four behaviours, the random start and your learned ones, in two conditions:
1. in the simple environment created during ”6COM1035 Practical Assignment 1: Constructing Autonomous Systems”
2. in a the complex environment created during ”6COM1035 Practical Assignment 1: Constructing Autonomous Systems”
Test the robot for at least 5 “runs” in both environments without further learning and no exploration. Each run should last no more than 5 minutes and no less than 2 minutes. All the runs should be of the same duration (unless the robot “dies” before the time has elapsed). Please note that, while testing
your robot, you need to log the relevant data to be able to perform a quantitative analysis, as explained below. Evaluate all the runs qualitatively and quantitatively.

For each of the runs in the original and in the complex environments, make notes of the behaviors that you observe. For example, cyclic behaviors, or if specific behaviors happens in a specific locations, how much of the environment your robot explores. Make a note of the differences you might observe in the behavior of the robot with the random policy in contrast the the robot with a learned policy. Are there behaviours the robot learned each time? Try to identify as many interesting behaviors as possible. Write a paragraph of between 100
and 200 words describing your observations for each type of environment and comparing the potential differences and similarities.

For each environment, perform a quantitative analysis of the robot’s performance in managing its internal variables. This must include plotting the logged internal physiological variables and the “wellbeing” calculated from these variables over time for each run.
Analyse the overall performance of the robot in each run by calculating and reporting two of the quantitative metrics seen in class: survival time for each run and viability.
In the report, include the plots or tables giving the above metrics for each run and a short paragraph (between 50 and 80 words) with the conclusions you draw from these metrics.