MATH50010 coursework 2024-25
26/11/2024
This coursework is due at 1pm on Monday 9th December. Please submit it via the turnitin link on blackboard. Your submission should contain your CID but not your name.
The Task
In this coursework, we will analyse the air quality in London. The dataset daqi_2023 .csv contains the daily air pollution index for greater London for every day in 2023. The first column contains the date of the measurement, and the Index column contains the index for that day. More details on the air pollution index can be found here: https://uk-air.defra.gov.uk/air-pollution/daqi?view=more-info. Importantly, an air pollution index of 1,2 or 3 is considered low. A low air pollution index means that there should not be enough pollutants in the air to cause any respiratory issues. An air pollution index of 4-6 is considered moderate which may affect those with respiratory conditions. An air pollution index of 7 or above is considered high and on such days there is a risk associated with strenuous outdoor activity.
The dataset daqi_2023 .csv is available to download from blackboard. In this coursework we will model the air pollution using Markov Chains constructed from this data set.
The following is a step-by-step workflow to guide you through the task. Your coursework submission should be written using RMarkdown, and compiled to a PDF for submission. All code should be commented clearly. For the highest marks, you should communicate to the marker clearly what you are trying to do, and justify any arbitrary choices. There are a total of 50 marks available for this coursework. 6/50 marks are available for an extension question, you can still get a good mark overall without attempting this question.
(4 marks) Loading and exploration
1. Read the data in to R.
2. (2 marks) We want to split the data into low, moderate and high index levels. A level of 3 of below is considered low, index level of 4-6 is moderate, and an index level of 7 or above is high. Create a new variable called ‘state’ indicating whether the pollution is high (2), moderate (1) or low (0) on each day.
3. (2 marks) Calculate the proportion of days in each of the states defined above.
(9 marks) A Markov Chain Model
We will now model the data as a Markov Chain.
4. (3 marks) We look at the transitions between states. Count the number of pairs in each of the possible pairs of successive states. Overlaps are OK, e.g. the sequence 0100 corresponds to one (0,1) transition, one (1,0) transition and one (0,0) transition.
5. (4 marks) Assume that the high/moderate/low air pollution index forms a three-state time-homogeneous Markov chain. Use the data to estimate the transition matrix of the chain.
6. (2 marks) Write a function that simulates draws of length m from a three state Markov chain.
(21 marks) Parameter Estimation
7. (4 marks) Use your function from question 6. to simulate n independent ‘years’ of daily air pollution index classifications using the transition probabilities from the data. For each of the n realizations of the chain, compute the estimates of the transition probabilities. Show that the estimators are approximately unbiased.
8. (3 marks) Calculate the variances of the estimates and comment on any differences.
9. (3 marks) Compute the correlation between your estimate of P (X1 = 1|X0 = 0) and your estimate of all other transition probabilities. Are there any other correlations you expect to be significant?
We will now investigate calculating the parameters of the transition matrix via maximum likelihood. Using the Chain Rule for probabilities and the Markov Property, we can write the likelihood of observing samples x0 , x1 , . . . , xn from a Markov chain as P (X0 = x0 ) n P (Xi = xi |Xi−1 = xi−1).
10. (2 marks) Write down the log-likelihood of the data in terms of the elements of the 3 state transition matrix (these will be the parameters we want to estimate).
11. (3 marks) When computing the maximum likelihood estimate, we want to make sure we are estimating the smallest possible number of parameters. Can you reduce the number of parameters of your model? Write down the log-likelihood with the smaller number of parameters. [Hint: use properties of transition matrices]
12. (5 marks) Calculate the maximum likelihood estimates of the transition probabilities, and verify that these are similar to your estimates in question 5. (Note that you do not need to estimate the initial state distribution)
13. (1 mark) Briefly explain why estimating the initial state distribution from the data via maximum likelihood is more challenging.
(14 marks) Estimating high/low air pollution events
For these questions, we will use our estimated transition matrix from question 5. or 11. to estimate the probability of high or low air pollution events. You should state which estimate you are using. You should find an expression for the quantities of interest analytically then substitute in the estimated transition matrix for the transition probabilities to create your estimates.
14. (3 marks) Given that we start in a low air pollution index day, estimate the probability that the air pollution index is low for the rest of the year (i.e. every other day that year has a low air pollution index).
15. (5 marks) Estimate the expected number of days in a year that have a high air pollution level given that we start in a low air pollution day. Compare your answer to the number of high air pollution days in the dataset.
16. (6 marks, extension) Estimate the probability that after a low air pollution index day, the next low air pollution day is exactly a week later (i.e. the probability that after a low air pollution index day, the next low air pollution index day is 7 days later with no low air pollution index days inbetween).
(2 marks) Conclusions
17. (2 marks) Comment on any limitations of your study and the Markov Chain model.