#
Quiz

The best way to learn and to avoid the illusion of competence is to test yourself. This will help you to find where you need to reinforce your knowledge.

###
Q1: Which of the following interpretations of bias-variance tradeoff is the most accurate in the field of Reinforcement Learning?

The bias-variance tradeoff reflects how my model is able to generalize the knowledge to previously tagged data we give to the model during training time. The bias-variance tradeoff reflects how well the reinforcement signal reflects the true reward the agent should get from the enviromment

###
Q2: Which of the following statements are true, when talking about models with bias and/or variance in RL?

An unbiased reward signal returns rewards similar to the real / expected ones from the environment A biased reward signal returns rewards similar to the real / expected ones from the environment A reward signal with high variance has much noise in it and gets affected by, for example, stochastic (non constant) elements in the environment A reward signal with low variance has much noise in it and gets affected by, for example, stochastic (non constant) elements in the environment

###
Q3: Which of the following statements are true about Monte Carlo method?

It's a sampling mechanism, which means we don't analyze all the possible states, but a sample of those It's very resistant to stochasticity (random elements in the trajectory) To reduce the impact of stochastic elements in Monte Carlo, we take `n` strategies and average them, reducing their individual impact

###
Q4: How would you describe, with your own words, the Actor-Critic Method (A2C)?

Solution

The idea behind Actor-Critic is that we learn two function approximations:

1. A `policy` that controls how our agent acts (π)
2. A `value` function to assist the policy update by measuring how good the action taken is (q)

###
Q5: Which of the following statements are true about the Actor-Critic Method?

The Critic does not learn any function during the training process The Actor learns a policy function, while the Critic learns a value function It adds resistance to stochasticity and reduces high variance

###
Q6: What is Advantage in the A2C method?

Solution

Instead of using directly the Action-Value function of the Critic as it is, we could use an `Advantage` function. The idea behind an `Advantage` function is that we calculate the relative advantage of an action compared to the others possible at a state, averaging them.

In other words: how taking that action at a state is better compared to the average value of the state

Congrats on finishing this Quiz 🥳, if you missed some elements, take time to read the chapter again to reinforce (😏) your knowledge.

Update on GitHub