From f34b3a261b74e5c0647534386399ab610300e79e Mon Sep 17 00:00:00 2001 From: Mateo Fidabel Date: Wed, 6 Oct 2021 17:39:24 -0300 Subject: [PATCH] Changed deprecated tf.multinomial to tf.categorical --- docs/spinningup/rl_intro.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/spinningup/rl_intro.rst b/docs/spinningup/rl_intro.rst index 604bc6584..5531cab7c 100644 --- a/docs/spinningup/rl_intro.rst +++ b/docs/spinningup/rl_intro.rst @@ -164,7 +164,7 @@ In what follows, we'll describe how to do these for both categorical and diagona A categorical policy is like a classifier over discrete actions. You build the neural network for a categorical policy the same way you would for a classifier: the input is the observation, followed by some number of layers (possibly convolutional or densely-connected, depending on the kind of input), and then you have one final linear layer that gives you logits for each action, followed by a `softmax`_ to convert the logits into probabilities. - **Sampling.** Given the probabilities for each action, frameworks like PyTorch and Tensorflow have built-in tools for sampling. For example, see the documentation for `Categorical distributions in PyTorch`_, `torch.multinomial`_, `tf.distributions.Categorical`_, or `tf.multinomial`_. + **Sampling.** Given the probabilities for each action, frameworks like PyTorch and Tensorflow have built-in tools for sampling. For example, see the documentation for `Categorical distributions in PyTorch`_, `torch.multinomial`_, `tf.distributions.Categorical`_, or `tf.random.categorical`_. **Log-Likelihood.** Denote the last layer of probabilities as :math:`P_{\theta}(s)`. It is a vector with however many entries as there are actions, so we can treat the actions as indices for the vector. The log likelihood for an action :math:`a` can then be obtained by indexing into the vector: @@ -207,7 +207,7 @@ In what follows, we'll describe how to do these for both categorical and diagona .. _`Categorical distributions in PyTorch`: https://pytorch.org/docs/stable/distributions.html#categorical .. _`torch.multinomial`: https://pytorch.org/docs/stable/torch.html#torch.multinomial .. _`tf.distributions.Categorical`: https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/distributions/Categorical -.. _`tf.multinomial`: https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/random/multinomial +.. _`tf.random.categorical`: https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/random/categorical .. _`torch.normal`: https://pytorch.org/docs/stable/torch.html#torch.normal .. _`tf.random_normal`: https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/random/normal .. _`torch.distributions.Normal`: https://pytorch.org/docs/stable/distributions.html#normal