perplexity

Hugging Face GPT-2 Perplexity

Activate python 3.7 Hugging Face environment:

conda activate hf37

System Info:

• NVIDIA Driver: 418.56
• CUDA version: 7.5.17
• Python version: 3.7
• PyTorch version: 0.4.1 (Compatible with CUDA 7.5)
• Tokenizers: 0.10.1
• Transformers 4.6.1

References:

https://github.com/Darg-Iztech/perplexity

https://www.cs.cornell.edu/~cristian/memorability.html