TypeError: a bytes-like object is required, not 'NoneType' for rainbow on treechop

[mattthelee]

I've put the log below. Appears maybe a None action is being passed into env.step?

INFO     - 2019-08-16 12:26:14,778 - [utils log_versions 9] 3.6.6 (default, Sep  3 2018, 15:31:46) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
INFO     - 2019-08-16 12:26:15,566 - [utils log_versions 10] absl-py==0.4.1,asn1crypto==0.24.0,astor==0.7.1,backcall==0.1.0,bcrypt==3.1.6,bitarray==0.9.3,bleach==2.1.4,boto==2.48.0,boto3==1.9.7,botocore==1.12.7,bunch==1.0.1,cached-property==1.4.3,cachetools==2.1.0,certifi==2019.3.9,cffi==1.12.3,chainer==4.4.0,chainerrl==0.7.0,chardet==3.0.4,cloudpickle==1.2.1,coloredlogs==10.0,crowdai-api==0.1.21,cryptography==2.7,cupy==4.0.0,cupy-cuda91==4.0.0,cycler==0.10.0,Cython==0.28.5,dask==0.19.0,decorator==4.4.0,Django==1.9,docutils==0.14,entrypoints==0.2.3,fastrlock==0.4,filelock==3.0.6,Flask==0.10.1,Flask-SocketIO==2.6,Flask-WTF==0.12,future==0.17.1,gast==0.2.0,getch==1.0,gevent==1.1.0,gevent-websocket==0.9.3,google-api-core==1.4.0,google-auth==1.5.1,google-cloud-core==0.28.1,google-cloud-dataproc==0.2.0,google-cloud-logging==1.7.0,google-cloud-storage==1.12.0,google-resumable-media==0.3.1,googleapis-common-protos==1.5.3,greenlet==0.4.14,grpcio==1.14.2,gym==0.14.0,h5py==2.6.0,html5lib==1.0.1,humanfriendly==4.18,idna==2.8,image==1.5.24,ipykernel==4.9.0,ipython==7.7.0,ipython-genutils==0.2.0,ipywidgets==7.4.1,itsdangerous==0.24,jedi==0.13.3,Jinja2==2.10,jmespath==0.9.3,jsonschema==2.6.0,jupyter==1.0.0,jupyter-client==5.2.3,jupyter-console==5.2.0,jupyter-core==4.4.0,kaggle==1.5.4,Keras==2.2.4,Keras-Applications==1.0.6,Keras-Preprocessing==1.0.5,kiwisolver==1.1.0,leveldb==0.194,lmdb==0.87,lxml==4.4.1,malmo==0.36.0.0,Markdown==2.6.11,MarkupSafe==1.0,marlo==0.0.1.dev15,matplotlib==3.0.3,minerl==0.2.3,mistune==0.8.3,mock==2.0.0,mrjob==0.6.5,nbconvert==5.3.1,nbformat==4.4.0,networkx==2.1,nose==1.3.7,notebook==5.6.0,numpy==1.17.0,olefile==0.45.1,openai==0.1,opencv-python==4.1.0.25,pandas==0.23.4,pandocfilters==1.4.2,paramiko==2.5.0,parso==0.4.0,pbr==5.1.3,pexpect==4.7.0,pickleshare==0.7.5,Pillow==6.0.0,pip==10.0.1,pipenv==2018.7.1,pkginfo==1.4.2,prometheus-client==0.3.1,prompt-toolkit==2.0.9,protobuf==3.6.1,psutil==5.6.3,ptyprocess==0.6.0,pyasn1==0.4.4,pyasn1-modules==0.2.2,pycparser==2.19,pydotplus==2.0.2,pyglet==1.3.2,Pygments==2.4.2,PyNaCl==1.3.0,pyparsing==2.4.0,PyQt5==5.9.1,PyQt5-sip==4.19.13,Pyro4==4.76,pyserial==3.4,pySmartDL==1.3.1,python-dateutil==2.8.0,python-engineio==2.2.0,python-gflags==3.1.2,python-gitlab==1.8.0,python-magic==0.2,python-slugify==3.0.2,python-socketio==2.0.0,pytz==2018.5,PyWavelets==1.0.0,PyYAML==3.13,pyzmq==17.1.2,qtconsole==4.4.1,redis==3.2.1,requests==2.22.0,requests-toolbelt==0.8.0,rsa==4.0,s3transfer==0.1.13,scikit-fmm==0.0.9,scikit-image==0.14.0,scikit-learn==0.19.2,scipy==1.3.0,seaborn==0.9.0,Send2Trash==1.5.0,serpent==1.28,setuptools==41.0.1,simplegeneric==0.8.1,sip==4.19.8,six==1.12.0,spyder-kernels==0.3.0,tensorboard==1.13.1,tensorflow==1.12.0,tensorflow-estimator==1.13.0,tensorflow-gpu==1.13.1,termcolor==1.1.0,terminado==0.8.1,testpath==0.3.1,text-unidecode==1.2,toolz==0.9.0,torch==0.4.1,torchvision==0.2.1,tornado==5.1,tqdm==4.33.0,traitlets==4.3.2,twine==1.11.0,typing==3.6.6,urllib3==1.25.3,virtualenv==16.0.0,virtualenv-clone==0.3.0,wcwidth==0.1.7,webencodings==0.5.1,Werkzeug==0.14.1,wheel==0.31.1,widgetsnbextension==3.4.1,WTForms==2.1,wurlitzer==1.0.2,XBee==2.3.2
INFO     - 2019-08-16 12:26:15,568 - [__main__ _main 137] The first `gym.make(MineRL*)` may take several minutes. Be patient!
INFO     - 2019-08-16 12:26:20,518 - [minerl.env.malmo.instance.04a184 _launch_minecraft 635] Starting Minecraft process: ['/tmp/tmpipots4mk/Minecraft/launchClient.sh', '-port', '9002', '-env', '-runDir', '/tmp/tmpipots4mk/Minecraft/run']
INFO     - 2019-08-16 12:26:20,643 - [minerl.env.malmo.instance.04a184 _launch_process_watcher 658] Starting process watcher for process 4524 @ localhost:9002
INFO     - 2019-08-16 12:31:11,890 - [minerl.env.malmo.instance.04a184 launch 498] Minecraft process ready
INFO     - 2019-08-16 12:31:11,904 - [__main__ wrap_env 151] Detected `gym.wrappers.TimeLimit`! Unwrap it and re-wrap our own time limit.
INFO     - 2019-08-16 12:31:11,908 - [minerl.env.malmo log_to_file 513] Logging output of Minecraft to results/MineRLTreechop-v0/rainbow/20190816T122613.869530/logs/mc_2.log
INFO     - 2019-08-16 12:31:11,942 - [env_wrappers __init__ 270] always pressing keys: []
INFO     - 2019-08-16 12:31:11,946 - [env_wrappers __init__ 276] reversed pressing keys: []
INFO     - 2019-08-16 12:31:11,947 - [env_wrappers __init__ 281] always ignored keys: ['back', 'left', 'right', 'sneak', 'sprint']
INFO     - 2019-08-16 12:31:11,968 - [env_wrappers __init__ 315] Dict(attack:Discrete(2), back:Discrete(2), camera:Box(2,), forward:Discrete(2), jump:Discrete(2), left:Discrete(2), right:Discrete(2), sneak:Discrete(2), sprint:Discrete(2)) is converted to Discrete(6).
INFO     - 2019-08-16 12:31:11,970 - [__main__ wrap_env 151] Detected `gym.wrappers.TimeLimit`! Unwrap it and re-wrap our own time limit.
INFO     - 2019-08-16 12:31:12,099 - [env_wrappers __init__ 270] always pressing keys: []
INFO     - 2019-08-16 12:31:12,110 - [env_wrappers __init__ 276] reversed pressing keys: []
INFO     - 2019-08-16 12:31:12,112 - [env_wrappers __init__ 281] always ignored keys: ['back', 'left', 'right', 'sneak', 'sprint']
INFO     - 2019-08-16 12:31:12,133 - [env_wrappers __init__ 315] Dict(attack:Discrete(2), back:Discrete(2), camera:Box(2,), forward:Discrete(2), jump:Discrete(2), left:Discrete(2), right:Discrete(2), sneak:Discrete(2), sprint:Discrete(2)) is converted to Discrete(6).
INFO     - 2019-08-16 12:52:51,814 - [chainerrl.experiments.train_agent train_agent 67] outdir:results/MineRLTreechop-v0/rainbow/20190816T122613.869530 step:8000 episode:0 R:0.0
INFO     - 2019-08-16 12:52:52,043 - [chainerrl.experiments.train_agent train_agent 68] statistics:[('average_q', 0.1425044723945982), ('average_loss', 3.8487588996725908), ('n_updates', 748)]
INFO     - 2019-08-16 13:10:50,996 - [chainerrl.experiments.train_agent train_agent 67] outdir:results/MineRLTreechop-v0/rainbow/20190816T122613.869530 step:16000 episode:1 R:0.0
INFO     - 2019-08-16 13:10:51,246 - [chainerrl.experiments.train_agent train_agent 68] statistics:[('average_q', 0.13654071304094723), ('average_loss', 3.7585677461190676), ('n_updates', 2748)]
INFO     - 2019-08-16 13:12:34,895 - [chainerrl.experiments.train_agent save_agent 274] Saved the agent to results/MineRLTreechop-v0/rainbow/20190816T122613.869530/16384_except
ERROR    - 2019-08-16 13:12:34,898 - [__main__ main 132] execution failed.
Traceback (most recent call last):
  File "dqn_family.py", line 130, in main
    _main(args)
  File "dqn_family.py", line 263, in _main
    outdir=args.outdir, eval_env=eval_env, save_best_so_far_agent=True,
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/experiments/train_agent.py", line 163, in train_agent_with_evaluation
    logger=logger)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/experiments/train_agent.py", line 54, in train_agent
    obs, r, done, info = env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 285, in step
    return self.env.step(self.action(action))
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/wrappers/continuing_time_limit.py", line 38, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/minerl/env/core.py", line 536, in step
    reward, done, sent = struct.unpack('!dbb', reply)
TypeError: a bytes-like object is required, not 'NoneType'
Traceback (most recent call last):
  File "dqn_family.py", line 271, in <module>
    main()
  File "dqn_family.py", line 130, in main
    _main(args)
  File "dqn_family.py", line 263, in _main
    outdir=args.outdir, eval_env=eval_env, save_best_so_far_agent=True,
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/experiments/train_agent.py", line 163, in train_agent_with_evaluation
    logger=logger)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/experiments/train_agent.py", line 54, in train_agent
    obs, r, done, info = env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 285, in step
    return self.env.step(self.action(action))
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/gym/core.py", line 261, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/chainerrl/wrappers/continuing_time_limit.py", line 38, in step
    observation, reward, done, info = self.env.step(action)
  File "/homes/mdl31/.local/lib/python3.6/site-packages/minerl/env/core.py", line 536, in step
    reward, done, sent = struct.unpack('!dbb', reply)
TypeError: a bytes-like object is required, not 'NoneType'
ERROR    - 2019-08-16 13:12:36,078 - [minerl.env.malmo _kill_minecraft_via_malmoenv 688] Attempted to send kill command to minecraft process and failed.
INFO     - 2019-08-16 13:12:36,164 - [minerl.env.malmo on_terminate 345] Minecraft process psutil.Process(pid=4524, status='terminated') terminated with exit code 0
/usr/bin/xvfb-run: line 186: kill: (4389) - No such process```