Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prometheus http: superfluous response.WriteHeader #319

Open
azimuth0x28 opened this issue Mar 30, 2020 · 0 comments
Open

prometheus http: superfluous response.WriteHeader #319

azimuth0x28 opened this issue Mar 30, 2020 · 0 comments

Comments

@azimuth0x28
Copy link
Contributor

azimuth0x28 commented Mar 30, 2020

Describe the bug
the node periodically stops working and gives an error
"http: superfluous response.WriteHeader"

To Reproduce
Steps to reproduce the behavior:
1 enable prometeus
2 request metrics every 3-5 sec

Expected behavior

Screenshots

Server :

  • OS: Ubuntu 18.04
  • Minter Version 1.1.3 - 1.1.5

Additional context
Mar 30 17:37:58 node-01-lab minterd[60007]: I[2020-03-30|17:37:58.652] Executed block module=state height=445440 validTxs=0 invalidTxs=0
Mar 30 17:37:58 node-01-lab minterd[60007]: I[2020-03-30|17:37:58.656] Updates to validators module=state updates=E43001EB90EA9CE73E63C93EC1F6CCDEF25CE3A2:12324051,E4656BE126AD3B0240FED35DC9218A680BD9D3A1:10746050,C8E0E3E3918BD6B7652AD526B2AA79E314BDBECC:9194007,37A2E1298F9BF90D64E177EEA09C1C2FF7526A03:6838495,C57F376FA419F7D3EFAF92546E2498D4E0CBF64A:5672001,8E36AD28D67B6C8E5A858EB89D8DADCE596ACF64:4314640,3DA17B6ADB8AD4E007C663B45C0592785CF529F5:3703563,E40B020EBF3B0F848054F7B552B377E2E84535D8:3390791,F314DA40E82365C30AAB918598D5BEA7C14ED18C:3170631,274A706B57D907B82BF2A8D09A2C4FC9CBA632A1:2822700,1FA57B4676E3BF71EDEB75AB8D721224F9B63139:2705289,2357D18EF1A7932D81D92997A2DC4EE4926A5167:2455321,6BD6070AD64ACC6AE6E9EBB57F8D1BED37924BA7:2347159,5B0D609F11C95EA6C8737B44098B45671B49C4D8:2144033,472241BE3DA8D2502EC42143207DF2BFBF6FB142:2053701,2F1643AC8F42A196E09D56604E96EABF40D31A7A:1986706,F8117B58FA10E888ADC41D7BD5BDDF69C4B679CE:1796873,48B4DCDF3CB995AF1A250BD9B9DAF0670BD8538F:1785527,1167B9CDA73ACF61DD4D547095A4EEA9E65FB57C:1736153,C56F4FCCE15670E95E747FCD35DE31C2FD165BF2:1399064,79F74D70EECEC249A22CB8F62C7DCBC6E6271272:1344410,76F198090EA6F028EFB2D0A4E3B8F17EBF9A481B:1284654,250AA5DDD60E04F9641880FCDC2C9F1B329B04AB:1113082,FE0366288EE5F77502977233B47AE9100EAC36AA:1079110,83ADD103E6E9958E731A2BC82B78A3D21D71E622:938978,2D8511E411B2DE903554642D62BC2AFB0E56901F:857231,FFFBFF90749719E024A4FE40C3574A3450F887EF:840604,D8C7836BC68DE509C4BA46DF9F0C8EE55A2DE4D4:739739,2EDE07A26576663CF724065B3631346112F3582F:735681,56C2F02DBC15E0E8EEEA06DB22BE110C128EDC9C:685901,3A310C4C18F96E1DB4C69CC0183BDF7868D25909:663410,881E474948E62A227728D157171ECE3D3B4D063F:553912,07945B7190495C2928C7882DE82EA6400B3A3158:537285,43760203B0A8F393256771497850AB8A48801B33:535722,CFBF359405F7DF2D56FDB981533628B5A5EA6B38:508972,AD4EA4CE2D4DCD7FF21A2EA7B303C515D9288ACC:508661,F3185036FC5C609AC240C64E161552FAC45A37EC:468739,B22B984386F6F474EBF711F1B4ADD8F5249553E2:445960,40E03221D1F80EC2E6D3AFEBFDF66C85802E20F6:363613,65B9E8D7CAD755F6FA48D1354E24C89FCC73661F:350103,622C70973D6DBEEA0EB4BDDA66CD4B2595532E17:326557,E766B0193777374B32E95B00A90C94D32F42AAD4:288620,BCAFB7535D8E808E1CCF12CD6BF39D7E5CBF20F1:269886,D44D9A6429BCB7A20897DB5408177FDC5B4DB5FC:269808,35B9BE575C9C427AFAB7141BBA1E0786BD029291:250599,61C35F269CCD2C6253D503BA49B2B339D71A48AD:204006,01959CC6CE0F1AC9A0E809D2DF23FC3AD2858920:192033,7C5CA666A60812AD69D5C0A772797BDAC4143E67:190108,D919FEB139AA85B875735C7EF395D22C24997EC4:160477,86805BF2E9BA3A71526C0A76A3536F805581C341:154085,6239FCBB81C8D1E8471A8A55E40D07CDB1A063B2:132127,4AED1026556A6ADA70E0296AFE92E3D3DE8A12F5:124131,EBEFB0AC72CA98CD7D12AF868A544614E33C5B2C:118663,E40398FAA915B0780576C99B44D715F9F6623930:97923,F1A19A8A8AF10A50AC86A70B2E0CD2CC91F830F6:46100,135AC0AC103089865E32E81621D6779E7235D235:32327
Mar 30 17:38:56 node-01-lab minterd[60007]: 2020/03/30 17:38:56 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:39:09 node-01-lab minterd[60007]: E[2020-03-30|17:39:09.208] Connection failed @ sendRoutine module=p2p peer=eb8d5dd2d0baf44e59859bed8a3b6591019e9f2b@xxxxxxxx:59218 conn=MConn{94.130.110.131:59218} err="pong timeout"
Mar 30 17:39:09 node-01-lab minterd[60007]: E[2020-03-30|17:39:09.208] Stopping peer for error module=p2p peer="Peer{MConn{xxxxxxxx:59218} eb8d5dd2d0baf44e59859bed8a3b6591019e9f2b in}" err="pong timeout"
Mar 30 17:39:15 node-01-lab minterd[60007]: E[2020-03-30|17:39:15.718] Connection failed @ sendRoutine module=p2p peer=b781f0e3e1bf44b15d71146332523995d1893bca@xxxxxxxx:57346 conn=MConn{95.216.114.103:57346} err="pong timeout"
Mar 30 17:39:15 node-01-lab minterd[60007]: E[2020-03-30|17:39:15.719] Stopping peer for error module=p2p peer="Peer{MConn{xxxxxxxx:57346} b781f0e3e1bf44b15d71146332523995d1893bca in}" err="pong timeout"
Mar 30 17:39:55 node-01-lab minterd[60007]: 2020/03/30 17:39:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.518] Committed state module=state height=445440 txs=0 appHash=096730F7890F3AE129FC9AB03D1C7171748F8B24310DBA307476D97AEB18FC58
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.522] Timed out module=consensus dur=-2m4.348303372s height=445441 round=0 step=RoundStepNewHeight
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.522] Added to lastPrecommits: VoteSet{H:445440 R:0 T:2 +2/3:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87(0.7511158827566531) BA{56:x___x_xx_xxxxxxxxxxx___xx_x__xx__xxx__x__xxx_xxxxxxxxx_x} map[b781f0e3e1bf44b15d71146332523995d1893bca:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87]} module=consensus
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.523] Added to lastPrecommits: VoteSet{H:445440 R:0 T:2 +2/3:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87(0.7514391528310052) BA{56:x__xx_xx_xxxxxxxxxxx___xx_x__xx__xxx__x__xxx_xxxxxxxxx_x} map[b781f0e3e1bf44b15d71146332523995d1893bca:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87]} module=consensus
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.523] Added to lastPrecommits: VoteSet{H:445440 R:0 T:2 +2/3:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87(0.8433795139772882) BA{56:x__xx_xx_xxxxxxxxxxx___xx_x__xx__xxx__xx_xxx_xxxxxxxxx_x} map[b781f0e3e1bf44b15d71146332523995d1893bca:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87]} module=consensus
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.524] Added to lastPrecommits: VoteSet{H:445440 R:0 T:2 +2/3:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87(0.8541704964592142) BA{56:x__xx_xx_xxxxxxxxxxx___xx_x__xx__xxx__xx_xxx_xxxxxxxxxxx} map[b781f0e3e1bf44b15d71146332523995d1893bca:AF2DDDBA147A87AF826E007249B8ECCB1BA743F35F829AEFCE0452A353EE9595:1:7F6342EE2E87]} module=consensus
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.524] enterNewRound(445441/0). Current: 445441/0/RoundStepNewHeight module=consensus height=445441 round=0
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.524] enterPropose(445441/0). Current: 445441/0/RoundStepNewRound module=consensus height=445441 round=0
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.622] Stopping gossipVotesRoutine for peer module=consensus peer="Peer{MConn{xxxxxxxx:57346} b781f0e3e1bf44b15d71146332523995d1893bca in}"
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.622] Stopping gossipDataRoutine for peer module=consensus peer="Peer{MConn{xxxxxxxx:59218} eb8d5dd2d0baf44e59859bed8a3b6591019e9f2b in}"
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.622] Stopping gossipVotesRoutine for peer module=consensus peer="Peer{MConn{xxxxxxxx:59218} eb8d5dd2d0baf44e59859bed8a3b6591019e9f2b in}"
Mar 30 17:40:05 node-01-lab minterd[60007]: I[2020-03-30|17:40:05.622] Stopping gossipDataRoutine for peer module=consensus peer="Peer{MConn{xxxxxxxx:57346} b781f0e3e1bf44b15d71146332523995d1893bca in}"
Mar 30 17:40:07 node-01-lab minterd[60007]: I[2020-03-30|17:40:07.522] Stopping queryMaj23Routine for peer module=consensus peer="Peer{MConn{xxxxxxxx:59218} eb8d5dd2d0baf44e59859bed8a3b6591019e9f2b in}"
Mar 30 17:40:07 node-01-lab minterd[60007]: I[2020-03-30|17:40:07.522] Stopping queryMaj23Routine for peer module=consensus peer="Peer{MConn{xxxxxxxx:57346} b781f0e3e1bf44b15d71146332523995d1893bca in}"
Mar 30 17:40:07 node-01-lab minterd[60007]: I[2020-03-30|17:40:07.524] Timed out module=consensus dur=2s height=445441 round=0 step=RoundStepPropose
Mar 30 17:40:07 node-01-lab minterd[60007]: I[2020-03-30|17:40:07.524] enterPrevote(445441/0). Current: 445441/0/RoundStepPropose module=consensus
Mar 30 17:40:07 node-01-lab minterd[60007]: I[2020-03-30|17:40:07.524] enterPrevote: ProposalBlock is nil module=consensus height=445441 round=0
Mar 30 17:40:55 node-01-lab minterd[60007]: 2020/03/30 17:40:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:41:55 node-01-lab minterd[60007]: 2020/03/30 17:41:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:42:56 node-01-lab minterd[60007]: 2020/03/30 17:42:56 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:43:55 node-01-lab minterd[60007]: 2020/03/30 17:43:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:44:56 node-01-lab minterd[60007]: 2020/03/30 17:44:56 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:45:55 node-01-lab minterd[60007]: 2020/03/30 17:45:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:46:55 node-01-lab minterd[60007]: 2020/03/30 17:46:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:47:55 node-01-lab minterd[60007]: 2020/03/30 17:47:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:48:55 node-01-lab minterd[60007]: 2020/03/30 17:48:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:49:55 node-01-lab minterd[60007]: 2020/03/30 17:49:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)
Mar 30 17:50:55 node-01-lab minterd[60007]: 2020/03/30 17:50:55 http: superfluous response.WriteHeader call from github.com/prometheus/client_golang/prometheus/promhttp.(*responseWriterDelegator).WriteHeader (delegator.go:58)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant