bug: authz store has same hash for different states

[maharifu]

What is happening?

Section description

Provide as much context as you can. Give as much context as you can to make it easier for the developers to figure what is happening.

After yesterday's the chain halt, the authz store is reporting the same hash for different states.

From our mainnet validator using iaviewer:

Got version: 24385692
Printing all keys with hashed values (to detect diff)
  0114:B2B58E311EB12BD21F250E3FFA8E161C76CF541445E387EC031E2AF7D3CC591A6B90E1B2BDDA12E92F636F736D6F732E7374616B696E672E763162657461312E4D7367556E64656C6567617465
    129FE0D8F232F820A1CCDD306ACC84E73E56DC70AED11A0B4441B910E3100758
Hash: 07BBF06E9935FCBCBAAB22689BF337A621A07092FA7AC5E8A563B97FD6C0CA01
Size: 1

And from our LCD:

Got version: 24385656
Printing all keys with hashed values (to detect diff)
  0114:B2B58E311EB12BD21F250E3FFA8E161C76CF541445E387EC031E2AF7D3CC591A6B90E1B2BDDA12E92F636F736D6F732E646973747269627574696F6E2E763162657461312E4D7367576974686472617744656C656761746F72526577617264
    5D1755FE6276CE5DD1F3CFFF41CEC675C0918A44A55427049A16FE29DEDB4EF0
  0114:B2B58E311EB12BD21F250E3FFA8E161C76CF541445E387EC031E2AF7D3CC591A6B90E1B2BDDA12E92F636F736D6F732E7374616B696E672E763162657461312E4D736744656C6567617465
    739F603BAAAAA40224066C30C34C71699736FD23DEBDF2DA5B1E0378C5B50929
  0114:B2B58E311EB12BD21F250E3FFA8E161C76CF541445E387EC031E2AF7D3CC591A6B90E1B2BDDA12E92F636F736D6F732E7374616B696E672E763162657461312E4D7367556E64656C6567617465
    129FE0D8F232F820A1CCDD306ACC84E73E56DC70AED11A0B4441B910E3100758
Hash: 07BBF06E9935FCBCBAAB22689BF337A621A07092FA7AC5E8A563B97FD6C0CA01
Size: 1

We also get different results when querying paloma.

On the validator:

palomad q authz grants-by-granter paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
grants:
- authorization:
    type: cosmos-sdk/GenericAuthorization
    value:
      msg: /cosmos.staking.v1beta1.MsgUndelegate
  grantee: paloma1gh3c0mqrrc40057vtydxhy8pk27a5yhffpsh23
  granter: paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
pagination:
  total: "1"

On the LCD:

palomad q authz grants-by-granter paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
grants:
- authorization:
    type: cosmos-sdk/GenericAuthorization
    value:
      msg: /cosmos.distribution.v1beta1.MsgWithdrawDelegatorReward
  grantee: paloma1gh3c0mqrrc40057vtydxhy8pk27a5yhffpsh23
  granter: paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
- authorization:
    type: cosmos-sdk/GenericAuthorization
    value:
      msg: /cosmos.staking.v1beta1.MsgDelegate
  grantee: paloma1gh3c0mqrrc40057vtydxhy8pk27a5yhffpsh23
  granter: paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
- authorization:
    type: cosmos-sdk/GenericAuthorization
    value:
      msg: /cosmos.staking.v1beta1.MsgUndelegate
  grantee: paloma1gh3c0mqrrc40057vtydxhy8pk27a5yhffpsh23
  granter: paloma182ettr33r6cjh5sly58rl75wzcw8dn65pmey6x
pagination:
  total: "3"

Paloma and pigeon versions and logs

Section description

Write down paloma version. Write down pigeon version. Copy and paste pigeon config file as well as relevant ENV variables.

Paloma v2.2.2

How to reproduce?

Section description

Please write detailed steps of what you were doing for this bug to appear.

{WRITE YOUR ANSWER HERE}

What is the expected behaviour?

Section description

If you know, please write down what is the expected behaviour. If you don't know, that's ok. We can have a discussion in comments.

{WRITE YOUR ANSWER HERE}