Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Odd readings from DWSB20.2TH: Display shows power -195W, vzlogger reports 460W #132

Closed
tanuva opened this issue Sep 10, 2023 · 15 comments · Fixed by #136
Closed

Odd readings from DWSB20.2TH: Display shows power -195W, vzlogger reports 460W #132

tanuva opened this issue Sep 10, 2023 · 15 comments · Fixed by #136

Comments

@tanuva
Copy link

tanuva commented Sep 10, 2023

I'm not sure if I'm doing it wrong or the computers. In some situations the reported power matches what the display shows, but most often when I expect grid feed-in (and the meter display shows -195W, for example), vzlogger reports significant draw from the grid instead (460W, at roughly the same time).

dwsb_-195w

vzlogger configuration:

{
  "retry": 0,
  "verbosity": 20,
  "log": "",
  "push": [],
  "local": {
    "enabled": true,
    "port": 8080,
    "index": true,
    "timeout": 0,
    "buffer": 0
  },
  "meters": [
    {
      "enabled": true,
      "allowskip": false,
      "interval": -1,
      "aggtime": -1,
      "aggfixedinterval": false,
      "channels": [
        {
          "api": "null",
          "uuid": "a61877f3-f289-4b7e-86f7-167b5fe07593",
          "identifier": "1-0:1.8.0"
        },
        {
          "api": "null",
          "uuid": "88ad4730-ee53-4236-8551-7daecc94f3a8",
          "identifier": "1-0:2.8.0"
        },
        {
          "api": "null",
          "uuid": "7ef5c3c6-92ae-44e7-a7c2-420cbad9b849",
          "identifier": "1-0:16.7.0"
        }
      ],
      "protocol": "sml",
      "device": "/dev/ttyUSB0",
      "pullseq": "",
      "baudrate": 9600,
      "parity": "8n1",
      "use_local_time": true
    }
  ]
}

Resulting JSON from the vzlogger web server:

{
  "version": "0.8.1",
  "generator": "vzlogger",
  "data": [
    {
      "uuid": "a61877f3-f289-4b7e-86f7-167b5fe07593",
      "last": 1694350497673,
      "interval": -1,
      "protocol": "sml",
      "tuples": [
        [
          1694350497673,
          2897323.8000000003
        ]
      ]
    },
    {
      "uuid": "88ad4730-ee53-4236-8551-7daecc94f3a8",
      "last": 1694350497673,
      "interval": -1,
      "protocol": "sml",
      "tuples": [
        [
          1694350497673,
          163375.6
        ]
      ]
    },
    {
      "uuid": "7ef5c3c6-92ae-44e7-a7c2-420cbad9b849",
      "last": 1694350497673,
      "interval": -1,
      "protocol": "sml",
      "tuples": [
        [
          1694350497673,
          461.96000000000004
        ]
      ]
    }
  ]
}

The OBIS code should do the same thing as the display is showing, right? (As in: current power in/-output in Watts)
Am I holding it wrong or is there something else going on?

vzlogger is running in a docker container on Raspbian (Debian 11), built from master.

@tanuva tanuva changed the title Odd readings from DWSB20.2TH: Display shows power -195W, vzlogger reports Odd readings from DWSB20.2TH: Display shows power -195W, vzlogger reports 460W Sep 10, 2023
@r00t-
Copy link
Collaborator

r00t- commented Sep 11, 2023

can you please attach an sml dump (ideally via https://github.com/devZer0/libsml-testing#readme ),
and verify the behaviour with sml_server from libsml?
otherwise we can't do much.
if anything, it's most likely a bug in libsml or your meter, unrelated to vzlogger.
(i will probably just move this ticket over to the libsml project.)
actually DZG meters are known to have such bugs: #59
but iirc the issue used to be that positive readings are errorously sent as negative ones, not negative ones as positive as in your example.
@jmberg: would appreciate a comment from you.

@r00t- r00t- transferred this issue from volkszaehler/vzlogger Sep 11, 2023
@jmberg
Copy link

jmberg commented Sep 11, 2023

Well that is a DZG meter per the photo, and bad encoding of -195.00 might also correspond to the 16-bit inverse 460.36 in a similar bug ... however, the original bug is that the firmware of the meter encodes positive values wrongly, not negative values.

They fixed it later, maybe they broke their own display while at it?

This should be pretty simple to figure out though - @tanuva please attach a sample raw SML as @r00t- requested, but please also check continuity: if the meter is anything like mine, it outputs a value every 1 or 2 seconds, please record all these values and check for jumps. In this kind of data, the original issue was very easy to detect because once it overflows a 16-bit value it would encode it correctly and you'd see very large jumps from relatively large negative to relatively large positive values and vice versa in such data. So if you record for an hour around whenever you usually cross the threshold of zero due to consumption/production being roughly equal, we should see some information there.

@tanuva
Copy link
Author

tanuva commented Sep 11, 2023

Thanks for the hints! Collecting the data may take me a while, I won't be ghosting you. ;)

@jmberg
Copy link

jmberg commented Sep 11, 2023

Oh ... wait!

Another thought - maybe libsml is erroneously applying the workaround? That'd be consistent with what you're seeing here.

So maybe this is just a duplicate of #105?

Have you made sure you actually have a version of libsml that includes the more specific workaround since #106?

@tanuva
Copy link
Author

tanuva commented Sep 28, 2023

Have you made sure you actually have a version of libsml that includes the more specific workaround since #106?

I rebuilt the docker container from master a few days ago, so the workaround should be included.

I also checked the meter serial number. Mine starts with 1 DZG 004 ... so it probably doesn't have the fixed firmware I guess.

So I started collecting the raw meter data. When I feed them into sml_server I can't get it to print more than the first (?) 8 lines though, is that expected? (Even if I just say cat my.bin | ./sml_server -.)

For reference, the next values I saw in Home Assistant/vzlogger after stopping the dump were 240W solar production and 510W draw from the grid. I expected about 100 - 150W feed into the grid instead at that time. The meter display also showed something in that ballpark.

DZG_DWSB-20.2TH_draw_to_feed.zip
(I can of course submit the files via PR, too if they turn out useful.)

@jmberg
Copy link

jmberg commented Sep 28, 2023

It looks like your meter is a fixed version despite the old serial ...

I tried to fix the multi-message parsing in sml-server (https://p.sipsolutions.net/d64f11fb627c91be.txt) and then it shows more data, and the data from your meter makes more sense if we assume it's fixed (e.g. https://p.sipsolutions.net/51824d601e5fccf0.txt)

@jmberg
Copy link

jmberg commented Sep 28, 2023

I plotted your data with

workaround-applied

and without the workaround applied

without-workaround

and clearly without makes more sense.

@tanuva
Copy link
Author

tanuva commented Sep 28, 2023

Sooo... DZG apparently fixed existing devices, looking at the serial number hint in the other ticket?! Curious. I wonder if there still is a way to automatically determine whether the workaround needs to be applied. Otherwise it would need to become a configuration option, wouldn't it? 🤔

@jmberg
Copy link

jmberg commented Sep 28, 2023

I don't think we can determine the FW version based on anything other than the serial #, and if that is wrong then I really don't know. I guess we can ask them again.

@jmberg
Copy link

jmberg commented Sep 28, 2023

Oh your meter is also a different model than mine. So maybe they use the same serial number for different meters, but from the SML you cannot distinguish them, and the "fixed since" only applies to the specific model I have? Not a clue ...

@tanuva
Copy link
Author

tanuva commented Sep 28, 2023

If we can‘t come up with a universal rule I could try and prepare a PR that introduces a configuration option in vzlogger to apply the workaround. (Without having checked what I‘m getting myself into by saying that. :))

However, do you have a contact at DZG or did you just use their contact form?

@jmberg
Copy link

jmberg commented Nov 17, 2023

Good morning everyone.

I'm sorry I dragged my feet so much on this, but I figured before we go down the route of having to configure it, I'd inquire again with the person I had been in contact with previously at DZG. And, what can I say, they immediately responded!

So yesterday I learned that in fact only the DVS74 meter type that I have has the original encoding bug, which is already good to know. They also said that while they assign serial numbers from a single space for all types of meters, even meters with different type. However, no serial number is assigned twice. That doesn't help much so far.

Crucially, though, they were willing to share the information that (at least currently, not guaranteed forever), the serial number ranges

  • 4200 0000 to 4899 9999
  • 5500 0000 to 5899 9999
    are reserved for DVS74 meters. Previously we had the information that from 6000 0000 the bug will be fixed, so I think we can safely assume that the workaround needs to be applied exactly for those two ranges.

I'll send a pull request to do that.

jmberg added a commit to jmberg/libsml that referenced this issue Nov 17, 2023
In volkszaehler#132 and volkszaehler/vzlogger#533 it was reported that there
are DZG meters that fall into the range currently deemed broken
(all DZG meters with serial numbers before 6000 0000), but aren't
actually broken. The reason for this seemed to be that they're a
different type than mine that was broken.

Further inquiry with DZG revealed that indeed only DVS74 meters
are affected by the encoding issue, and then _those_ only for
serial numbers before 6000 0000. They also said that (currently)
the serial number ranges
 - 4200 0000 to 4899 9999, and
 - 5500 0000 to 5899 9999
are reserved for DVS74 meters.

As such, make the workaround detection more specific, and we no
longer need to have the check for 6000 0000 either, just need to
have a list of affected ranges. To achieve that, refactor the
code into a separate function that actually decodes the serial
number to an integer, so we can make the comparison more easily
and have the list of affected numbers more clearly in the code.

This should fix volkszaehler#132 since the serial number there starts with
4005, so it's clearly not in the affected ranges, as explained
by my contact at DZG. For volkszaehler/vzlogger#533 I don't have
the serial number, but it's the same type of meter as in volkszaehler#132.
@fhwe71
Copy link

fhwe71 commented Nov 17, 2023

Hello jmberg,
thanks for taking care and linking to the issue with 2/3 byte which I brought up last year.
Nevertheless I cannot say whether it's related or not.
Roughly reading this issue, I learned there is a bug in some DZG meters which must be compensated or not, according to serial number.
But in my issue I really manually analyzed the bytestream and figured out, that DZG changed the byte length of measurement from 3 bytes to 2 bytes in case measured values were small. According to my understanding of the protocol specification, everything was correct from DZG side.
But libsml did not consider that correctly and mixed up the sign of the value. Unfortunately I cannot reproduce, as I have a private smart meter meanwhile and my raspberry image containing vzlogger is destroyed.
Hope this helps,
Felix

@jmberg
Copy link

jmberg commented Nov 17, 2023

Well, the thing is, in some versions of some meters (see the pull request) DZG messed up the encoding, and libsml compensates for that.

However, it compensates too much, even on meters (like the one you had) that don't have the messed up encoding, since we didn't know. We figured out earlier in this thread that disabling the workaround fixed the issue. Pretty sure that causes/caused your issue as well, since it was the same meter model, and the person I was in contact with at DZG confirmed it was only some meters that were affected, not the DWSB series.

IOW, I think volkszaehler/vzlogger#533 is a duplicate of this bug, which should be fixed by #136.

jmberg added a commit to jmberg/libsml that referenced this issue Nov 27, 2023
In volkszaehler#132 and volkszaehler/vzlogger#533 it was reported that there
are DZG meters that fall into the range currently deemed broken
(all DZG meters with serial numbers before 6000 0000), but aren't
actually broken. The reason for this seemed to be that they're a
different type than mine that was broken.

Further inquiry with DZG revealed that indeed only DVS74 meters
are affected by the encoding issue, and then _those_ only for
serial numbers before 6000 0000. They also said that (currently)
the serial number ranges
 - 4200 0000 to 4899 9999, and
 - 5500 0000 to 5899 9999
are reserved for DVS74 meters.

As such, make the workaround detection more specific, and we no
longer need to have the check for 6000 0000 either, just need to
have a list of affected ranges. To achieve that, refactor the
code into a separate function that actually decodes the serial
number to an integer, so we can make the comparison more easily
and have the list of affected numbers more clearly in the code.

This should fix volkszaehler#132 since the serial number there starts with
4005, so it's clearly not in the affected ranges, as explained
by my contact at DZG. For volkszaehler/vzlogger#533 I don't have
the serial number, but it's the same type of meter as in volkszaehler#132.
jmberg added a commit to jmberg/libsml that referenced this issue Dec 11, 2023
In volkszaehler#132 and volkszaehler/vzlogger#533 it was reported that there
are DZG meters that fall into the range currently deemed broken
(all DZG meters with serial numbers before 6000 0000), but aren't
actually broken. The reason for this seemed to be that they're a
different type than mine that was broken.

Further inquiry with DZG revealed that indeed only DVS74 meters
are affected by the encoding issue, and then _those_ only for
serial numbers before 6000 0000. They also said that (currently)
the serial number ranges
 - 4200 0000 to 4899 9999, and
 - 5500 0000 to 5899 9999
are reserved for DVS74 meters.

As such, make the workaround detection more specific, and we no
longer need to have the check for 6000 0000 either, just need to
have a list of affected ranges. To achieve that, refactor the
code into a separate function that actually decodes the serial
number to an integer, so we can make the comparison more easily
and have the list of affected numbers more clearly in the code.

This should fix volkszaehler#132 since the serial number there starts with
4005, so it's clearly not in the affected ranges, as explained
by my contact at DZG. For volkszaehler/vzlogger#533 I don't have
the serial number, but it's the same type of meter as in volkszaehler#132.
r00t- pushed a commit that referenced this issue Dec 11, 2023
In #132 and volkszaehler/vzlogger#533 it was reported that there
are DZG meters that fall into the range currently deemed broken
(all DZG meters with serial numbers before 6000 0000), but aren't
actually broken. The reason for this seemed to be that they're a
different type than mine that was broken.

Further inquiry with DZG revealed that indeed only DVS74 meters
are affected by the encoding issue, and then _those_ only for
serial numbers before 6000 0000. They also said that (currently)
the serial number ranges
 - 4200 0000 to 4899 9999, and
 - 5500 0000 to 5899 9999
are reserved for DVS74 meters.

As such, make the workaround detection more specific, and we no
longer need to have the check for 6000 0000 either, just need to
have a list of affected ranges. To achieve that, refactor the
code into a separate function that actually decodes the serial
number to an integer, so we can make the comparison more easily
and have the list of affected numbers more clearly in the code.

This should fix #132 since the serial number there starts with
4005, so it's clearly not in the affected ranges, as explained
by my contact at DZG. For volkszaehler/vzlogger#533 I don't have
the serial number, but it's the same type of meter as in #132.
@tanuva
Copy link
Author

tanuva commented Feb 26, 2024

I have vzlogger logging with the more specific workaround linked above by now. I'm just waiting for a bit of sunshine so I can see the momentary power usage cross the pull/feed threshold to verify it works correctly.

Edit: Confirmed the fix in a few-minute-long glimpse of solar power.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants