Welcome to OTGW-firmware Discussions!

[rvdbreemen]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[Roos-AID]

Hi, I am Rob, 23 years experience in home automation (KNX). Some rusty programming skills and experience in electronics.
OTGW in use with Nefit heating combined with ELGA heatpump and Honeywell thermostat.

OTGW sw linked to Openhab for home automation and logging of data in Influx to show graphs with Grafana.

I would like to get OTGW to the next level, get it working between the Heatpump and Nefit and between Thermostat and Heatpump, so 2 OTGW streams to record.
I have no intention to modify OT data.

[rvdbreemen]

Yes you can have multiple OpenTherm Gateways in series and just monitor the traffic of OT messages.

[SiriosDev]

Like this:


or Like this:

[SiriosDev]

??? @rvdbreemen

[STemplar]

Like this, with a gateway between, you can put multiple gateways in series.

[SiriosDev]

Like this, with a gateway between, you can put multiple gateways in series.

so if I understand correctly, do I need gt1 to have two terminals for gt2, or can I put the 2 "positives" together? (obviously I know it's a data cable, it doesn't have +/-)

like this?

[Roos-AID]

Robert, No it is not possible to connect 2 different OT devices (CV boiler and heatpump fi) at same time to the OTGW as the connection is one to one. The OTGW is positioned between thermostat (master) and the OT device (heatpump, ELGA in my case). The heatpump is then in a different circuit managing the boiler and is the master to the CV boiler (in my case a Nefit). I use a second OTGW between the heatpump and the CV boiler to track the boiler behaviour.

[rvdbreemen]

Hi Rob,
I kind of figured that.
Robert

[MrSnuggels]

Hi, I'm Erik (background hardware engineer and some basic programming skills) and I'm working on getting some more insight in my home appliances. Now running otmonitor in deamon on a RPi with a nodeMCU on the OTGW. However, the serial over tcp tends to drop out now and then. Therefore I would like to run the protocol-decoding locally, and preferably on the nodeMCU.

At the moment I'm trying to install the OTGW-firmware on a NodeMCU Amica flash id 4016h. Flashing works and I can configure the wifi, but once it is configured web-connections to the nodeMCU are refused. Pinging however is possible and yields quick responses (1~10ms). This occurs both with the 0.60 and 0.61 firmware binaries.

[bheuvel]

@rvdbreemen (nice work!), @MrSnuggels ,
If this is about the api/v1/otgw/otmonitor/, I had some problems with the JSON layout, e.g. {"name": "airtemp", "value": "On",.

But actually my problem started with getting all data (separate topics) into InfluxDB, using Telegraf. As I'm a bit lazy, I want a simple config and little maintenance.

When using MQTT (plugin for Telegraf) I need to specify each topic in telegraf, and create dedicated instances for each data type. The "ON"/"OFF" already a problem as it's not a number or boolean and as a string a bit less useful in InfluxDB, but do understand how common it is in most HA software.

Then I thought of using api/v1/otgw/otmonitor/ and the Telegraf HTTP plugin. Telegraf has a JSON parser, but then the fields would end up in InfluxDB as otmonitor_0_value (and numbered on). Telegraf has some pre-processing and other stuff, but could not find a way to get {"name: "x", "value": "y"} into {"x": "y"}. One other result was that everything would end up in one field called "value" (as ALL data comes from the keys "value")...

My final solution to get api/v1/otgw/otmonitor/ properly into InfluxDB using Telegraf:

[[inputs.exec]]
    interval = "5m"
    name_override = "OTGW"
    commands = [
        'sh -c "if ! command -v jq; then apk add jq; fi > /dev/null"',
        'sh -c "wget -q -O-  http://opentherm/api/v1/otgw/otmonitor/ | jq \".otmonitor | map( { (.name): .value } ) | map( map_values( if . == \\\"On\\\" then 1 elif . == \\\"Off\\\" then 0 else . end ))\""'
        ]
    data_format = "json"

As I'm using containers (telegraf:1.14.0-alpine), I had to cheat a bit (installing jq) to get it working with one simple config (mostly make it more acceptable to the JSON interpreter of Telegraf ;-)

So re-arranged (example):

{
  "otmonitor": [
    { "name": "flamestatus", "value": "Off", "unit": "" },
    { "name": "chmodus", "value": "Off", "unit": "" },
    { "name": "roomtemperature", "value": 16.650, "unit": "°C" },
    { "name": "roomsetpoint", "value": 15.000, "unit": "°C" }
  ]
}

using jq to:

[
  { "flamestatus": 0 },
  { "chmodus": 0 },
  { "roomtemperature": 16.650 },
  { "roomsetpoint": 15.000 }
]

Actually I would have liked the 0 to be false, but the JSON processor of Telegraf doesn't understand that (valid) JSON type, while InfluxDB does!

Unfortunately not all topics are published in the REST API, and personally would like to see the more descriptive names in the JSON, e.g.:

[
  { "Flame status": 0 },
  { "Central Heating Status": 0 },
  { "Room Temperature": 16.650 },
  { "Room Temperature Setpoint": 15.000 }
]

(Perhaps even "Room Temperature (°C)")

I could write a Python script for this and use the homeassistant info from MQTT, but this was quicker. But perhaps providing a api/v1/otgw/telegraf/ with all published topics is an intermediate step to building explicit publishing functionality to InfluxDB (as Telegraf is commonly used anyway when InfluxDB is present)?

If the api/v1/otgw/telegraf/ sounds like a plan (or whatever name for a condensed JSON with everything), I might have a go at it (not having programmed C++ in 20 years...but will manage (having used PowerShell, Perl, Ruby, Python, Bash).

One advantage of using REST over MQTT is that I can reduce the amount of (sometimes less useful) data (updates) into InfluxDB; instead of each second, once every 5 or even 15 minutes) (Not sure it actually matters much for InfluxDB)....

[rvdbreemen]

What if I just add a InfluxDB writer? That would simplify your setup even further. It's on the backlog, and I have already done the work my other project DSRM-next. It just got on the back burner so far.

For now I will add a restapi that matches your need.

[rvdbreemen]

Some comments on you post...

1. OTmonitor API versus single message rest api
   Did you know you can simply get every message you want by calling the values one by one?

This is the API for that:
/api/v1/otgw/id/{msgid}

That API is there so you can poll any value. The OTmonitor restAPI is there for one single reason, it's there for the use with the WebUI. If you find that particular api usefull, then you can use it as it is.

2. mixing unit and label
   I don't agree with you on mixing the label and the unit in one label. InfluxDB parses your data into label, value, unit. If you make a single label, you actually make it harder.

3. using spaces in label
   The reason I do not use "spaces" in the OTmonitor API has a simple reason, once you use spaces in json strings, then the parsing in javascript is more complicated. So that is the reason I have used the flatlabelwithoutaspace rest api call.

4. making the rest api for telegraf
   I think the example you give is with an array, why would you want to use an array. Why not a simple key value pair.

5. InfluxDB and storing values
   To be honest, influxDB was made for way more data ingestion than we will do with this single iot device. Actually using a rest API to poll the otgw firmware will mean you will miss out on a lot of data. As you only snapshot the data, and do not see the stream. Thats actually the reason why I plan to extend the firmware with influxdb writing in the first place.

If you start polling every 5 - 15m you actually will not be reliably monitor you boiler/thermostat. Think about it, within 5 minutes hot water can be turned on and off, boiler can go on/off, you would simply no be monitoring the otgw values reliable.

The main reason for the rest api I had was that the UI would benefit from it, decoupling the UI and ESP8266....

let me know you thoughts,
Robert

PS. you are welcome to join my discord server, invite link: https://discord.gg/zjW3ju7vGQ

[bheuvel]

Hi @rvdbreemen ,

You make valid points!
I just recently bought the OTGW and also just recently started using InfluxDB to collect data (from Openhab, ESMR5, Power meter).
Regarding 2 and 3, was more about getting more descriptive name in/from InfluxDB for graphs.
For now, building and connecting my infra, my Telegraf solution provides enough info for some simple graphs.

If you can "just add a InfluxDB writer", that would indeed simplify everything. Then don't add a Telegraf endpoint on my behalf.

And regarding the polling/"realtime"; also right indeed! I also plan to optimize my heating system in the near future and at that point having all data (changes/spikes/) would be very important to have.

So thanks for the informative answers, and looking forward to the InfluxDB writer.

Greetz,
Bob

[rvdbreemen]

@bheuvel check out the dev-telegraf branch. Added an rest api based on your idea.
Also if you need a compiled binary, drop a line.

[ESPmonster]

Hi all,

First of all pardon my ignorance and thanks to the developer for making this. I just finished my Nodo OTGW, installed this firmware and currently trying to make it work in Home Assistant. OTGW.local is successfully publishing heater status etc. via MQTT. But how have you guys configured everything in home assistant? E.g. setting the override temperature.
Does this work with HA's Opentherm integration (https://www.home-assistant.io/integrations/opentherm_gw/)? I tried connecting via this integration with Path: socket://mynodemcuIP:serialoutputport but without success. If so, how have you configured it? And if I may ask, what is the benefit of using this firmware vs the ESPeasy route except for the use of MQTT?

Thank you in advance for any reply! Cheers, Mats

[rvdbreemen]

Hi Mats,
I have started this project less than a year ago realizing the EasyESP route was not really adding any value, except a serial connection to the OTGW. While this is fine, in my personal situation I found it to be unstable over time. And that made me think, specifically the fact that the Nodoshop version has a full blown ESP8266 which is capable of way more than just forwarding bytes to a serial port.

Using a seperate CPU / microboard to just to MQTT was overkill (some use RasPI), so that got me thinking. And for my personal enjoyment I started to work on this during a short break from work. It has been progressing nicely and I met most of my initial goals. Like a standalone MQTT and OT protocol interpretation, as a serial 2 network (for OT monitor app) support. I am not trying to "double" the PIC firmware functionality, but more enhance the already great solution it is.

For me personally I wanted some features that EasyESP would probably never do, like:

• WebUI standlone to just monitor
• MQTT messages
• REST API

The configuration with Home Assitant is automatically created when using MQTT, it uses the MQTT Discovery protocol to configure the sensors.

You can also ignore the MQTT function completly, and just connect Home Assistant integration to port 1023 of your ESP device. It should just work nicely, as it is a simple serial port forwarder. At this time I have not been able to find the time to test that myself.

By change Bron Schelte just updated his firmware, which made my firmware unique, simply because I can interpret ALL msgid's in the 8266 and have topics for all of them. I think the ESP8266 solution in that way is unique, as it uses the raw OT messages to interpret the data going back and forth.

I do not use any of the 2 letter commands at this point in time (yet).

Hope this helps you to clearify the solution,
Robert

[ESPmonster]

Hi Robert, thanks for the reply! An honest apology for under-appreciating your development. I get it now, and it's quite brilliant! I can confirm the port fowarder function: I have successfully integrated the OTGW via port 1023 using the previously mentioned integration so that I can set a override temperature. I am now using the MQTT function to gather the sensor data. I also have influxdb installed for storage (actually, is the otgw mqtt data automatically saved to influxdb?). Thank you again. Really looking forward to see how this developments will grow.

[Haraldvdh]

Hi i am trying to write the outside temperature to the otgw via mqtt, but it does not work, and i cannot see why. In mqtt browser the message is displayed correctly. Here is the HA script, maybe someone can help me out. Thanks a lot!

alias: MQTT set outsidetemp
description: ""
trigger:

• platform: state
  entity_id:
  • sensor.slaapkamer_outside_temperature
    condition: []
    action:
• service: mqtt.publish
  metadata: {}
  data:
  retain: false
  topic: OTGW/command
  payload_template: "-m "OT={{ states('sensor.slaapkamer_outside_temperature') | int }}""
  mode: single

[woudenbe]

Hi there,
I am investigating the use of your firmware for the NodeMCU.
Is this already working with the latest ( version 5) firmware for the OTGW ?

br Hans

[rvdbreemen]

@woudenbe yes, the firmware will work with version 5. The reason why it will work is simply that the firmware interprets the open therm message itself, any message from our boiler or thermostat is parsed and used. You can use the OTGW advanced commands too if you like to play around with more. So it's a brigde for serial 2 network, but it does way more.

Hope you will enjoy it, come back with questions any time,
Robert

[hubertjanhickinson]

Hello Robert, nice work! Can you please explain to me how I can send commands to the gateway/boiler. I would like to test MSG ID 14, the max relative modulation level. I am using MQTT in Home Assistant so I would like to use the "sending commands thru MQTT (topic: OTGW/command)" option but I don't know how. Also I made a pull request, can you please check it?
#18

[rvdbreemen]

@hubertjanhickinson there are multiple ways to send commands to the opentherm gateway.

1. With MQTT the topic is: OTGW/command
2. Using the RESTAPI
3. Using serial port interface.

I am considering a way to send a wait for response (and retry max number of times) over MQTT and RESTAPI. So you do not need to implement logic yourself, but you can tell the nodemcu to take that task of reliable send the commands.

I checked your PR and closed it. I actually removed the sensors.yaml file, as since a very early version I started to support the Home Assistant Auto Discovery integration. So the sensor.yaml file is no londer needed if you use MQTT and HA.

By removing it I hope to clear that misunderstanding up, it was actualy an artifact that I had before the auto discovery integration. Just forgot to remove it.

[ESPmonster]

I updated to the latest firmware. Unfortunately the OTGW integration in HA is not yet compatible with the PIC 5.0 firmware so I reverted. Either way excellent work @rvdbreemen. How can we give back? Where is your buymeacoffee page?

[rvdbreemen]

If you mean the latest PIC firmware, 5.1 or 5.0. Then you need to thank Schelte Bron, he is the creator of the amazing assembler code for the PIC.

I focus with my NodeMCU firmware on futher integration with the Home Automation platforms, and I am trying to use the power of the ESP platform that is available on the Nodoshop version.

Also the integrations seem to have broken with the release of the 5.x as it has a breaking change in the PS=1 reports. The integrations use the serial interface and the "advanced" commands to control the opentherm interface, that are part of the PIC. Anyway, it's just time until they catch up.

So if you want to thank Schelte, you can go to his page at the bottom of his page is his donation link: https://otgw.tclcode.com/

Just in case you did mean to by me a coffee, just check out the bottom of my readme ;-)

[ESPmonster]

Although I'm also very thankful to Bron, I was actually referring to your work for the nodeMCU =)! I must be blind: just bought you that well-deserved cup. Indeed, I'm sure the pyotgw lib will be updated soon.

[rvdbreemen]

@ESPmonster thank you so much, much appreciated.

[georgeavd]

@rvdbreemen
Thanks for your good work.
I'm currently running 0.7.6 with PIC firmware 5.1 and connecting the OTGW to Node-Red via MQTT.
Durring the first tests I see a huge amount of MQTT passing by.

It would be good if E.g. the otmonitor MQTT-messages could be switched off via the web-GUI thus to reduce the MQTT-traffic when otmonitoring via MQTT is not used.

[rvdbreemen]

@georgeavd turning on and off MQTT integration can be done thru the settings already. Just uncheck the MQTT enable option, then MQTT function is just turned off.

[georgeavd]

I need MQTT to communicate between OTGW and NodeRed. Thus the MQTT function must be enabled.
But when not using the otmonitor it would be nice to be able to switch off otmonitor realated MQTT-messages.

[chezlong]

Got the Nodo board and ESP8266 board working fine between my cranky Vokera Linea boiler and the equally cranky Rellio BeSmart thermostat/hub. I've enabled MQTT on the gateway (firmware 0.10.2+50c3ed2 and PIC version 6.5).
Using the otmonitor, I can see all the nice graphs and logs, but it appears some of the opentherm parameters don't get read much (or not at all), by the thermostat, specifically, return temp and relative modulation.
I've used the REST API to request the values, and get a sensible result, like

 curl 192.168.1.212/api/v1/otgw/id/28
{
  "label": "Tret",
  "value": 52.1,
  "unit": "°C"
}

but I don't see this query appearing in the otmonitor log file as a read request of opentherm MSG_ID 28, and the return temperature value doesn't appear on the graph, or indeed in the mqtt Explorer
[EDIT] My question is should I expect the API to issue a opentherm read to the boiler and for MQTT/otmonitor to record this traffic. If not, is there another way to do this that would ?