Odd UART behaviour

[drzee99]

I have build a simple RS422 converter to interface with a ModBus Device using the UART interface.

The electrical diagram looks similar to this: https://github.com/dkjonas/Wavin-AHC-9000-mqtt/blob/master/electronics/schematic.png with the exception that I replaced the NodeMCU ESP8266 with the PICO and connected TX to GPIO0, TX to GPIO1, 3V3 to VSYS and the connection on the drawing going to GPIO5 to GPIO10 on the PICO.

I measured the voltages and its spot on 3.3V. The PICO boots up fine and runs stable.

I initialize the UART with:

self._uart = UART(0, baudrate=38400, bits=8, parity=None, stop=1, timeout_char=10, tx=Pin(0), rx=Pin(1), rxbuf=49, invert=0, timeout=10, txbuf=49, flow=0)

(note the RX buf 49 is sized to meet the requirements for the largest/longest message I can receive from the modbus device - baudrate. bits, partity etc. are dictated by the ModBus device spec).

I then send a command to trigger a response from the ModBus device using this routine (data is a bytearray containing the command and a CRC code at the end - the modbus will calculate the CRC again and see if it matched. If it does not match it sends an error back.)

def _raw_send(self, data):
    # flush the Rx FIFO
    while self._uart.any():
        self._uart.read()
    if self._ctrlPin:
        self._ctrlPin(1)
    self._uart.write(data)
    if self._ctrlPin:
        while not self._uart.txdone():
            time.sleep_us(100)
        #We wait a bit extra before closing the control pin as txdone exits when the last char starts to be send. So we have to wait until we are sure its send.
        time.sleep_us(int(self._char_time_ms*1000))    
        self._ctrlPin(0)

Then I wait the specified silent period between the send and receive operation (again its a modbus spec, 1.75ms) and start reading the incoming data:

def _raw_read(self):
    response = bytearray()
    start = time.ticks_ms()
    while (len(response) < RX_buffer_Size) and (time.ticks_diff(time.ticks_ms(), start) < 1000):
        if self._uart.any():
            response.extend(self._uart.read())
            #we check if we got everything, after we have the first 3 bytes we can read byte 3 which should contain the length of the data we are expecting
            #The total response length expected in bytes is 2 header bytes, 1 byte containing the expected data length, the N data byte and finally 2 bytes for CRC thus "N + 5 bytes" is the total
            if (len(response) >= 3) and (response[2] + 5 == len(response)):
                #we got it all and break the while loop
                break
            time.sleep_us(int(self._char_time_ms/2*1000))
    return response

On the ESP8266 this worked flawlessly and I had clean communication with the ModBus device. However moving this to the PICO i sometimes get "random data" in the RX buffer and sometimes the write/read cycle happens without any issue.

If i get "random data" and thus a CRC error calculation i resend the command and try reading again. Usually it then succeeds after the 1st or second attempts (sometimes up to 4 tries) and I get the correct response

I have checked if the "random data" in the RX buffer could be an error signal from the ModBus device, however its not. If the Modbus device did not understand what I send it does not respond at all. If i send a correct command but where the CRC does not match, the modbus device will send the applicable error code back. But none of the data on the RX buffer makes any sense in relation to the ModBus spec/error codes.

I have done some testing and can see that the RX buffer starts containing data right after I finish writing (despite me flushing it before the write operation) and while the control pin (GPIO10) is still high (which prevents the MAX3072E chip to actually send anything to the PICO).

The modbus specification clearly states that we only can expect a response when the Modbus device as received and replied to a correct command. It will not just send random things out. But where is the data coming from then?

Is this a buffer overflow of some kind in the internal micro code or micropython implementation? Where random data can makes its way into the RX buffers memory?

As mentioned the same code functions without any issue and no random data in the RX buffer on the ESP8266.

Im running: rp2-pico-w-20230203-unstable-v1.19.1-852-g9ea64a36a

[robert-hh]

I guess that the garbage on RX is related to not switching the direction at the right time, maybe too late.
You may have to experiment with the wait time before switching the direction, making it shorter or longer. For things like these, getting a simple 10$ logic analyzer is a great help.

Hint: Instead of the while not self._uart.txdone() loop you can simply call self._uart.flush(). That includes the loop and returns more predictable when just the last character is to be sent.

[robert-hh]

• There no need to change rxbuf to the largest size of the message. The default for rxbuf is 256.
• If the RX channel opens too late, the the first low pulse will be taken as start bit and subsequent received bytes are garbage. It then takes a silent period of at least 1 character time to re-sync the receiver to character frames.

[drzee99]

I used flush() before - same result.

At 38400 baud the character time, time to transmit a character, is calculated in ms to: char_time_ms = ((data_bits + stop_bits + 2) / float(baudrate))*1000 = 0.286 ms

After I see txdone (or flush they behave the same as in they exit while the last char is still being send) I wait for "char_time_ms" before reversing the direction.

In ModBus the standard defines the minimum silent period (between sending a command and expecting the response) to 1.75 ms or 3.5*char_time_ms - which ever is longer. At 38400 baud 0,286 * 3.5 = 1.0ms. Thus the minimum period is 1.75ms.

So I have plenty of time to turn off GPIO10 and reverse the direction by changing GPIO10. As I mentioned I already see data in the RX buffer before even reversing direction.

I don't think that is it ... but will see if I can improve the timing.

[robert-hh]

If the character time is 0.286ms, what is the value of self._char_time_ms ? Being in integer 0.286 maps to 0. It should be better to calculate the character time as µs. Like:
self._char_time_us = ((data_bits + stop_bits + 2) * 1000000) // baudrate
Why do you use 2 in the formula? There is 1 start bit. Do you use parity?

[drzee99]

I use:

time.sleep_us(int(self._char_time_ms*1000)) where _char_time_ms is float and return 0.286 converted to u-seconds it comes out to 286. So should all be god.

Well you may be right about the '2' in the formula. But even changing this to 1 does not make a sufficient difference to anything. In that case the _char_time_ms would be 0.260 ms which is marginally shorter but not short enough to matter for the silent period.

I read somewhere that there could be an issue if you happen to switch between the two CPUs. Could it be that the write operation runs on one CPU and the read then on the other?

[robert-hh]

After writing it came to my mind, that you might use float for self._char_time_ms. I had assumed to do that calculation as int. The other possible cause may be the call of MICROPY_EVENT_POLL_HOOK somwhere in the binaries used by the execution path of the code. MICROPY_EVENT_POLL_HOOK may stop code execution for up to 1 ms, which is about 3.8 character times at 38400 baud and may kind of randomly delay the call to _raw_read() too much. That happens definitely in the internal wait of uart.flush(), but may happen in other situations as well.
Which Python code do you run between the calls to _raw_send() and _raw_read()?

[robert-hh]

Just a note: my test showed that uart.flush() may return from the call up to 1 ms after the transmission of the last byte has started. At 38400 baud that would be ~750µs after the end of transmission. You do not use that here, but it should better be changed to make it more dependable, that just exactly one character time delay has to be added.
That does not help you here, but your problem may be caused by a similar effect.

[robert-hh]

It is the usual case that a GPIO input has a high impedance. Only if not, a documentation would be required. So ESP8266 is different from the standard, and there it is documented, even if it is hard to find.

[cve2022]

It is the usual case that a GPIO input has a high impedance. Only if not, a documentation would be required. So ESP8266 is different from the standard, and there it is documented, even if it is hard to find.

In the case of RP2040, the default of the GPIOs, according to the data sheet, is the weak pull-downs enabled (I looked at it after posting the above). Thus, a stronger case for the break condition as the pin does not float, instead it stays low. Other microcontrollers have pull-ups by default (IIRC it is the case of ST's STM32). I did not check the data sheet but probably it is also the case of ESP32 and that could explain why the break does not happen with it.

I did a quick test here with the code below:

from machine import UART, Pin
uart = UART(1, baudrate=38400, bits=8, parity=None, stop=1, timeout_char=10, tx=Pin(4), rx=Pin(5, pull=Pin.PULL_UP), rxbuf=49, invert=0, timeout=10, txbuf=49, flow=0)

Without the pull=Pin.PULL_UP I measured ~0V with a multimeter. After adding it, RX voltage is close to 3.3V. So the pull-up/pull-down state is preserved and it makes unnecessary adding the resistor, at least for testing.

[drzee99]

Thank you all for the input and suggestions! This was really helpful and basically solved my problems :) Big thanks to robert-hh and cve2022.

I can confirm that using Pin.PULL_UP works perfectly.

Question - what is the best "long term" approach a pull up resistor or using Pin.PULL_UP?

[cve2022]

Question - what is the best "long term" approach a pull up resistor or using Pin.PULL_UP?

If it is going into some sort or product, and a baseboard will be developed (using the Pico as component) or a full PCB development based on the RP2040, then I would add the resistor as it provides higher noise immunity. It is easier to not assemble it than having to rework the board later. Ofc, this is my opinion and it would be my approach.

OTOH, for testing or for use in controlled conditions, the internal pull-up is perfectly fine.

[drzee99]

This is for a hobby project of controlling my home heating system through AWS IoT and ultimately an app. For now the internal pull up is fine. But I may still add the external at a later stage - its easy on the prototype board that I use (with some soldering).

I will also be collecting temperature statistics from the room thermostats belonging to the system to get a time series of the temp in the room to make nice graphs :)

Because I decided to use Micropython I had to move from the ESP8266. It does not have enough memory for the code that I need to run (both to interface with the ModBus device as well as with MQTT and some logic in between). I could almost fit it if I do precomp bytecode, but then maintaining it and making changes was too cumbersome. The ESP8266 CPU is also barely powerful enough to run the MQTT TLS certificate based connection to AWS IoT - in particular when I have a connection loss and need to reconnect the process would eat a lot of memory and almost put the CPU to its knee.

The PICO seems to handle everything well due to plenty of mem and a powerful CPU that could be clocked higher if needed. My code was just not as portable as I thought :)

And then the PICO board is actually cheaper than most ESP boards .... when I have put more of it together I may throw it on GitHub or GitLab.

[robert-hh]

Just a note: if you still have a wait loop with calling machine.idle() in your _raw_send function, replace idle() by a short sleep (like sleep_us(100)). machine.idle() calls MICROPY_EVENT_POLL_HOOK, and that will wait up to 1 ms. That may still cause the wrong receives.