HELP: Nonblocking socket server odd behavior

[haldark]

Hi,
I am trying to test the non blocking behavior of socket server. I took https://github.com/troublegum/micropyserver as my base, and then added concept from https://forum.micropython.org/viewtopic.php?f=16&t=4211 to make it non-blocking.

However, when the Poller is a local variable, it works beautifully, but if I make poller as a class variable, it does not work.

Here is my code. The class has 2 start methods, start1() and start2(). start1() uses a local variable to hold poll object, whereas start2() stores the poll object in the class member variable.

SSID = <SSID>
PASS = <PASS>

import network
import esp
import time

#Copyright (c) 2019 troublegum. https://github.com/troublegum/micropyserver
import usocket as socket
import uselect as select
import sys
import io

class MicroPyServer_nb(object):

    def __init__(self, host="0.0.0.0", port=80, name=""):
        """ Constructor """
        self._host = host
        self._port = port
        self._name = name
        self._routes = []
        self._connect = None
        self._on_request_handler = None
        self._on_not_found_handler = None
        self._on_error_handler = None
        self._sock = None
        self._poller=None

    def do_something_else(self):
        print("Something Else")
        
    def handle_request(self, sock):
        print(sock)

    def start1(self):
        """ Start server """
        print("Starting '",self._name,"' NB Server at ", self._host,":",self._port)
        self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self._sock.bind((self._host, self._port))
        self._sock.listen(1)
        print(self._name," NB Server started")
        poller = select.poll()
        poller.register(self._sock, select.POLLIN)
        while True:
            if self._sock is None:
                break
            #https://forum.micropython.org/viewtopic.php?f=16&t=4211
            #we poll every 1 sec to check for socket activity.
            events = poller.poll(1000) #timeout=1sec
            if events: #only if there is socket activity, we accept the incoming connection
                for e in events:
                    #e is a tuple. e[0] is socket object
                    try:
                        if (e[0] == self._sock):
                            print("activity in my socket")
                            self._connect, address = e[0].accept()
                            self.handle_request(self._connect)
                    except Exception as e:
                        print("Exception occured",e)
                    finally:
                        self._connect.close()
            else:
                print ("No incoming socket activity")
                
            self.do_something_else()


    def start2(self):
        """ Start server """
        print("Starting '",self._name,"' NB Server at ", self._host,":",self._port)
        self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self._sock.bind((self._host, self._port))
        self._sock.listen(1)
        print(self._name," NB Server started")
        self._poller = select.poll()
        self._poller.register(self._sock, select.POLLIN)
        while True:
            if self._sock is None:
                break
            #https://forum.micropython.org/viewtopic.php?f=16&t=4211
            #we poll every 1 sec to check for socket activity.
            events = self._poller.poll(1000) #timeout=1sec
            if events: #only if there is socket activity, we accept the incoming connection
                for e in events:
                    #e is a tuple. e[0] is socket object
                    try:
                        if (e[0] == self._sock):
                            print("activity in my socket")
                            self._connect, address = e[0].accept()
                            self.handle_request(self._connect)
                    except Exception as e:
                        print("Exception occured",e)
                    finally:
                        self._connect.close()
            else:
                print ("No incoming socket activity")
                
            self.do_something_else()



#### MAIN CODE ####
esp.osdebug(None)

import gc
gc.collect()

sta=network.WLAN(network.STA_IF)

if not sta.isconnected():
    print("connecting to network...")
    sta.active(True)
    sta.connect(SSID,PASS)
    while not sta.isconnected():
        print(".", end=""),
        time.sleep(1)

print(sta.ifconfig())
sta_ip=sta.ifconfig()[0]
cfg_server = MicroPyServer_nb(host=sta_ip, port=80, name="Test")
#cfg_server.start1()
cfg_server.start2()

When I use start1() of the class, everything works fine. Both "Something Else" and "activity in my socket" shows up in th elog.

Here is the output, when I do 'curl 192.168.210.254' after the NB server starts.

MPY: soft reboot
RAM free  34000 , alloc  4016
('192.168.210.254', '255.255.255.0', '192.168.210.172', '192.168.210.172')
Starting ' Test ' NB Server at  192.168.210.254 : 80
Test  NB Server started
No incoming socket activity
Something Else
No incoming socket activity
Something Else
No incoming socket activity
Something Else
activity in my socket
**<socket state=3 timeout=-1 incoming=0 off=0>**
Something Else
No incoming socket activity
Something Else
No incoming socket activity
Something Else

However, when I comment start1() and use start2(), it becomes unreliable. The socket activity is detected sometimes. But when it stops detecting, it never detects ever again and I have to revert back to start1() to get it working.

Here is the output when it gets into that state. I did multiple 'curl 192.168.210.254' after the NB server starts, but it has no effect.

MPY: soft reboot
RAM free  34000 , alloc  4016
('192.168.210.254', '255.255.255.0', '192.168.210.172', '192.168.210.172')
Starting ' Test ' NB Server at  192.168.210.254 : 80
Test  NB Server started
No incoming socket activity
Something Else
No incoming socket activity
Something Else
No incoming socket activity
Something Else
No incoming socket activity
Something Else
No incoming socket activity
Something Else

it is baffling me. What mistake I am doing here? Any idea why I am seeing this behavior?

[peterhinch]

Interesting issue.

I tried your code (the problematic bound variable version) on a Raspberry Pico W also on an ESP32-S3 and it worked without lockups You may have found an ESP8266 bug here.

[shariltumin]

I ran your test script on an ESP8266 with cfg_server.start2() without any of the problems you mentioned.

>>> 
MPY: soft reboot
MicroPython v1.24.0-preview.112.g154813297.dirty on 2024-07-18; ESP8266 with ESP8266
>>>

The only change I made is to add "self._connect.send(b'hello from two\n')" after "self.handle_request(self._connect)" so that the connected client gets a response.

I have tried with three different types of clients:

$ curl --http0.9 192.168.4.98 80
hello from two
^C

$ telnet 192.168.4.98 80
Trying 192.168.4.98...
Connected to 192.168.4.98.
Escape character is '^]'.
hello from two
Connection closed by foreign host.

$ wget 192.168.4.98 80
--2024-08-02 17:35:14--  http://192.168.4.98/
Connecting to 192.168.4.98:80... connected.
HTTP request sent, awaiting response... 200 No headers, assuming HTTP/0.9
Length: unspecified
Saving to: ‘index.html’

index.html              [ <=>                ]      15  --.-KB/s    in 0s      

2024-08-02 17:35:14 (819 KB/s) - ‘index.html’ saved [15]

--2024-08-02 17:35:14--  http://80/
Resolving 80 (80)... 0.0.0.80
Connecting to 80 (80)|0.0.0.80|:80... ^C

$ cat index.html 
hello from two

As you can see, all tests received valid responses from the server. The "start1" has self._connect.send(b'hello from one\n') to be doubly sure what I am doing.