-
Notifications
You must be signed in to change notification settings - Fork 40
Home
Adapted from Instructables: ESP8266 WiFi OTA Remote File Management
Briefly, the work flow is as follows:
PHP files served via web browser will collect the users intentions to either: Read the controller status, upload a file, reboot the controller, remove files, compile files or run files on the ESP.
Once the command is collected from the user, PHP formats the data/information and sends it to the ESP via fsocket.
When the data/information is received at the ESP, LUA files parse the data and carry out the commands.
The possibility of managing ESPs over WiFi rests with the fact that the dofile() command loads the script into memory. Once in memory, the file can be removed or replaced in the ESPs file system with an edited file.
As an example. The init.lua file is automatically loaded into memory on boot up. If there is a command in this file to blink a LED or read a pin, the ESP will continue to carry out those tasks even if the init.lua file is deleted from the ESP. If a new init.lua file is uploaded or sent via WiFi that doesn't contain commands to blink a LED, nothing will happen until the file is executed ( dofile() ) or the ESP is rebooted. Then the new init.lua file will be executed and the LED will stop blinking.
We will use a web browser to send the new init.lua file and a command to restart.
There are some caveats to this solution, we will cover those later.
What you will need:
- A webserver running on the same LAN as your ESP8266 or the ESP8266 exposed to the Internet
- Access to the webserver's root. You will need to upload the PHP files to a root directory in under
www/html(apache) or\Inetpub\wwwroot(IIS). Simple website business - An ESP8266 flashed with NodeMCU and the ability to upload initial files via an IDE
- Some PHP and LUA understanding
- Source files for this project
The ESP running NodeMCU has a number of ways to communicate with the outside world. TCP/UDP Client, TCP/UDP Server, telnet and MQTT. Since we will be sending simple text strings to the ESP, any of these options can be used. MQTT would be a great solution for this project. However MQTT has a few more requirements, we will be using NodeMCUs TCP Server.
An ESP flashed with NodeMCU and configured as a TCP Server is capable of sending and receiving data to and from a web browser. This allows us to send files and commands to the ESP and also to query the ESP for certain information. For our purposes, when queried, the ESP can tell us how much memory is left and list the files in its file system.
Sending data to the ESP TCP server can be sent via number of methods. Some of the more conventional are GET and POST. Both of these include lots of header information and other information data of no use to us. The GET protocol is limited 256 characters in most configurations. ~256 characters would be a small .lua file. Although it could be done, sending files via multiple GETs would be difficult and error prone. These methods are slow and browsers can be troublesome.
PHP's fsockopen command is the tool of choice for this project. It is a one to one connection and very fast. We write to the socket just like writing to a file. And!! it only sends data written to it. No header info!
There is a limit to the number of characters sent in a fsockopen session. This is controlled by different TCP layer and most connections are limited to ~1460 characters. If the lua file we are sending is larger that 1200 characters (ASCII), our PHP file will close the session, open another, and continue to send the next 1200 characters of the file until EOF is reached. The LUA file on the ESP will either write or append to the file as it receives the data.
The PHP part of the project resides on the web server attached to the common LAN. This collection of scripts format the data and manage the transfer to the ESP. Files in /filbin folder can be sent to the ESP.
The php files for this project consist of:
-
index.php-- The GUI file for the user to send files/commands to the ESP -
controllerIP.txt-- Text file containing the IP number of the target ESP -
writeIP.php-- Saves the IP number tocontrollerIP.txt -
send_Compile.php-- Sends the filename and 'compile' command to ESP -
send_Delfile.php-- Sends the filename and 'delete file ' command to ESP -
send_Dofile.php-- Sends the filename and 'dofile' command to ESP -
send_file.php-- Sends the filename, command to write or append and a string containing the file to ESP -
send_Restart.php-- Sends a restart command to ESP
These files along with an empty sub-directory called filebin will need to be uploaded to the server root directory of the machine hosting your web server.
The directory /filebin contains files to be uploaded the ESP are stored.
In my previous example discussing the init.lua file blinking a LED, the /filebin directory is where you will put the edited/new version of the init.lua file. When you are finished editing, simply click on the file name and it will be sent to the ESP.
The index.php file (image above) uses simple forms and buttons to send information to the associated php execute file.
Clicking the "ReBoot ESP" button sends the IP number to the send_Restart.php file. The send_Restart.php file opens the socket to the IP, formats the message **command**restart** and writes that string to the ESP. META http-equiv= refresh is used to redirect back to index.php after 5 seconds.
All the commands are similar with the exception of writeIP and Controller Status.
The 'Controller Status' button is simply linked to the controllers IP number with /status appended to the IP.
Clicking on any of the files in Files in '/filebin' available for upload: area sends the filename and IP number to the send_file.php file.
The send_file.php reads the target file into an array, then begins to reassemble the file into one string prepended with the command **command**Newfile**\n. If/when the string exceeds 1200 characters the socket is closed. Another socket is opened and the string continues, this time pre-pended with **command**Apdfile**\n
The .lua files on the ESP8266 do the heavy lifting of parsing the strings coming over via fsockets. They 'catch' the payload (pl) coming from the webserver and parse for instructions.
In this example four files are required:
init.luaservernode.luawifi_tools.luagetstatus.lua
Initially, these files have to be uploaded to the ESP via FTDI and COM port. init.lua, servernode.lua and wifi_tools.lua must ALWAYS be in the file system of the ESP. They have to be available to dofile(). This is one of the caveats mentioned earlier.
When the ESP boots, it MUST to boot into a server mode and it MUST be able to access dofile() wifi_tools.lua.
If you 'format' the controller, its back to a FTDI and a COM port. Yes, you can format via WiFi.
With the exception of wifi_tools.lua and servernode.lua, these files can be removed and uploaded between reboots.
!! wifi_tools.lua and servernode.lua can only be overwritten, never removed !!
The size of wifi_tools.lua and servernode.lua normally, cannot exceed 1200 bytes (ASCII characters, including whitespace and linefeeds)
If wifi_tools.lua is larger that 1200 bytes, it will be partially written and will cause 'panic' when the ESP tries to dofile from the servernode.lua to append the rest of the file.
servernode.lua is a generic TCP server file, code can be found on the NodeMCU api site with this addition:
if string.sub(pl, 0, 11) == "**command**" then
dofile("wifi_tools.lua")
end
if string.find(pl, "status") then
dofile("getstatus.lua")
tmr.delay(250)
file.open("info.lua", "r")
conn:send("\n")
conn:send(file.read())
file.close("info.lua")
end
The variable pl is the payload from the fsocket() connection, this is the string generated by the PHP files server-side.
If **command** is found then the data string is parsed by wifi_tools.lua for exact details and commands.
The getstatus.lua routine is a bit convoluted, however it allows us to see which files are actually on the ESP in a near real time fashion. It also demonstrates the ability to create and serve files dynamically.
There is a learning curve associated with this project and stable managing of the ESP. One has to bear in mind how files are affected by other files.
For example, in this setup the init.lua calls the servernode.lua file, the servernode.lua file may call the wifi_tools.lua file. If this chain is broken, the ESP crashes and restarts.
If init.lua or servernode.lua are causing the panic, the ESP will loop forever. Probably nothing short of re-flashing the chip will cure this.
Only add the dofile() command when you are sure its working correctly by manual debugging.
It is good practice to run files manually during debugging to avoid these loops. With this project that isn't always possible and some other form of debugging must be employed. Many print() commands come in handy. Adding print commands and other debugging code to wifi_tools.lua is not a problem. The file can be as large as you like.
In fact you can add more commands to such as starting and stopping timers and even Format if you like. :)
Just remember you will not be able to upload/overwrite via WiFi if wifi_tools.lua or servernode.lua exceeds 1200 bytes. FTDI and COM ports still work if you have a wifi_tools.lua or servernode.lua file larger that 1200 bytes.
There is no size limit on any other files.
A lot of things have to happen just right for this to work. I have tested this on my setup with a number of different ESPs but I can't possibly foresee all combinations of variables.
My setup is running Apache2 and PHP Version 5.4.on Linux. I believe linux based Apache2 systems are similar enough to not cause issues.
Running on IIS or IOS will probably work, however, if it doesn't, look hard at line feeds and EOF handling. Possibly forward and backslashes in file addressing.
Chrome, IE and Safari seem to be ok. Chrome/Safari on my Iphone have no problems. Yes, I update/configure my ESPs from my Iphone.
I am connecting to my ESPs over a private LAN. I have not tested this over the Internet.