Windows Service cannot find required configuration files after creating .exe using cx_Freeze

[dqcontrols]

Hi,

I am attempt to create a few executables that will be run as a Windows Service. Here is my code for creating the executables:

from __future__ import annotations
import sys
from cx_Freeze import setup, Executable

build_exe_options = {
    "build_exe": "build/tool",
    "packages": ["logging"],  # Replace with your logging package name
    "includes": ["ServiceHandler", "cx_Logging"],
    "include_files": [("devicePrefs.conf", "conf/devicePrefs.conf"), ("appVersion.conf", "conf/appVersion.conf"), ("loggingPrefs.conf", "conf/loggingPrefs.conf")],  # Add your .config file
}

base = None
if sys.platform == "win32": # Create the Windows .exe
    base = "Win32Service"
    executables = [Executable("MainCode.py", base=base, icon = "Logo.ico", target_name="MainSoftware.exe"),
                   Executable("Reset.py", base=base, icon = "ResetLogo.ico", target_name="Reset.exe"),
                   Executable("Search.py", base=base, icon = "SearchLogo.ico", target_name = "Search.exe")]
elif sys.platform == "darwin": # Create the MAC OS .exe
    base = "MacOSX" 
else: # Create the Linux OS .exe
    base = None
    executables = [Executable("MainCode.py", base=base, icon = "DQLogo.ico", target_name="MainSoftware"),
                   Executable("Reset.py", base=base, icon = "ResetLogo.ico", target_name="Reset"),
                   Executable("Search.py", base=base, icon = "Search.ico", target_name = "Search")]

setup(
    name="MainSoftware",
    version = "1.1",
    description="Main Software",
    options={"build_exe": build_exe_options},
    executables=executables,
)

This creates the executables correctly. However, when I attempt to initialize MainSoftware.exe as a windows service using the following command:

sc create MainSoftware start= delayed-auto binPath= "C:\work\build\tool\MainSoftware.exe C:\work\build\tool"

The windows service is created but it cannot start due to an error "error 1503 the service didn't respond in a timely fashion". Checking the log file, it seems to be because the service cannot access the configuration files required for my code to run (devicePrefs.conf, appVersion.conf, and loggingPrefs.conf located in the conf folder - see "include_files" declaration above).

How can I amend this issue, where the service is able to pick up these config files? Here is how my ServiceHandler.py script is set up:

from __future__ import annotations
"""Implements a simple service using cx_Freeze.

See below for more information on what methods must be implemented and how they
are called.
"""
import threading
import os
import sys

class Handler:
    # no parameters are permitted; all configuration should be placed in the
    # configuration file and handled in the initialize() method
    def __init__(self):
        self.stopEvent = threading.Event()
        self.stopRequestedEvent = threading.Event()

    # called when the service is starting
    def initialize(self, configFileName):
        pass

    # called when the service is starting immediately after initialize()
    # use this to perform the work of the service; don't forget to set or check
    # for the stop event or the service GUI will not respond to requests to
    # stop the service
    def run(self):
        self.stopRequestedEvent.wait()
        self.stopEvent.set()

    # called when the service is being stopped by the service manager GUI
    def stop(self):
        self.stopRequestedEvent.set()
        self.stopEvent.wait()

Thanks in advance!

[marcelotduarte]

Here is a example service that starts and stops.
Please note that your MainCode.py should be a Config.py equivalent.
base="Win32Service" should be used only for windows service. If Reset and Search aren't windows services, use a different base (None, 'console' or 'gui').

[dqcontrols]

Yep, MainCode.py has the attributes required from Config.py. I have adjusted the issue where Reset and Search are using base=Console. I am still running into the same issue, where the configuration files required for my code to run (devicePrefs.conf, appVersion.conf, and loggingPrefs.conf) are not being accessed by MainCode.py, preventing the service from starting. Specifically, MainCode seems to crash when it attempts to read from the configuration files. See log file entry when attempting to start the service below:

[14744] 2024/06/07 12:41:41.212 starting logging at level ERROR
[14744] 2024/06/07 12:41:41.927 Python exception encountered:
[14744] 2024/06/07 12:41:41.927 Internal Message: initialization script didn't execute properly
[14744] 2024/06/07 12:41:41.927 Type => <class 'KeyError'>
[14744] 2024/06/07 12:41:41.927 Value => 'NAME'
[14744] 2024/06/07 12:41:41.929 Traceback (most recent call last):

[14744] 2024/06/07 12:41:41.929 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts_startup_.py", line 141, in run
module_init.run(name + "main")

[14744] 2024/06/07 12:41:41.930 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts\console.py", line 25, in run
exec(code, main_globals)

[14744] 2024/06/07 12:41:41.930 File "MainCode.py", line 569, in

[14744] 2024/06/07 12:41:41.930 File "C:\Users\sk\anaconda3\envs\dq\Lib\configparser.py", line 979, in getitem
raise KeyError(key)

[14744] 2024/06/07 12:41:41.930 KeyError: 'NAME'

From the service side, I get this error:

Is there anything I can/should change in ServiceHandler.py to ensure that the configuration files I require are read? Please see above for the ServiceHandler.py script setup

[marcelotduarte]

Use this information to test: https://github.com/marcelotduarte/cx_Freeze/tree/main/samples/service#run-the-sample

Run in a command prompt or powershell with admin privileges.

MainSoftware --install test
MainSoftware --uninstall test

I have an automated test, and tested it now, and it it ok.

I recommend using Python 3.10+ because previous versions have bugs with Python.

[dqcontrols]

Here is my execution on a shell with admin privileges:

C:\Users\sk\Desktop\testservice\build\agent>MainCode --install test
C:\Users\sk\anaconda3\envs\dq\Lib\zoneinfo_tzpath.py:44: InvalidTZPathWarning: Invalid paths specified in PYTHONTZPATH environment variable. Paths should be absolute but found the following relative paths:
.\share\zoneinfo
warnings.warn(
SelfService Directory provided: --install
Configuration file [conf] does not exist is provided directory. Ensure that the folder exists and contains devicePrefs.conf and appVersion.conf, and/or that the correct directory was provided.
Exiting application...
Service not installed. See log file for details.
C:\Users\sk\Desktop\testservice\build\agent>

The output of the log file is as:

[20452] 2024/06/07 14:14:41.300 starting logging at level ERROR
[20452] 2024/06/07 14:14:41.834 Python exception encountered:
[20452] 2024/06/07 14:14:41.834 Internal Message: initialization script didn't execute properly
[20452] 2024/06/07 14:14:41.834 Type => <class 'SystemExit'>
[20452] 2024/06/07 14:14:41.834 Value =>
[20452] 2024/06/07 14:14:41.835 Traceback (most recent call last):

[20452] 2024/06/07 14:14:41.835 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts_startup_.py", line 141, in run
module_init.run(name + "main")

[20452] 2024/06/07 14:14:41.836 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts\console.py", line 25, in run
exec(code, main_globals)

[20452] 2024/06/07 14:14:41.836 File "MainCode.py", line 503, in

[20452] 2024/06/07 14:14:41.836 SystemExit

[20452] 2024/06/07 14:14:41.837 ending logging

The persistant issue remains that the service cannot access the configuration files needed to start MainCode which exist in a file titled conf in the directory. I have attempted to amend the test line in shell as such:

MainCode C:\Users\sk\Desktop\testservice\build\agent --install test

Such that the configuration files exist in the directory C:\Users\sk\Desktop\testservice\build\agent\conf

[marcelotduarte]

Configuration file [conf] does not exist is provided directory. Ensure that the folder exists and contains devicePrefs.conf and appVersion.conf, and/or that the correct directory was provided.

I think this message says it all. Check that these files are being copied to the correct folder, and adjust your include_files accordingly.

[dqcontrols]

The files appear to be in the correct folder:

As you can see the conf folder exists in the build/agent directory, and all three required files are in the conf folder

My include_files are set up such that when the service/executables are created using cx_Freeze, the three required config files are sent to the conf folder under build/agent/conf (see above).

build_exe_options = {
"build_exe": "build/agent",
"packages": ["logging"], # Replace with your logging package name
"includes": ["ServiceHandler", "cx_Logging"],
"include_files": [("devicePrefs.conf", "conf/devicePrefs.conf"), ("appVersion.conf", "conf/appVersion.conf"), ("loggingPrefs.conf", "conf/loggingPrefs.conf")], # Add your .config file
}

This is a bizarre issue. Setting up MainCode.py as an executable using base=Console works as intended, with the config files being read. However, setting it up as a service prevents the config files from being read, despite the files being in the right place.

[marcelotduarte]

Sorry, I think I understand your problem. Look this:
#885 (comment)
Basically, the problem is with the program's home directory, which must be configured as in the link above.
In your case, changing the directory in initialize or run should solve it. Something like:
os.chdir(os.path.dirname(sys.executable))

Edit: This information is also relevant: https://cx-freeze.readthedocs.io/en/stable/faq.html#using-data-files

[dqcontrols]

I have attempted to resolve the issue using Issue #885 and did not have much success. See below for my updated ServiceHandler.py

from future import annotations
"""Implements a simple service using cx_Freeze.

See below for more information on what methods must be implemented and how they
are called.
"""
import threading
import os
import sys

class Handler:
# no parameters are permitted; all configuration should be placed in the
# configuration file and handled in the initialize() method
def init(self):
self.stopEvent = threading.Event()
self.stopRequestedEvent = threading.Event()

# called when the service is starting
def initialize(self, configFileName):
    self.directory = os.path.dirname( os.path.abspath( os.getcwd() + "\_file_"))

# called when the service is starting immediately after initialize()
# use this to perform the work of the service; don't forget to set or check
# for the stop event or the service GUI will not respond to requests to
# stop the service
def run(self):
    self.stopRequestedEvent.wait()
    self.stopEvent.set()

# called when the service is being stopped by the service manager GUI
def stop(self):
    self.stopRequestedEvent.set()
    self.stopEvent.wait()

Running MainCode --install test yields the following:

Directory provided: --install
Configuration file [conf] does not exist is provided directory. Ensure that the folder exists and contains devicePrefs.conf and appVersion.conf, and/or that the correct directory was provided.
Exiting application...
Service not installed. See log file for details.

And the service creation log file shows the following:
[15116] 2024/06/10 13:29:07.562 starting logging at level ERROR
[15116] 2024/06/10 13:29:12.673 Python exception encountered:
[15116] 2024/06/10 13:29:12.673 Internal Message: initialization script didn't execute properly
[15116] 2024/06/10 13:29:12.673 Type => <class 'KeyError'>
[15116] 2024/06/10 13:29:12.674 Value => 'NAME'
[15116] 2024/06/10 13:29:12.686 Traceback (most recent call last):

[15116] 2024/06/10 13:29:12.686 File "C:\Users\sk\anaconda3\envs\dqagent\Lib\site-packages\cx_Freeze\initscripts_startup_.py", line 141, in run
module_init.run(name + "main")

[15116] 2024/06/10 13:29:12.686 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts\console.py", line 25, in run
exec(code, main_globals)

[15116] 2024/06/10 13:29:12.686 File "MainCode.py", line 573, in

[15116] 2024/06/10 13:29:12.686 File "C:\Users\sk\anaconda3\envs\dq\Lib\configparser.py", line 979, in getitem
raise KeyError(key)

[15116] 2024/06/10 13:29:12.686 KeyError: 'NAME'

Line 573 in my script is the following line of code:

log_filename = (logging_config['NAME']['agent_logfile']+".log")

Where the log_filename variable is being created using the filename provided in the config file.

The issue remains that the configuration files cannot be accessed and read by the service (the script uses the configparser library to read the config files), despite the following line of code to ServiceHandler.py in the initialize function():

self.directory = os.path.dirname( os.path.abspath( os.getcwd() + "_file_"))

I also attempted to create the service using the following shell command, and start the service manually from Services, with the same result:
sc create MainCode start=delayed-auto binPath= "C:\Users\sk\Desktop\testservice\agent\build\agent\MainCode.exe C:\Users\sk\Desktop\testservice\agent\build\agent"

Is there another way for me to ensure that the service knows/can access the configuration files? The service .exe is in the path C:\Users\sk\Desktop\testservice\agent\build\agent\MainCode.exe and the conf folder holding the configuration files are in C:\Users\sk\Desktop\testservice\agent\build\agent\conf

[marcelotduarte]

self.directory = os.path.dirname( os.path.abspath( os.getcwd() + "_file_"))

Does not work, check the value of getcwd...

Basically, the problem is with the program's home directory, which must be configured as in the link above. In your case, changing the directory in initialize or run should solve it. Something like: os.chdir(os.path.dirname(sys.executable))

Edit: This information is also relevant: https://cx-freeze.readthedocs.io/en/stable/faq.html#using-data-files

Following my advice, you should have tried:
self.directory = os.path.dirname(sys.executable)

[dqcontrols]

I tried that approach, which didn't work on my end at which point I tried: self.directory = os.path.dirname( os.path.abspath( os.getcwd() + "file"))

Here are the results with: self.directory = os.path.dirname(sys.executable)

Here is my ServiceHandler.py
from future import annotations
"""Implements a simple service using cx_Freeze.

See below for more information on what methods must be implemented and how they
are called.
"""
import threading
import os
import sys

class Handler:
# no parameters are permitted; all configuration should be placed in the
# configuration file and handled in the initialize() method
def init(self):
self.stopEvent = threading.Event()
self.stopRequestedEvent = threading.Event()

# called when the service is starting
def initialize(self, configFileName):
    self.directory = os.path.dirname(sys.executable)

# called when the service is starting immediately after initialize()
# use this to perform the work of the service; don't forget to set or check
# for the stop event or the service GUI will not respond to requests to
# stop the service
def run(self):
    self.stopRequestedEvent.wait()
    self.stopEvent.set()

# called when the service is being stopped by the service manager GUI
def stop(self):
    self.stopRequestedEvent.set()
    self.stopEvent.wait()

After creating the .exe attempting to run MainCode --install test yields the following console result:
Directory provided: --install
Configuration file [conf] does not exist is provided directory. Ensure that the folder exists and contains devicePrefs.conf and appVersion.conf, and/or that the correct directory was provided.
Exiting application...
Service not installed. See log file for details.

And the logfile reads the following, indicating that the configuration files could not be found:
[24436] 2024/06/10 14:42:54.702 starting logging at level ERROR
[24436] 2024/06/10 14:42:55.249 Python exception encountered:
[24436] 2024/06/10 14:42:55.249 Internal Message: initialization script didn't execute properly
[24436] 2024/06/10 14:42:55.249 Type => <class 'SystemExit'>
[24436] 2024/06/10 14:42:55.249 Value =>
[24436] 2024/06/10 14:42:55.250 Traceback (most recent call last):

[24436] 2024/06/10 14:42:55.250 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts_startup_.py", line 141, in run
module_init.run(name + "main")

[24436] 2024/06/10 14:42:55.253 File "C:\Users\sk\anaconda3\envs\dq\Lib\site-packages\cx_Freeze\initscripts\console.py", line 25, in run
exec(code, main_globals)

[24436] 2024/06/10 14:42:55.253 File "MainCode.py", line 507, in

[24436] 2024/06/10 14:42:55.253 SystemExit

[24436] 2024/06/10 14:42:55.254 ending logging

Attempting to run the service with another shell command shown before (sc create MainCode start=delayed-auto binPath= "C:\Users\sk\Desktop\testservice\agent\build\agent\MainCode.exe C:\Users\sk\Desktop\testservice\agent\build\agent" then manually starting the service in Services) yields the same result.

[marcelotduarte]

I'll try to explain to you what I think it is another way. When you start a common program, the current folder (getcwd) is the starting directory, but in "service" the current directory is not the current folder or the installation folder. It's probably C:\windows\system32.
Therefore, I recommended changing the directory (chdir) because if you have the program searching for a file in a relative way, you won't find it. The self.directory is only for when you are going to use it in 'run'.

So, please try it:

In your case, changing the directory in initialize or run should solve it. Something like:
os.chdir(os.path.dirname(sys.executable))