[GRPC_CLIENT] LoadModel with serialized model raise an error: "Poll failed for model directory" on native windows triton.

[FilipLeja]

Hi,

I prepared very simple onnx model and tried to load model in this way:

    std::vector<char> model;
    std::ifstream in("simple_model.onnx", std::ios::in | std::ios::binary);

    in.seekg(0, std::ios::end);
    model = std::vector<char>(in.tellg());
    in.seekg(0, std::ios::beg);
    in.read(model.data(), model.size());
    in.close();

    auto modelConfig = prepareModelConfig("");

    auto err = tritonClient->loadModel("debug",
              triton::client::Headers(),
              modelConfig,
              {{"file:1/model.onnx", model}});

simple_model.zip

and I am getting following error:

C:\tmp\tritonbuild\tritonserver\build\_deps\repo-core-src\src\model_repository_manager\model_repository_manager.cc:1460] "Poll failed for model directory 'debug': failed to open binary file for write C:\\Users\\FilipLeja\\AppData\\Local\\Temp\\fol8BC4.tmp\\1\\model.onnx: No such file or directory"

Above code works perfectly on other platforms like docker, wsl and native linux. Nevertheless we have to use triton on windows natively.

Do you know what can cause above error?

Additional Info about triton:

+----------------------------------+----------------------------------------------------------------------------------+
| Option                           | Value                                                                            |
+----------------------------------+----------------------------------------------------------------------------------+
| server_id                        | triton                                                                           |
| server_version                   | 2.50.0                                                                           |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) sch |
|                                  | edule_policy model_configuration system_shared_memory cuda_shared_memory binary_ |
|                                  | tensor_data parameters statistics trace logging                                  |
| model_repository_path[0]         | C:\Users\FilipLeja\AppData\Local\model_repository                |
| model_control_mode               | MODE_EXPLICIT                                                                    |
| startup_models_0                 | *                                                                                |
| strict_model_config              | 0                                                                                |
| model_config_name                |                                                                                  |
| rate_limit                       | OFF                                                                              |
| pinned_memory_pool_byte_size     | 268435456                                                                        |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                         |
| min_supported_compute_capability | 6.0                                                                              |
| strict_readiness                 | 1                                                                                |
| exit_timeout                     | 30                                                                               |
| cache_enabled                    | 0                                                                                |
+----------------------------------+----------------------------------------------------------------------------------+

I1106 14:22:27.501000 12884 C:\workspace\src\grpc\grpc_server.cc:2370]
+----------------------------------------------+---------+
| GRPC KeepAlive Option                        | Value   |
+----------------------------------------------+---------+
| keepalive_time_ms                            | 7200000 |
| keepalive_timeout_ms                         | 20000   |
| keepalive_permit_without_calls               | 0       |
| http2_max_pings_without_data                 | 2       |
| http2_min_recv_ping_interval_without_data_ms | 300000  |
| http2_max_ping_strikes                       | 2       |
+----------------------------------------------+---------+

Triton run command:
.\bin\tritonserver --model-repository="C:\Users\FilipLeja\AppData\Local\model_repository" --backend-directory="C:\Repos\Utils\tritonserver2.50.0-win\tritonserver2.50.0-win\backends" --log-verbose=3 --model-load-thread-count 1 --model-control-mode=explicit --load-model=*

Thanks in advance!

P.S.
I've created similar topic on triton-inference-server/client, but didn't know where is a valid place for this one .