Exception / Segfault while using turbodbc

[zacqed]

Environment details:

Docker
Turbodbc==4.5.2
Python ==3.10.6
MSSQL 17
Pyarrow = 8.0.0
Flask application running with gunicorn
gcc (Debian 10.2.1-6) 10.2.1 20210110

Turbodbc installation

WORKDIR /root/.local/lib/python3.10/site-packages/pyarrow
RUN ln -s libarrow.so.800 libarrow.so
RUN ln -s libarrow_python.so.800 libarrow_python.so
RUN --mount=type=cache,target=/root/.cache/pip CFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0" pip install --user --no-cache-dir turbodbc==4.5.3

Turbodbc configuration

   connection = connect(
        driver="ODBC Driver 17 for SQL Server",
        server=server + "," + port,
        database=database,
        uid=username,
        pwd=password,
        turbodbc_options=make_options(prefer_unicode=True, use_async_io=True, autocommit=True),
    )
   cursor = connection.cursor()
    table = cursor.execute(sql_query).fetchallarrow().to_pandas(use_threads=True)
    cursor.close()

On certain occasions gunicorn exits on sql query and a core is generated. I am unable to figure out what is leading to that. The core provides following details:

bt

#0  0x00007fd88d19c8ca in ?? () from /opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.10.so.1.1
#1  0x00007fd8a1af4609 in SQLBindCol (statement_handle=0x7fd86c0bac10, column_number=<optimized out>, target_type=<optimized out>, target_value=0x7fd86c0c3c50, buffer_length=130,
    strlen_or_ind=0x7fd86cac3c00) at SQLBindCol.c:327
#2  0x00007fd88e12c231 in cpp_odbc::level2::level1_connector::do_bind_column (this=0x7fd86c068310, handle=..., column_id=<optimized out>, column_type=<optimized out>, column_buffer=...)
    at src/cpp_odbc/level2/level1_connector.cpp:233
#3  0x00007fd88e1371e4 in turbodbc::result_sets::bound_result_set::rebind (this=0x7fd86c0c2218) at src/turbodbc/result_sets/bound_result_set.cpp:62
#4  0x00007fd88e138244 in turbodbc::result_sets::(anonymous namespace)::reader_thread (read_requests=..., read_responses=..., batches=...)
    at src/turbodbc/result_sets/double_buffered_result_set.cpp:28
#5  0x00007fd8a19eded0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fd8a3ef1ea7 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#7  0x00007fd8a4008def in clone () from /lib/x86_64-linux-gnu/libc.so.6

bt full

#0  0x00007fd88d19c8ca in ?? () from /opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.10.so.1.1
No symbol table info available.
#1  0x00007fd8a1af4609 in SQLBindCol (statement_handle=0x7fd86c0bac10, column_number=<optimized out>, target_type=<optimized out>, target_value=0x7fd86c0c3c50, buffer_length=130,
    strlen_or_ind=0x7fd86cac3c00) at SQLBindCol.c:327
        statement = 0x7fd86c0bac10
        ret = <optimized out>
#2  0x00007fd88e12c231 in cpp_odbc::level2::level1_connector::do_bind_column (this=0x7fd86c068310, handle=..., column_id=<optimized out>, column_type=<optimized out>, column_buffer=...)
    at src/cpp_odbc/level2/level1_connector.cpp:233
        return_code = <optimized out>
#3  0x00007fd88e1371e4 in turbodbc::result_sets::bound_result_set::rebind (this=0x7fd86c0c2218) at src/turbodbc/result_sets/bound_result_set.cpp:62
        column = @0x7fd86c0073e0: {statement_ = @0x7fd86c033100, one_based_index_ = 1, description_ = std::unique_ptr<const turbodbc::description> = {get() = 0x7fd86c022b90}, buffer_ = {
            element_size_ = 130, data_ = std::vector of length 10485670, capacity 10485670 = {84 'T', 0 '\000', 70 'F', 0 '\000', 83 'S', 0 '\000', 73 'I', 0 '\000', 78 'N', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
              0 '\000', 0 '\000', 0 '\000'...}, indicators_ = std::vector of length 80659, capacity 80659 = {10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0...}}}
        __for_range = std::vector of length 1, capacity 1 = {{statement_ = @0x7fd86c033100, one_based_index_ = 1, description_ = std::unique_ptr<const turbodbc::description> = {
              get() = 0x7fd86c022b90}, buffer_ = {element_size_ = 130, data_ = std::vector of length 10485670, capacity 10485670 = {84 'T', 0 '\000', 70 'F', 0 '\000', 83 'S', 0 '\000',
                73 'I', 0 '\000', 78 'N', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000',
                0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000', 0 '\000'...}, indicators_ = std::vector of length 80659, capacity 80659 = {10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0...}}}}
Python Exception <class 'gdb.error'> value has been optimized out:
        __for_begin =
Python Exception <class 'gdb.error'> value has been optimized out:
        __for_end =
#4  0x00007fd88e138244 in turbodbc::result_sets::(anonymous namespace)::reader_thread (read_requests=..., read_responses=..., batches=...)
--Type <RET> for more, q to quit, c to continue without paging--
    at src/turbodbc/result_sets/double_buffered_result_set.cpp:28
        promise = {_M_future = std::shared_ptr<std::__future_base::_State_baseV2> (use count 1610614992, weak count 32727) = {get() = 0x7fd860000c80},
          _M_storage = std::unique_ptr<std::__future_base::_Result<unsigned long>> = {get() = 0x0}}
        batch_id = <optimized out>
#5  0x00007fd8a19eded0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#6  0x00007fd8a3ef1ea7 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#7  0x00007fd8a4008def in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.

[alippai]

Turbodbc 4.5.2 is not compatible with pyarrow 8.0. Upgrading to the latest (4.5.5) can help

[zacqed]

src/turbodbc/result_sets/double_buffered_result_set.cpp:28

Still leads to the same segfault:

Program terminated with signal SIGSEGV, Segmentation fault.

#0  0x00007fc92c77e8ca in ?? () from /opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.10.so.1.1
[Current thread is 1 (Thread 0x7fc903fff700 (LWP 286))]
(gdb) bt
#0  0x00007fc92c77e8ca in ?? () from /opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.10.so.1.1
#1  0x00007fc9417fe609 in SQLBindCol (statement_handle=0x7fc9080422c0, column_number=<optimized out>, target_type=<optimized out>, target_value=0x7fc90817fb50, buffer_length=130,
    strlen_or_ind=0x7fc908b7fb00) at SQLBindCol.c:327
#2  0x00007fc92d80e231 in cpp_odbc::level2::level1_connector::do_bind_column (this=0x7fc908057390, handle=..., column_id=<optimized out>, column_type=<optimized out>, column_buffer=...)
    at src/cpp_odbc/level2/level1_connector.cpp:233
#3  0x00007fc92d8191e4 in turbodbc::result_sets::bound_result_set::rebind (this=0x7fc9080afc88) at src/turbodbc/result_sets/bound_result_set.cpp:62
#4  0x00007fc92d81a244 in turbodbc::result_sets::(anonymous namespace)::reader_thread (read_requests=..., read_responses=..., batches=...)
    at src/turbodbc/result_sets/double_buffered_result_set.cpp:28
#5  0x00007fc9416f7ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fc943bfbea7 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#7  0x00007fc943d12def in clone () from /lib/x86_64-linux-gnu/libc.so.6