RavenDB-27141 Sync the Python test driver with the C# test driver

[poissoncorp]

Syncs the Python test driver with src/Raven.TestDriver, from a line-by-line audit of both drivers. Version goes to 7.2.5.post3.

python -m unittest discover -s tests

67 tests, nothing skipped. CI is green on all four jobs: embedded on ubuntu, embedded on windows, attach against Docker with no .NET installed, and the self-contained download.

---

New features

Embedded test servers run in memory

Every get_document_store() call creates a database and deletes it when the store closes. That cycle now happens in memory, because the driver appends --RunInMemory=true to the server it boots, which is what the .NET driver has always done. Only the server log is written, into a scratch directory removed when the test process exits.

The second half matters as much: a data directory left at the ravendb-embedded default used to point inside the installed package, so a test run wrote into site-packages/ravendb_embedded/RavenDB. The driver now redirects that to a temporary directory, and logs follow it.

The tradeoff is real, so there are two ways out. Either say so on the options object:

from ravendb_test_driver import RavenTestDriver, TestServerOptions

options = TestServerOptions()
options.run_in_memory = False
options.data_directory = "/path/you/choose"     # left alone once you set it
RavenTestDriver.configure_server(options)

or put the argument on the command line yourself, which the driver never overrides:

options = TestServerOptions()
options.command_line_args.append("--RunInMemory=false")
RavenTestDriver.configure_server(options)

Secured embedded servers actually work

Point the driver at a server certificate and the client certificate your tests authenticate with. The driver takes the client material the embedded server was started with and puts it on every store it hands out, so sessions just work.

from ravendb_test_driver import RavenTestDriver, TestServerOptions

options = TestServerOptions()
options.secured("server.pfx", "client.pem", ca_certificate_path="ca.crt")
RavenTestDriver.configure_server(options)

with RavenTestDriver() as driver:
    with driver.get_document_store() as store:
        assert store.urls[0].startswith("https://")
        assert store.certificate_pem_path == "client.pem"   # the driver put it there

        with store.open_session() as session:
            session.store({"name": "John"}, "people/1")
            session.save_changes()

Before this, the driver built its stores without any certificate, so the first request against a secured embedded server failed. A secured server configured with no client certificate is now rejected before the server starts, instead of failing later on the first request.

Runnable end to end in the new labs/05_secured_embedded.py, which generates its own certificates.

pre_configure_database: change a database before it is created

setup_database has always been the place to seed documents and create indexes, but it runs on a database that already exists. pre_configure_database runs one step earlier, on the DatabaseRecord itself, before CreateDatabaseOperation goes out. That is where database settings, revisions, expiration, encryption and topology live. It is a straight port of PreConfigureDatabase from the .NET driver, which the Python and JVM drivers never had.

from ravendb import GetDatabaseRecordOperation

from ravendb_test_driver import RavenTestDriver


class PeopleTestDriver(RavenTestDriver):
    def pre_configure_database(self, database_record):
        database_record.settings["Indexing.MapTimeoutInSec"] = "30"


with PeopleTestDriver() as driver:
    with driver.get_document_store() as store:
        record = store.maintenance.server.send(GetDatabaseRecordOperation(store.database))
        assert record.settings["Indexing.MapTimeoutInSec"] == "30"

The three override points now run in this order, and each gets a different thing:

class MyDriver(RavenTestDriver):
    def pre_configure_database(self, database_record):
        ...   # 1. the DatabaseRecord, before the database exists

    def pre_initialize(self, document_store):
        ...   # 2. the DocumentStore, before initialize() runs on it

    def setup_database(self, document_store):
        ...   # 3. the initialized store: seed documents, create indexes

TestServerOptions: one place for test-server defaults

configure_server used to take a raw ravendb_embedded.ServerOptions, which is the same type you would use to run a production embedded server. TestServerOptions is a subclass of it that names the intent and carries the knobs that only make sense for a test server, starting with run_in_memory.

from ravendb_test_driver import RavenTestDriver, TestServerOptions

options = TestServerOptions()
options.with_auto_downloaded_server()    # everything ServerOptions can do still works
options.run_in_memory = False            # plus what only a test server needs
RavenTestDriver.configure_server(options)

configure_server still accepts a plain ServerOptions, and the driver applies the same test defaults either way, so no existing call site has to change:

from ravendb_embedded import ServerOptions

RavenTestDriver.configure_server(ServerOptions())   # still fine, still gets the test defaults

Database names that say which test they came from

The .NET driver names each database after the calling test method. Python can now do the same. Off by default, because it changes every generated name.

from unittest import TestCase

from ravendb_test_driver import RavenTestDriver


class PeopleTestDriver(RavenTestDriver):
    use_caller_name_for_database = True


class TestPeople(TestCase):
    def test_stores_a_person(self):
        with PeopleTestDriver() as driver:
            with driver.get_document_store() as store:
                assert store.database.startswith("test_stores_a_person_")   # was "test_3"

Names are sanitized to [A-Za-z0-9_.-], and a caller with no usable name (a lambda, module level) falls back to test.

Per-process unique database names

The counter alone restarts in every process, so two runners against one shared server hand out the same names and delete each other's databases. Setting this adds the process id.

RAVENDB_TEST_UNIQUE_DB_NAMES=1 pytest -n auto

# test_stores_a_person_31241_7   instead of   test_stores_a_person_7

Off by default, because it changes every generated name.

stop_test_server()

The test server is shared by the whole process and nothing used to close it, so its shutdown cost landed at interpreter exit, after your runner had already printed its summary. Now you can put it where it belongs. It is idempotent, and the server starts again on the next get_document_store().

# conftest.py
from ravendb_test_driver import RavenTestDriver


def pytest_sessionfinish(session, exitstatus):
    RavenTestDriver.stop_test_server()

---

Breaking changes

Explicit configuration now beats RAVENDB_TEST_SERVER_URL

The driver picks its server from three places: configure_external_server(...), configure_server(...), and the RAVENDB_TEST_SERVER_URL environment variable. The environment used to win over everything, silently. That is dangerous, because the driver creates and hard-deletes databases on whichever server it ends up using, so a suite pinned to a local embedded server could be redirected onto a shared one and delete databases there.

Now the code wins and the driver says what it ignored.

RavenTestDriver.configure_server(TestServerOptions())
# with RAVENDB_TEST_SERVER_URL also set, you now get:
#   UserWarning: Ignoring RAVENDB_TEST_SERVER_URL='http://shared:8080' because
#   configure_server() was called explicitly. [...]

If you relied on the environment overriding your code, drop the configure_server() call and let the environment pick the server:

# before: configure_server(...) in the test file, RAVENDB_TEST_SERVER_URL in CI
# after:  pick one. For a CI-chosen server, call nothing and set only the variable.

Test data no longer lands on disk

This is the other side of the in-memory feature above. A suite that seeded a large fixture and let it spill to disk now holds it in RAM, the configured data directory stays empty, and there are no files to inspect after a failing run.

options = TestServerOptions()
options.run_in_memory = False        # the escape hatch, if any of that mattered
RavenTestDriver.configure_server(options)

---

Fixes

Closing a driver that still held an open store crashed

close() iterated _document_stores while each store's after-close callback popped itself out of that same dict. Python raises on that, and because the exception came from the for statement rather than from inside the try, it escaped uncaught: the driver was left half-closed, remaining stores were never closed and on_driver_closed never fired.

driver = RavenTestDriver()
driver.get_document_store()      # not closed by hand
driver.close()
# RuntimeError: dictionary changed size during iteration

The loop now iterates a snapshot, which is what the .NET driver does with _documentStores.Keys.ToList(). This is the contract the README always advertised: whatever you forget to close, the driver closes and its database is deleted.

wait_for_indexing returned early during a side-by-side index swap

When you change the definition of an existing index, RavenDB does not replace it in place. It builds a second index next to it, named ReplacementOf/YourIndex, and swaps it in once that one has caught up. Python excluded those replacement indexes from the staleness check, which is the exact inverse of the .NET driver: the wait returned while the swap was still pending, so the next query read the pre-swap index and the test passed on stale results.

store.execute_index(People_ByName())     # deployed once
# ... later in the same test, with a changed definition:
store.execute_index(People_ByName())     # server starts ReplacementOf/People/ByName

driver.wait_for_indexing(store)          # before: returned immediately, old index still serving
                                         # after:  blocks until the swap completes

A suite that never redeploys an index definition sees no difference at all, because no replacement index ever exists. One that does gets correct results instead of stale ones. The one new failure mode: if a swap never completes, because the replacement errored out or one was left behind, this now raises TimeoutException where it used to return. Give it more room if your swap is genuinely slow:

driver.wait_for_indexing(store, timeout=timedelta(minutes=2))

Two live drivers asked for the same database name

self._INDEX += 1 (now _DATABASE_COUNTER) read a class attribute and writes an instance one, so every driver instance restarted numbering at 1 and two live drivers both asked for test_1. The second create was rejected, and closing the first store deleted the database the other one was still using.

first = RavenTestDriver()
second = RavenTestDriver()

first.get_document_store().database     # before: test_1     after: test_1
second.get_document_store().database    # before: test_1     after: test_2

Generated names therefore shift: a suite that used to see test_1 three times now sees test_1, test_2, test_3. They were never a documented contract, so read store.database if you pinned one. RavenTestDriver._DATABASE_COUNTER = 0 still works if you reset it between suites.

Teardown failures were flattened into a string

Closing a driver used to flatten every teardown failure into one string, losing the types and the tracebacks. It now raises a typed error that keeps the originals.

try:
    driver.close()
except DriverCloseError as e:
    for original in e.exceptions:        # the real exceptions, not a joined string
        print(type(original).__name__, original)

DriverCloseError subclasses RuntimeError, so this keeps working unchanged:

except RuntimeError:
    ...

If you matched on the message text, that text changed: it is now counted and type-prefixed.

Debug/Done was loaded instead of checked, and never deleted

wait_for_user_to_continue_the_test loaded the whole marker document into the session and left it in the database, so a second wait on the same store returned immediately.

# before: session.load("Debug/Done", dict)      tracks a document, leaves it behind
# after:
if session.advanced.exists("Debug/Done"):
    session.delete("Debug/Done")
    session.save_changes()
    break

wait_for_user_to_continue_the_test could hang a CI job forever

It was an unconditional while True with no timeout, so a call left in committed code blocked until the CI job's own wall-clock limit. It is now bounded, and there is a kill switch.

driver.wait_for_user_to_continue_the_test(store)                      # 5 minutes, then TimeoutException
driver.wait_for_user_to_continue_the_test(store, timeout=None)        # wait forever, on purpose

RAVENDB_TEST_WAIT_FOR_USER=0 python -m unittest discover -s tests     # skip it entirely

A wait is also unbounded automatically while a debugger is attached, which is how the .NET driver behaves.

An explicit zero timeout was ignored

timeout or timedelta(seconds=60) and if options.wait_for_indexing_timeout: both treat timedelta(0) as "not set", so asking for no wait at all gave you either a full minute or no check.

driver.get_document_store(GetDocumentStoreOptions.with_timeout(timedelta(0)))
# before: the indexing wait was skipped entirely
# after:  the wait runs with a zero timeout, exactly as asked

Temporary files were left behind

The generated empty settings file was never removed, and the cleanup loop read shutil.rmtree's None return as a failure flag, so a successful delete still cost a retry pass.

# before: if not shutil.rmtree(dir_, ignore_errors=True):   # rmtree always returns None
# after:
shutil.rmtree(directory, ignore_errors=True)
if os.path.exists(directory):
    ...   # the only real signal

Proven by a test that runs a full driver session in a child interpreter and asserts no temp directory or settings file survives its exit.

A no-leader failure during teardown failed the test

Deleting a test database goes through the cluster, which can be between leaders. The .NET driver swallows NoLeaderException there; Python did not, so a leadership hiccup failed an otherwise green test.

except (DatabaseDoesNotExistException, NoLoaderException):
    pass
except RavenException as e:
    # The client maps the server's NoLeaderException under a misspelled key, so a real
    # no-leader failure currently arrives untyped.
    if "NoLeaderException" not in str(e):
        raise

Error messages, cause chains and a silent browser failure

The configuration error named the Java driver's methods, wrapped exceptions dropped __cause__, and open_browser raised an empty RuntimeError while ignoring webbrowser.open returning False on a headless machine.

# before: "Please call 'configureServer' method before any 'getDocumentStore' is called."
# after:  "Call 'configure_server' before any 'get_document_store'."

# before: raise RuntimeError()
# after:  raise RuntimeError(f"Failed to open a browser at {url}") from e

Option preparation mutated the caller's list

run_server inserted arguments straight into options.command_line_args, the list you passed in. Harmless while there was one writer, not harmless once in-memory injection added a second. The driver now builds its own copy.

---

Deprecations

run_server, default_server_options and cleanup_temp_dirs are now private

All three are absent from the .NET driver and private in the JVM one. They were public here by accident.

RavenTestDriver.run_server()             # still works for one release, warns
RavenTestDriver._run_server()            # the new name

DeprecationWarning: RavenTestDriver.run_server() is internal and will be removed in a
future release; use _run_server() if you really need it.

Nothing breaks in this release: the old names still resolve and only warn. They go away in a later one.

---

Not in this PR

Dump-based seeding (DatabaseDumpFilePath / DatabaseDumpFileStream) is the one .NET feature still missing. It is blocked in the client, not here: ravendb/documents/store/definition.py still has a bare # todo: database smuggler, so there is no import to call. Tracked separately, to be done in the client first.

Verification

• 67 tests, nothing skipped. tests/test_end_to_end.py boots a real server; the rest are hermetic.
• In-memory is proven, not assumed: after writing 100 documents, the server's data directory holds only Logs/server.log and zero .voron files.
• Secured embedded is proven end to end: a test generates certificates, boots a secured server through configure_server, then writes and reads through the driver's store.
• Database deletion is asserted against GetDatabaseNamesOperation, both when a store closes and when the driver closes stores left open.
• Cleanup is asserted from outside: a child interpreter runs a full session and leaves no temp directory or settings file.
• The attach path no longer skips locally. With no RAVENDB_TEST_SERVER_URL it boots a second embedded server and attaches to that; CI still runs it against Docker with no .NET installed.
• black --check clean, labs 01 to 05 pass.