Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: LVM2 module uevent processing optimization #814

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from

Commits on Oct 21, 2020

  1. tests: Add 'loadtest' test tag

    Supposed to be used for heavy load environment preparation or
    for running load tests in general. Should not be included in
    regular test runs.
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    4ce5a10 View commit details
    Browse the repository at this point in the history
  2. test: Add LVM snapshots load test

    This creates 1000 LVM snapshots over a persistent image file
    attached as loop device. Splitting the test case into "setup"
    and "teardown" parts allows debugging on such heavy environment.
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    1910d5c View commit details
    Browse the repository at this point in the history
  3. udiskslinuxprovider: Add flag for a secondary coldplug

    Just like a primary coldplug that is run upon daemon startup this secondary
    coldplug indicates modules a secondary initialization over existing drive
    and block objects. Upon activating a module it's desirable to attach extra
    module interfaces over existing objects, perform additional probing and
    expose all available information in a synchronous fashion so that upon
    retuning from the org.freedesktop.UDisks2.Manager.EnableModule() method call
    all exported objects and their properties are up-to-date.
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    677f82d View commit details
    Browse the repository at this point in the history
  4. lvm2: Make use of the secondary coldplug flag

    It is possible to distinguish a coldplug phase now so remove
    the dirty workaround. This will change timing of object properties
    validity and availability of additional objects at the expense
    of longer coldplug phase.
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    f53a6cf View commit details
    Browse the repository at this point in the history
  5. udiskslinuxdevice: Save timestamp of a uevent for the device

    As long as GUdevDevice doesn't carry timestamp of the reported uevent,
    let's use system monotonic time and bind a timestamp to it as soon as possible.
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    e983562 View commit details
    Browse the repository at this point in the history
  6. udiskslinuxprovider: Store timestamp of a last uevent received

    Betting on natural atomicity of the 64-bit value, may need explicit
    measures to be added around (TODO).
    tbzatek committed Oct 21, 2020
    Configuration menu
    Copy the full SHA
    f124da4 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2020

  1. lvm2: Run update sync on coldplugging

    Updates should be done sync while coldplugging and async updates should
    be rate limited. This is done by comparing timestamps of (filtered) uevents,
    keeping the update job limited to a single instance, optionally
    queueing another update once the previous one finished.
    
    We rely on uevents being generated for any change to lvm2 block devices,
    otherwise this wouldn't work.
    tbzatek committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    ddf857a View commit details
    Browse the repository at this point in the history
  2. lvm2: Implement 'scalable mode' for large number of LVs in the system

    For consistency reasons we would like to run updates sync, however
    with large number of LVs in the system this can take considerable
    amount of time, blocking the serialized daemon uevent processing.
    
    As a workaround this change introduces an arbitrary threshold
    that will switch updates back to async as was historically observed.
    tbzatek committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    94a3c25 View commit details
    Browse the repository at this point in the history
  3. tests: Remove lingering lvm2 VG entries

    There's a known bug somewhere between lvm2 and udev, keeping
    /dev entries around while the block devices are actually gone.
    This causes problems on subsequent test runs while it seems
    nothing else breaks down.
    tbzatek committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    2238da5 View commit details
    Browse the repository at this point in the history
  4. udiskslinuxprovider: Reorder uevent processing for modules

    This is a major change and will need to be followed up to make
    things right. This essientially reorders the moment the uevent
    is propagated to modules and may affect availability of information
    on exposed D-Bus objects.
    
    This is all about dependencies, i.e. in case a module gathers
    information from existing block objects, it needs to know whether
    that block object has already finished processing the particular
    uevent or not. This commit defines that order and it appears to be
    sensible to guarantee the "core" daemon objects and its interfaces
    are updated by the time udisks_module_new_object() is called. That
    happens as a last step of uevent processing chain which includes
    block object specific module callbacks for exposing extra D-Bus
    interfaces (run prior to udisks_module_new_object()).
    
    However it may not suit all cases and would be perhaps wise to provide
    pre- and post- udisks_module_new_object() callbacks for finer
    granularity and precise definition of the moment of uevent processing.
    
    For the moment let's run udisks_module_new_object() as the last
    in the chain and make modules aware of that.
    tbzatek committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    99bd6dc View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2020

  1. lvm2: Use udisks_daemon_util_trigger_uevent()

    This might trigger some critical warnings from the daemon as
    the VG/LV might not exist. Mostly harmless as until now
    the return code of open() wasn't actually checked.
    
    Subject to further fixes in the future.
    tbzatek committed Nov 13, 2020
    Configuration menu
    Copy the full SHA
    4efd894 View commit details
    Browse the repository at this point in the history