Fix key performance issues in 6.17 with byte range locks - #202
Open
hbirth wants to merge 17 commits into
Open
Conversation
hbirth
force-pushed
the
redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
branch
from
August 25, 2026 11:27
588473f to
7ffa485
Compare
|
What does this patch do? Do I still need to rebase my PR 201 by merge the range lock interval tree with DLM cache interval? |
hbirth
force-pushed
the
redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
branch
6 times, most recently
from
August 26, 2026 09:52
839c405 to
9fb952d
Compare
Collaborator
Author
|
This is doing pretty well in xfstests |
fuse_dlm_buffered_write() sends the unaligned edges of a write to the server itself so no partly written block is dirtied, and cuts at PAGE_SIZE. Nothing checked that a block is a page: fc->blkbits keeps whatever blksize= asked for, and attr->blksize sets inode->i_blkbits per inode. iomap then goes back for the remainder of an edge block and dirties what the write already sent. Refuse the connection at FUSE_INIT when a writeback cache is negotiated on a block that is not a page, and pin inode->i_blkbits afterwards. st_blksize still reports what the server named. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
Readahead fills the page cache past the range fuse_cache_read_iter() locked, so those folios get no revoke when a remote node writes them. Request a read grant over the whole window in fuse_readahead() before any folio is consumed, and skip the window when the request fails. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
hbirth
force-pushed
the
redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
branch
3 times, most recently
from
August 31, 2026 10:14
5b89c6b to
3ee09db
Compare
The record was an interval tree of ranges carrying a lock mode and a revoke generation, walked under one lock per inode. Splitting and merging ranges on every grant made it the contended structure on a file several threads write. Record coverage as two bitmaps per aligned file region instead, a bit per page for granted and for granted-for-write. A region is a fixed span, so recording a grant neither allocates nor rearranges anything, and the regions live in an xarray, so threads writing far apart never touch the same one. A grant still on the wire stays out of the bitmaps and waits on a pending list where a revoke marks it, which keeps a reply that crosses a revoke from recording coverage the server took back. Adapt the paths to it: reads and readahead take a read grant, a cached write takes the write grant for the range it dirties and sends its unaligned edges straight through, writeback confirms the grant for every run it sends, and a NOTIFY invalidate drops the record for the range it revokes before it drops the page cache. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_writeback_write_iter() published its AOP_TRUNCATED_PAGE retry state in a connection-wide xarray keyed by task pointer, and erased it again, once per buffered write. A task pointer is a high index, so every write built and tore down the whole node chain down to it under one lock shared by every writer of the mount, and its GFP_KERNEL allocation ran with i_rwsem held. iomap already carries a private pointer through to fuse_iomap_read_folio_range(), where fuse put the struct file. Put the file and the flag in one struct on the stack and pass that instead. The store could fail and take the write down with it, and the load could come back empty and drop a needed retry. Neither is reachable now. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A server answers a contended range with -EDEADLK, which fuse_do_readfolio() turns into AOP_TRUNCATED_PAGE so the read is retried. __fuse_get_dlm_lock() returned it instead, and its writeback caller has no way to hold on to a folio it reports an error for: iomap takes the dirty flag off before offering the folio and clears its dirty ranges whatever the callback returns, so the bytes are dropped without ever reaching the server. A NOTIFY invalidate then drops the now clean folio and the next read gets the pre-write content. Ask again on -EDEADLK and -EAGAIN, on the retry budget the killed-in- flight grant already uses. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
iomap clears the dirty flag before it offers a folio to ->writeback_range and clears its dirty ranges whatever the callback returns, and nothing puts either back. A run that reports an error has therefore dropped its bytes: the folio stays in the page cache, clean and uptodate, holding data the server never received, until an invalidate drops it and the next read returns what the server has. The comment claimed the opposite. Redirty the folio instead, for the grant and the allocation, while there is still a connection to send it on. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
The DLM buffered write holds i_rwsem shared and claims its i_size extension before it dirties anything. fuse_attr_cache_mask() decides whether the server's size wins from an i_size it reads before that claim, and sleeps in the lock tree query before fuse_change_attributes_i() applies the answer under fi->lock. A GETATTR that left while i_size still matched the server's is therefore applied over every claim made since, and i_size drops by exactly the writes in flight. truncate_pagecache() then zeroes the tail of the page holding the new size and drops what is above, which writeback sends as zeros. Move attr_version on the claim so those replies are dropped, the same reason fuse_write_update_attr() moves it, and count the writers whose claim is not yet dirty for a reply that leaves after one. FUSE_I_SIZE_UNSTABLE cannot serve as the count: it is a single bit and every writer clears it. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_reverse_inval_inode() drives this inode's page cache, before the revoke to flush it and after the revoke to drop it, on the thread that wrote the NOTIFY into /dev/fuse. Both reach fuse_iomap_writeback_range(), which holds the folio locked and under writeback while fuse_dlm_regrant_range() sends FUSE_DLM_WB_LOCK and waits. The range is the one being revoked, so the server cannot answer until the revoke completes, and the revoke cannot complete until the handler returns. A folio dirtied under a grant a NOTIFY took away is reached this way, which fuse_cache_write_iter() says is allowed to happen. Mark the task across that work and report the run as a hole when the grant is not already held: the folio goes back on the dirty list and an ordinary writeback sends it with a grant of its own. Reporting a hole means the run needs the type reset that had no user before. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_iomap_writeback_range() runs with the folio locked and, since iomap_writeback_folio() starts the writeback before the run loop, under writeback as well. Asking the server for a grant there is the ordering Documentation/filesystems/fuse/fuse-AOP_TRUNCATED_PAGE-reason.txt exists to forbid: no cluster lock may be taken while a page lock is held. The read path has AOP_TRUNCATED_PAGE to unlock and retry with, and ->writeback_range has nothing of the sort, so the violation sits where the remedy does not reach. Skip a run whose grant has gone: report it as a hole, put the folio back on the dirty list and remember the range. fuse_iomap_writeback_submit() takes it back once the pass has let go of every folio, and the pass that follows sends it. Not from fuse_launder_folio(), which arrives with the folio locked by folio_unmap_invalidate(), nor from a revoke handler, which would ask for the range it is revoking. There the skip stands. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_dlm_request_begin() publishes the range asked for, and fuse_dlm_kill_pending() can only test that one, but the commit records what the server granted, which may reach FUSE_DLM_MAX_EXTRA_GRANT either side of it. A revoke processed while the request was on the wire and landing in that excess marks nothing: the request does not overlap it, and the shards hold no bit for it yet because the grant is not recorded. The excess is then recorded over the revoked range and no later NOTIFY takes it back, which is the case the pending list exists to catch. Publish the widest bounds the commit could record as well, and separate the two outcomes. A revoke over the range asked for still kills the grant. One over the excess alone leaves the range asked for recorded and drops the excess, which only costs a re-request. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
The comment claimed the round trip happens before any folio of the window is locked. ->readahead is entered with all of them locked and readahead_folio() is what unlocks them, so it happens under them. Say so, and say why it does not close a cycle: the lock in the way belongs to the node the revoke is sent to, not to this one. Name the case that does not follow, a window this already holds part of. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_iomap_writeback_range() unions every run it had to skip, and fuse_iomap_writeback_submit() asks for the lot in one grant. A pass sweeping a large file skips runs gigabytes apart, so the union grows to the whole sweep and the request covers a range nothing wanted. Stop extending at one shard. The runs left out stay dirty and a later pass asks for them. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_writeback_redirty() dirtied the folio from inside ->writeback_range, and iomap_writeback_folio() runs iomap_clear_range_dirty() over the whole folio once that returns. For a folio one block wide there is no iomap_folio_state and the call does nothing, so it worked. A large folio has one, and the folio is left with the dirty flag and no dirty block under it: the next pass finds nothing to write and the folio goes clean with its bytes never sent. Hold the folio instead and dirty it once iomap has released it, on the next call or in the submit. fuse_writepage_need_send() also reached wbc->range_end with no wbc, which only a folio of a single run kept out of reach; guard it. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_fill_write_pages() carried the offset of the next copy across iterations as a within page residue and turned it into a folio offset by adding the page delta. A write that starts inside a folio it does not begin covers the rest of that folio, and the residue left from it is then added to the next folio's own delta: the copy lands past the end of the folio, copy_folio_from_iter_atomic() warns and returns nothing, and the fault in loop retries it for good. A folio one page wide cannot start before the write does, which is what kept it out of reach. Take the offset from @pos, which is always current, and read the loop's carry on test off the folio it just filled. fuse_page_mkwrite() locked the page that faulted. The whole folio is dirtied on the way out, in fault_dirty_shared_page(), so the DLM lock has to cover the folio. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
Two ways a cached write reached the write check as a hole. fuse_iomap_writeback_range() returns -EIO when the inode has no file open for writing, and dropped the folio doing so. fuse_open() invalidates the whole mapping unless the server sets FOPEN_KEEP_CACHE, and reaches fuse_launder_folio() that way; by the time a reader opens the file the last writer has closed, so fuse_write_file_get() finds nothing, warns, and the bytes go. They are still the newest there are, so keep them. And a run whose grant had gone is skipped, its range taken back in the submit, and the folio left dirty for a later pass. A data integrity writeback has no later pass, so fsync() and close() reported bytes written that were still only in the page cache. Go round again while anything is left deferred. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A writer confirms its grant and then copies and dirties, and a revoke landing in between is answered while those bytes are in no page cache and on no wire. The flush the revoke runs cannot find them, so they go out later under a fresh grant, after the server has handed the lock to another node. Add a pin: the range a writer is about to dirty, on a list a revoke walks before it removes anything. A revoke publishes the range it takes away on a second list and waits for the pins overlapping it, and a pin overlapping a published range is refused. Refusal and wait test the same overlap, so the wait converges, and ranges that do not overlap never meet: a notify leaves the rest of the file writable and a write outside the notified range does not hold it up. Both nodes are caller storage, so nothing is allocated to take a pin and the writeback path can take one with a folio held. A pin is found again by owner, since iomap hands ->put_folio the inode and nothing of the iteration. Confirm the grant under the pin, never before, in the three places bytes become the server's: the iomap interior through new ->get_folio and ->put_folio hooks, the writethrough edges around their FUSE_WRITE, and the writeback run until the folio is under writeback. A grant that has gone is asked for again with the pin dropped, since that request is answered by the server the revoke came from. With the writers fenced, the flush a revoke runs needs a single pass: nothing can turn up dirty behind it. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A folio wider than a block carries per-block dirty state, and iomap_writeback_folio() clears all of it after ->writeback_range. A run fuse defers and puts back with folio_mark_dirty() comes back dirty over the whole folio, since iomap_dirty_folio() sets the whole range, so the next pass sends blocks this client never wrote: data read from the server, or zeros laid down above the size the file had at the time. Order-0 folios carry no per-block state, so a dirty folio is one this client wrote whole and the widening cannot happen. The parameter stays writable for anyone opting back in. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
hbirth
force-pushed
the
redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
branch
from
September 1, 2026 12:22
b8ea0ff to
b901618
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.