FT-701 keep all FT files consistency before next checkpoint for backup #331
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[summary] When we do hot backup like below: 1) take checkpoint lock 2) lock table for read 3) wirte binary log postion 4) copy redo log to backup dir 5) unlock tables 6) start to copy datas files 6.1) write data by another client 6.2) copy file1 to backup dir 7) release checkpoint lock Between 6.1) and 6.2) the FT may does ft_close and update FT header to new, the new writes are visible in the backup instance, this broke the consistency. It would be better for TokuDB to update FT header only in checkpoint, keep all FT files consistency when we are backing up. Copyright (c) 2015, BohuTANG All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Jun-Yuan
added a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Sep 4, 2017
This is an attempt based on @BohuTANG's patch percona#331 in particular, the ctest is all his credits. I included his commit history in the git log, 1) The issue is real. 2) @bohu's patch works for mediating the issue but somewhat introduces a ft leak that the ft with ref=0 may be left until next open/close or checkpoint. 3) If ft_close should be held off upon a backup, then it simply means a backup should hold a ref to the ft and responsible for closing ft up if it is the last one to hold it, like any other possible referencer. including the checkpoint, the txn and ft handlers. 4) mtr test contributed by @BohuTANG still passes
Jun-Yuan
added a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Sep 4, 2017
…der.
This is an attempt based on @BohuTANG's patch
percona#331
in particular, the ctest is all his credits.
Summary:
1) The issue is real.
2) @bohu's patch works for mediating the issue but somewhat introduces
a ft leak that the ft with ref=0 may be left until next open/close or
checkpoint.
3) If ft_close should be held off upon a backup, then it simply means
a backup should hold a ref to the ft and responsible for closing ft up
if it is the last one to hold it, like any other possible referencer.
including the checkpoint, the txn and ft handlers.
4) mtr test contributed by @BohuTANG still passes
Jun-Yuan
added a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Sep 4, 2017
…der.
This is an attempt based on @BohuTANG's patch
percona#331
in particular, the ctest is all his credits.
Summary:
1) The issue is real.
2) @bohu's patch works for mediating the issue but somewhat introduces
a ft leak that the ft with ref=0 may be left until next open/close or
checkpoint.
3) If ft_close should be held off upon a backup, then it simply means
a backup should hold a ref to the ft and responsible for closing ft up
if it is the last one to hold it, like any other possible referencer.
including the checkpoint, the txn and ft handlers.
4) mtr test contributed by @BohuTANG still passes
Jun-Yuan
added a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Sep 14, 2017
This is an attempt based on @BohuTANG's patch percona#331 in particular, the ctest is all his credits. Summary: 1) The issue is real. 2) @bohu's patch works for mediating the issue but somewhat introduces a ft leak that the ft with ref=0 may be left until next open/close or checkpoint. 3) If ft_close should be held off upon a backup, then it simply means a backup should hold a ref to the ft and responsible for closing ft up if it is the last one to hold it, like any other possible referencer. including the checkpoint, the txn and ft handlers. 4) mtr test contributed by @BohuTANG still passes
Jun-Yuan
added a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Sep 19, 2017
…der.
This is an attempt based on @BohuTANG's patch
percona#331
in particular, the ctest is all his credits.
Summary:
1) The issue is real.
2) @bohu's patch works for mediating the issue but somewhat introduces
a ft leak that the ft with ref=0 may be left until next open/close or
checkpoint.
3) If ft_close should be held off upon a backup, then it simply means
a backup should hold a ref to the ft and responsible for closing ft up
if it is the last one to hold it, like any other possible referencer.
including the checkpoint, the txn and ft handlers.
4) mtr test contributed by @BohuTANG still passes
Jun-Yuan
pushed a commit
to Jun-Yuan/PerconaFT
that referenced
this pull request
Mar 29, 2018
[Updated the comments and formats from the last PR.]
This is an attempt based on @BohuTANG's patch
percona#331
in particular, the ctest mostly comes from his with minor updates.
Summary:
1) The issue is real.
2) @bohu's patch works for mediating the issue but somewhat introduces
a ft leak that the ft with ref=0 may be left until next open/close or
checkpoint.
3) If ft_close should be held off upon a backup, then it simply means
a backup should hold a ref to the ft and responsible for closing ft up
if it is the last one to hold it, like any other possible referencer.
including the checkpoint, the txn and ft handlers.
4) mtr test contributed by @BohuTANG still passes. (in a different PS PR).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[summary]
When we do hot backup like below:
6.1) write data by another client
6.2) copy file1 to backup dir
Between 6.1) and 6.2) the FT may does ft_close and update FT header to new, the new writes are visible in the backup instance, this broke the consistency.
It would be better for TokuDB to update FT header only in checkpoint, keep all FT files consistency when we are backing up.
Copyright (c) 2015, BohuTANG
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.