Add faulty file system for io failure injections #9457
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
xiaoxmeng
force-pushed
the
scan
branch
8 times, most recently
from
57630af to
544c6d9
Compare
xiaoxmeng
changed the title
xiaoxmeng
requested review from
mbasmanova,
oerling,
tanjialiang and
kewang1024
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
tanjialiang
reviewed
Apr 13, 2024
Comment on lines
74
to
75
| /// 'opTypes' is empty, it injects error for all kinds of file operation | ||
| /// types. |
There was a problem hiding this comment.
I wonder if we want to be explicit here. I could be easy to get confused. If none is provided, should we just not inject? Can we add Type::kAll to indicate that we want to inject to all operations?
There was a problem hiding this comment.
I think it is common to treat an empty set to match all. User will only call this method to inject fault.
Comment on lines
91
to
93
| // Defines the file injection setup and only one type of injection can be set | ||
| // at a time. | ||
| struct FileInjections { |
There was a problem hiding this comment.
Does this mean either delay type or exception type can be set? If so can we make members const? Otherwise direct access to members can make this struct in an inconsistent state
There was a problem hiding this comment.
There is a default case used to copy the injection settings. I will add some sanity check on the execution path. This is a private structure used internally.
xiaoxmeng
force-pushed
the
scan
branch
3 times, most recently
from
49bd2c3 to
98a300c
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
xiaoxmeng
force-pushed
the
scan
branch
2 times, most recently
from
a51d498 to
21f5227
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
tanjialiang
approved these changes
Apr 13, 2024
xiaoxmeng
force-pushed
the
scan
branch
3 times, most recently
from
b043084 to
3af7411
Compare
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
Apr 14, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56079500 |
|
@xiaoxmeng merged this pull request in d796cfc. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
Jun 7, 2024
) Summary: Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test. It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate the file operation to the corresponding underlying file system. We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters and each file api will extend with its own operation type. This PR only supports fault injection for read file operations. Next we will extend to other file operation types as well as fs operation types. Pull Request resolved: facebookincubator#9457 Reviewed By: tanjialiang, oerling Differential Revision: D56079500 Pulled By: xiaoxmeng fbshipit-source-id: e7cccbf0b51b3f94c161949b22a88cb560d58b7d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add faulty file system to allow us to inject failure for low-level file operations in unit and fuzzer test.
It is implemented under velox filesystem. It is wrapper on top of the real file system, it either inject
errors or delegate the file operations to the underlying file system. We extend TempDirectoryPath or
TempFilePath to allow user to specify whether enable fault injection or not in test. If it does, it returns
the file or directory path with faulty file system scheme prefix (faulty:). This allows the velox file system
to open the file through faulty file system and by removing the faulty fs scheme prefix, it can delegate
the file operation to the corresponding underlying file system.
We support three kinds of file fault injections: (1) error injection which throws for a set (or all) file operation
types; (2) delay injection for a set (or all) file operation types; (3) custom injection with user provided fault
injection hook. We define the base structure FaultFileOperation to capture the file fault injection parameters
and each file api will extend with its own operation type.
This PR only supports fault injection for read file operations. Next we will extend to other file operation types
as well as fs operation types.