Skip to content

Commit 5db8a24

Browse files
committed
run pre-commit
1 parent f1d8309 commit 5db8a24

File tree

3 files changed

+78
-65
lines changed

3 files changed

+78
-65
lines changed

docs/source/index.rst

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -157,8 +157,8 @@ Python's standard `logging framework`_.
157157
Errors
158158
------
159159

160-
The ``s3fs`` library includes a built-in mechanism to automatically retry
161-
operations when specific transient errors occur. You can customize this behavior
160+
The ``s3fs`` library includes a built-in mechanism to automatically retry
161+
operations when specific transient errors occur. You can customize this behavior
162162
by adding specific exception types or defining complex logic via custom handlers.
163163

164164
Default Retryable Errors
@@ -176,7 +176,7 @@ By default, ``s3fs`` will retry the following exception types:
176176
Registering Custom Error Types
177177
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
178178

179-
To include additional exception types in the default retry logic, use the
179+
To include additional exception types in the default retry logic, use the
180180
``add_retryable_error`` function. This is useful for simple type-based retries.
181181

182182
.. code-block:: python
@@ -188,10 +188,10 @@ To include additional exception types in the default retry logic, use the
188188
Implementing Custom Error Handlers
189189
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
190190

191-
For more complex scenarios, such as retrying based on an error message rather than
191+
For more complex scenarios, such as retrying based on an error message rather than
192192
just the type, you can register a custom error handler using ``set_custom_error_handler``.
193193

194-
The handler should be a callable that accepts an exception instance and returns ``True``
194+
The handler should be a callable that accepts an exception instance and returns ``True``
195195
if the error should be retried, or ``False`` otherwise.
196196

197197
.. code-block:: python
@@ -203,9 +203,9 @@ if the error should be retried, or ``False`` otherwise.
203203
Handling AWS ClientErrors
204204
~~~~~~~~~~~~~~~~~~~~~~~~~
205205

206-
``s3fs`` provides specialized handling for ``botocore.exceptions.ClientError``.
207-
While ``s3fs`` checks these against internal patterns (like throttling),
208-
you can extend this behavior using a custom handler. Note that the internal
206+
``s3fs`` provides specialized handling for ``botocore.exceptions.ClientError``.
207+
While ``s3fs`` checks these against internal patterns (like throttling),
208+
you can extend this behavior using a custom handler. Note that the internal
209209
patterns will still be checked and handled before the custom handler.
210210

211211
.. code-block:: python

s3fs/core.py

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ def setup_logging(level=None):
7373
if ClientPayloadError is not None:
7474
S3_RETRYABLE_ERRORS += (ClientPayloadError,)
7575

76+
7677
def add_retryable_error(exc):
7778
"""
7879
Add an exception type to the list of retryable S3 errors.
@@ -86,18 +87,21 @@ def add_retryable_error(exc):
8687
----------
8788
>>> class MyCustomError(Exception): # doctest: +SKIP
8889
... pass # doctest: +SKIP
89-
>>> add_retryable_error(MyCustomError) # doctest: +SKIP
90+
>>> add_retryable_error(MyCustomError) # doctest: +SKIP
9091
"""
9192
global S3_RETRYABLE_ERRORS
9293
S3_RETRYABLE_ERRORS += (exc,)
9394

95+
9496
CUSTOM_ERROR_HANDLER = lambda _: False
97+
98+
9599
def set_custom_error_handler(func):
96100
"""Set a custom error handler function for S3 retryable errors.
97101
98102
The function should take an exception instance as its only argument,
99103
and return True if the operation should be retried, or False otherwise.
100-
This can also be used for custom behavior on `ClientError` exceptions,
104+
This can also be used for custom behavior on `ClientError` exceptions,
101105
such as retrying other patterns.
102106
103107
Parameters
@@ -118,6 +122,7 @@ def set_custom_error_handler(func):
118122
global CUSTOM_ERROR_HANDLER
119123
CUSTOM_ERROR_HANDLER = func
120124

125+
121126
_VALID_FILE_MODES = {"r", "w", "a", "rb", "wb", "ab"}
122127

123128
_PRESERVE_KWARGS = [
@@ -151,13 +156,14 @@ def set_custom_error_handler(func):
151156
}
152157
buck_acls = {"private", "public-read", "public-read-write", "authenticated-read"}
153158

159+
154160
async def _error_wrapper(func, *, args=(), kwargs=None, retries):
155161
if kwargs is None:
156162
kwargs = {}
157163
err = None
158164
for i in range(retries):
159165
wait_time = min(1.7**i * 0.1, 15)
160-
166+
161167
try:
162168
return await func(*args, **kwargs)
163169
except S3_RETRYABLE_ERRORS as e:
@@ -177,7 +183,7 @@ async def _error_wrapper(func, *, args=(), kwargs=None, retries):
177183
if pattern in str(e):
178184
matched = True
179185
break
180-
186+
181187
if matched:
182188
await asyncio.sleep(wait_time)
183189
else:

0 commit comments

Comments
 (0)