feat: update storage extension to v2.0.0#1561
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1561 +/- ##
==========================================
- Coverage 92.22% 92.03% -0.20%
==========================================
Files 55 55
Lines 8414 8622 +208
Branches 973 1004 +31
==========================================
+ Hits 7760 7935 +175
- Misses 466 486 +20
- Partials 188 201 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
gadomski
left a comment
There was a problem hiding this comment.
I think we have unnecessary new extensions here ... both schemes and refs can just be classes, not extensions themselves?
| class StorageSchemeType(StringEnum): | ||
| AWS_S3 = "aws-s3" | ||
| CUSTOM_S3 = "custom-s3" | ||
| AZURE = "ms-azure" |
There was a problem hiding this comment.
Can you also add GCS = "gcp-gs"
There was a problem hiding this comment.
generally, having parity to the v1 Enum would be great
There was a problem hiding this comment.
I didn't see best practice GCP storage defined here, so did not add as a possible enum here, but as written schemes can be set to any string. I think opening a PR in the extension w/ a suggested schema to use for GCP storage would make sense before implemented here.
1b0a9ab to
a0267df
Compare
gadomski
left a comment
There was a problem hiding this comment.
Maybe I missed it, but we should test the migration path from v1 to v2.
docs/api/extensions.rst
Outdated
| storage.StorageSchemesExtension | ||
| storage.StorageRefsExtension |
There was a problem hiding this comment.
Looks like the docs need to be reverted back to a single class?
pystac/extensions/storage.py
Outdated
| Returns the dictionary encoding of this object | ||
|
|
||
| Returns: | ||
| dict[str, Any |
I'm taking a look at what adding a migration hook would require, but I think there will be some cases that will be harder to migrate cleanly. (e.g. azure where we need account id, or the platforms that exist in v1 but not v2 e.g. IBM, alibaba) |
I think raising a warning in those cases and then leaving the data unchanged will be sufficient. |
gadomski
left a comment
There was a problem hiding this comment.
Sorry about the long delay, thanks for the contribution!
Description:
StorageScheme platform property allows templated uri's, so arbitrary additional properties are allowed to be set on StorageScheme objects
Includes best-effort a migration from v1 to v2, with currently defined best-practices
PR Checklist:
pre-commit run --all-files)pytest)