v2.1.1

What's Changed

• Added key range validation by @alex268 in #43
• Added support of SUBSTRING push down by @alex268 in #41
• Fix datetime pushdown by @alex268 in #44

Full Changelog: v2.1.0...v2.1.1

v2.1.0

What's Changed

• Fix timestamp and datetime processing by @zinal in #29
• Fixes rename table and drop table functions by @zinal in #30
• Types enhancements by @zinal in #31
• Added options to customize batch request size by @alex268 in #32
• Removed notes.md by @alex268 in #33
• Added retries for writing by @alex268 in #35
• Retry management by @zinal in #34
• Fix YdbContext hashCode() by @zinal in #36
• Added support of custom YQL scripts by @alex268 in #37
• Added basic support of predicate pushdown by @alex268 in #38
• Added option to use signed datetypes for table creating by @alex268 in #39
• Use YQL functions for startsWith/endsWith/contains by @alex268 in #40

Full Changelog: v2.0.1...v2.1.0

v2.0.1

What's Changed

• Updated options to autocreate tables by @alex268 in #24
• Removed thread-locks while stream processing by @alex268 in #25
• Update to the latest version of Java SDK & YC AuthProvider by @alex268 in #26

Full Changelog: v2.0.0...v2.0.1

v2.0.0

What's Changed

• Split project to two different artifacts by @alex268 in #8
• Updated tests and added code coverage by @alex268 in #9
• Removed usage of JavaConversions by @alex268 in #10
• Memory limited implementation of YdbScanReadTable by @alex268 in #11
• Reorganize classes by packages by @alex268 in #12
• New implementation by @alex268 in #13
• Added convertation of UTF8String to ydb types by @alex268 in #14
• Added support of JDBC URL parameters by @alex268 in #15
• Fixed inconsistend reads by @alex268 in #16
• Added tests for YdbCatalog by @alex268 in #17
• Added implicit session for BulkUpsert & ReadTable by @alex268 in #18
• Added Scan impelementation based on QueryService by @alex268 in #19
• Performance fixes by @alex268 in #20
• New implementation of QueryService reader by @alex268 in #21
• Fixed key range filter by @alex268 in #22
• Added github actions by @alex268 in #23

New Contributors

• @alex268 made their first contribution in #8

Full Changelog: v1.3...v2.0.0

v1.3 - alpha release update

What's Changed

• TRUNCATE implementation
• Support for table creation on external metadata
• Preparing for columnar tables support
• Basic integration tests
• Switched to YDB SDK version 2.3.6

Full Changelog: v1.2...v1.3

v1.2 - alpha release update

Minor updates: dependency versions bumped, enhanced shading settings.

Full Changelog: v1.1...v1.2

v1.1 - alpha release update

This is the first alpha release of YDB Connector for Apache Spark. It allows Spark applications to connect to YDB, and read and write row tables. All previously stated limitations still apply.

Changes:

• fixed the GRPC dependency to match the YDB SDK version used. This is necessary to avoid the java.lang.IllegalAccessError exceptions on YDB discovery calls.

v1.0 - initial alpha release

This is the first alpha release of YDB Connector for Apache Spark. It allows Spark applications to connect to YDB, and read and write row tables. Known limitations:

• Only row tables access has been tested. Reading and writing columnar tables is expected to fail.
• The connector cannot read YDB system tables (e.g. those under the .sys directory).
• Query results may be inconsistent in case of concurrent writes to the tables being read by the Connector.
• In case of any error during the write operation performed through the Connector, the target table may be left inconsistent (some rows may have been written, and some have not).

These limitations are to be addressed in the future releases of the Connector.