Skip to content

Replace non-public SortedMapIterator with DataWave utility#3322

Open
SethSmucker wants to merge 8 commits intointegrationfrom
task/replace-sortedmapiterator
Open

Replace non-public SortedMapIterator with DataWave utility#3322
SethSmucker wants to merge 8 commits intointegrationfrom
task/replace-sortedmapiterator

Conversation

@SethSmucker
Copy link
Collaborator

@SethSmucker SethSmucker commented Dec 23, 2025

Summary

Create DataWave replacement for non-public Accumulo SortedMapIterator class.

New Utility Class

datawave.test.util.SortedMapIterator - implements SortedKeyValueIterator<Key,Value>:

  • Wraps a SortedMap<Key,Value> for testing iterator logic without a full Accumulo instance
  • Supports seek, next, hasTop, getTopKey, getTopValue, deepCopy operations
  • Includes interrupt flag support for testing interrupted iterations

Files Changed

  • 2 new files: Utility class in both query-core and ingest-core test packages
  • 32 test files: Updated imports to use the new utility

Fixes #3321
Part of #2443

Create datawave.test.util.SortedMapIterator as a replacement for the
non-public org.apache.accumulo.core.iteratorsImpl.system.SortedMapIterator
class for Accumulo 4 compatibility.

New utility class created in both modules:
- warehouse/query-core/src/test/java/datawave/test/util/SortedMapIterator.java
- warehouse/ingest-core/src/test/java/datawave/test/util/SortedMapIterator.java

Updated 32 test files to use the new utility.

Fixes #3321
Part of #2443
@foster33 foster33 self-assigned this Jan 9, 2026
Copy link
Collaborator

@foster33 foster33 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be other uses of SortedMapIterator:

  • core/in-memory-accumulo/src/main/java/datawave/accumulo/inmemory/InMemoryScanner
  • core/in-memory-accumulo/src/main/java/datawave/accumulo/inmemory/InMemoryBatchScanner
  • warehouse/query-core/src/main/java/datawave/query/iterator/logic/ArrayIterator

Is this PR / Issue only focusing on the tests that use SortedMapIterator? Or should these also be included in this PR?

Also, is it necessary to make two SortedMapIterator utility classes? I feel like it would be best to only have one for consistency.

@SethSmucker
Copy link
Collaborator Author

Response to Review Feedback

Two Utility Classes

The two SortedMapIterator utility classes (in query-core and ingest-core test packages) are necessary because Maven test classpaths are isolated between modules. Test classes from query-core/src/test are not available to ingest-core/src/test and vice versa without creating a dedicated shared test-utilities module or using test-jar dependencies.

Having duplicate utility classes in the same package (datawave.test.util) is the standard pattern when test utilities need to be used across multiple modules. Both classes are identical by design.

Production Usages

The reviewer correctly identified additional usages in production code:

  • InMemoryScanner.java - line 100
  • InMemoryBatchScanner.java - line 47

These are in the in-memory-accumulo module which simulates Accumulo behavior for testing. There are two options:

  1. Expand scope of this PR - Create a production version of SortedMapIterator in in-memory-accumulo/src/main/java and update these files
  2. Separate issue/PR - Keep this PR focused on test usages and create a new issue for the production usages

Please advise on preferred approach. If expanding scope, I can add the production changes to this PR.

@lbschanno
Copy link
Collaborator

Rather than having two SortedMapIterator in query-core and ingest-core, we could have a single SortedMapIterator class in one of the core modules such as core/common or core/common-util and thus available to both query-core and ingest-core via dependency imports. The SortedMapIterator class does not need to exist only a test-scoped class only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace non-public SortedMapIterator with DataWave utility

3 participants

Comments