SYS-2069: Add API endpoint to get all records serialized as JSON

[henry-r18]

Implements SYS-2069

Acceptance criteria

• Add a /records/ endpoint which allows retrieval of all SheetImport records as JSON, the same format as the specific /records/<record_id>
  • This endpoint also requires basic authentication.
• Records are retrieved in ascending numerical order by record_id
• The endpoint accepts the following parameters:
  • offset: Offset of the results returned. Optional. Default value: 0, which means that the first results will be returned
  • limit: Limits the number of results. Optional. Default value: 100. Valid values: 0-??? - maybe 250, maybe 500, depends on performance. If even 100 is too slow, decrease the default, but this means we also need to do some performance work
• The endpoint uses the following response format: JSON, with two top-level elements:
  • records: A list of dictionaries, 1 for each record in the specified range
  • total_records: An integer with the total number of records which will be retrieved, so the caller knows how to manage offset and limit to get all records

Description

This PR adds an API endpoint to retrieve all records, serialized as JSON via our particular logic, through paginated results. The endpoint takes the form records/?offset={OFFSET}&limit={LIMIT}, with defaults of 0 and 100 for the two query parameters respectively.

The underlying view for this endpoint utilizes our transform_record_to_dict utility, which handles serialization and joining of some data from objects other than SheetImport. This seems to work fine performance-wise in my local setup—retrieving 100 records takes ~0.5 seconds, and 10,000 records takes ~6 seconds.

Testing

I added 3 tests to cover the criteria that the results be in ascending order by ID and that the limit and offset params work as expected. Total tests now stand at 74.

[akohler]

@henry-r18 This is great! The code does what's needed, simply and efficiently. Tests are relevant and all pass. I tested via curl requests and by writing a simple python program to retrieve all records in batches of 1000. Everything I saw worked as expected, and performance against my local system was very good.

Please add release notes and a version bump for deployment, then I'll merge and we'll try it in production.

Thanks --Andy

[henry-r18]

@akohler, just pushed the version bump and release note—thanks!

[akohler]

Merging now.
@ztucker4 FYI