Feat/beacon manifest #581

dmitrytrager · 2026-02-03T15:51:45Z

Implement GET /api/v1/beacons/manifest to build and return the full manifest JSON (language, region, tags, providers with nested topics and files, totals, checksum). Implement HEAD support with If-None-Match (304 Not Modified) and If-Match (412 Precondition Failed) for lightweight change detection during sync.

Manifest versioning uses lazy evaluation: content checksum is compared with stored value on the beacon, and the version is incremented only when content actually changes. Conditional requests use the stored version directly, avoiding a full manifest build for 304/412 responses.

What Issue Does This PR Cover, If Any?

Resolves
#566
#567
#573

What Changed? And Why Did It Change?

Manifest controller with change detection and corresponding response statuses
Manifest building logic, saving version and checksum inside beacon
Files and assignments tracking logic that will update manifest data for beacon once triggered

How Has This Been Tested?

Rspec

Please Provide Screenshots

Additional Comments

For changed associations (such as topic/providers) rebuild is triggered from callbacks now.
This logic should be later moved into controllers/mutators.

Implement GET /api/v1/beacons/manifest to build and return the full manifest JSON (language, region, tags, providers with nested topics and files, totals, checksum). Implement HEAD support with If-None-Match (304 Not Modified) and If-Match (412 Precondition Failed) for lightweight change detection during sync. Manifest versioning uses lazy evaluation: content checksum is compared with stored value on the beacon, and the version is incremented only when content actually changes. Conditional requests use the stored version directly, avoiding a full manifest build for 304/412 responses. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Introduce Beacons::RebuildManifestJob and wire it into Topics::Mutator (create, update, archive, unarchive, destroy), BeaconTopic/BeaconProvider join model callbacks, and SynchronizeCognatesOnTopicsJob so that beacon manifests stay current without waiting for a GET request. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

dmitrytrager · 2026-02-03T15:53:45Z

app/controllers/api/v1/beacons/manifests_controller.rb

+
+          # During sync, beacon sends If-Match with its current version.
+          # If the manifest changed since sync started, return 412 so beacon aborts and restarts.
+          if stale_by_match?(stored_etag)


Couple of checks here to protect from unnecessary rebuild

dmitrytrager · 2026-02-03T15:54:32Z

app/models/beacon_provider.rb

  belongs_to :beacon
  belongs_to :provider
+
+  after_commit :rebuild_beacon_manifest, on: [ :create, :destroy ]


When we add controller/mutator for managing association, move this logic there

dmitrytrager · 2026-02-03T15:54:38Z

app/models/beacon_topic.rb

  belongs_to :beacon
  belongs_to :topic
+
+  after_commit :rebuild_beacon_manifest, on: [ :create, :destroy ]


When we add controller/mutator for managing association, move this logic there

Could this logic go in a concern shared by the two models?

It will not stay in models. It will go to controller/mutator when we add managing associations

seanmarcia · 2026-02-04T09:56:20Z

Some nice things we might want to store for the admins are some statistics on the current deployed manifest as well as the upcoming version. So we can have some nice summary facts like:
This beacon is providing data with the following configurations:
Language: Spanish
Region: Europe
Tags: ...
Providers:
Total Files on Beacon: 1522
Total Size of Files: 40gb
On the next update this beacon will receive:
78 files for a total of 1600 files and the new total size of the beacon will be 64gb
(Or something like that)

dmitrytrager · 2026-02-04T10:41:35Z

We can store manifest itself inside beacon model. It will help to show total files number or size.
It won't help with diff however, we can only see full content this way.
Working with diffs will make things much more complicated.

I will save manifest into beacon

seanmarcia · 2026-02-04T10:45:49Z

How about storing statistics on the most recent deployed manifest? Size, Number of files, etc? Then we can just do some math and compare.

Save the manifest content (without version/checksum metadata) as jsonb on the beacon record so admins can inspect the full manifest state. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Rotate manifest_data into previous_manifest_data before saving new content, so admins can compare current and previous manifests. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

dmitrytrager · 2026-02-04T11:36:21Z

Added manifest_data and previous_manifest_data to beacon

oatkins8 · 2026-02-04T13:48:43Z

app/controllers/api/v1/beacons/manifests_controller.rb

+          ::Beacons::ManifestBuilder.new(Current.beacon)
+        end
+
+        def stale_by_match?(etag)


I'm wondering if the naming could be a little clearer here, maybe version_mismatches?.

But it is important to underline here that version can become stale during process. And later that is can be outdated. Just match/mismatch does not cover that these are different checks. But I am open to better naming for sure

Could go with sync_version_stale? and cached_version_fresh?.

oatkins8 · 2026-02-04T13:50:13Z

app/controllers/api/v1/beacons/manifests_controller.rb

+          if_match != etag
+        end
+
+        def fresh_by_none_match?(etag)


Same here, could we go for something like version_matches?.

oatkins8 · 2026-02-04T13:54:27Z

app/models/beacon_topic.rb

  belongs_to :beacon
  belongs_to :topic
+
+  after_commit :rebuild_beacon_manifest, on: [ :create, :destroy ]


Could this logic go in a concern shared by the two models?

oatkins8

Looks good 👍

dmitrytrager and others added 2 commits February 3, 2026 15:45

dmitrytrager commented Feb 3, 2026

View reviewed changes

dmitrytrager marked this pull request as ready for review February 3, 2026 15:57

dmitrytrager requested review from FionaLMcLaren, devjona and oatkins8 February 3, 2026 16:11

dmitrytrager and others added 2 commits February 4, 2026 12:03

Store manifest content on beacon for admin visibility

d88a5be

Save the manifest content (without version/checksum metadata) as jsonb on the beacon record so admins can inspect the full manifest state. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Preserve previous manifest data for admin diff visibility

f677129

Rotate manifest_data into previous_manifest_data before saving new content, so admins can compare current and previous manifests. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

oatkins8 reviewed Feb 4, 2026

View reviewed changes

Rename header checking method names

da357cc

oatkins8 approved these changes Feb 4, 2026

View reviewed changes

Feat/beacon manifest #581

Are you sure you want to change the base?

Feat/beacon manifest #581

Uh oh!

Conversation

dmitrytrager commented Feb 3, 2026

What Issue Does This PR Cover, If Any?

What Changed? And Why Did It Change?

How Has This Been Tested?

Please Provide Screenshots

Additional Comments

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seanmarcia commented Feb 4, 2026

Uh oh!

dmitrytrager commented Feb 4, 2026

Uh oh!

seanmarcia commented Feb 4, 2026

Uh oh!

dmitrytrager commented Feb 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oatkins8 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oatkins8 left a comment •

edited

Loading