Skip to content

Shrink meta index by quantizing word count #998

@bglw

Description

@bglw

Banking this issue for some future where Pagefind reaches 2.0. (If that ever happens).

The meta index that is loaded up front for every site isn't enormous, but reducing it is always a goal. For a 30-40,000 page site this sits around 250kb. One element in this index is the word count for each page, which is used in result ranking, and is also returned as part of the initial search results.

In a quick test, quantizing this down to 5 bits (32 buckets) cuts ~60kb from this meta index. This would be totally fine for ranking, but since this is also returned from the search API, making this count imprecise would be a breaking change. So we won't do it.

But if we ever do make a breaking change elsewhere, this should be rolled in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions