Skip to content

running pg-benchmark does not take into account dataset splits. #114

@karel-w

Description

@karel-w

Summary

I realise we don't have the splitting functionality in the dataset yet, but should not forget to implement this in the benchmark when possible.

Current examples do not take a split dataset, they take the full dataset and set this to the supervise/zero-shot benchmark.

Motivation

To avoid confusion we should only utilize:

  • train, validation, test datasets in supervised.
  • test dataset in zero-shot.

Proposed solution

Depends on execution of ProteinGym/proteingym-base#294

Metadata

Metadata

Assignees

No one assigned

    Labels

    refineIssues with this tag needs refinement by team. Typically used to brain dump issues that pop up

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions