[feat] Introduce cross-modality and multi-modality support; modularize CrossEncoder class

.github/workflows/tests.yml

@@ -67,15 +67,17 @@ jobs:
           # The --break-system-packages flag is used to allow pip to upgrade or install packages
           # even in environments where system-managed packages might otherwise block such operations.
           python -m pip install --upgrade pip --break-system-packages
-          python -m pip install '.[train, onnx, openvino, dev]' 'transformers<5.0.0'
+          # We require the not-yet-released https://github.com/huggingface/accelerate/pull/3850, so install from GitHub for now
+          python -m pip install '.[train, onnx, openvino, dev]' 'transformers<5.0.0' 'git+https://github.com/huggingface/accelerate'
 
       - name: Install dependencies (transformers >= 5.0.0)
         if: ${{ matrix.transformers-version == '>=5.0.0' }}
         run: |
           # The --break-system-packages flag is used to allow pip to upgrade or install packages
           # even in environments where system-managed packages might otherwise block such operations.
           python -m pip install --upgrade pip --break-system-packages
-          python -m pip install '.[train, dev]' 'transformers>=5.0.0rc0'
+          # We require the not-yet-released https://github.com/huggingface/accelerate/pull/3850, so install from GitHub for now
+          python -m pip install '.[train, dev]' 'transformers>=5.0.0rc0' 'git+https://github.com/huggingface/accelerate'
 
       - name: Install model2vec
         run: python -m pip install model2vec

.pre-commit-config.yaml

@@ -1,7 +1,10 @@
 repos:
-  - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.14.5
-    hooks:
-      - id: ruff
-        args: [--exit-non-zero-on-fix]
-      - id: ruff-format
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  # Ruff version.
+  rev: v0.14.10
+  hooks:
+    # Run the linter.
+    - id: ruff-check
+      args: [ --fix, --show-fixes ]
+    # Run the formatter.
+    - id: ruff-format

README.md

@@ -112,7 +112,7 @@ print(scores)
 # => [8.607139 5.506266 6.352977]
 ```
 
-And we're good to go. You can also use [`model.rank`](https://sbert.net/docs/package_reference/cross_encoder/cross_encoder.html#sentence_transformers.cross_encoder.CrossEncoder.rank) to avoid having to perform the reranking manually:
+And we're good to go. You can also use [`model.rank`](https://sbert.net/docs/package_reference/cross_encoder/cross_encoder.html#sentence_transformers.cross_encoder.model.CrossEncoder.rank) to avoid having to perform the reranking manually:
 
 ```python
 # 2b. Rank a list of passages for a query

docs/.htaccess

@@ -71,6 +71,10 @@ Redirect 301 /examples/unsupervised_learning/README.html /examples/sentence_tran
 Redirect 301 /examples/unsupervised_learning/SimCSE/README.html /examples/sentence_transformer/unsupervised_learning/SimCSE/README.html
 Redirect 301 /examples/unsupervised_learning/TSDAE/README.html /examples/sentence_transformer/unsupervised_learning/TSDAE/README.html
 
+# Moved API Reference pages for v5.3.0
+Redirect 301 /docs/package_reference/sentence_transformer/quantization.html /docs/package_reference/util/quantization.html
+Redirect 301 /docs/package_reference/util.html /docs/package_reference/util/index.html
+
 # Redirect to index.html when request file does not exist
 # RewriteCond %{REQUEST_FILENAME} !-f
 # RewriteCond %{REQUEST_FILENAME} !-d

docs/cross_encoder/pretrained_models.md

@@ -85,7 +85,7 @@ These models have been trained on the [Quora duplicate questions dataset](https:
 
 ```{eval-rst}
 .. note::
-    The model don't work for question similarity. The question "How to learn Java?" and "How to learn Python?" will get a low score, as these questions are not duplicates. For question similarity, a :class:`~sentence_transformers.SentenceTransformer` trained on the Quora dataset will yield much more meaningful results.
+    The model don't work for question similarity. The question "How to learn Java?" and "How to learn Python?" will get a low score, as these questions are not duplicates. For question similarity, a :class:`~sentence_transformers.model.SentenceTransformer` trained on the Quora dataset will yield much more meaningful results.
 ```
 
 ## NLI

docs/cross_encoder/usage/backend_export_sidebar.rst

@@ -0,0 +1,5 @@
+.. sidebar:: Export, Optimize, and Quantize Hugging Face models
+
+   This Hugging Face Space provides a user interface for exporting, optimizing, and quantizing models for either ONNX or OpenVINO:
+
+   - `sentence-transformers/backend-export <https://huggingface.co/spaces/sentence-transformers/backend-export>`_

docs/cross_encoder/usage/efficiency.rst

@@ -145,14 +145,14 @@ All keyword arguments passed via ``model_kwargs`` will be passed on to :meth:`OR
 
 .. tip::
 
-   It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() <sentence_transformers.cross_encoder.CrossEncoder.save_pretrained>` if your model was local:
+   It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() <sentence_transformers.cross_encoder.model.CrossEncoder.save_pretrained>` if your model was local:
 
    .. code-block:: python
 
       model = CrossEncoder("path/to/my/model", backend="onnx")
       model.save_pretrained("path/to/my/model")
 
-   or with :meth:`model.push_to_hub() <sentence_transformers.cross_encoder.CrossEncoder.push_to_hub>` if your model was from the Hugging Face Hub:
+   or with :meth:`model.push_to_hub() <sentence_transformers.cross_encoder.model.CrossEncoder.push_to_hub>` if your model was from the Hugging Face Hub:
 
    .. code-block:: python
 
@@ -354,14 +354,14 @@ If the model path or repository already contains a model in OpenVINO format, Sen
 
 .. tip::
 
-   It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() <sentence_transformers.cross_encoder.CrossEncoder.save_pretrained>` if your model was local:
+   It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() <sentence_transformers.cross_encoder.model.CrossEncoder.save_pretrained>` if your model was local:
 
    .. code-block:: python
 
       model = CrossEncoder("path/to/my/model", backend="openvino")
       model.save_pretrained("path/to/my/model")
 
-   or with :meth:`model.push_to_hub() <sentence_transformers.cross_encoder.CrossEncoder.push_to_hub>` if your model was from the Hugging Face Hub:
+   or with :meth:`model.push_to_hub() <sentence_transformers.cross_encoder.model.CrossEncoder.push_to_hub>` if your model was from the Hugging Face Hub:
 
    .. code-block:: python
 

docs/cross_encoder/usage/usage.rst

@@ -13,12 +13,12 @@ Once you have `installed <../../installation.html>`_ Sentence Transformers, you
 
 .. sidebar:: Documentation
 
-   1. :class:`~sentence_transformers.cross_encoder.CrossEncoder`
-   2. :meth:`CrossEncoder.predict <sentence_transformers.cross_encoder.CrossEncoder.predict>`
-   3. :meth:`CrossEncoder.rank <sentence_transformers.cross_encoder.CrossEncoder.rank>`
+   1. :class:`~sentence_transformers.cross_encoder.model.CrossEncoder`
+   2. :meth:`CrossEncoder.predict <sentence_transformers.cross_encoder.model.CrossEncoder.predict>`
+   3. :meth:`CrossEncoder.rank <sentence_transformers.cross_encoder.model.CrossEncoder.rank>`
 
    .. note::
-      MS Marco models return logits rather than scores between 0 and 1. Load the :class:`~sentence_transformers.cross_encoder.CrossEncoder` with ``activation_fn=torch.nn.Sigmoid()`` to get scores between 0 and 1. This does not affect the ranking.
+      MS Marco models return logits rather than scores between 0 and 1. Load the :class:`~sentence_transformers.cross_encoder.model.CrossEncoder` with ``activation_fn=torch.nn.Sigmoid()`` to get scores between 0 and 1. This does not affect the ranking.
 
 ::