[first stage] Upgrade faiss to 1.13.2+ (commit 1aea3e934745f37)#1430
[first stage] Upgrade faiss to 1.13.2+ (commit 1aea3e934745f37)#1430sre-ci-robot merged 1 commit intozilliztech:mainfrom
Conversation
|
@alexanderguzhva 🔍 Important: PR Classification Needed! For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:
For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”. Thanks for your efforts and contribution to the community!. |
4d62de3 to
001b7e6
Compare
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
001b7e6 to
5008edf
Compare
|
issue: #1439 |
|
/kind improvement |
|
@alexanderguzhva After this PR, does knowhere still need to bear the maintenance cost of keeping code in sync with Faiss? It currently looks like that’s the case. |
|
@foxspy yes |
Could we maintain two versions of Faiss: one that directly references https://github.com/facebookresearch/faiss, and another one under thirdparty/faiss? The former would be integrated into Knowhere without any modifications, so we can avoid the cost of keeping it in sync. |
|
@foxspy we cannot, because we still need to modify faiss a bit. For example, I needed to add extra metric types, which is impossible to do if the code is just referenced |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alexanderguzhva, foxspy The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This is the first patch of the following series. I expect it to be committed ASAP, because it allows starting the work on integrating some other Faiss parts, such as SVS, cuVS, etc. into Milvus. And these integration processes can be started independently from a fork-related refactoring process.
The code has been reorganized in the following way:
/thirdparty/faisshas been replaced with a clone of the baseline Faiss version, 1.13.2+, commit 1aea3e934745f37e8d60cfff6c5900b175c42709/thirdparty/faiss/faiss/cppcontrib/knowheredirectory under aC++namespacefaiss::cppcontrib::knowhere./src/simd/) has been altered to use aC++namespacefaiss::cppcontrib::knowhereIt is expected to gradually remove duplicate components from a knowhere-fork a replace them with the baseline faiss components. If this is not possible, then a new component must be created and put into
cppcontrib/knowheredirectory.Additional patches with the refactoring of the fork are expected.