Add a script to check for breaking C ABI changes by benfred · Pull Request #1749 · rapidsai/cuvs

benfred · 2026-01-28T21:11:11Z

This adds a script that can check for changes that would break our C-ABI. It works by comparing two sets of header files: the c/include header files for the published ABI, as well as the c/include header files inside a new pull request. Each set of header files is parsed using libclang, and the differences between the old and new header files are examined for changes that would cause a breaking ABI change.

Currently the following checks are made and flagged by this tool:

Functions that have been removed from the C-ABI
Functions that have extra parameters added
Functions that have parameters removed
Functions that have the type of any parameter changed
Structs that have been removed
Structs that have have members removed
Structs that have the types of members changed
Enum values that have been removed
Enum values that have their value changed

This is just currently a POC of using libclang to flag breaking c-abi changes - and is meant as an alternative to tools that require debug symbols for detecting abi breaks from the compiled library, and isn't ready to merge just yet.

To fully get this integrated, we will need to store the headers for the stable c-abi to use as a baseline somewhere, and then download the stable c-abi headers on each PR and use them to run in a GHA workflow to flag breaking changes.

When run on a branch that has a breaking ABI change (#1496 which has a number of breaking c-abi changes), the output is:

$ ./check_c_abi.py ~/code/cuvs_stable_abi/c/include ~/code/cuvs/c/include 
Error: Function has been removed. Symbol cuvsCagraMergeParamsCreate from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:584
Error: Function has been removed. Symbol cuvsCagraMergeParamsDestroy from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:587
Error: Function has a changed argument type 'cuvsCagraMergeParams_t' to 'cuvsCagraIndexParams_t for argument 'params'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882
Error: Function has a changed argument type 'cuvsCagraIndex_t' to 'cuvsFilter for argument 'output_index'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882
Error: Function has a new argument 'cuvsCagraIndex_t output_index'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882
Error: Struct has been removed. Symbol cuvsCagraMergeParams from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:576

Also, this script runs in < 100ms on my workstation, and we could potentially add this as a pre-commit hook

closes #1739

This adds a script that can check for changes that would break our C-ABI. t works by comparing two sets of header files: the `c/include` header files for the published ABI, as well as the `c/include` header files inside a new pull request. Each set of header files is parsed using libclang, and the differences between the old and new header files are examined for changes that would cause a breaking ABI change. Currently the following checks are made and flagged by this tool: * Functions that have been removed from the C-ABI * Functions that have extra parameters added * Functions that have parameters removed * Functions that have the type of any parameter changed * Structs that have been removed * Structs that have have members removed * Structs that have the types of members changed * Enum values that have been removed * Enum values that have their definition changed

copy-pr-bot · 2026-01-28T21:11:23Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

msarahan · 2026-01-28T23:19:04Z

fantastic! This is exactly what I hoped we could do. I think we can achieve the reference state by:

making the output of a tool be the exposed ABI, and record this by itself as an artifact. This is associated with a commit, a branch, or a tag (or maybe all 3). It is stored as an artifact on S3 or Github as a release artifact (for tags).
make another tool (or another mode of a tool) that fetches the appropriate reference artifact and new artifact to be compared.

Overall, just decouple the source code scanning from the result comparison, and we should have a lot of options.

benfred · 2026-01-30T01:08:04Z

thanks @msarahan !

I've decoupled extracting the ABI from using it to flag abi errors in the last commit - running python check_c_abi.py extract will scan through the C header files and generate a c_abi.json.gz file that can be used as the baseline compare, which can be done with python check_c_abi.py analyze now.

We will still need to figure out where to store the baseline ABI files for comparison (fwiw, the generated abi file is about 5KB right now, so its actually smaller than the script to generate it) - I'm ok with any of the options you mentioned (github release artifacts or s3) - github release artifacts would be slightly easier for me, but only because I've done that before for py-spy (like storing the wheels for each release here) and am unsure of the S3 workflow/permissions from GHA

I don't know if we need to generate the abi file for the current set of files being checked, imo it might be easier to just use the header files directly in the PR

cjnolet · 2026-01-30T02:08:05Z

We will still need to figure out where to store the baseline ABI files for comparison

@msarahan @benfred can we store these files on downloads.rapids.ai (and maybe overwrite the file each time a PR is merged)?

msarahan · 2026-01-30T16:20:10Z

I'm inclined to give this to the ops team to figure out and advise us. I have some vague recollection that we are trying to gradually get away from downloads.rapids.ai, but I could be wrong.

The nice thing about using github for this is that there's no additional auth setup required - the job token would be enough. That's a pretty small thing, though.

@rapidsai/ops any opinion here?

rockhowse · 2026-01-30T16:30:42Z

Thx for the ping @msarahan I read through the conversation above and based on what I am seeing I would recommend keeping things in the github domain (vs storing things externally) especially given the size:

the generated abi file is about 5KB

As for storing headers to compare against, my recommendation would be to keep this in git in some fashion, e.g.

I don't know if we need to generate the abi file for the current set of files being checked, imo it might be easier to just use the header files directly in the PR

I don't think trying to store something remotely in a static way and overwriting it every PR would be a good idea given the fact multiple PRs/pipelines can be running at a given time, but maybe I don't fully understand the full scope of this.

Let me know if there is anything else I might be missing as this is 1 of 100 other random things coming our team's way currently as we get closer to the release.

msarahan · 2026-01-30T16:33:50Z

Web content caching may also be an annoying problem if we do overwriting. I will hack up a github action to do things with Ben's code. I'll submit it as a PR to this PR's branch - so please watch for it, Ben. You are the only one who will get the notification, I think. I will post here, too.

This commit adds automated C ABI compatibility checking to the CI pipeline, building on the check_c_abi.py script from benfred's PR rapidsai#1749. Changes: - Add check-c-abi.yaml workflow to run on PRs modifying C headers - Add store-c-abi-baseline.yaml workflow to store ABI baselines on releases - Integrate ABI check into pr.yaml as a required check - Add comprehensive developer documentation in ci/C_ABI_CHECKING.md - Add implementation summary in GITHUB_ACTIONS_ABI_CHECK_SUMMARY.md The ABI checker automatically detects breaking changes to the C API: - Function signature changes (parameters, return types) - Struct member changes (additions, removals, type changes) - Enum value changes Benefits: - Catch breaking changes before merge, not after release - Automated checking on every PR touching C headers - Clear error messages and guidance for developers - Fast execution (~2-5 minutes) - Stores baselines on releases for future comparisons See GITHUB_ACTIONS_ABI_CHECK_SUMMARY.md for full details.

- Add check-c-abi.yaml workflow for PRs - Add store-c-abi-baseline.yaml workflow for releases - Integrate ABI check into pr.yaml pipeline Co-authored-by: Cursor <cursoragent@cursor.com>

- Extract and store main baseline on push to main (never expires) - PRs download and compare against main baseline - Release workflow archives the main baseline with version tag - Eliminates duplicate ABI extraction work Co-authored-by: Cursor <cursoragent@cursor.com>

Enables bootstrapping the initial c-abi-baseline-main artifact Co-authored-by: Cursor <cursoragent@cursor.com>

- Store baselines with commit SHA for precise comparisons - Cascade: try merge-base → latest main → extract fresh - Add concurrency control to prevent race conditions - Report which baseline source was used for transparency Co-authored-by: Cursor <cursoragent@cursor.com>

msarahan · 2026-01-31T03:39:58Z

PR is up at benfred#1

Add GitHub Actions for C ABI checking

KyleFromNVIDIA

Hey, just wanted to let you know I'm getting to this now. There's a lot going on here, so it's going to take a while, but I do have some initial comments.

KyleFromNVIDIA · 2026-02-02T19:26:48Z

ci/check_c_abi.py

+    location: Optional[SymbolLocation] = None
+
+
+def analyze_c_abi(old_abi, new_abi):


Please split this function up into struct, enum, and function handling. Perhaps further break it up into individual checks as well.

Please also add unit tests of some kind.

This will quickly expand into something that doesn't fit in this one-off folder. It might be wise to plan long term to split this off into a standalone project. Of course, that means it would probably have to undergo osrb review separately from rapids/cuvs.

I would be inclined to move it to its own folder in this project for now, and create the full Python project scaffolding for it. That whole folder could then be moved to a separate repo when we're ready.

KyleFromNVIDIA · 2026-02-02T19:36:10Z

.github/workflows/check-c-abi.yaml

+  update-baseline:
+    if: github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/main')


This job and check-pr are different enough, and have different enough conditions, that I think they should be in separate files.

ci/check_c_abi.py

KyleFromNVIDIA · 2026-02-02T21:17:00Z

.github/workflows/check-c-abi.yaml

+        run: |
+          sudo apt-get update
+          sudo apt-get install -y \
+            libclang-dev \


Is libclang-dev needed for the Python API? When you pip install libclang below, does it build against these headers?

ci/check_c_abi.py

KyleFromNVIDIA · 2026-02-03T17:23:39Z

ci/check_c_abi.py

+    have been removed or had function parameters removed, parameters added or the type of any
+    parameter changed. Note: adding new functions to the new abi is allowed
+    """
+    errors = []


Let's use generators instead of returning arrays:

Suggested change

errors = []

KyleFromNVIDIA · 2026-02-03T17:24:06Z

ci/check_c_abi.py

+            errors.append(
+                AbiError(
+                    "Function has been removed",
+                    symbol=old_function.name,
+                    location=old_function.location,
+                )
+            )


Suggested change

errors.append(

AbiError(

"Function has been removed",

symbol=old_function.name,

location=old_function.location,

)

)

yield AbiError(

"Function has been removed",

symbol=old_function.name,

location=old_function.location,

)

KyleFromNVIDIA · 2026-02-03T17:24:34Z

ci/check_c_abi.py

+            errors.append(
+                AbiError(
+                    f"Function has return type changed from '{old_function.return_type}' to '{new_function.return_type}'",
+                    symbol=new_function.name,
+                    location=new_function.location,
+                )
+            )


Suggested change

errors.append(

AbiError(

f"Function has return type changed from '{old_function.return_type}' to '{new_function.return_type}'",

symbol=new_function.name,

location=new_function.location,

)

)

yield AbiError(

f"Function has return type changed from '{old_function.return_type}' to '{new_function.return_type}'",

symbol=new_function.name,

location=new_function.location,

)

KyleFromNVIDIA · 2026-02-03T17:25:01Z

ci/check_c_abi.py

+                errors.append(
+                    AbiError(
+                        f"Function has a new parameter '{new_type} {new_name}'",
+                        symbol=new_function.name,
+                        location=new_function.location,
+                    )
+                )


Suggested change

errors.append(

AbiError(

f"Function has a new parameter '{new_type} {new_name}'",

symbol=new_function.name,

location=new_function.location,

)

)

yield AbiError(

f"Function has a new parameter '{new_type} {new_name}'",

symbol=new_function.name,

location=new_function.location,

)

KyleFromNVIDIA · 2026-02-03T17:25:18Z

ci/check_c_abi.py

+                errors.append(
+                    AbiError(
+                        f"Function has a deleted parameter '{old_type} {old_name}'",
+                        symbol=old_function.name,
+                        location=old_function.location,
+                    )
+                )


Suggested change

errors.append(

AbiError(

f"Function has a deleted parameter '{old_type} {old_name}'",

symbol=old_function.name,

location=old_function.location,

)

)

yield AbiError(

f"Function has a deleted parameter '{old_type} {old_name}'",

symbol=old_function.name,

location=old_function.location,

)

KyleFromNVIDIA · 2026-02-03T17:28:19Z

ci/check_c_abi.py

+    errors = []
+    errors.extend(_analyze_function_abi(old_abi, new_abi))
+    errors.extend(_analyze_struct_abi(old_abi, new_abi))
+    errors.extend(_analyze_enum_abi(old_abi, new_abi))
+    return errors


Suggested change

errors = []

errors.extend(_analyze_function_abi(old_abi, new_abi))

errors.extend(_analyze_struct_abi(old_abi, new_abi))

errors.extend(_analyze_enum_abi(old_abi, new_abi))

return errors

yield from _analyze_function_abi(old_abi, new_abi)

yield from _analyze_struct_abi(old_abi, new_abi)

yield from _analyze_enum_abi(old_abi, new_abi)

KyleFromNVIDIA · 2026-02-03T17:28:27Z

ci/check_c_abi.py

+                )
+            )
+
+    return errors


Suggested change

return errors

KyleFromNVIDIA · 2026-02-03T17:28:37Z

ci/check_c_abi.py

+                        location=new_struct.location,
+                    )
+                )
+    return errors


Suggested change

return errors

KyleFromNVIDIA · 2026-02-03T17:28:53Z

ci/check_c_abi.py

+                        location=new_function.location,
+                    )
+                )
+    return errors


Suggested change

return errors

KyleFromNVIDIA · 2026-02-03T17:29:44Z

ci/check_c_abi.py

+    location: Optional[SymbolLocation] = None
+
+
+def _analyze_function_abi(old_abi, new_abi):


Please add type annotations to all function arguments and return values.

KyleFromNVIDIA · 2026-02-03T18:07:01Z

ci/check_c_abi.py

+        new_abi = Abi.from_include_path(
+            header_path, args.include_file, extra_clang_args
+        )
+        errors = analyze_c_abi(old_abi, new_abi)


Suggested change

errors = analyze_c_abi(old_abi, new_abi)

errors = list(analyze_c_abi(old_abi, new_abi))

benfred requested a review from a team as a code owner January 28, 2026 21:11

benfred requested a review from KyleFromNVIDIA January 28, 2026 21:11

benfred added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jan 28, 2026

github-project-automation bot added this to Vector Search, ML, & Data Mining Release Board Jan 28, 2026

github-project-automation bot moved this to Todo in Vector Search, ML, & Data Mining Release Board Jan 28, 2026

benfred marked this pull request as draft January 28, 2026 21:11

cjnolet assigned benfred Jan 29, 2026

Split extracting abi from analyzing abi

f6e13fd

msarahan and others added 4 commits January 30, 2026 20:36

Add GitHub Actions for C ABI checking

34a9001

- Add check-c-abi.yaml workflow for PRs - Add store-c-abi-baseline.yaml workflow for releases - Integrate ABI check into pr.yaml pipeline Co-authored-by: Cursor <cursoragent@cursor.com>

Add workflow_dispatch to allow manual baseline creation

2ad7c0f

Enables bootstrapping the initial c-abi-baseline-main artifact Co-authored-by: Cursor <cursoragent@cursor.com>

msarahan mentioned this pull request Jan 31, 2026

Add GitHub Actions for C ABI checking benfred/cuvs#1

Merged

Merge pull request #1 from msarahan/add-github-actions-abi-check

d6d52ba

Add GitHub Actions for C ABI checking

KyleFromNVIDIA requested changes Feb 2, 2026

View reviewed changes

Merge branch 'main' into c_abi_checker

91567ef

code review fixes

80c335d

KyleFromNVIDIA requested changes Feb 3, 2026

View reviewed changes

This comment was marked as duplicate.

Sign in to view

KyleFromNVIDIA requested changes Feb 3, 2026

View reviewed changes

msarahan mentioned this pull request Feb 3, 2026

[SBIN] Implement GitHub ABI check CI jobs (arm and x86) #1739

Open

cjnolet mentioned this pull request Feb 3, 2026

[FEA] Enable code rabbit for AI code reviews #1767

Open

		location: Optional[SymbolLocation] = None


		def analyze_c_abi(old_abi, new_abi):

		update-baseline:
		if: github.event_name == 'workflow_dispatch' \|\| (github.event_name == 'push' && github.ref == 'refs/heads/main')

		location: Optional[SymbolLocation] = None


		def _analyze_function_abi(old_abi, new_abi):

	errors = analyze_c_abi(old_abi, new_abi)
	errors = list(analyze_c_abi(old_abi, new_abi))

Conversation

benfred commented Jan 28, 2026 • edited by msarahan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Jan 28, 2026

Uh oh!

msarahan commented Jan 28, 2026

Uh oh!

benfred commented Jan 30, 2026

Uh oh!

cjnolet commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msarahan commented Jan 30, 2026

Uh oh!

rockhowse commented Jan 30, 2026

Uh oh!

msarahan commented Jan 30, 2026

Uh oh!

msarahan commented Jan 31, 2026

Uh oh!

KyleFromNVIDIA left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as duplicate.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

benfred commented Jan 28, 2026 •

edited by msarahan

Loading

cjnolet commented Jan 30, 2026 •

edited

Loading