Skip to content

Add record_ecosystem_meta API support#407

Merged
jakecoffman merged 3 commits intodependabot:mainfrom
rhyskoedijk:feature/enable-record-ecosystem-meta
May 7, 2025
Merged

Add record_ecosystem_meta API support#407
jakecoffman merged 3 commits intodependabot:mainfrom
rhyskoedijk:feature/enable-record-ecosystem-meta

Conversation

@rhyskoedijk
Copy link
Contributor

@rhyskoedijk rhyskoedijk commented Mar 9, 2025

In dependabot/dependabot-core#10905, a new record_ecosystem_meta API was added. This API is not implemented by dependabot-cli.

When performing updates for an ecosystem that invokes the API (e.g. pip), the API server will throw 501 errors until updater retry limit is reached:

cli | 2025/03/09 21:40:17 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:17 [024] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:17 [024] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:22 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:22 [025] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:22 [025] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:27 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:27 [026] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:27 [026] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
cli | 2025/03/09 21:40:31 unexpected output type: record_ecosystem_meta
proxy | 2025/03/09 21:40:31 [027] POST http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 21:40:31 [027] 501 http://host.docker.internal:44891/update_jobs/update_0_pip_all/record_ecosystem_meta
updater | 2025/03/09 21:40:31 ERROR <job_update_0_pip_all> Failed to record ecosystem meta after 3 retries

This API is called multiple times per dependency update and is compounded by the 3 retries per call. The volume of these errors causes the update process to take significantly longer than normal.

After this change, the API server returns 200 and the updater doesn't report error:

proxy | 2025/03/09 22:15:29 [128] POST http://host.docker.internal:35303/update_jobs/update_0_pip_all/record_ecosystem_meta
proxy | 2025/03/09 22:15:29 [128] 200 http://host.docker.internal:35303/update_jobs/update_0_pip_all/record_ecosystem_meta

The output scenario file now contains the record_ecosystem_meta data:

image

@rhyskoedijk
Copy link
Contributor Author

@kbukum1, @amazimbe, @abdulapopoola tagging for visibility as you were involved with the original change dependabot/dependabot-core#10905.

@robaiken robaiken force-pushed the feature/enable-record-ecosystem-meta branch from c0e79f5 to f6788c3 Compare March 17, 2025 14:17
@jakecoffman jakecoffman force-pushed the feature/enable-record-ecosystem-meta branch from f6788c3 to 8c7b98e Compare May 7, 2025 15:36
Copy link
Member

@jakecoffman jakecoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@jakecoffman jakecoffman added this pull request to the merge queue May 7, 2025
Merged via the queue into dependabot:main with commit 65e5e09 May 7, 2025
76 checks passed
@rhyskoedijk rhyskoedijk deleted the feature/enable-record-ecosystem-meta branch May 7, 2025 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants