Replies: 28 comments 28 replies
-
|
This is a nice clear example of how another project is handling it (HT @geakstr for pointing this out) |
Beta Was this translation helpful? Give feedback.
-
There's a beauty of simplicity in this policy. I would be curious if there are any learnings from that community around the implementation / enforcement of the policy. |
Beta Was this translation helpful? Give feedback.
-
|
As much as I understand the need to protect the project from AI-related issues, in particular IP theft / missing credits, or the multiplication of dubious Pull Requests with a limited understanding of what's going on from the authors, I find this policy very strict, and I'm not sure this is going in the right direction when looking at the current trends in programming. Let's be clear: I don't have an ideal solution in mind, it's hard to find the right balance. But to me, it feels like this policy basically tells users “You must disable Copilot (or your other LLMs) in your IDE when working on Cilium. Using Cursor or another one of the kind? Change your IDE”. As I see it:
Again, I'm not saying we should blindly approve all contributions, and working on a proper policy is an excellent initiative (thanks for all your work and reflection, Joe!). At the same time, rejecting all forms of generated content sounds extreme to me, in today's context. I see that Dylan expressed a similar opinion as I was typing these words ( 🙌 ), I relate to his sentiment. |
Beta Was this translation helpful? Give feedback.
-
|
It appears that https://lwn.net/ml/all/20251001140310.527097-1-christian.couder@gmail.com/ |
Beta Was this translation helpful? Give feedback.
-
|
Credit and legal implications 🧵
The current approach in the Cilium project is that we rely heavily on DCO and trust in contributors to read this document and sign off that they are following its steps. Parts (a) or (b) seem quite difficult to honestly certify without either creating the contribution yourself or building on work from a reputable website like GitHub that clearly denotes the appropriate licensing. Part (c) is of course a delegation to someone else who certified one of (a) or (b). Do we know of any LLM tools that limit the training data to licenses compatible with the project? Is there an audit trail of the source material to provide a reasonable basis to state that the work is covered by an appropriate open source license? |
Beta Was this translation helpful? Give feedback.
-
|
We require all contributors to certify their work, and at least in the cilium/cilium repository there is a bot that checks for this certification and links to what it means / how to solve it. This is a bit less direct and may betray the seriousness of the requirement for the contributor to follow the DCO. We might consider making this stronger by adjusting the bot to respond with the full DCO inline in the PR when sign-off is omitted, and also include the current link to the docs page that helps to describe how to certify your changes. |
Beta Was this translation helpful? Give feedback.
-
|
IMO the license issue and low code quality is two separate kind of problems. If the issue is about licensing I would say that the debate should be centered around legals issue and that if we ban LLM the documentation should clearly be stating this as first reason to do so. If the main problem is low quality contribution/vibe coded contribution like I think @xmulligan is implying in the OP, the potential checkbox might (debatable) be centered more around "yes I used an LLM but I did double checked things and I know what I am doing". For instance, one could use copilot as a slightly more intelligent auto completion tool or use LLM chat bot in a way that only influence indirectly his contribution (for instance analyzing some error/code and still writing his actual contribution without direct LLM help) and maybe those shouldn't have the same treatment as someone who (vibe) coded mostly by discussing with an LLM and changing almost none of the output. Although this is super debatable and also assuming there is no legal issue in the first place! |
Beta Was this translation helpful? Give feedback.
-
|
I would like to raise a situation we have observed in relation to the DCO, sign-off and legal aspects. PR 1 was posted to a Cilium project repository. Some time later, a different author posted PR 2 to the repository, and the content had a significant resemblance to PR 1. When we asked the author of PR 2 about the content, they explained that they did not realize it was almost the same and that the submission was created using an LLM IDE. The author had signed the DCO but it appears they had unknowingly copied another author's work to claim it as their own. The author of PR 2 ultimately withdrew the submission. From what I understand, some of the newer LLM IDEs provide the ability to pull content from the web as part of an agentic / iterative development process. I suspect this capability may have been used in this case. My take away from the above case is that LLM IDEs will copy others' work without proper attribution, and that contributors signing off a DCO for such submissions may be falsely asserting the origin or rights to submit the content to the project. The case outlined above is already alarming as attribution is an important part of open source development, but it could be worse - submissions could be based on code that cannot be accepted given the licensing of the original source. I am aware that there are tools to attempt to detect such cases, and I do not trust them at this time. |
Beta Was this translation helpful? Give feedback.
-
|
I came across this guidance from the Apache Software Foundation that recommends all commits that were authored with generative AI tooling should disclose the usage with a commit tag like |
Beta Was this translation helpful? Give feedback.
-
Yes. One possible approach: Include a checkbox in the PR template with language that reflects a default assumption that someone is using LLM tooling. Ensure that contributors must take explicit action to certify that an LLM was not used. |
Beta Was this translation helpful? Give feedback.
-
|
^ If we do this, we should also be clear about what "LLM usage" means. I think there are at least two distinct categories: Using an LLM to write contributions or using an LLM as a fancy search engine to learn. Both may suffer from well-known accuracy issues, but the latter seems less likely to raise some of the legal issues. |
Beta Was this translation helpful? Give feedback.
-
|
I'll split "write contributions" above into a few extra categories as I think this may also be useful for discussion purposes:
|
Beta Was this translation helpful? Give feedback.
-
|
@zacharysarah had shared this "AI Influence Level" system which would provide an indicator of the degree to which AI was used to create work. In an ideal world I think there'd be direct tooling integration to reflect this AIL metric automatically. Levels 0-5 indicating increasing levels of involvement by AI, where 3-5 are driven AI-first with human guidance. We could add an opt-in item into the PR template that asks "Which AI Influence Level was used to create this change?". |
Beta Was this translation helpful? Give feedback.
-
One idea is to require high context before reviewing a PR. That is to say, reviewers should do an initial 2m triage of the motivation for the PR before reviewing the work. If the context is insufficient, request context and move on. In cilium/cilium we often go through this pattern anyway just as a way to assist reviewer workload, since we already face similar problems with sharing the understanding between contributors and reviewers. |
Beta Was this translation helpful? Give feedback.
-
I don't have a full answer, here, but I have some thoughts. Your question highlights in particular a category of contributors: those who are unaware of the norms, expectations and policies of the project. On that note I think it would help if the project has a clear policy that states what is acceptable behavior with respect to the use of AI/LLM tooling so that we can take action on an objective basis as situations arise. As a broader discussion point, we should not assume any level of trust as a baseline from new contributors. I'll pull in references to the University of Minnesota debacle with the Linux kernel (TechCrunch coverage) and the xz-utils exploit (Ars Technica coverage) here for some examples of misplaced trust in the open source world. These are not AI/LLM examples, but they provide some broader context. We could consider taking inspiration around the handling of these incidents from these other projects. |
Beta Was this translation helpful? Give feedback.
-
|
Using LLMs to summarize code in order to make it easier to start working on Cilium 🧵 Someone shared with me a workflow where they interactively analyzed logic in the Cilium tree using some modern LLM-based tooling. This was not for drafting content changes for the project, but rather to inquire about how something works in Cilium and to locate the relevant code. For newer contributors who are not familiar with the Cilium architecture, how to navigate the codebase, or how to answer questions they have about how Cilium operates, it seems like this may provide a way to "rubber duck" and explore the codebase more easily. |
Beta Was this translation helpful? Give feedback.
-
|
I also used the cursor to help me contribute the actual code in my previous PRs. From my experience,
I personally feel the most helpful way to use it is to ask questions around how this works when you pointing to certain lines so you dont have to the keyword to search. To be honest, I started to pick up golang/programming this year and my personal feel is LLM sometimes can generate the better code than I can write. |
Beta Was this translation helpful? Give feedback.
-
|
I personally use Copilot regularly and I find it very helpful. I do think that it is important to clarify how I use it, which is essentially as a better form of auto complete (which I realize is the most basic way to use LLMs). It helps me type out verbose and predictable parts of my code such as function signatures, error handling and similar additions to code which I want to write already, but instead of doing it manually I press . I feel that the way I use tools like this does not effect the quality of my code, since I will always end up with what I had in mind. The same goes for copyright, concerns. Output is similar if not exactly the same to what I would have typed manually, just faster. LLMs are a tool, which like any other can be used for good and evil, it all depends on the user. I personally feel that outright banning LLMs is not the way to go. I would rather hold people accountable for all code they submit no matter how they made it. |
Beta Was this translation helpful? Give feedback.
-
|
This is my suggestion for a policy: To maintain the high quality and trustworthiness of contributions to the Cilium project, we provide the following guidance on the use of large language models (LLMs), such as GitHub Copilot, ChatGPT, or similar tools: Acceptable UseWe recognize that LLMs can be helpful tools, for example:
These uses are acceptable provided the contributor:
Unacceptable UseIt is not acceptable to:
Transparency & AttributionWe do not require contributors to declare when LLMs were used for basic productivity support. However, if LLMs were used to generate non-trivial text or code (e.g., new features, documentation pages, complex design proposals), contributors are encouraged to note that LLMs were used and explain the human review process applied. DCO and LicensingThe Developer Certificate of Origin (DCO) still applies. AI tools often lack transparency into their training data and may produce copyrighted output. If you submit AI-assisted code, you are certifying that it does not infringe on any licenses and that you have the right to contribute it under the project's license. If you’re unsure, don’t submit it. Guiding PrincipleCilium is a community-driven project that values expertise, clear communication, and personal responsibility. Whether or not an LLM was involved, contributors are fully accountable for the correctness, security, and clarity of their contributions. |
Beta Was this translation helpful? Give feedback.
-
|
Not sure if it would be too heavy handed, but we could also include something about blocking/banning if AI is continually misused. |
Beta Was this translation helpful? Give feedback.
-
|
I find the line for these specific items is a bit blurry:
First I'll consider non-generative approaches for autocomplete. I think that the general sense within the industry around traditional forms of autocomplete are that the templates and tables used to provide autocomplete information are reasonable for use in OSS code. We haven't even entertained the idea of banning such non-generative usage of autocomplete. So, I raise the question of how generative approaches to autocomplete are different? If it is a matter of auto-completing well-known templated forms of code such as a Kubernetes resource, that seems functionally very close to existing non-generative approaches. As the autocomplete becomes more sophisticated, I think that the line of responsibility between the contributor and the LLM becomes less clear in the moment of using the tool. I can imagine some common scenarios where the contributor may defer too far to the tooling:
I completely agree that ultimately the contributor should be held responsible for the contributions, and I do think that the text above accounts for this concern. At the same time, I worry that it is ultimately the reviewers that will be burdened with more work any time that the contributor has a lapse in responsibility regarding LLM use. From that perspective I would personally be in favor of an approach that discourages LLM use by default in creating content unless we are confident to delegate trust to the contributor. Regarding unit tests, I think this is one of the more problematic cases. It is very easy to slip into poor habits with writing unit tests of generating too much boilerplate and making the codebase harder to change because the tests are too tied to the implementation. There are many examples out there that already inhibit development, including a lot of cases within the Cilium codebase . All of these examples are now fed into LLM tools to generate yet more poorly considered unit testing logic. It is also very common that for each line of "business logic" there are multiple lines of testing logic, which exacerbates review workloads. |
Beta Was this translation helpful? Give feedback.
-
What is "basic productivity support"? As a reviewer, I want to see an honest answer to "Is any of the content I review generated by an LLM?". I don't care if you used LLMs to search for information or to summarize/explain some code during design / analysis (in this case, answer "no"), but if an LLM was used to create any of the code, writing, etc. that are part of the submission, I want to be aware. The next step is "Recognizing that I have used an LLM to create some of the content of this submission, I have reviewed every line and every word of this submission to ensure it meets the project guidelines". I want this to be the ambition even for submissions not generated with LLM tooling. I think that the last part of the suggested policy above covers this point. Maybe one practical point I might add to the existing text is "If in doubt, label your submission as containing LLM-generated content". |
Beta Was this translation helpful? Give feedback.
-
|
I don't quite understand why documentation and unit tests would get exceptions. They are as tricky to write properly as the main logic itself and they are also susceptible to all of the LLM issues Joe mentioned. |
Beta Was this translation helpful? Give feedback.
-
I would even to go as far and require this. I know this is hard to enforce, but I think without making declaration mandatory, it will be hard to gauge if the author even used LLMs, and if so followed the "acceptable use" (i.e. only used LLMs to produce code in areas they understand). As a reviewer, it is both time-consuming and frustrating trying to understand the reasoning behind a piece of odd looking code, only to realize that there is no reason or logic behind it. If the code is LLM-generated, then I have no interest in suggesting changes to auto-generated non-sense line by line. However, if the author did not use LLMs and perhaps is a junior developer (everyone makes mistakes after all), I'm more than happy to provide in-depth feedback on how certain subsystems work, how to improve things and link to learning resources. Having the author declare that they used LLM to generate a certain section of the code (e.g. Helm boilerplate) helps me as a reviewer to disengage early if it becomes clear that code doesn't make sense, indicating that author didn't follow acceptable use. |
Beta Was this translation helpful? Give feedback.
-
|
Contributors must take responsibility for the full content they submit in a PR 🧵
The first and last points are a bit tricky here: I agree to some degree around the sentiment, but I think it's not a very practical position to take. For the sake of discussion, let's take LLM tooling out of the picture and just consider contributions more generally. It is somewhat common that someone does not fully understand the codebase or their own contribution. They may not be able to fully explain / justify the code. Part of the point of the review process is to identify gaps where the contributor may lack context or may not fully understand the project and have a discussion about the abstractions, shape of the problem, possible solutions and so on. Although I think it would be ideal if everyone had a full picture of what changes need to be made and could explain the contribution all the way, this is a high bar to set. We need to recognize that there will be contributors with different experience levels and we'll need to meet those contributors where they're at. Ideally that interaction will develop the contributions to the point that they generally improve the project and do not create significant problems, and while we're at it, provide a positive experience for the contributors to help motivate them to continue to contribute in a healthy way. The most practical response to my feedback above would be to loosen the language a little bit to set expectations that when you submit a change, you are trying to improve something in the project to the benefit of the community, and that you will honestly engage in the review process to the best of your ability. Something I would be happy to see written as unacceptable would be to use LLMs as part of review discussion.
We've already seen people violate this without even knowing. We can state that this is the policy, but I think we have to recognize that if we set expectations that LLM usage is generally OK, there will be submissions in this category that everyone will miss. Maybe if we're lucky that will not cause an incident, but I have very little interest in having to deal with such an incident because we set a policy that is too liberal. I guess to summarize.. I agree with this sentence, but I think the only practical way to enforce it is to ban non-trivial use of LLMs (probably, anything more than a line or two here or there). |
Beta Was this translation helpful? Give feedback.
-
|
Contributors should not use LLMs to engage in pull request discussion 🧵 via @xmulligan
I'd like to share another experience I've had in the Cilium community recently. A new contributor submitted a pull request to fix a "good first issue". When I reviewed the submission, I asked a specific technical question and provided a reference link to what I was talking about so that the contributor could understand the basis of my question. I was curious to learn about the technical solution, but I also wanted to check whether the contributor had experienced the issue they were fixing and whether they were doing a drive-by contribution or whether there is potential to mentor the contributor to become a regular contributor. The contributor responded to my question with some text that was clearly generated by an LLM. The response did not make sense, and it did not match the communication style that the contributor used elsewhere in the PR. I asked the contributor whether they used an LLM to generate the answer as the response was very clearly not written by them. The contributor responded in their own voice that they had not used an LLM. That is to say, they directly lied in response to my question. We currently do not have a policy regarding LLM usage, so if the contributor acknowledged usage and engaged in discussion directly then I think we might have been able to navigate the pull request process. However, I think now it is a matter for the code of conduct. I feel that there is a trend recently where developers are trying to generate a good-looking CV/Resume by flooding OSS projects with low-quality submissions. These contributors attempt to get involved in the project not for the benefit of the community but only for their own enrichment. As we are starting to see abuse of the Cilium community of this kind, I would like to see clear guidance in place that highlights that such behavior is unacceptable, and provides guidance to the maintainers and committers about how to respond to such incidents. I recognize that this incident is not caused by the use of LLM, but that LLM tooling lowers the cost for abusive behavior in OSS communities. From my perspective if the project has a blanket rejection on accepting content created using LLM tools, then the line is crystal clear about whether such submissions cross the line and I could point to the policy and close the PR. |
Beta Was this translation helpful? Give feedback.
-
|
It seems likely that a lot of folks will be using tools like Copilot for "fancy autocomplete" (as Dylan & Quentin pointed out). But what if we ban LLM-generated text in GitHub issue/PR text and comments? (As well as a requirement to state if you used LLM in the coding). That way the contributor would have to describe their change and interact with the community in their own voice rather than that of an LLM |
Beta Was this translation helpful? Give feedback.
-
|
For what it's worth, I find that Liz's proposal is not unreasonable. I could imagine some gain of productivity by using LLMs to write commit descriptions, although I've not really used it so far; but this is an area where I'd be ready to make a compromise, and accept a policy that says “no generated content in commit/PR description”, for the sake of clarity. Comments in discussions on GitHub Pull Requests' threads are a different thing, I don't think AI-generated have their place in there at all (unless very clearly indicated, such as marking some snippet as a quote and telling where it comes from to illustrate a technical argument, or having some clearly-identified bot take part of the conversation such as Copilot reviews, were the project to use that). |
Beta Was this translation helpful? Give feedback.
-
|
I've definitely seen comments and PR text that smelled strongly of being completely AI-generated. Both the formatting and the content felt very mechanical. |
Beta Was this translation helpful? Give feedback.
-
|
Are we trying to define the "Turing Test"? I would be in favor of accepting anything regardless of its origin as long as the intent of the PR is clear, no low effort (typos etc), no huge 'automatic' refactoring like change |
Beta Was this translation helpful? Give feedback.
-
|
I agree that intent and quality are important components to this discussion. However, I also think that LLMs create materially different properties in the contribution process. Most prominently, LLMs have been trained on a wide range of content without respect for consent, attribution or legal process. LLMs will happily copy significant content that has been made available under some license, violate the license and hide the attribution. I think this creates legal and ethical risks for any project that accepts nontrivial contributions built using generative AI tools. |
Beta Was this translation helpful? Give feedback.
-
|
To be honest, I agreed on this one. In last few months, I started to use cursor to help me understand the codebase and write the PR. I'm new to golang and cilium codebase, and I personally feel LLM helps me to get a better quality of my PR. Sometimes, the boundary of understanding, suggesting and writing the code can be blurry. I also have to admit that LLM lower the bar for people to raise the PR but I think have an extra check box doesn't help. People can always lie to that check box. |
Beta Was this translation helpful? Give feedback.
-
|
I honestly don't think this one is too bad either https://github.com/agentgateway/agentgateway/blob/main/CODE_OF_CONDUCT.md |
Beta Was this translation helpful? Give feedback.
-
|
There's similar ideas in this blog post: https://distantprovince.by/posts/its-rude-to-show-ai-output-to-people/ . |
Beta Was this translation helpful? Give feedback.
-
|
Additional project proposals around AI usage: |
Beta Was this translation helpful? Give feedback.
-
|
Airflow has an AGENTS.md https://github.com/apache/airflow/blob/main/AGENTS.md "Somehow I have a feeling that quality of the contributing docs is nowadays super important - precisely because the AI agents that people use actuallly read those instructions - as opposed to many humans who often did not read instructions before contributions." |
Beta Was this translation helpful? Give feedback.
-
|
I also quite like this one from OTel https://github.com/open-telemetry/community/blob/main/policies/genai.md |
Beta Was this translation helpful? Give feedback.
-
Seems like Harbor has an https://github.com/goharbor/harbor/pulls?q=is%3Apr+author%3AArvindhworks+is%3Aclosed |
Beta Was this translation helpful? Give feedback.
-
|
Labeling something as slop certainly evokes certain emotions. As carthartic as that may be, I don't think it sets the right tone for participation in a project. That said, labeling something as containing generative AI content could be useful. For any given label, it allows people with write privileges to communicate a property of the labelled PR/issue, and additionally later filter issues for similar matches. If we were to have a label for content that we think either may or does include LLM generated content, what would we use that label for? Ideas:
|
Beta Was this translation helpful? Give feedback.
-
|
The git Summer of Code AI guidelines are pretty well written and clearly outline many of the aspects of the problem space: https://git.github.io/General-Application-Information/ |
Beta Was this translation helpful? Give feedback.
-
|
I quite like the way that RedHat frames the issues here https://www.redhat.com/en/blog/ai-assisted-development-and-open-source-navigating-legal-issues |
Beta Was this translation helpful? Give feedback.
-
|
I can't agree with the plagiarism and DCO aspects here. I think the argument is "I haven't seen it so it doesn't exist". As I documented above, we have seen LLM tools directly take entire PRs from GitHub and recommend to a naive contributor to post it as their own. |
Beta Was this translation helpful? Give feedback.
-
|
Saw this elsewhere and I agree with the sentiment: Our big concern right now is AI-led contributions, not the AI-assisted ones. We have tons of people who see our issues and think they can just delegate 99% of the work to the AI and arrive at a good result. |
Beta Was this translation helpful? Give feedback.
-
|
I kind of like this line "Human review is required for all code that is generated or assisted by a large language model." |
Beta Was this translation helpful? Give feedback.
-
|
I like how Zulip focuses on the why not the how and succinctness https://github.com/zulip/zulip/blob/main/CONTRIBUTING.md#ai-use-policy-and-guidelines |
Beta Was this translation helpful? Give feedback.
-
|
This is a particularly good sentence, I think:
|
Beta Was this translation helpful? Give feedback.
-
|
Zulip have just updated the policy with a new section targeted at LLM misinformation propagated by unwitting contributors: 
|
Beta Was this translation helpful? Give feedback.
-
|
I actually really like the way the Zulip policy focuses on the contributor and uses the imperative form to directly instruct contributors how they should interact with the project in context of GenAI/LLMs. |
Beta Was this translation helpful? Give feedback.
-
|
Python splits it clearly into "acceptable uses", "unacceptable uses" and "considerations for success": |
Beta Was this translation helpful? Give feedback.
-
|
In light of security incidents like camoleak, I think we should strongly recommend against committers from using generative AI tools. The lack of determinism in the tooling combined with classic security bugs presents a risk that is greater than the benefits of the tooling. If committers take responsibility for those risks, they must have at least one secondary deterministic control to limit the risk. For instance, when granting GitHub API access, create a token with read-only permissions only on the required repositories. I note that while this would mitigate problems associated with write tokens, it could still leak content of private repositories through read access. It would also be very easy to misconfigure this to grant too much access, so this should not be taken lightly. If the tooling doesn't provide secondary controls, the tooling must not be used. Any actions undertaken by generative tooling must be reviewed by the user of those tools (for instance if generating text, the contributor must review the content). While the camoleak exploit has been patched by GitHub, I don't currently see the ability to configure secondary controls in GitHub Copilot today, so I think we should avoid all Copilot use through the GitHub website. |
Beta Was this translation helpful? Give feedback.
-
|
Overall like a lot of the sentiment in this write up |
Beta Was this translation helpful? Give feedback.
-
|
Could be an interesting talk here too |
Beta Was this translation helpful? Give feedback.
-
|
Relevant issue: cilium/cilium#43656 |
Beta Was this translation helpful? Give feedback.
-
|
Not a bad take either ghostty-org/ghostty#10412 |
Beta Was this translation helpful? Give feedback.
-
|
I like this one, it feels like quite a good fit with our "trusted group of committers" model. |
Beta Was this translation helpful? Give feedback.
-
|
Seems like we are starting to coalesce on some themes. I'll ask to add this to the Cilium Dev Summit to discuss there. Maybe we can come up with a policy we can all agree on later this year. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
We currently have no written policy around the use of generative AI in code contributions. We may want to establish a policy that does things like require contributors to disclose when AI tooling was used, holding AI-assisted submissions to the same standard of originality, quality, and attribution as any other contribution,
and encourages contributors to learn, understand, and reason through the code they're submitting, rather than deferring entirely to tooling.
AI tooling in general can be a useful accelerant but if a contributor cannot explain the code they’re submitting, or isn’t able to reason through feedback, that’s a red flag.
A few things we might want to consider when developing:
How do we ensure contributors appropriately credit prior work, whether human- or AI-generated?
Should we require contributors to disclose if AI tools were used? If so, how do we define acceptable usage?
How do we handle low-context, potentially AI-generated PRs that add to the review burden and may reflect limited understanding of the codebase?
How do we handle cases where trust is broken through copy-paste or AI-mediated submissions, particularly when the contributor appears early in their OSS journey?
Beta Was this translation helpful? Give feedback.
All reactions