doc: create ai-guidelines and include to CONTRIBUTING#62105
doc: create ai-guidelines and include to CONTRIBUTING#62105RafaelGSS wants to merge 4 commits intonodejs:mainfrom
Conversation
Co-Authored-By: Beth Griggs <bethanyngriggs@gmail.com>
|
Review requested:
|
|
There may be some ideas we can borrow from https://llvm.org/docs/AIToolPolicy.html - for example "good first issue" should not be picked up by AI is a good one. |
Co-authored-by: Aditi <62544124+Aditi-1400@users.noreply.github.com> Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
I took inspiration from https://github.com/zulip/zulip/blob/main/CONTRIBUTING.md#ai-use-policy-and-guidelines |
| * **Verify accuracy** of any LLM-generated content before including it in a | ||
| PR description or comment. | ||
| * **Complete pull request templates fully** rather than replacing them with | ||
| LLM-generated summaries. |
There was a problem hiding this comment.
Do we have a template? I thought those are for issues, not PRs.
There was a problem hiding this comment.
Not strictly a template: https://github.com/nodejs/node/blob/main/.github/PULL_REQUEST_TEMPLATE.md?plain=1
There was a problem hiding this comment.
It's not possible to fulfil the instructions "Complete pull request templates fully" based on the contents of https://github.com/nodejs/node/blob/main/.github/PULL_REQUEST_TEMPLATE.md?plain=1 so it looks like this sentence needs to be removed.
Co-authored-by: Tobias Nießen <tniessen@tnie.de> Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com> Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> Co-authored-by: Efe <dogukankrskl@gmail.com>
There was a problem hiding this comment.
I'd be against this contribution policy update. While many different opinions exist on the licensing terms of the code produced by LLMs, my opinion is that the generated code isn't explicitly licensed and attributed to the original authors so it cannot be considered open source regardless of the used prompt.
| * **Verify accuracy** of any LLM-generated content before including it in a | ||
| PR description or comment. | ||
| * **Complete pull request templates fully** rather than replacing them with | ||
| LLM-generated summaries. |
There was a problem hiding this comment.
| * **Verify accuracy** of any LLM-generated content before including it in a | |
| PR description or comment. | |
| * **Complete pull request templates fully** rather than replacing them with | |
| LLM-generated summaries. | |
| * **Verify accuracy** of any LLM-generated content before including it in a | |
| PR description or comment. |
|
In #61478 (comment) , regarding the usage of Claude Code, @mcollina suggested:
I added it to the TSC agenda tomorrow for awareness/context collection before moving to a proper vote. @indutny sorry about the short notice since this is just one day ahead of the meeting, but if you'd like to join the meeting to present your points please let us know. |
mcollina
left a comment
There was a problem hiding this comment.
lgtm with a sentence removed
|
@joyeecheung thanks for considering me for this! I'd be happy to join if the time isn't in conflict with my work meetings tomorrow. Could you send me an invite, please? |
|
nodejs/TSC#1830 this is the issue. It might be reschedueld to the next meeting if there is low attedance given the timezone. I think we can schedule the discussion for April 1st which will be in the morning PT so a lot of people can join, and I would table the vote for that session. I would try to get an answer from the Board by then. |
|
Let's make sure it's on the agenda for a time more convenient for the majority of TSC members. That said, I can say right now that I'm -1 on blocking this on those grounds. |
|
I've updated the agenda tomorrow to mark it FYI (as in, it doesn't need to be discussed this week, but worth taking a look and do some homework before discussions happen). In any case, decisions won't be made in meetings, and this likely needs a proper vote. |
| * **Own every line you submit.** You are responsible for all code in your | ||
| pull request, regardless of how it was generated. Be prepared to explain | ||
| any change in detail during review. |
There was a problem hiding this comment.
| * **Own every line you submit.** You are responsible for all code in your | |
| pull request, regardless of how it was generated. Be prepared to explain | |
| any change in detail during review. | |
| * **Own every line you submit.** You are responsible for all code in your | |
| pull request, regardless of how it was generated. This includes ensuring | |
| that AI-generated or AI-assisted contributions satisfy the project's | |
| [Developer's Certificate of Origin][] and licensing requirements. Be | |
| prepared to explain any change in detail during review. |
(would need a DCO link added at the bottom of the page also)
|
[from vote on AI and in context of quoted there PR] If someone uses an advanced tool, like LLM, should everyone have a higher baseline for expectations? Take a look at initial review in github.com/#61478 , looks like begging to use node's style for ... core node. This tool, LLM, is supposed to do things in style. Instead there was a re-implementation of path.normalize() ? User of an advanced tool has responsibility to provide more advanced result. Of course, in coding, LoC count is not a metric for being better. For my 25 years, usage of more advanced tooling brought better results. Should LLM's code be considered a starting point, not a ready PR? May I note, as user of node.js, an unhealthy dynamic regarding this, and a need to frame constructive approach, also as an example for whole industry. nodejs/TSC#1831 (comment) is opened with, quote: "In order to surpass the blocks of ..." as if we are here to score social points, using heavy words. Please, snap out of it. We need to re-frame topic in nuances that matter to code. Otherwise, we either get more hack-able artifacts, or, throw away baby with the bath water. And you all here seem to have enough emotional shrapnel to swing either way. Please don't. For the sake of us, your users. |
|
|
||
| ## Core principle | ||
|
|
||
| Node.js expects contributors to understand and take full responsibility for |
There was a problem hiding this comment.
| Node.js expects contributors to understand and take full responsibility for | |
| Node.js requires contributors to understand and take full responsibility for |
There was a problem hiding this comment.
sorry for resolve, hit the wrong button in UI.
If required, what's the consequence for failing this requirement? We've implemented so many policies without consequence and had to correct that later, we shouldn't do that here.
If this is merged (I'm unconvinced it should be), we have to define consequences.
There was a problem hiding this comment.
It seems like the obvious consequence is that the contribution will not be accepted.
| ## Core principle | ||
|
|
||
| Node.js expects contributors to understand and take full responsibility for | ||
| every change they propose. The answer to "Why is X an improvement?" should |
There was a problem hiding this comment.
| every change they propose. The answer to "Why is X an improvement?" should | |
| every change they propose. The answer to "Why is X an improvement?" can |
| AI tools may assist contributors, but must not replace contributor judgment. | ||
| When using AI as a coding assistant: |
There was a problem hiding this comment.
| AI tools may assist contributors, but must not replace contributor judgment. | |
| When using AI as a coding assistant: | |
| Contributors may use AI tools to assist with contributions, but such tools | |
| never replace contributor judgement. | |
| When using AI as a coding assistant: |
| Node.js values concise, precise communication that respects collaborator time. | ||
|
|
||
| * **Do not post messages generated entirely by AI** in pull requests, issues, or the | ||
| project's communication channels. |
There was a problem hiding this comment.
I'm less convinced this one is necessary. It's also difficult to enforce. These should follow the same rules as contributions... whatever is posted, you're responsible for, so use appropriate discretion.
There was a problem hiding this comment.
It's only difficult to enforce if it's low quality prose, though, in which case this item seems like it'd be needed.
There was a problem hiding this comment.
I think this also disenfranchises people that use an AI to help them write English, as it might not be their first language
| contributor has not personally understood, tested, and verified might be closed | ||
| without review. |
There was a problem hiding this comment.
| contributor has not personally understood, tested, and verified might be closed | |
| without review. | |
| contributor has not personally understood, tested, and verified will likely be closed | |
| without review. |
|
|
||
| ## Using AI for code contributions | ||
|
|
||
| AI tools may assist contributors, but must not replace contributor judgment. |
There was a problem hiding this comment.
| AI tools may assist contributors, but must not replace contributor judgment. | |
| AI tools may assist contributors, but must not replace human judgment. |
| * **Understand the codebase first.** Do not skip familiarizing yourself with | ||
| the relevant subsystem. LLMs frequently produce inaccurate descriptions of | ||
| Node.js internals — always verify against the actual source. When using an AI | ||
| tool, ask it to cite the exact source files/PRs/docs it’s relying on, and then |
There was a problem hiding this comment.
| tool, ask it to cite the exact source files/PRs/docs it’s relying on, and then | |
| tool, ask it to cite the exact source it’s relying on, and then |
|
|
||
| ## Using AI for communication | ||
|
|
||
| Node.js values concise, precise communication that respects collaborator time. |
There was a problem hiding this comment.
| Node.js values concise, precise communication that respects collaborator time. | |
| Node.js values concise, precise communication that respects collaborator and contributor time. |
| * **Link to primary sources** — code, documentation, specifications — rather | ||
| than quoting LLM answers. |
There was a problem hiding this comment.
| * **Link to primary sources** — code, documentation, specifications — rather | |
| than quoting LLM answers. | |
| * **Link to primary sources** — code, documentation, specifications — rather | |
| than quoting LLM answers or linking to LLM chats. |
| feedback and iterate until the work lands or is explicitly closed. If you | ||
| can no longer pursue it, close the PR. Stalled PRs block progress. | ||
|
|
||
| * **Edit generated comments critically.** LLM-produced comments are often |
There was a problem hiding this comment.
I would recommend adding an extra rule to not push "linter" changes that LLM usually does, it's pretty frequent at meteor, and dirty the commits and PR history
|
Given this is receiving a lot of attention, I had AI prepare a summary of all discussions within the Linux Kernel: https://gist.github.com/mcollina/8a4f2ee2e64d38edb90760016e89f919. I don't see anything more critical than the Linux Kernel, and it's also the flagship project of our parent Foundation. Given that this proposal follows that concept and is currently blocked on a similar basis, I think documenting our position on what has already been debated elsewhere is relevant. On a more practical note, Chrome/Chromium (which V8 is part of) allows AI assistance. As a result, we are likely already including code developed with the assistance of AI in our tree. |
|
@indutny can you please send a PR to add youself back to active contributors and use the “request chages” button in both PRs? Thanks. |
|
Hi all, I wanted to weigh in as a community member and former Node.js core contributor in favor of these proposed guidelines. And, in particular, to say that it's important that Node.js not try to instate a blanket ban on AI-assisted contributions, as I've seen some calls for. A lot of large open-source projects, including ones I've worked on like Chromium, V8, jsdom, and Undici, are seeing huge benefit from AI. This is clearest when the AI is wielded by core contributors working in their element, and any policy which prevented core contributors from taking advantage of AI uplift would hurt the project a lot. But it's also important for a project that wants to stay relevant and welcome outside contributors. I know of at least one large open source project, Servo, which has lost long-time community contributors due to their anti-AI stance. When a modern, AI-using developer wants to contribute to a nascent browser engine, and they have a choice between Servo (AI-unfriendly) and Ladybird (AI-welcoming), they choose Ladybird. If Node.js were to ban AI assistance, I think it would find itself in a similar situation with regard to other competing runtimes. I fully acknowledge that AI-generated contributions have created new problems and there needs to be some reckoning with it. I like the Rust community survey's common ground section for summarizing these issues. The guidelines proposed here seem like a good framework to start with, but surely evolution will occur over time. Regardless, I'm heartened to see this PR as a starting point, for establishing clearly that while AI assistance needs to be treated with care, it's also very much a part of modern software development. Thanks! |
which long-time community contributors were those? as far as i’m aware, even the most vocally AI-positive contributors in Servo have continued doing great work on the project. |
|
|
||
| ## [AI Use Policy and Guidelines](./doc/contributing/ai-guidelines.md) | ||
|
|
||
| Node.js expects contributors to understand and take full responsibility for |
There was a problem hiding this comment.
this first sentence seems intentionally to be the same as the full policy document, which now has a suggested change. marking this here to resolve whether or not the language should continue syncing, if it should change
| be removed or modified without human verification. Do not rely on the LLM | ||
| to assess correctness. | ||
|
|
||
| * **Do not disappear.** If you open a PR, follow it through. Respond to |
There was a problem hiding this comment.
What is / can we link to our official policy on "stale" or stalled PRs? Like regardless if its involving AI do we have a policy to close out PRs after a certain amount of time? If not, should we?
There was a problem hiding this comment.
Right now our automation closes PRs if they are both older than a year and have had no activity in the past sixth months, but IMO we should make that just six months (no year clause)
|
@RafaelGSS Can you please add a link to https://openjsf.cdn.prismic.io/openjsf/aca4d5GXnQHGZDiZ_OpenJS_AI_Coding_Assistants_Policy.pdf (official AI policy)? This document already aligns for the most part, which should mention the |
As discussed in today's TSC meeting.
cc: @nodejs/tsc @BridgeAR