Some Commits not appearing in issue data

[Leo-Send]

@bockthom discovered that some commit_added events are missing in the final issue data. After looking into it, I found out that this is the result of the author post-processing, where all commits identified as being authored by github are removed. It turns out, that all commits committed by a bot provided by github, such as dependabot, are represented as being authored by github in our data. Github is, however, in this case only the user, and not the author. Therefore, to my understanding, it is not intended to remove these commits.

There are multiple ways to fix this:

1. Simply change what is written into the issue data file, which is later read again in the author post-processing. This can be done with a ternary expression in the print_to_disk function. Here is the Python 3 code that solves the issue after upgrading the file from python 2 to python 3 (lines 811-812):

event["commit"]["author"]["name"] if event["event"] == "commit_added" else event["user"]["name"],
event["commit"]["author"]["email"] if event["event"] == "commit_added" else event["user"]["email"],

This would solve the problem and would leave everything else as is, but does not remove the github user from the data - if the skript is ever extended, this problem may occur again.

2. In merge__issue__events, change the user field of all commit_added events. This would result in the field being correct from the beginning, but would also remove the original user before it is added to the user dictionary or the function insert__user__data is called.

3. In reformat__events, change the user field of all commit_added events. This would retain the data in the user dictionary, but this would still be before insert_user_data is called.

Since I can't run the skript using a codeface database, I can not estimate the impact of not having the correct user for insert_user_data.
Also, all of these points do not specifically target commit_added events made by a github bot - they all would change the listed user behind a commit_added event to the author of the commit for all commit_added events (though a simple if statement could theoretically change that.. ). I am unsure what would be the correct and intended behaviour here.

[bockthom]

Thanks @Leo-Send for digging deeper into this issue.

I guess all three solutions would make sense, but let's discuss them in our meeting tomorrow. Let's pick an example and orally discuss how each of the three solutions would affect the user database.

Also, all of these points do not specifically target commit_added events made by a github bot - they all would change the listed user behind a commit_added event to the author of the commit for all commit_added events (though a simple if statement could theoretically change that.. ). I am unsure what would be the correct and intended behaviour here.

That depends. In general, we don't only care about bots but also potential other cases in which user and author are different. Could you maybe figure out whether there are cases in your example data in which user and author differ? Then we can discuss them tomorrow. Maybe we can also have a look at commits that have more than one author? In many projects that use squash merging, it could be that there are multiple "authors" for a commit - for example this one:
igraph/rigraph@661cb38
(but this may appear also in projects for which you already have the data). Could you please check how the multiple authors that are mentioned for this commit in GitHub's web interface are internally stored in the REST API?

Based on such examples, we can orally discuss the impact of the different suggested solutions on the expected output.