Add User-Agent header to API requests

[twsl]

Without providing a user agent, cloudflare, which runs in front of my gramps web instance, returns an error 1010 which results in a 403 access denied

[DavidMStraub]

Did you check this actually works? If so, I wonder why.

The official Python documentation states

headers should be a dictionary, and will be treated as if add_header() was called with each key and value as arguments. This is often used to “spoof” the User-Agent header value, which is used by a browser to identify itself – some HTTP servers only allow requests coming from common browsers as opposed to scripts. For example, Mozilla Firefox may identify itself as "Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11", while urllib’s default user agent string is "Python-urllib/2.6" (on Python 2.6).

So even without setting a user agent, the library will send one. So just setting this to any non-empty value cannot be the solution. It could be that Cloudflare explicitly rejects user agents starting with Python-urllib. In that case, your solution might work. What is more likely (IMO) is that it rejects all non-browser user agents. What we definitely don't want to do is to spoof a browser.

See e.g. a similar issue with Nextcloud sync: nextcloud/desktop#7070

[twsl]

I ran the code block for token retrieval quickly in command line and it worked. Initially it worked in gramps without user agent as well but at some point I started getting the 403 error message despite using the same password.
I wasn't aware urllib sets a user agent internally, I am mostly working with requests package and set one explicitly.
Since issues are disabled I figured I just post a PR as proposed solution and start a discussion this way. Obviously I'm fine with using a browser specific user agent as well, but I think this is specific to urllib header. Curl works as well

[DavidMStraub]

I mean, it doesn't hurt to add this, so if it helps you, I think it's ok. But I'd like to understand better why this actually works - so if CF is explicitly blocking the default agent.

[twsl]

A quick google search suggest that the Python-urllib/x.x is the culprit, but information is rare as Cloudflare doesn't publish anything about their bot detection. Setting a dedicated user agent seems like a reasonable thing to do regardless.
We could consider adding the plugin version number as well

[DavidMStraub]

Ok in that case I'm ok to merge it. Let me know if you want to modify anything else. I don't think version number is easily doable, we don't wanna have to keep it in sync manually

[twsl]

Moving the plugin version from the grampswebsync.gpr.py to const.py and then import it in grampswebsync.gpr.py and webapihandler.py should work. however I'm fine to just merge this as a hotfix as is

[GaryGriffin]

Moving the plugin version from the grampswebsync.gpr.py to const.py and then import it in grampswebsync.gpr.py and webapihandler.py should work.

Not sure I follow. The plugin version belongs in the gpr. It gets auto-updated when a new version is published, so no need to make that change in the PR. In this case, the current plugin version is 1.3.6 . Once the PR is merged/published, it will be 1.3.7 .

[twsl]

I wasn't aware this is generated. Then I'd say this is good to go

[DavidMStraub]

I can't formally submit a review, but I approve the changes.