Document changing host for rotating-url and assume same version based only on checksum and file size

[TuxCoding]

Description

Hey there,

Motivation

The Flatpak projects for Discord, War Thunder seen multiple PR for the same version number, because of a changed URL. This behavior could indicate load-balancing. The hash and version number stays the same, which you can see in the referenced PRs.

-        url: https://cdnnow-distr.gaijinent.com/wt_launcher_linux_1.0.3.15.tar.gz
+        url: https://aws-yup-distr-02.gaijinent.com/wt_launcher_linux_1.0.3.15.tar.gz
         sha256: c89e4a41185bfaab634a978a0eac9752c42fea1afb6c094692063ceb40760e15

Changes

Documentation

in this PR I added some documentation on how to use non-capturing groups to allow the pattern to match despite the changing host. The use of non-capturing groups then doesn't cause conflicts with the version extraction of the first capturing version group.

urlchecker

I adjusted the is_update parameter to compare the version numbers if they could be extracted to reflect that it's still the same version. Nevertheless, this change doesn't stop this project from reporting a new update without the changes below.

externaldata

I dropped the URL comparison in ExternalFiles to rely only on checksum and file size to report identical files. Without this change matches and is_same_version both uses the URL and therefore report False and program flow goes into self.new_version = new_version and so reports ... got new version although the version is still up to date.

flatpak-external-data-checker/src/lib/externaldata.py

Lines 284 to 290 in a01c775

	elif is_update:
	log.info(
	"Source %s: got new version %s", self.filename, new_version.version
	)
	self.state |= self.State.OUTDATED
	self.new_version = new_version

Alternatives

The changes mentioned will affect all sources that uses ExternalFile to rely only on checksum and size. An alternative would be to modify ExternalFile.is_same_version which uses url compares too, which seems to be a better based on the name. However, this wouldn't work for AppImages, because then we would need to download the old/current version again to extract that version number.

Testing

I used Discord's project for validation with the following changes:

[...]
                    "x-checker-data": {
[...]
                        "pattern": "https://(?:dl|stable.dl\\d).discordapp.net/apps/linux/([0-9.]+)/discord-([0-9.]+).tar.gz"
                    }
[...]

Related

• rotating-url checker should respect input pattern #438
• Rotating URL does not respect pattern #373

Fixes #438

[bbhtt]

I guess this works, but I think checking that the final URL matches the pattern would be better?

Also can you also clean-up the commit history (squash the formatting commit) and rebase to main?

[bbhtt]

Looks like you updated your GitHub username and thus the email. I changed it to your current ones to get the authorship correctly rendered.