aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
Hi aur-general,

as a maintainer of fifty-something AUR packages I found it difficult
to keep track of new upstream versions. I would subscribe to various
RSS feeds to keep track of some.

Recently, I sat down and wrote a tool "aur-out-of-date" which, given a
username or a list of packages, checks upstream for newer versions. It
does so by inspecting the upstream URL and/or source URL, and calling
a suitable API (github.com, registry.npmjs.org, pypi.python.org,
cpan.org for now) to determine the latest released version.
Out-of-date packages are output in red color on the command line.

Give it a try:
https://github.com/simon04/aur-out-of-date
https://aur.archlinux.org/packages/aur-out-of-date/

I'm happy to hear you feedback :)
Simon
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

Michael Kogan
Very nice and useful! It works for me, though just a few of my PKGBUILDs
are displaying status information. I am wondering, what about PKGBUILDs
whose source is hosted at Launchpad or Sourceforge? Those are quite common
hosters, are there plans to add them? Thanks!

2018-02-01 17:08 GMT+01:00 Simon Legner via aur-general <
[hidden email]>:

> Hi aur-general,
>
> as a maintainer of fifty-something AUR packages I found it difficult
> to keep track of new upstream versions. I would subscribe to various
> RSS feeds to keep track of some.
>
> Recently, I sat down and wrote a tool "aur-out-of-date" which, given a
> username or a list of packages, checks upstream for newer versions. It
> does so by inspecting the upstream URL and/or source URL, and calling
> a suitable API (github.com, registry.npmjs.org, pypi.python.org,
> cpan.org for now) to determine the latest released version.
> Out-of-date packages are output in red color on the command line.
>
> Give it a try:
> https://github.com/simon04/aur-out-of-date
> https://aur.archlinux.org/packages/aur-out-of-date/
>
> I'm happy to hear you feedback :)
> Simon
>
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
In reply to this post by tur-users mailing list
On 02/01/2018 11:08 AM, Simon Legner via aur-general wrote:

> Hi aur-general,
>
> as a maintainer of fifty-something AUR packages I found it difficult
> to keep track of new upstream versions. I would subscribe to various
> RSS feeds to keep track of some.
>
> Recently, I sat down and wrote a tool "aur-out-of-date" which, given a
> username or a list of packages, checks upstream for newer versions. It
> does so by inspecting the upstream URL and/or source URL, and calling
> a suitable API (github.com, registry.npmjs.org, pypi.python.org,
> cpan.org for now) to determine the latest released version.
> Out-of-date packages are output in red color on the command line.
>
> Give it a try:
> https://github.com/simon04/aur-out-of-date
> https://aur.archlinux.org/packages/aur-out-of-date/
>
> I'm happy to hear you feedback :)
> Simon
This looks quite cool!

2 slight questions:

1) How does this handle PKGBUILDs that, say, have their homepage on
github, but do not tag releases there -- because :( :( :( -- and instead
push releases to PyPI or npmjs or something?

2) It looks like your AUR package is downloading a source tarball, then
completely ignoring it by using go get and downloading the bleeding-edge
master. I'm not really sure how this crazy language handles packaging :p
but I think *maybe* you should be cloning
git+https://github.com/simon04/$pkgname.git#tag=v$pkgver and symlinking
"$srcdir/$pkgname" --> "$srcdir/src/github.com/simon04/aur-out-of-date/"

You should also probably add github.com/mikkeloscar/gopkgbuild/ and
github.com/mikkeloscar/aur to your sources.

I've also heard that https://github.com/golang/dep is meant to somehow
solve all this.

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
In reply to this post by Michael Kogan
Hi,

On Thu, Feb 1, 2018 at 5:20 PM, Michael Kogan <[hidden email]> wrote:
> I am wondering, what about PKGBUILDs
> whose source is hosted at Launchpad or Sourceforge? Those are quite common
> hosters, are there plans to add them? Thanks!

I see the following difficulties with Launchpad or Sourceforge (as a
infrequent user I might be wrong):
- Sometimes, more than one software/package is released within a
project, such as [1]
- The files are listed in chronological order, but there's no clear
concept of the latest release

Since only few of my packages are hosted there, extracting version
from there is not high priority for me. However, if you came up with a
PR or a way to reliably determine the latest release, I'd happily
include that.

Simon

[1] https://sourceforge.net/projects/sevenzip/files/
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
In reply to this post by tur-users mailing list
Hi,

On Thu, Feb 1, 2018 at 5:44 PM, Eli Schwartz via aur-general
<[hidden email]> wrote:
> This looks quite cool!

Thanks :)

> 1) How does this handle PKGBUILDs that, say, have their homepage on
> github, but do not tag releases there -- because :( :( :( -- and instead
> push releases to PyPI or npmjs or something?

This should work: unless a version can be determined from the upstream
URL, the first source URL is used for a second attempt. I haven't
tested the described situation, though.
>
> 2) It looks like your AUR package is downloading a source tarball, then
> completely ignoring it by using go get and downloading the bleeding-edge
> master. I'm not really sure how this crazy language handles packaging :p
> but I think *maybe* you should be cloning
> git+https://github.com/simon04/$pkgname.git#tag=v$pkgver and symlinking
> "$srcdir/$pkgname" --> "$srcdir/src/github.com/simon04/aur-out-of-date/"

You're right. I hopefully fixed that in aur-out-of-date 0.6.0-2

Simon
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

Michael Kogan
Unfortunately, I don't know Go and in general have too little programming
skills and experience to propose any code modifications. But I have an
idea: What about introducing some variables which the package maintainers
would put into their PKGBUILD and which are then parsed by aur-out-of-date?
The variables might be commented out in order to not interfere with
whatever happens when makepkg is run. Something like

# aur-out-of-date-check-url =
https://sourceforge.net/projects/sevenzip/files/7-Zip/

for the above example. There could be sane defaults defined for such
problematic hosters and only if they fail the user-defined values put into
the PKGBUILD by its maintainer would be used. Also this would require some
documentation to tell the maintainers which variables are necessary and how
to give them the desired values.

Just an idea, I don't know if it makes much sense.

2018-02-01 20:43 GMT+01:00 Simon Legner via aur-general <
[hidden email]>:

> Hi,
>
> On Thu, Feb 1, 2018 at 5:44 PM, Eli Schwartz via aur-general
> <[hidden email]> wrote:
> > This looks quite cool!
>
> Thanks :)
>
> > 1) How does this handle PKGBUILDs that, say, have their homepage on
> > github, but do not tag releases there -- because :( :( :( -- and instead
> > push releases to PyPI or npmjs or something?
>
> This should work: unless a version can be determined from the upstream
> URL, the first source URL is used for a second attempt. I haven't
> tested the described situation, though.
> >
> > 2) It looks like your AUR package is downloading a source tarball, then
> > completely ignoring it by using go get and downloading the bleeding-edge
> > master. I'm not really sure how this crazy language handles packaging :p
> > but I think *maybe* you should be cloning
> > git+https://github.com/simon04/$pkgname.git#tag=v$pkgver and symlinking
> > "$srcdir/$pkgname" --> "$srcdir/src/github.com/simon04/aur-out-of-date/"
>
> You're right. I hopefully fixed that in aur-out-of-date 0.6.0-2
>
> Simon
>
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
On 02/01/2018 02:56 PM, Michael Kogan wrote:

> Unfortunately, I don't know Go and in general have too little programming
> skills and experience to propose any code modifications. But I have an
> idea: What about introducing some variables which the package maintainers
> would put into their PKGBUILD and which are then parsed by aur-out-of-date?
> The variables might be commented out in order to not interfere with
> whatever happens when makepkg is run. Something like
>
> # aur-out-of-date-check-url =
> https://sourceforge.net/projects/sevenzip/files/7-Zip/
>
> for the above example. There could be sane defaults defined for such
> problematic hosters and only if they fail the user-defined values put into
> the PKGBUILD by its maintainer would be used. Also this would require some
> documentation to tell the maintainers which variables are necessary and how
> to give them the desired values.
>
> Just an idea, I don't know if it makes much sense.
Well, lots of people already do this, via urlwatch or pkgbuild-urlwatch.
I think the idea was supposed to be, finding a way to do this without
introducing complex tricks that need to be manually added to the
PKGBUILD and then fragilely parsed.

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

Simon Doppler
On Thu, Feb 01, 2018 at 03:36:28PM -0500, Eli Schwartz via aur-general wrote:

> [...]
> Well, lots of people already do this, via urlwatch or pkgbuild-urlwatch.
> I think the idea was supposed to be, finding a way to do this without
> introducing complex tricks that need to be manually added to the
> PKGBUILD and then fragilely parsed.
>
> --
> Eli Schwartz
> Bug Wrangler and Trusted User
>
Repology[1] has taken the approach of comparing the versions in various
repositories (as well as some other sources like PyPi).

Their parser collection[2] could maybe be a source of inspiration.

[1] https://repology.org/
[2] https://github.com/repology/repology/tree/master/repology/parsers

--
Simon Doppler (dopsi)
E: [hidden email]
AUR: https://aur.archlinux.org/account/dopsi
GitHub: https://github.com/dopsi
PGP: DE9F 7688 CACF 04FE B81A 6C59 0AEE C907 55DA 7B5A

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

Marcin Wieczorek
In reply to this post by tur-users mailing list

On 02/01/2018 11:08 AM, Simon Legner via aur-general wrote:

> Hi aur-general,
>
> as a maintainer of fifty-something AUR packages I found it difficult
> to keep track of new upstream versions. I would subscribe to various
> RSS feeds to keep track of some.
>
> Recently, I sat down and wrote a tool "aur-out-of-date" which, given a
> username or a list of packages, checks upstream for newer versions. It
> does so by inspecting the upstream URL and/or source URL, and calling
> a suitable API (github.com, registry.npmjs.org, pypi.python.org,
> cpan.org for now) to determine the latest released version.
> Out-of-date packages are output in red color on the command line.
>
> Give it a try:
> https://github.com/simon04/aur-out-of-date
> https://aur.archlinux.org/packages/aur-out-of-date/
>
> I'm happy to hear you feedback :)
> Simon
Awesome project. I made myself a similar one, a bash script that
checks GitHub API for new versions and updates the PKGBUILD. I was
planning on making something bigger with ruby, so it would be easily
extensible for non-regular cases (custom download pages etc.).
So I'm testing it at the moment. Looks cool.
First of all please modify the name of the
source in AUR. Some people use common source directories and that causes
conflicts. Secondly I think the program is too verbose. It would be
better readable displaying only brief information like:
"[OUT-OF-DATE] [$pkgname] Flagged: 1.0.0 -> 1.0.1"
or something like that. An option to ignore [UNKNOWN] packages would be
great too. I might be wrong, but it seems to go crazy with metapackages.
I don't know Go, but I'll try to patch if anything else comes to my mind.

Eli Schwartz via aur-general <[hidden email]> writes:
> 1) How does this handle PKGBUILDs that, say, have their homepage on
> github, but do not tag releases there -- because :( :( :( -- and instead
> push releases to PyPI or npmjs or something?

Imho such cases should be always reported upstream as that is a very bad
practice.

Marcin Wieczorek

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: aur-out-of-date: determine out-of-date AUR packages w.r.t. upstream

tur-users mailing list
On 02/02/2018 04:48 PM, Marcin Wieczorek wrote:
> Eli Schwartz via aur-general <[hidden email]> writes:
>> 1) How does this handle PKGBUILDs that, say, have their homepage on
>> github, but do not tag releases there -- because :( :( :( -- and instead
>> push releases to PyPI or npmjs or something?
>
> Imho such cases should be always reported upstream as that is a very bad
> practice.

It certainly is, but there is a difference between reporting it upstream
and actually getting them to fix it...

See, for example, this upstream request for a Github tag (which has been
carefully ignored since October):
https://github.com/jakubroztocil/httpie/issues/620

Which would fix these two outstanding bugs:
https://bugs.archlinux.org/task/55786
https://bugs.archlinux.org/task/55881

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment