Kernel source URL change

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Kernel source URL change

arch general mailing list-2
Hi,

Recently the way kernel sources are retrieved was changed in the linux
package [1]. Now the sources are fetched from
https://github.com/archlinux/linux.

I see a few problems with this:

- Previously the list of applied patches was very transparent. You could
immediately see that the kernel and kernel patch tarballs come from
kernel.org, and view individual extra patches. Now the code comes from a
non-kernel source, and cannot be verified as easily.

- Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.

- An often cited Arch policy is to use software as released by upstream
with minimal patching. What becomes of this policy if one of the core
packages builds from a technical fork instead of upstream?


If the patches from kernel.org will no longer be signed, as announced in
[2], then an alternative would be git tags from [3] and [4]. It's
understandable if it may make development harder, nonetheless it would
allow for better transparency and follow upstream closer — just one
user's opinion.


[1]
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
[2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
[4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git

--
Regards,
Andrey
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
On 08/01, Andrey Vihrov via arch-general wrote:
> - Previously the list of applied patches was very transparent. You could
> immediately see that the kernel and kernel patch tarballs come from
> kernel.org, and view individual extra patches. Now the code comes from a
> non-kernel source, and cannot be verified as easily.

Just run `git log v$_srcver`. That will show you all the patches that
have been applied to the upstream release.

That the sources do no longer come from kernel.org is irrelevant, you
should _always_ verify the gpg signature for auditing purposes.

> - Previously, if a new kernel version is released and is not yet in the
> repos, you could more or less take the official linux PKGBUILD, change
> one number and build it yourself. With the new layout it is not clear
> how to achieve this.

git-rebase(1) will accomplish that.

> - An often cited Arch policy is to use software as released by upstream
> with minimal patching. What becomes of this policy if one of the core
> packages builds from a technical fork instead of upstream?

Nobody forked the linux kernel, and no new patches have been added. The
resulting package is exactly the same, so nothing changed in that
regard.

Regards,
Tharre

--
PGP fingerprint: 42CE 7698 D6A0 6129 AA16  EF5C 5431 BDE2 C8F0 B2F4

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
In reply to this post by arch general mailing list-2
On 01/08/18, Andrey Vihrov via arch-general wrote:

> Hi,
>
> Recently the way kernel sources are retrieved was changed in the linux
> package [1]. Now the sources are fetched from
> https://github.com/archlinux/linux.
>
> I see a few problems with this:
>
> - Previously the list of applied patches was very transparent. You could
> immediately see that the kernel and kernel patch tarballs come from
> kernel.org, and view individual extra patches. Now the code comes from a
> non-kernel source, and cannot be verified as easily.
>
> - Previously, if a new kernel version is released and is not yet in the
> repos, you could more or less take the official linux PKGBUILD, change
> one number and build it yourself. With the new layout it is not clear
> how to achieve this.
>
> - An often cited Arch policy is to use software as released by upstream
> with minimal patching. What becomes of this policy if one of the core
> packages builds from a technical fork instead of upstream?
>
>
> If the patches from kernel.org will no longer be signed, as announced in
> [2], then an alternative would be git tags from [3] and [4]. It's
> understandable if it may make development harder, nonetheless it would
> allow for better transparency and follow upstream closer — just one
> user's opinion.
>
>
> [1]
> https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
> [2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
> [4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
>
> --
> Regards,
> Andrey

Also now to build the package locally you download the whole repository
(~2 Gb compared to the ~110 Mb previously).

What's the reasoning behind this change?

Regards,

--
Leonidas Spyropoulos

A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
The size problem can be solved using hollow clone.

Regards.

On Thu, 2 Aug 2018, 07:28 Leonidas Spyropoulos via arch-general, <
[hidden email]> wrote:

> On 01/08/18, Andrey Vihrov via arch-general wrote:
> > Hi,
> >
> > Recently the way kernel sources are retrieved was changed in the linux
> > package [1]. Now the sources are fetched from
> > https://github.com/archlinux/linux.
> >
> > I see a few problems with this:
> >
> > - Previously the list of applied patches was very transparent. You could
> > immediately see that the kernel and kernel patch tarballs come from
> > kernel.org, and view individual extra patches. Now the code comes from a
> > non-kernel source, and cannot be verified as easily.
> >
> > - Previously, if a new kernel version is released and is not yet in the
> > repos, you could more or less take the official linux PKGBUILD, change
> > one number and build it yourself. With the new layout it is not clear
> > how to achieve this.
> >
> > - An often cited Arch policy is to use software as released by upstream
> > with minimal patching. What becomes of this policy if one of the core
> > packages builds from a technical fork instead of upstream?
> >
> >
> > If the patches from kernel.org will no longer be signed, as announced in
> > [2], then an alternative would be git tags from [3] and [4]. It's
> > understandable if it may make development harder, nonetheless it would
> > allow for better transparency and follow upstream closer — just one
> > user's opinion.
> >
> >
> > [1]
> >
> https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
> > [2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
> > [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
> > [4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
> >
> > --
> > Regards,
> > Andrey
>
> Also now to build the package locally you download the whole repository
> (~2 Gb compared to the ~110 Mb previously).
>
> What's the reasoning behind this change?
>
> Regards,
>
> --
> Leonidas Spyropoulos
>
> A: Because it messes up the order in which people normally read text.
> Q: Why is it such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing on usenet and in e-mail?
>
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

Jens John-2
On Thu, 2 Aug 2018, at 09:11, Joan Aymà via arch-general wrote:
> The size problem can be solved using hollow clone.

Not in a well-formed PKGBUILD. There's ample discussion on this topic available. Of course you're welcome to hack together whatever you want.
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
In reply to this post by arch general mailing list-2
> Also now to build the package locally you download the whole repository
> (~2 Gb compared to the ~110 Mb previously).

Oh...


Cloning into bare repository '/tmp/4.17-arch/archlinux-linux'...
remote: Counting objects: 6135977, done.
remote: Compressing objects: 100% (1099/1099), done.
remote: Total 6135977 (delta 984), reused 385 (delta 385), pack-reused
6134493
Receiving objects: 100% (6135977/6135977), 2.07 GiB | 1.03 MiB/s, done.
Resolving deltas: 100% (5111928/5111928), done.


33 minutes to fetch kernel.

If this changeset is good for arch infrastructure builds, may be tar.xz
will be also provided, via github releases for example.




k
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
Now we know that the change is a solution in search of a problem.

The fact that they won't sign the patch tarballs anymore isn't a problem.

All that heftig must do, is to make it download the complete tarball and its signature.

Here you go:
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On August 7, 2018 8:31 AM, Konstantin Shalygin via arch-general <[hidden email]> wrote:

> > Also now to build the package locally you download the whole repository
> > (~2 Gb compared to the ~110 Mb previously).
>
> Oh...
>
> Cloning into bare repository '/tmp/4.17-arch/archlinux-linux'...
> remote: Counting objects: 6135977, done.
> remote: Compressing objects: 100% (1099/1099), done.
> remote: Total 6135977 (delta 984), reused 385 (delta 385), pack-reused
> 6134493
> Receiving objects: 100% (6135977/6135977), 2.07 GiB | 1.03 MiB/s, done.
> Resolving deltas: 100% (5111928/5111928), done.
>
> 33 minutes to fetch kernel.
>
> If this changeset is good for arch infrastructure builds, may be tar.xz
> will be also provided, via github releases for example.
>
> k
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
On 08/07/2018 07:31 PM, W B via arch-general wrote:
> Now we know that the change is a solution in search of a problem.
>
> The fact that they won't sign the patch tarballs anymore isn't a problem.
>
> All that heftig must do, is to make it download the complete tarball and its signature.
>
> Here you go:
> https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
> https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign

We do not listen to your orders.

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
It isn't an order.

Can you tell us why this change was required, please?

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On August 8, 2018 3:11 AM, Eli Schwartz via arch-general <[hidden email]> wrote:

> On 08/07/2018 07:31 PM, W B via arch-general wrote:
>
> > Now we know that the change is a solution in search of a problem.
> > The fact that they won't sign the patch tarballs anymore isn't a problem.
> > All that heftig must do, is to make it download the complete tarball and its signature.
> > Here you go:
> > https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
> > https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign
>
> We do not listen to your orders.
>
> -----------------------------------
>
> Eli Schwartz
> Bug Wrangler and Trusted User
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
Em agosto 7, 2018 23:31 W B via arch-general escreveu:
> It isn't an order.
>
> Can you tell us why this change was required, please?
>

Have you read the original post to the list? Specially this [0]?

Those tar files you just linked are not signed by Linus anymore, they are signed
instead by Greg Kroah-Hartman. You would have known this if you bothered to actually
download them and check the signature.

Another reason for this move is to apply our patches as commits. You can use any other
kernel if you want.

[0] https://www.kernel.org/minor-changes-to-tarball-release-format.html

Cheers,
Giancarlo Razzolini

attachment0 (887 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
In reply to this post by arch general mailing list-2
On 08/07/2018 10:31 PM, W B via arch-general wrote:
> It isn't an order.
>
> Can you tell us why this change was required, please?

Because heftig decided it was easier *for him* to do it this way.

Because downloading 100 MB for every single patchlevel release quickly
builds up to just as much as a full git clone.

Can you tell us why you believe the move to git was "a solution in
search of a problem"?
Can you tell us why you replied to this list with the message that there
are full tarballs, as though you think heftig did not know this and
reject this already? Who are you trying to convince here?

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

Joakim Hernberg-2
On Tue, 7 Aug 2018 22:55:55 -0400
Eli Schwartz via arch-general <[hidden email]> wrote:

> Because heftig decided it was easier *for him* to do it this way.
>
> Because downloading 100 MB for every single patchlevel release quickly
> builds up to just as much as a full git clone.
>
> Can you tell us why you believe the move to git was "a solution in
> search of a problem"?

I wonder if this isn't going to complicate my life maintaining the
linux-rt kernel on AUR.  So far I've tried to track the linux package
closely, but I wonder what repercussions it will have.  Many times it's
also out of sync with the main distro kernel, but I always used to pick
up the patches etc used by the main kernel.

Oh well, nor really a complaint, let's see how it works out in practice.

--

   Joakim
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
In reply to this post by arch general mailing list-2
On August 8, 2018 4:54 AM, Giancarlo Razzolini via arch-general <[hidden email]> wrote:

> Em agosto 7, 2018 23:31 W B via arch-general escreveu:
>
> > It isn't an order.
> >
> > > Can you tell us why this change was required, please?
>
> Have you read the original post to the list? Specially this [0]?
>

The author of original post was only speculating about possible reasons for the recent
changes. He also asked few questions which weren't answered.

> Those tar files you just linked are not signed by Linus anymore, they are signed
> instead by Greg Kroah-Hartman. You would have known this if you bothered to actually
> download them and check the signature.
>

Greg Kroah-Hartman PGP key was already included as validpkgkey inside PKGBUILD so there
is no real argument here.

> Another reason for this move is to apply our patches as commits. You can use any other
> kernel if you want.
>

There is no tradition in Arch to self-host package sources as Debian does unless upstream has
completely broken release process. This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
which didn't exist before.

Is it general Arch move to self-host sources and applying patches as commits or will linux kernel
package stay as outlier?

> [0] https://www.kernel.org/minor-changes-to-tarball-release-format.html
>
> Cheers,
> Giancarlo Razzolini

Yours sincerely

G. K.
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

Ralf Mardorf-4
On Wed, 8 Aug 2018 13:43:08 +0200 (CEST), Geo Kozey wrote:
>The author of original post [snip] asked few questions which weren't
>answered.

Hi,

the OP did ask how to build a custom kernel based on the official linux
package [1]. Perhaps somebody with unobjectionable knowledge could
correct related Wiki pages, at least
https://wiki.archlinux.org/index.php/Kernels/Arch_Build_System .

TIA,
Ralf

[1]
On Wed, 1 Aug 2018 23:41:12 +0300, Andrey Vihrov via arch-general wrote:
>Previously, if a new kernel version is released and is not yet in the
>repos, you could more or less take the official linux PKGBUILD, change
>one number and build it yourself. With the new layout it is not clear
>how to achieve this.
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

Jonathon Fernyhough
In reply to this post by arch general mailing list-2
On 08/08/18 12:43, Geo Kozey via arch-general wrote:
> This can impose security risks on Arch as we now have to
> trust their github infra rather than kernel.org (we all know what happened to gentoo recently)

Just to provide some perspective, kernel.org itself had a major issue a
few years back [1][2][3]. kernel.org was down for several weeks after
that incident, and IIRC this prompted them to start using GitHub (at
least as a mirror; my memory is fuzzy as I wasn't paying all that much
attention to that sort of thing seven years ago).

If you don't trust the Arch-run/administered infrastructure you can't
really trust any of the packages in the repos either.

[1] https://www.theregister.co.uk/2011/08/31/linux_kernel_security_breach/
[2] https://en.wikipedia.org/wiki/Kernel.org
[3] https://www.linuxfoundation.org/blog/2011/08/the-cracking-of-kernel-org/


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
> From: Jonathon Fernyhough <[hidden email]>
> Sent: Wed Aug 08 18:09:30 CEST 2018
> To: <[hidden email]>
> Subject: Re: [arch-general] Kernel source URL change
>
>
> On 08/08/18 12:43, Geo Kozey via arch-general wrote:
> > This can impose security risks on Arch as we now have to
> > trust their github infra rather than kernel.org (we all know what happened to gentoo recently)
>
> Just to provide some perspective, kernel.org itself had a major issue a
> few years back [1][2][3]. kernel.org was down for several weeks after
> that incident, and IIRC this prompted them to start using GitHub (at
> least as a mirror; my memory is fuzzy as I wasn't paying all that much
> attention to that sort of thing seven years ago).
>

IIRC in 2011 Arch didn't even used gpg for signing packages so it's quite ancient time.

> If you don't trust the Arch-run/administered infrastructure you can't
> really trust any of the packages in the repos either.
>

The point was that before changes no user had to care about https://github.com/Archlinux
and now it's critical infrastructure for self-hosting package sources.

> [1] https://www.theregister.co.uk/2011/08/31/linux_kernel_security_breach/
> [2] https://en.wikipedia.org/wiki/Kernel.org
> [3] https://www.linuxfoundation.org/blog/2011/08/the-cracking-of-kernel-org/
>

Yours sincerely

G. K.
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
On 08/08, Geo Kozey via arch-general wrote:
> There is no tradition in Arch to self-host package sources as Debian does unless upstream has
> completely broken release process. This can impose security risks on Arch as we now have to
> trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
> I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
> which didn't exist before.
[...]
> The point was that before changes no user had to care about https://github.com/Archlinux
> and now it's critical infrastructure for self-hosting package sources.

No, nobody has to trust github or for that fact kernel.org. The
commits/tags are *signed* and thus makepkg will check if that signature
matches one of those specified in the validpgpkeys array.

From a security standpoint, it's irrelevant if the sources come from
arch hosted infra, from github, or from kernel.org.

Regards,
Tharre

--
PGP fingerprint: 42CE 7698 D6A0 6129 AA16  EF5C 5431 BDE2 C8F0 B2F4

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel source URL change

arch general mailing list-2
On 8/8/18 4:11 PM, Tharre via arch-general wrote:

> On 08/08, Geo Kozey via arch-general wrote:
>> There is no tradition in Arch to self-host package sources as Debian does unless upstream has
>> completely broken release process. This can impose security risks on Arch as we now have to
>> trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
>> I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
>> which didn't exist before.
> [...]
>> The point was that before changes no user had to care about https://github.com/Archlinux
>> and now it's critical infrastructure for self-hosting package sources.
>
> No, nobody has to trust github or for that fact kernel.org. The
> commits/tags are *signed* and thus makepkg will check if that signature
> matches one of those specified in the validpgpkeys array.
>
> From a security standpoint, it's irrelevant if the sources come from
> arch hosted infra, from github, or from kernel.org.
I'm all for hosting it through bittorrent TBH.

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment