[RFC] Make PKGBUILD attributes configurable

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[RFC] Make PKGBUILD attributes configurable

Dustin Falgout

It would be great if there was a way to customize the list of PKGBUILD attributes supported by makepkg without having to edit the installed copy of makepkg. My use-case is mainly for use with the --printsrcinfo option but I'm sure it would be useful in other areas as well. I'd like to submit a patch for it but I thought it best to see what everyone thought about the feature before spending time on it. My initial thinking is that that simple text files with one attribute per line could be placed in /etc/makepkg.d. Perhaps something along these lines: 

/etc/makepkg.d/attributes.single
/etc/makepkg.d/attributes.multi
/etc/makepkg.d/attributes-march.single
/etc/makepkg.d/attributes-march.multi

Looking forward to your comments and also any guidance on preferred implementation details.


-- 

Dustin Falgout

Email: [hidden email]
Github: lots0logs

     
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Allan McRae
On 23/04/17 09:36, Dustin Falgout wrote:

> I would like a way to include custom attributes from the PKGBUILD in the output of the --printsrcinfo option. So basically, this...
>
> pkgbase = pacman
> pkgdesc = A library-based package manager with dependency support
> pkgver = 5.0.1
> pkgrel = 4
> url = http://www.archlinux.org/pacman/
> arch = i686
> arch = x86_64
> ...
> _custom_attribute1 = some value
> _custom_attribute2 = some value
> _custom_attribute3 = some value
> ...
>
> pkgname = pacman
>

Bringing the mailing list back into this  (seems our reply field is
broken...)

Can you give an example of what a custom attribute would be?   Your
example is still to vague to judge whether this would be something we
wish to support.

Thanks,
Allan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Dustin Falgout
Sure, no problem. Currently, our build server uses some custom attributes in the PKGBUILD for additional metadata needed for things like release monitoring. I would like to start using .SRCINFO files on the server because they are easier to parse and also because it would be better convention-wise.  Here's an example[1].

[1] https://github.com/Antergos/antergos-packages/blob/master/antergos/mate/mate-desktop/PKGBUILD

--
Dustin Falgout

E-mail: [hidden email]
Github: lots0logs




From: pacman-dev <[hidden email]> on behalf of Allan McRae <[hidden email]>
Sent: Saturday, April 22, 2017 6:51 PM
To: Discussion list for pacman development
Subject: Re: [pacman-dev] [RFC] Make PKGBUILD attributes configurable
   
On 23/04/17 09:36, Dustin Falgout wrote:

> I would like a way to include custom attributes from the PKGBUILD in the output of the --printsrcinfo option. So basically, this...
>
> pkgbase = pacman
>        pkgdesc = A library-based package manager with dependency support
>        pkgver = 5.0.1
>        pkgrel = 4
>        url = http://www.archlinux.org/pacman/
>        arch = i686
>        arch = x86_64
>        ...
>        _custom_attribute1 = some value
>        _custom_attribute2 = some value
>        _custom_attribute3 = some value
>        ...
>
> pkgname = pacman
>

Bringing the mailing list back into this  (seems our reply field is
broken...)

Can you give an example of what a custom attribute would be?   Your
example is still to vague to judge whether this would be something we
wish to support.

Thanks,
Allan
   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Dustin Falgout

Here is a basic POC for what I had in mind. Please let me know what you think. 


From e7f9293f42d8d069b7a57165b25e79e248abd680 Mon Sep 17 00:00:00 2001
From: Dustin Falgout <[hidden email]>
Date: Sat, 22 Apr 2017 19:54:51 -0500
Subject: [PATCH] [POC] makepkg :: Configurable pkgbuild attributes

Signed-off-by: Dustin Falgout <[hidden email]>
---
 scripts/libmakepkg/conf/attributes-march.multi |  0
 scripts/libmakepkg/conf/attributes.multi       |  0
 scripts/libmakepkg/conf/attributes.single      |  0
 scripts/libmakepkg/srcinfo.sh.in               | 61 +++++++++++++++++++++-----
 4 files changed, 49 insertions(+), 12 deletions(-)
 create mode 100644 scripts/libmakepkg/conf/attributes-march.multi
 create mode 100644 scripts/libmakepkg/conf/attributes.multi
 create mode 100644 scripts/libmakepkg/conf/attributes.single

diff --git a/scripts/libmakepkg/conf/attributes-march.multi b/scripts/libmakepkg/conf/attributes-march.multi
new file mode 100644
index 00000000..e69de29b
diff --git a/scripts/libmakepkg/conf/attributes.multi b/scripts/libmakepkg/conf/attributes.multi
new file mode 100644
index 00000000..e69de29b
diff --git a/scripts/libmakepkg/conf/attributes.single b/scripts/libmakepkg/conf/attributes.single
new file mode 100644
index 00000000..e69de29b
diff --git a/scripts/libmakepkg/srcinfo.sh.in b/scripts/libmakepkg/srcinfo.sh.in
index 99f5628a..96783850 100644
--- a/scripts/libmakepkg/srcinfo.sh.in
+++ b/scripts/libmakepkg/srcinfo.sh.in
@@ -23,8 +23,53 @@ LIBMAKEPKG_SRCINFO_SH=1
 
 LIBRARY=${LIBRARY:-'@libmakepkgdir@'}
 
+SINGLE_VALUED_ATTRS=
+MULTI_VALUED_ATTRS=
+MARCH_MULTI_VALUED_ATTRS=
+
 source "$LIBRARY/util/pkgbuild.sh"
 
+set_attributes() {
+ local makepkgd="${confdir}/makepkg.d"
+ local singlevalued_default= multivalued_default= singlevalued= multivalued=
+ local tmparray=()
+
+ mapfile -t singlevalued_default < "${LIBRARY}/conf/attributes.single"
+ mapfile -t multivalued_default < "${LIBRARY}/conf/attributes.multi"
+
+ [[ -f "${makepkgd}/attributes.single" ]] && mapfile -t singlevalued < "${makepkgd}/attributes.single"
+ [[ -f "${makepkgd}/attributes.multi" ]] && mapfile -t multivalued < "${makepkgd}/attributes.multi"
+
+ for key in "${singlevalued_default[@]}" "${singlevalued[@]}"; do
+ [[ -n "${key}" ]] && tmparray["${key}"]=1
+ done
+
+ SINGLE_VALUED_ATTRS=("${!tmparray[@]}")
+
+ tmparray=()
+
+ for key in "${multivalued_default[@]}" "${multivalued[@]}"; do
+ [[ -n "${key}" ]] && tmparray["${key}"]=1
+ done
+
+ MULTI_VALUED_ATTRS=("${!tmparray[@]}")
+}
+
+set_march_attributes() {
+ local makepkgd="${confdir}/makepkg.d"
+ local multivalued_default= multivalued=
+
+ mapfile -t multivalued_default < "${LIBRARY}/conf/attributes.multi"
+
+ [[ -f "${makepkgd}/attributes-march.multi" ]] && mapfile -t multivalued < "${makepkgd}/attributes-march.multi"
+
+ for key in "${multivalued_default[@]}" "${multivalued[@]}"; do
+ [[ -n "${key}" ]] && tmparray["${key}"]=1
+ done
+
+ MARCH_MULTI_VALUED_ATTRS=("${!tmparray[@]}")
+}
+
 srcinfo_open_section() {
  printf '%s = %s\n' "$1" "$2"
 }
@@ -61,15 +106,12 @@ pkgbuild_extract_to_srcinfo() {
 
 srcinfo_write_section_details() {
  local attr package_arch a
- local multivalued_arch_attrs=(source provides conflicts depends replaces
-                              optdepends makedepends checkdepends
-                              {md5,sha{1,224,256,384,512}}sums)
 
- for attr in "${singlevalued[@]}"; do
+ for attr in "${SINGLE_VALUED_ATTRS[@]}"; do
  pkgbuild_extract_to_srcinfo "$1" "$attr" 0
  done
 
- for attr in "${multivalued[@]}"; do
+ for attr in "${MULTI_VALUED_ATTRS[@]}"; do
  pkgbuild_extract_to_srcinfo "$1" "$attr" 1
  done
 
@@ -78,19 +120,14 @@ srcinfo_write_section_details() {
  # 'any' is special. there's no support for, e.g. depends_any.
  [[ $a = any ]] && continue
 
- for attr in "${multivalued_arch_attrs[@]}"; do
+ for attr in "${MARCH_MULTI_VALUED_ATTRS[@]}"; do
  pkgbuild_extract_to_srcinfo "$1" "${attr}_$a" 1
  done
  done
 }
 
 srcinfo_write_global() {
- local singlevalued=(pkgdesc pkgver pkgrel epoch url install changelog)
- local multivalued=(arch groups license checkdepends makedepends
-                   depends optdepends provides conflicts replaces
-                   noextract options backup
-                   source validpgpkeys {md5,sha{1,224,256,384,512}}sums)
-
+ set_attributes
  srcinfo_open_section 'pkgbase' "${pkgbase:-$pkgname}"
  srcinfo_write_section_details ''
  srcinfo_close_section
--
2.12.2
 
     




From: pacman-dev <[hidden email]> on behalf of Dustin Falgout <[hidden email]>
Sent: Saturday, April 22, 2017 7:03 PM
To: Discussion list for pacman development
Subject: Re: [pacman-dev] [RFC] Make PKGBUILD attributes configurable
   
Sure, no problem. Currently, our build server uses some custom attributes in the PKGBUILD for additional metadata needed for things like release monitoring. I would like to start using .SRCINFO files on the server because they are easier  to parse and also because it would be better convention-wise.  Here's an example[1].

[1]  https://github.com/Antergos/antergos-packages/blob/master/antergos/mate/mate-desktop/PKGBUILD

--
Dustin Falgout

E-mail: [hidden email]
Github: lots0logs




From: pacman-dev <[hidden email]> on behalf of Allan McRae <[hidden email]>
Sent: Saturday, April 22, 2017 6:51 PM
To: Discussion list for pacman development
Subject: Re: [pacman-dev] [RFC] Make PKGBUILD attributes configurable
   
On 23/04/17 09:36, Dustin Falgout wrote:

> I would like a way to include custom attributes from the PKGBUILD in the output of the --printsrcinfo option. So basically, this...
>
> pkgbase = pacman
>        pkgdesc = A library-based package manager with dependency support
>        pkgver = 5.0.1
>        pkgrel = 4
>        url = http://www.archlinux.org/pacman/
>        arch = i686
>        arch = x86_64
>        ...
>        _custom_attribute1 = some value
>        _custom_attribute2 = some value
>        _custom_attribute3 = some value
>        ...
>
> pkgname = pacman
>

Bringing the mailing list back into this  (seems our reply field is
broken...)

Can you give an example of what a custom attribute would be?   Your
example is still to vague to judge whether this would be something we
wish to support.

Thanks,
Allan
       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Eli Schwartz
In reply to this post by Dustin Falgout
On 04/22/2017 06:53 PM, Dustin Falgout wrote:

> It would be great if there was a way to customize the list of
> PKGBUILD attributes supported by makepkg without having to edit the
> installed copy of makepkg. My use-case is mainly for use with the
> --printsrcinfo option but I'm sure it would be useful in other areas
> as well. I'd like to submit a patch for it but I thought it best to
> see what everyone thought about the feature before spending time on
> it. My initial thinking is that that simple text files with one
> attribute per line could be placed in /etc/makepkg.d. Perhaps
> something along these lines:
>
> /etc/makepkg.d/attributes.single /etc/makepkg.d/attributes.multi
> /etc/makepkg.d/attributes-march.single
> /etc/makepkg.d/attributes-march.multi
>
> Looking forward to your comments and also any guidance on preferred
> implementation details.
Regardless of the suitability of such a thing, wouldn't it make a lot
more sense to try implementing it by making the attributes fully
libmakepkg-ified and extending it via drop-in components for libmakepkg,
rather than inventing some /etc configuration directory for what is,
after all, adding new functionality rather than merely configuring the
behavior?

Generically speaking, this is the kind of thing that motivated splitting
makepkg into a library in the first place.

> Sure, no problem. Currently, our build server uses some custom
> attributes in the PKGBUILD for additional metadata needed for things
> like release monitoring. I would like to start using .SRCINFO files
> on the server because they are easier to parse and also because it
> would be better convention-wise.  Here's an example[1].

Convention-wise, possibly... but for static key-variable assignments
like that, parsing is actually precisely as easy as .SRCINFO parsing.
The reason for .SRCINFO is because the PKGBUILD fields can contain any
valid bash -- and pretty much always do, e.g. $pkgname-$pkgver.tar.gz in
the sources.

--
Eli Schwartz


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Dustin Falgout
> Regardless of the suitability of such a thing, wouldn't it make a lot
> more sense to try implementing it by making the attributes fully
> libmakepkg-ified and extending it via drop-in components for libmakepkg,
> rather than inventing some /etc configuration directory for what is,
> after all, adding new functionality rather than merely configuring the
> behavior?

That sounds sensible to me. I didn't realize there was already a drop-in replacement system for makepkg. How does it work?

> The reason for .SRCINFO is because the PKGBUILD fields can contain any
> valid bash -- and pretty much always do, e.g. $pkgname-$pkgver.tar.gz in
> the sources.

Indeed, that's one of the things I was referring to when I said it was easier. I should have elaborated, sorry about that.


-- 
Dustin Falgout



From: pacman-dev <[hidden email]> on behalf of Eli Schwartz <[hidden email]>
Sent: Saturday, April 22, 2017 9:00 PM
To: [hidden email]
Subject: Re: [pacman-dev] [RFC] Make PKGBUILD attributes configurable
   
On 04/22/2017 06:53 PM, Dustin Falgout wrote:

> It would be great if there was a way to customize the list of
> PKGBUILD attributes supported by makepkg without having to edit the
> installed copy of makepkg. My use-case is mainly for use with the
> --printsrcinfo option but I'm sure it would be useful in other areas
> as well. I'd like to submit a patch for it but I thought it best to
> see what everyone thought about the feature before spending time on
> it. My initial thinking is that that simple text files with one
> attribute per line could be placed in /etc/makepkg.d. Perhaps
> something along these lines:
>
> /etc/makepkg.d/attributes.single /etc/makepkg.d/attributes.multi
> /etc/makepkg.d/attributes-march.single
> /etc/makepkg.d/attributes-march.multi
>
> Looking forward to your comments and also any guidance on preferred
> implementation details.

Regardless of the suitability of such a thing, wouldn't it make a lot
more sense to try implementing it by making the attributes fully
libmakepkg-ified and extending it via drop-in components for libmakepkg,
rather than inventing some /etc configuration directory for what is,
after all, adding new functionality rather than merely configuring the
behavior?

Generically speaking, this is the kind of thing that motivated splitting
makepkg into a library in the first place.

> Sure, no problem. Currently, our build server uses some custom
> attributes in the PKGBUILD for additional metadata needed for things
> like release monitoring. I would like to start using .SRCINFO files
> on the server because they are easier to parse and also because it
> would be better convention-wise.  Here's an example[1].

Convention-wise, possibly... but for static key-variable assignments
like that, parsing is actually precisely as easy as .SRCINFO parsing.
The reason for .SRCINFO is because the PKGBUILD fields can contain any
valid bash -- and pretty much always do, e.g. $pkgname-$pkgver.tar.gz in
the sources.

--
Eli Schwartz

   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Eli Schwartz
On 04/22/2017 10:18 PM, Dustin Falgout wrote:
> That sounds sensible to me. I didn't realize there was already a
> drop-in replacement system for makepkg. How does it work?

I don't even know what you are talking about, there is no "replacement"
system and I never suggested there was. What I said was that libmakepkg
is by definition extendable by simply dropping in new components, which
shouldn't be news to anyone...

The source contains many examples of how to extend a bash array. :)

...

Also, your emails look a bit wonky -- your top-posted responses to
selected quotes appear above what seems to be a *forwarded* message,
which is honestly kind of confusing to read.

Also, your paragraphs don't line-wrap properly, which should only happen
in HTML emails since HTML emails have better ways of doing text layout.

--
Eli Schwartz


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Dustin Falgout
> I don't even know what you are talking about, there is no "replacement"
> system and I never suggested there was.

Okay, sure, so I added the term "replacement" to what you said because that's how I understood it when I read it (as in "drop-in replacement"). My mistake.


> What I said was that libmakepkg
> is by definition extendable by simply dropping in new components, which
> shouldn't be news to anyone...

Well, TBH, it's news to me. I've never seen any documentation about what you are describing nor have I ever seen it mentioned anywhere online that I frequent. Do you have any links I can reference to know what you are talking about. I'm trying to understand how I should go about implementing what I proposed in the way that you suggested implementing such a feature should be done if it's done at all.


> Also, your emails look a bit wonky -- your top-posted responses to
> selected quotes appear above what seems to be a *forwarded* message,
> which is honestly kind of confusing to read.

Sorry, there's not much I can do about it. I don't have any option to reply in the style that you are using with my client.


  -- 
 Dustin Falgout
 
     




From: pacman-dev <[hidden email]> on behalf of Eli Schwartz <[hidden email]>
Sent: Saturday, April 22, 2017 10:03 PM
To: [hidden email]
Subject: Re: [pacman-dev] [RFC] Make PKGBUILD attributes configurable
   
On 04/22/2017 10:18 PM, Dustin Falgout wrote:
> That sounds sensible to me. I didn't realize there was already a
> drop-in replacement system for makepkg. How does it work?

I don't even know what you are talking about, there is no "replacement"
system and I never suggested there was. What I said was that libmakepkg
is by definition extendable by simply dropping in new components, which
shouldn't be news to anyone...

The source contains many examples of how to extend a bash array. :)

...

Also, your emails look a bit wonky -- your top-posted responses to
selected quotes appear above what seems to be a *forwarded* message,
which is honestly kind of confusing to read.

Also, your paragraphs don't line-wrap properly, which should only happen
in HTML emails since HTML emails have better ways of doing text layout.

--
Eli Schwartz

   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Eli Schwartz
On 04/22/2017 11:33 PM, Dustin Falgout wrote:
> Well, TBH, it's news to me. I've never seen any documentation about
> what you are describing nor have I ever seen it mentioned anywhere
> online that I frequent. Do you have any links I can reference to know
> what you are talking about. I'm trying to understand how I should go
> about implementing what I proposed in the way that you suggested
> implementing such a feature should be done if it's done at all.

Well, to people hacking on the source anyway.

The whole idea of libmakepkg was that you can drop in new *.sh files
into the libmakepkg directory (because it is a "library") rather than
editing all that tightly-woven logic in /usr/bin/makepkg

Have you actually looked at what, for example,
/usr/share/makepkg/tidy.sh does to allow extendable tidy options?

How many different ways can I say "add code to automatically source
certain types of files, and expect those files to append items to an
array" without actually writing the code on your behalf?

> Sorry, there's not much I can do about it. I don't have any option to
> reply in the style that you are using with my client.

Use a client that doesn't suck, then.

--
Eli Schwartz


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFC] Make PKGBUILD attributes configurable

Allan McRae
In reply to this post by Dustin Falgout
On 23/04/17 10:03, Dustin Falgout wrote:
> Sure, no problem. Currently, our build server uses some custom attributes in the PKGBUILD for additional metadata needed for things like release monitoring. I would like to start using .SRCINFO files on the server because they are easier to parse and also because it would be better convention-wise.  Here's an example[1].
>
> [1] https://github.com/Antergos/antergos-packages/blob/master/antergos/mate/mate-desktop/PKGBUILD
>

I'm going to suggest that you look at scripts/libmakepkg/srcinfo.sh.in
in the pacman code base (or /usr/share/makepkg/srcinfo.sh when installed).

It would be very easy to create a script that sources that file, adds a
function to print your extend attributes and adjusts write_srcinfo() to
output the additional fields too.

Unless significant other demand for this happens, I will not be adding
this feature to makepkg itself.

Allan
Loading...