Switching the bugtracker to Bugzilla

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Switching the bugtracker to Bugzilla

Jelle van der Waa-2
It's time to switch our to something which is maintained and can be extended to
our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2].

It seems to be time to move on, and Bugzilla is one of the more active and
maintained bugtrackers out there. Used by several big projects such as
Gnome, LLVM and Mozilla. One of the benefits of moving would be the
possibility of default assignees per 'component'.

# Migration

There are several options for migrating the bug history to Bugzilla and a few options are under
debate. (input welcome)

* No migration at all
* Migrate open bugs
* Migrate open bugs and auto-closing them
* Migrate all bugs
* Migrate all bugs and auto-closing them

In either case, I believe it would be nice to "archive" the current bugtracker and make it read
only.

# User migration

User migration should be possible as well, except migrating the password, a mass password reset
would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.

# Migration Projects

Bugzilla has a concept of products with components, so for all our packages we can create a
component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer
information from archweb.

Possible products would be.

# Products

* Arch packages (core/extra or split this up)
* Community packages (community)
* Pacman
* AURWeb
* Keyring
* Archweb (new)
* Arch VM / Docker images (new)
* Release engineering

Input would be welcome, on what we should migrate from flyspray and what products we should define.

[1] https://github.com/Flyspray/flyspray/releases/tag/v1.0-rc6
[2] https://github.com/Flyspray/flyspray/releases/tag/v1.0-rc4

--
Jelle van der Waa

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

arch dev mailing list
Quoting Jelle van der Waa (2017-11-14 20:30:21)

> There are several options for migrating the bug history to Bugzilla and a few options are under
> debate. (input welcome)
>
> * No migration at all
> * Migrate open bugs
> * Migrate open bugs and auto-closing them
> * Migrate all bugs
> * Migrate all bugs and auto-closing them
>
> In either case, I believe it would be nice to "archive" the current bugtracker and make it read
> only.
>
My first reaction is that it'd be nice to not have a bunch of old cruft
around, but if we autoclose them and we could get the migrated bugs to
have the same ID it would simplify having the old links still work with
just a simple redirect to the new URL with the same ID, and it would
mean that we wouldn't need to keep the current bugtracker around for the
indefinite future.

> # Products
>
> * Arch packages (core/extra or split this up)
> * Community packages (community)
> * Pacman
> * AURWeb
> * Keyring
> * Archweb (new)
> * Arch VM / Docker images (new)
> * Release engineering
>
I feel like having core and extra as separate products would make
searching and filling in the component field a *lot* easier, because
that's pretty much my biggest annoyance with e.g. the GNOME bugzilla,
their component lists are huge and it's really hard to browse through
them.

--
Sincerely,
  Johannes Löthberg
  PGP Key ID: 0x50FB9B273A9D0BB5
  PGP Key FP: 5134 EF9E AF65 F95B 6BB1  608E 50FB 9B27 3A9D 0BB5
  https://theos.kyriasis.com/~kyrias/

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Eli Schwartz-2
In reply to this post by Jelle van der Waa-2
On 11/14/2017 02:30 PM, Jelle van der Waa wrote:

> There are several options for migrating the bug history to Bugzilla
> and a few options are under debate. (input welcome)
>
> * No migration at all
> * Migrate open bugs
> * Migrate open bugs and auto-closing them
> * Migrate all bugs
> * Migrate all bugs and auto-closing them
>
> In either case, I believe it would be nice to "archive" the current
> bugtracker and make it read only.
We've never autoclosed bugs just because they are old, why start now
*just because* we are migrating?

Then too there are not-yet-implemented feature requests for
pacman/aurweb/$project.

I'd prefer we migrated all open bugs and left them open, possibly
excluding old bugs filed under the kernel category as they have an
alarming tendency to be upstream issues and/or hardware issues and
anyway are questionably relevant considering the numerous updates.

Possibly also extend that to also excluding open bugs that are assigned
as upstream issues, as those usually have no real purpose other than to
maybe track upstream and backport a fix as soon as one is available.

If we have too many unresolved open bugs (which we do), they need to be
triaged and dealt with (something I'd like to get around to doing and
which I have already picked some low-hanging fruit).

> # Migration Projects
>
> Bugzilla has a concept of products with components, so for all our
> packages we can create a component counterpart. It should be possible
> to auto-assign bugs with the pkgname <-> maintainer information from
> archweb.
>
> Possible products would be.
>
> # Products
>
> * Arch packages (core/extra or split this up)
> * Community packages (community)
> * Pacman
> * AURWeb
> * Keyring
> * Archweb (new)
> * Arch VM / Docker images (new)
> * Release engineering
>
> Input would be welcome, on what we should migrate from flyspray and
> what products we should define.
netctl/mkinitcpio/devtools/dbscripts/namcap/pyalpm

These are currently grouped together alongside core/extra as a component
in the Arch Linux project, but if they don't each deserve their own
project they can probably be components in an "Arch Projects" project.

--
Eli Schwartz


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Bartłomiej Piotrowski-3
In reply to this post by Jelle van der Waa-2
On 2017-11-14 20:30, Jelle van der Waa wrote:
> Used by several big projects such as Gnome, LLVM and Mozilla

GNOME will probably end up switching to Gitlab. (Not dismissing the fact
that bugzilla is rather popular choice.)

> # Migration
>
> There are several options for migrating the bug history to Bugzilla and a few options are under
> debate. (input welcome)

As I said multiple times on IRC, I'm for starting from scratch. There
are way too many inactive or/and incorrect bugs open, and honestly any
effort to review that list is a waste of time. With no bugs open we can
1) pretend everything works fine 2) hopefully avoid zombie-bugs
apocalypse that we have now. Flyspray could be mirrored with wget for
read-only version.

> Bugzilla has a concept of products with components, so for all our packages we can create a
> component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer
> information from archweb.

Packages come and go. Some never will have a bug reported because they
just work fine or nobody uses them. Component per package sounds
overboard unless it's going to be automated.

> * Arch packages (core/extra or split this up)
+1 for splitting.

> * Archweb (new)
> * Arch VM / Docker images (new)

These two are already primarily developed on GitHub and I think bugs
should be reported there as well.

Bartłomiej
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Gaetan Bisson-2
In reply to this post by arch dev mailing list
[2017-11-14 22:11:02 +0100] Johannes Löthberg via arch-dev-public:
> My first reaction is that it'd be nice to not have a bunch of old cruft
> around,

For what it's worth: I completely agree.

My choice would be to start over with a clean bug tracker and not
migrate anything. Everyone who cares will create their account on the
new tracker and every bug that matters will be recreated. And in the
process we'll get rid of hundreds of very old bug no one cares about.

> but if we autoclose them and we could get the migrated bugs to
> have the same ID it would simplify having the old links still work with
> just a simple redirect to the new URL with the same ID,

If that's straightforward to implement, sure, that'd be nice.

But if not I don't think it's worth patching bugzilla for.

> and it would
> mean that we wouldn't need to keep the current bugtracker around for the
> indefinite future.

We can convert the current tracker into a bunch of static pages.
Personally I'd find that a very clean archival solution...

Cheers.

--
Gaetan

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Allan McRae
In reply to this post by Bartłomiej Piotrowski-3
On 15/11/17 07:34, Bartłomiej Piotrowski wrote:
>> There are several options for migrating the bug history to Bugzilla and a few options are under
>> debate. (input welcome)
> As I said multiple times on IRC, I'm for starting from scratch. There
> are way too many inactive or/and incorrect bugs open, and honestly any
> effort to review that list is a waste of time. With no bugs open we can
> 1) pretend everything works fine 2) hopefully avoid zombie-bugs
> apocalypse that we have now. Flyspray could be mirrored with wget for
> read-only version.

If you don't migrate pacman bugs, don't bother creating a pacman component.

A
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Bartłomiej Piotrowski-3
On 2017-11-14 22:59, Allan McRae wrote:

> On 15/11/17 07:34, Bartłomiej Piotrowski wrote:
>>> There are several options for migrating the bug history to Bugzilla and a few options are under
>>> debate. (input welcome)
>> As I said multiple times on IRC, I'm for starting from scratch. There
>> are way too many inactive or/and incorrect bugs open, and honestly any
>> effort to review that list is a waste of time. With no bugs open we can
>> 1) pretend everything works fine 2) hopefully avoid zombie-bugs
>> apocalypse that we have now. Flyspray could be mirrored with wget for
>> read-only version.
>
> If you don't migrate pacman bugs, don't bother creating a pacman component.
>
> A
>

I mean packaging bugs here, not standalone projects.

Bartłomiej
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Giancarlo Razzolini-2
In reply to this post by Jelle van der Waa-2
Em novembro 14, 2017 17:30 Jelle van der Waa escreveu:
> It's time to switch our to something which is maintained and can be extended to
> our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2].

Being the person responsible for writing our flyspray ansible's role and having spent an
awful amount of time writing nginx redirects to overcome the fact that flyspray can't handle
decent URL's, I might be a little biased against it.

>
> It seems to be time to move on, and Bugzilla is one of the more active and
> maintained bugtrackers out there. Used by several big projects such as
> Gnome, LLVM and Mozilla. One of the benefits of moving would be the
> possibility of default assignees per 'component'.
>

I agree with the move to bugzilla. I advocated in the past for us to use something like gitlab, but
we are using github and I think we should embrace it and make it "official". But I think neither
gitlab nor github's issue trackers are that great. At least not for some of the things we do. We
can use them just fine for our projects, though.

> # Migration
>
> There are several options for migrating the bug history to Bugzilla and a few options are under
> debate. (input welcome)
>
> * No migration at all
> * Migrate open bugs
> * Migrate open bugs and auto-closing them
> * Migrate all bugs
> * Migrate all bugs and auto-closing them
>
> In either case, I believe it would be nice to "archive" the current bugtracker and make it read
> only.
I'm for starting clean and making the current flyspray read-only.

>
> # User migration
>
> User migration should be possible as well, except migrating the password, a mass password reset
> would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.

Agreed.

Regards,
Giancarlo Razzolini

attachment0 (887 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Lukas Fleischer-2
In reply to this post by Jelle van der Waa-2
On Tue, 14 Nov 2017 at 20:30:21, Jelle van der Waa wrote:
> It's time to switch our to something which is maintained and can be extended to
> our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2].
> [...]

Great!

> # Migration
>
> There are several options for migrating the bug history to Bugzilla and a few options are under
> debate. (input welcome)
>
> * No migration at all
> * Migrate open bugs
> * Migrate open bugs and auto-closing them
> * Migrate all bugs
> * Migrate all bugs and auto-closing them
>
> In either case, I believe it would be nice to "archive" the current bugtracker and make it read
> only.

Doing no migration and having an archive of the current tickets sounds
like a good idea to me -- I can only talk about the aurweb project and
the packages projects, though -- maintainers of other components might
disagree.

After the migration, I would go through the open aurweb tickets on the
Flyspray archive, create new tickets for the most important issues
(should not be more than 10), fix/implement some of the easy stuff and
close the tickets which I think are not worth fixing or implementing.

> # User migration
>
> User migration should be possible as well, except migrating the password, a mass password reset
> would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.

Sounds good to me, although I do not see the benefit of doing this if we
do not migrate tickets. If users need to reset their passwords, they can
as well reregister. There aren't too many things to configure and the
profile settings in Flyspray will likely not match what you can
configure in Bugzilla anyways.

Generally speaking, I am in favor of setting up a fresh Bugzilla
instance and not doing any sort of migration. The idea of switching to a
new bug tracker came up at least four years ago and my feeling is that
all this migration work is what was holding it back for a long time.
However, we should make sure that the old Flyspray URLs still work (and
redirect to the Flyspray archive).

>
> # Migration Projects
>
> Bugzilla has a concept of products with components, so for all our packages we can create a
> component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer
> information from archweb.
>
> Possible products would be.
>
> # Products
>
> * Arch packages (core/extra or split this up)
> * Community packages (community)
> * Pacman
> * AURWeb
> * Keyring
> * Archweb (new)
> * Arch VM / Docker images (new)
> * Release engineering

Sounds good to me. I would split [core] and [extra]. I would also call
the packages projects "[core] packages", "[extra] packages" and
"[community] packages"; unless there are technical limitations
preventing the use of brackets. These brackets make it much clearer that
the names are referring to pacman repositories; and to a novice Arch
user, it may not be 100% clear what a "community package" or an "extra
package" is.

Best regards,
Lukas
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Jelle van der Waa-2
On 11/15/17 at 09:07am, Lukas Fleischer wrote:
> On Tue, 14 Nov 2017 at 20:30:21, Jelle van der Waa wrote:
> > It's time to switch our to something which is maintained and can be extended to
> > our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2].
> > [...]
>
> Great!

Thanks! Most of the credit goes to Bluewind for coming up with the idea
and poking me about the status ;-)

Note that Bugzilla, also has an REST API, which unlocks some more
possibilities.

>
> > # Migration
> >
> > There are several options for migrating the bug history to Bugzilla and a few options are under
> > debate. (input welcome)
> >
> > * No migration at all
> > * Migrate open bugs
> > * Migrate open bugs and auto-closing them
> > * Migrate all bugs
> > * Migrate all bugs and auto-closing them
> >
> > In either case, I believe it would be nice to "archive" the current bugtracker and make it read
> > only.
>
> Doing no migration and having an archive of the current tickets sounds
> like a good idea to me -- I can only talk about the aurweb project and
> the packages projects, though -- maintainers of other components might
> disagree.
Ok, but it seems Allan would love to migrate all of his bugs, so we
could see if that would be possible for aurweb as well. The biggest
migration would be for packages.

> After the migration, I would go through the open aurweb tickets on the
> Flyspray archive, create new tickets for the most important issues
> (should not be more than 10), fix/implement some of the easy stuff and
> close the tickets which I think are not worth fixing or implementing.

> > # User migration
> >
> > User migration should be possible as well, except migrating the password, a mass password reset
> > would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.
>
> Sounds good to me, although I do not see the benefit of doing this if we
> do not migrate tickets. If users need to reset their passwords, they can
> as well reregister. There aren't too many things to configure and the
> profile settings in Flyspray will likely not match what you can
> configure in Bugzilla anyways.
We currently have 12821 users in flyspray, of which there are ~ 50 with
invalid email addresses and I'm not sure how many are actually 'active',
since flyspray doesn't log the last logged in date.

>
> Generally speaking, I am in favor of setting up a fresh Bugzilla
> instance and not doing any sort of migration. The idea of switching to a
> new bug tracker came up at least four years ago and my feeling is that
> all this migration work is what was holding it back for a long time.
> However, we should make sure that the old Flyspray URLs still work (and
> redirect to the Flyspray archive).

Setting up Bugzilla in Ansible, and figuring how to auto-create and sync
products will still be some effort. And we have to make sure that
Bugzilla, isn't as slow as it's showcased ;-)

My todolist is here:

https://wiki.archlinux.org/index.php/User:jelly/Bugzilla#TODO

> > * Release engineering
>
> Sounds good to me. I would split [core] and [extra]. I would also call
> the packages projects "[core] packages", "[extra] packages" and
> "[community] packages"; unless there are technical limitations
> preventing the use of brackets. These brackets make it much clearer that
> the names are referring to pacman repositories; and to a novice Arch
> user, it may not be 100% clear what a "community package" or an "extra
> package" is.

I'm in favor, this should make it more clear where to report bugs, also
with the automatic product selection.

--
Jelle van der Waa

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

arch dev mailing list
In reply to this post by Lukas Fleischer-2
On 15.11.2017 09:07, Lukas Fleischer wrote:
> Generally speaking, I am in favor of setting up a fresh Bugzilla
> instance and not doing any sort of migration. The idea of switching to a
> new bug tracker came up at least four years ago and my feeling is that
> all this migration work is what was holding it back for a long time.

That's not really true. The most important problem back then was that
bugzilla only supported cgi and mod_perl. cgi performed really badly
when I tested it and mod_perl didn't perform much better because it just
hid the cost of compiling everything by using preforked workers. At
least until the pool of preforked workers was used up and it needed to
start new ones.

In the end the server was able to compile bugzilla roughly 8 times per
second using all cores and with a sufficient number of requests that
also seemed to affect mod_perl.

Nowadays bugzilla finally has psgi support where the perl code is
compiled once and requests are sent to it. At least without data this
leaves us with a few hundred requests per second so the base performance
is much much better.

Sadly it looks like the migration script, that once worked pretty well,
no longer works out of the box. Jelle has been working on reviving it.

Florian


signature.asc (875 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Giancarlo Razzolini-2
In reply to this post by Lukas Fleischer-2
Em novembro 15, 2017 6:07 Lukas Fleischer escreveu:
> However, we should make sure that the old Flyspray URLs still work (and
> redirect to the Flyspray archive).

Even though flyspray URL's are not really flyspray's and are more nginx's [0],
if there is no clash with bugzilla, I don't see any reason why we would not
be able to redirect them to the flyspray archive.

But what I have in mind for this is to have flyspray hosted under a different
domain name. Something like old-bugs-meant-to-die.archlinux.org.

Regards,
Giancarlo Razzolini

[0] https://git.archlinux.org/infrastructure.git/tree/roles/flyspray/templates/nginx.d.conf.j2

attachment0 (887 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Morten Linderud-2
In reply to this post by Jelle van der Waa-2
On Tue, Nov 14, 2017 at 08:30:21PM +0100, Jelle van der Waa wrote:

> [snip]
> # Products
>
> * Arch packages (core/extra or split this up)
> * Community packages (community)
> * Pacman
> * AURWeb
> * Keyring
> * Archweb (new)
> * Arch VM / Docker images (new)
> * Release engineering
>
I think core and extra should be split up. There is also some work being done to
help the communication between the testers, currently discussed at
#archlinux-testing. I propose another component could be `testing` that
encompasses bugs found from testing, community-testing and multilib-testing.

It would make be a lot simpler for testers and others using the testing repos to
find bugs and issues holding back package releases.

--
Morten Linderud

PGP: 9C02FF419FECBE16

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

Jelle van der Waa-2
On 11/16/17 at 02:01pm, Morten Linderud wrote:

> On Tue, Nov 14, 2017 at 08:30:21PM +0100, Jelle van der Waa wrote:
> > [snip]
> > # Products
> >
> > * Arch packages (core/extra or split this up)
> > * Community packages (community)
> > * Pacman
> > * AURWeb
> > * Keyring
> > * Archweb (new)
> > * Arch VM / Docker images (new)
> > * Release engineering
> >
>
> I think core and extra should be split up. There is also some work being done to
> help the communication between the testers, currently discussed at
> #archlinux-testing. I propose another component could be `testing` that
> encompasses bugs found from testing, community-testing and multilib-testing.
>
> It would make be a lot simpler for testers and others using the testing repos to
> find bugs and issues holding back package releases.
Sounds good, I intent to start with a small PoC. But migration is a
little bit more troublesome now, since I used an old db for conversion
while the flyspray db changed it's schema.

--
Jelle van der Waa

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Switching the bugtracker to Bugzilla

David Runge
In reply to this post by Jelle van der Waa-2
Sorry for being late to the party.

On 2017-11-14 20:30:21 (+0100), Jelle van der Waa wrote:
> It seems to be time to move on, and Bugzilla is one of the more active and
> maintained bugtrackers out there. Used by several big projects such as
> Gnome, LLVM and Mozilla. One of the benefits of moving would be the
> possibility of default assignees per 'component'.
I guess the decision has already been made towards bugzilla, but if not,
MantisBT [1] is another "only bug tracker" web application.
It has quite improved over the recent versions concerning usability
(nicely scalable on small screens) and is actively maintained.
Default assignies can be made "by category" afaik.
I can't say, if it fits all or any of the other requirements you're
seeking though.

btw: I'm suggesting this not because I package mantisbt in AUR, but
because I don't like using bugzilla too much ;-)

> # Migration
I would probably also opt for archiving and starting from scratch, if it
indeed takes too long to sieve through and find relevant open bugs.

> # User migration
This would be great, but will be an ugly script job in any case ;)
Additionally: If the relation to the user's bugs can be transferred,
don't bother.

> # Products
Separating core/extra and a separate testing would be great.
Although some "products" have found new bug tracking homes, I'd much
rather see them unified under bugs.archlinux.org, than scattered.

> Input would be welcome, on what we should migrate from flyspray and
> what products we should define.
Sorry for probably adding more noise.

In any case, thanks for the work!

Best,
David


[1] https://www.mantisbt.org/

--
https://sleepmap.de

signature.asc (849 bytes) Download Attachment