gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

David C. Rankin
Houston -- we have a problem.

  The problem is gcc is mis-applying optimizations, and it seems Arch
specific. See discussion:

[Why does not execute printf inside function?]
https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-inside-function/49249533#49249533

  The basic issue can be summarized in this short snippet:

#include <stdio.h>

int arr[10];

int func()
{
    printf ("In func\n");
    return 0;
}

int main (void)
{
    if (&arr[func()])
        printf("In main\n");
    return 0;
}

  Compile (with or without optimization) and the output on Archlinux (all
packages current as of 1hr ago) and you get:

$ ./bin/if2
In main

  That is wrong. `func()` has been completely optimized out of the procedure
in main(). The assembly generated omits any call to func. (full assembly
posted as part of the answer on SO)

  Debian gcc 7.3 does not exhibit this same behavior. I do not know where to
start looking to find why this is happening, but it does not take much of a
stretch to see how this could be the root cause of some of the "unexplained"
errors I and others have seen in the past few days on Arch.

  Let me know if I need to file a bug here. Since it "works on Debian" and not
here, I suspect the bug should start here and so the default optimization we
package with can be eliminated.

  What say the devs?


--
David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
Hi,

AFAIK this is the exact case of gcc bugzilla #84607:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607

As this is an upstream bug this should affect all distributions, maybe
the commenter on Debian is using 7.3.1 so he can't reproduce the
issue?

On Tue, Mar 13, 2018 at 2:36 PM, David C. Rankin
<[hidden email]> wrote:

> Houston -- we have a problem.
>
>   The problem is gcc is mis-applying optimizations, and it seems Arch
> specific. See discussion:
>
> [Why does not execute printf inside function?]
> https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-inside-function/49249533#49249533
>
>   The basic issue can be summarized in this short snippet:
>
> #include <stdio.h>
>
> int arr[10];
>
> int func()
> {
>     printf ("In func\n");
>     return 0;
> }
>
> int main (void)
> {
>     if (&arr[func()])
>         printf("In main\n");
>     return 0;
> }
>
>   Compile (with or without optimization) and the output on Archlinux (all
> packages current as of 1hr ago) and you get:
>
> $ ./bin/if2
> In main
>
>   That is wrong. `func()` has been completely optimized out of the procedure
> in main(). The assembly generated omits any call to func. (full assembly
> posted as part of the answer on SO)
>
>   Debian gcc 7.3 does not exhibit this same behavior. I do not know where to
> start looking to find why this is happening, but it does not take much of a
> stretch to see how this could be the root cause of some of the "unexplained"
> errors I and others have seen in the past few days on Arch.
>
>   Let me know if I need to file a bug here. Since it "works on Debian" and not
> here, I suspect the bug should start here and so the default optimization we
> package with can be eliminated.
>
>   What say the devs?
>
>
> --
> David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
On 3/13/18, PkmX via arch-general <[hidden email]> wrote:
> Hi,
>
> AFAIK this is the exact case of gcc bugzilla #84607:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
>
> As this is an upstream bug this should affect all distributions, maybe
> the commenter on Debian is using 7.3.1 so he can't reproduce the
> issue?

Let's not forget that Debian is patch happy (as is Ubuntu, logically),
especially for gcc. Mostly for good reason but quite a bit is too
many diffs:

https://sources.debian.org/src/gcc-7/7.3.0-11/debian/patches/

It's quite possible that the snapshot plus diffs fixes/hides the bug
on Debian. Fedora is another distro that is very happy to carry
many diffs for some packages. Thinks of the version string like
git-gc-7.3.1+dirty in mainstream distros. It's like a downstream
branch.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

Jelle van der Waa-2
In reply to this post by arch general mailing list-2
On 03/13/18 at 03:17pm, PkmX via arch-general wrote:

> Hi,
>
> AFAIK this is the exact case of gcc bugzilla #84607:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
>
> As this is an upstream bug this should affect all distributions, maybe
> the commenter on Debian is using 7.3.1 so he can't reproduce the
> issue?
>
BTW, cross checking with clang was also an option :-)

[jelle@starfighter][/tmp]%clang foo.c
[jelle@starfighter][/tmp]%./a.out
In func
In main

Feel free to make a bugreport on https://bugs.archlinux.org

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

David C. Rankin
In reply to this post by arch general mailing list-2
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:

> Hi,
>
> AFAIK this is the exact case of gcc bugzilla #84607:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
>
> As this is an upstream bug this should affect all distributions, maybe
> the commenter on Debian is using 7.3.1 so he can't reproduce the
> issue?
>

That is good to know that it is being addressed. I can see this current
handling of optimizations causing many of subtle non-traceable and just plain
"wonky" errors that have been see these past few days. I thought the guy who
posted it was nuts... It built and had the correct output on SuSE, but when I
tried it on Arch - WTF? they was right. Who knows in how many subtle ways that
could manifest itself. I've convinced it is as the root of the (Out of IOMMU
space) error I have in the logs, and more than likely it is responsible for my
hardlocks on the big servers. We will be looking for a fix -- hopefully soon.

Thank you for letting me know.

--
David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607

Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.


​Sent with ProtonMail Secure Email.​
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

David C. Rankin
On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
> On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
>
> Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
>
>
> ​Sent with ProtonMail Secure Email.​
>

Yes, confirmed it was indeed a bug:

  $ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
  $ ./bin/infunc2
  In main

And, with Jelle van der Waa's suggestion, I loaded clang (which I never
otherwise install) and it confirms the issue further:

  $ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
  $ ./infunc2
  In func
  In main

The only part I'm unclear on is where in the fix process
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
when that will trickle down to us.

--
David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

Jelle van der Waa-2
On 03/13/18 at 08:44pm, David C. Rankin wrote:

> On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
> > On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
> >> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
> >
> > Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
> >
> >
> > ​Sent with ProtonMail Secure Email.​
> >
>
> Yes, confirmed it was indeed a bug:
>
>   $ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
>   $ ./bin/infunc2
>   In main
>
> And, with Jelle van der Waa's suggestion, I loaded clang (which I never
> otherwise install) and it confirms the issue further:
>
>   $ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
>   $ ./infunc2
>   In func
>   In main
>
> The only part I'm unclear on is where in the fix process
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
> when that will trickle down to us.
In the comments it reads: "Fixed on the trunk." In commit https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=258061

As I suggested, create a bugtracker issue for gcc on our bugtracker
otherwise you'll have to wait on a new gcc minor release.

--
Jelle van der Waa

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

Bastian Beischer
In reply to this post by David C. Rankin
Hi David,

On Wed, Mar 14, 2018 at 2:44 AM, David C. Rankin
<[hidden email]> wrote:

>
> On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
> > On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
> >> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
> >
> > Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
> >
> >
> > Sent with ProtonMail Secure Email.
> >
>
> Yes, confirmed it was indeed a bug:
>
>   $ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
>   $ ./bin/infunc2
>   In main
>
> And, with Jelle van der Waa's suggestion, I loaded clang (which I never
> otherwise install) and it confirms the issue further:
>
>   $ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
>   $ ./infunc2
>   In func
>   In main
>
> The only part I'm unclear on is where in the fix process
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
> when that will trickle down to us.
>

The fix is available in the trunk and the gcc7 branch, but there
hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
they come. If you need the fix before that you need to open a ticket
on bugs.archlinux.org and ask for a backport of that particular
bugfix.

However, whether this will actually solve any real world problems in
your case: I am doubtful. Do you have any specific evidence which
links this particular bug to your symptoms? Please have a look around
the gcc bug tracker: You will see that there are hundreds of compiler
bugs, many of which more severe than the one you have quoted, but that
doesn't mean that Arch actually backports all of these individual
fixes all the time. I'm not saying it's impossible that the fix will
help you, I'm just saying it is not very likely. And if indeed your
symptoms are due to a gcc miscompilation, you would need recompiled
binaries of whatever software causes your problems as well.

>
> --
> David C. Rankin, J.D.,P.E.

Cheers
Bastian
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
In reply to this post by Jelle van der Waa-2
On 03/14/2018 04:26 AM, Jelle van der Waa wrote:
> As I suggested, create a bugtracker issue for gcc on our bugtracker
> otherwise you'll have to wait on a new gcc minor release.

Someone already did.

https://bugs.archlinux.org/task/57823

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

David C. Rankin
In reply to this post by Bastian Beischer
On 03/14/2018 03:44 AM, Bastian Beischer wrote:
> The fix is available in the trunk and the gcc7 branch, but there
> hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
> they come. If you need the fix before that you need to open a ticket
> on bugs.archlinux.org and ask for a backport of that particular
> bugfix.

Yes, thanks, I see that somebody beat me to it. Thank you.

https://bugs.archlinux.org/index.php?do=details&action=details.addvote&task_id=57823

--
David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
On 03/14/2018 07:03 PM, David C. Rankin wrote:

> On 03/14/2018 03:44 AM, Bastian Beischer wrote:
>> The fix is available in the trunk and the gcc7 branch, but there
>> hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
>> they come. If you need the fix before that you need to open a ticket
>> on bugs.archlinux.org and ask for a backport of that particular
>> bugfix.
>
> Yes, thanks, I see that somebody beat me to it. Thank you.
>
> https://bugs.archlinux.org/index.php?do=details&action=details. ohnoes addvote&task_id=57823
>
You just gave the exact same link as I gave 12 hours ago, except your
link (which I have broken in my quote by adding the string " ohnoes ")
will cause people who are logged in, to inadvertently interact with the
bugtracker by adding their vote for the bug.

This was *probably* an accident (flyspray links are sort of weird for
reloading the page and replacing the current url with this at all);
nevertheless, please don't do this again...

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

David C. Rankin
On 03/14/2018 06:52 PM, Eli Schwartz via arch-general wrote:

<quote>
This was *probably* an accident (flyspray links are sort of weird for
reloading the page and replacing the current url with this at all);
nevertheless, please don't do this again...
</quote>

Oh, sorry, and yes this was definitely an accident. While I see the logic
behind the *probably* comment, I assure you that wasn't intention, I simply
double-clicked the content of the address window and pasted -- will make sure
that does not happen again....

--
David C. Rankin, J.D.,P.E.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
On 03/14/2018 08:40 PM, David C. Rankin wrote:

> On 03/14/2018 06:52 PM, Eli Schwartz via arch-general wrote:
>
> <quote>
> This was *probably* an accident (flyspray links are sort of weird for
> reloading the page and replacing the current url with this at all);
> nevertheless, please don't do this again...
> </quote>
>
> Oh, sorry, and yes this was definitely an accident. While I see the logic
> behind the *probably* comment, I assure you that wasn't intention, I simply
> double-clicked the content of the address window and pasted -- will make sure
> that does not happen again....
:p

Good news, I guess, is that we will almost certainly switch to bugzilla
as soon as Harmony brings stable psgi support (Soon™).

--
Eli Schwartz
Bug Wrangler and Trusted User


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
On 3/15/18, Eli Schwartz via arch-general <[hidden email]> wrote:

> Good news, I guess, is that we will almost certainly switch to bugzilla
> as soon as Harmony brings stable psgi support (Soon™).

Harmony?
PSGI?

I like Bugzilla's features, and I don't use arch's tracker enough to have
a voice, or maintain it to have a say, but I would still like to provide
an opinion. It's just an opinion. Having used large bugzilla installations
like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla
can be fast. I've found Mantis and to some extend Redmine to be fast and
lean trackers. These may or may not suit Arch's needs, and I still have
hope Bugzilla will at some point become snappy.

Just an opinion based on experience, the arch admins/devs will know why
bugzilla is the right choice. I trust them.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

arch general mailing list-2
I should note that Jira takes the position of being the slowest bugtracker.
Nobody would use git if it was slow like Jira or Bugzilla.
Reply | Threaded
Open this post in threaded view
|

Re: gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems

Jelle van der Waa-2
In reply to this post by arch general mailing list-2
On 03/15/18 at 09:29pm, Carsten Mattner via arch-general wrote:

> On 3/15/18, Eli Schwartz via arch-general <[hidden email]> wrote:
>
> > Good news, I guess, is that we will almost certainly switch to bugzilla
> > as soon as Harmony brings stable psgi support (Soon™).
>
> Harmony?
> PSGI?
>
> I like Bugzilla's features, and I don't use arch's tracker enough to have
> a voice, or maintain it to have a say, but I would still like to provide
> an opinion. It's just an opinion. Having used large bugzilla installations
> like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla
> can be fast. I've found Mantis and to some extend Redmine to be fast and
> lean trackers. These may or may not suit Arch's needs, and I still have
> hope Bugzilla will at some point become snappy.
>
> Just an opinion based on experience, the arch admins/devs will know why
> bugzilla is the right choice. I trust them.
Please don't de-rail the thread. (Obviously the Arch devs also want a
performing bugtracker, but Flyspray has to go either way)

--
Jelle van der Waa

signature.asc (499 bytes) Download Attachment