tmux/systemd: tmux user service exits on detach/logout

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

tmux/systemd: tmux user service exits on detach/logout

David Runge
Hey all,

I got a very strange behavior with tmux and systemd on my server and
maybe someone knows a way around it.

The way I start tmux is as follows:
Setting TMUX_TMPDIR to my user's XDG_RUNTIME_DIR and starting tmux in a
systemd user service [1]. I do the same for other things, such as
weechat [2]. They both use the same tmux.conf [3].
Lingering is activated for my user, so the enabled user services will
start with the system and stay running, even when it's disconnected...
in theory (and this used to work very well, without a problem for
years).

With some changes in systemd however, this seems to fail now, and I
can't seem to find the reason why. Tmux just exits with "server lost"
once I do a detach.
At first I could only observe this behavior when I had a shared ssh
session to the server (so I thought), that I then quit, or when I
attached and detached the tmux session quickly.
Now I'm getting this nearly every time I detach from the tmux session!

I tried several things by now, and it doesn't seem to make any
difference:
- using legacy cgroups [4], because of maybe being affected by this [5]
- switching to single connections instead of shared connections for ssh
  (lowered the amount of random tmux quits!)
- using hardened kernel (just to try another one)
- logging a tmux session, without getting any output, that would
  explain, why this happens
- crossing fingers (actually helps, because the bug seems completely
  random ;/ )
- using a system scope unit (tmux@username) [6], getting the same results
- downgrading tmux to >=2.3 (no difference)
- downgrading systemd to <=233-7 (seems to happen less frequently)

For some awkward reason, it seems to have worked again for a short time,
when Arch first switched to systemd 233 a few days back [6], but ever
since it moved to systemd-stable [7], the problem is back (and even
worse than before). Me thinking that might only be due to a "less
frequent occurence" though.

I'm just wondering, how to solve this. Maybe someone has a great idea to
share? It's getting beyond annoying by now and I can't seem to find out,
if it's tmux, systemd or some kernel problem... or all of them together.

David


P.S.: Doesn't seem to happen with screen. I don't want to adapt
everything to screen now though ;)


[1] https://git.sleepmap.de/software/uenv.git/tree/user/tmux.service
[2] https://git.sleepmap.de/software/uenv.git/tree/user/weechat.service
[3] https://git.sleepmap.de/config/dotfiles.git/tree/.tmux.conf
[4] https://www.freedesktop.org/software/systemd/man/systemd.html#systemd.legacy_systemd_cgroup_controller
[5] https://github.com/systemd/systemd/issues/3388
[6] https://wiki.archlinux.org/index.php/Tmux#Autostart_with_systemd
[7] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/systemd&id=e0ee38b0c90d0a0da7baad4f472f38dd451326a6
[8] https://github.com/systemd/systemd-stable


--
https://sleepmap.de

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: tmux/systemd: tmux user service exits on detach/logout

arch general mailing list-2
Not sure if related, but try setting KillUserProcesses=no in
/etc/systemd/logind.conf

https://askubuntu.com/questions/802189/how-to-run-tmux-screen-with-systemd-230

--
André Vitor de Lima Matos


signature.asc (883 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: tmux/systemd: tmux user service exits on detach/logout

David Runge
Hey Andre Vitor,

On 2017-07-07 10:53:32 (-0300), André Vitor de Lima Matos wrote:
> Not sure if related, but try setting KillUserProcesses=no in
> /etc/systemd/logind.conf
>
> https://askubuntu.com/questions/802189/how-to-run-tmux-screen-with-systemd-230
That is an Arch default setting (commented variables in the systemd
config files reflect compile-time defaults).

Check with:
$ loginctl show-session|grep Kill

Also: The issue at hand is not a logind problem (at least not a
login/logout issue), as far as I can see, as it happens on detach from
tmux (but only sometimes).
That one often times disconnects from the machine after a tmux detach
doesn't help to figure out why it happens unfortunately :>

Thanks anyhow!

Best,
David


--
https://sleepmap.de

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: tmux/systemd: tmux user service exits on detach/logout

David Runge
In reply to this post by David Runge
Hey again,

On 2017-07-07 15:24:39 (+0200), David Runge wrote:

> I got a very strange behavior with tmux and systemd on my server and
> maybe someone knows a way around it.
>
> The way I start tmux is as follows:
> Setting TMUX_TMPDIR to my user's XDG_RUNTIME_DIR and starting tmux in a
> systemd user service [1]. I do the same for other things, such as
> weechat [2]. They both use the same tmux.conf [3].
> Lingering is activated for my user, so the enabled user services will
> start with the system and stay running, even when it's disconnected...
> in theory (and this used to work very well, without a problem for
> years).
>
> With some changes in systemd however, this seems to fail now, and I
> can't seem to find the reason why. Tmux just exits with "server lost"
> once I do a detach.
> At first I could only observe this behavior when I had a shared ssh
> session to the server (so I thought), that I then quit, or when I
> attached and detached the tmux session quickly.
> Now I'm getting this nearly every time I detach from the tmux session!
>
> I tried several things by now, and it doesn't seem to make any
> difference:
> - using legacy cgroups [4], because of maybe being affected by this [5]
> - switching to single connections instead of shared connections for ssh
>   (lowered the amount of random tmux quits!)
> - using hardened kernel (just to try another one)
> - logging a tmux session, without getting any output, that would
>   explain, why this happens
> - crossing fingers (actually helps, because the bug seems completely
>   random ;/ )
> - using a system scope unit (tmux@username) [6], getting the same results
> - downgrading tmux to >=2.3 (no difference)
> - downgrading systemd to <=233-7 (seems to happen less frequently)
After a week of testing: tmux - in above mentioned setup - works much
more reliable with systemd 233-7.

Maybe any of the package maintainers can comment on it?

> For some awkward reason, it seems to have worked again for a short time,
> when Arch first switched to systemd 233 a few days back [6], but ever
> since it moved to systemd-stable [7], the problem is back (and even
> worse than before). Me thinking that might only be due to a "less
> frequent occurence" though.
So, in conclusion, the issue is most likely related to some changes
applied to the 233 branch in systemd-stable.

> [1] https://git.sleepmap.de/software/uenv.git/tree/user/tmux.service
> [2] https://git.sleepmap.de/software/uenv.git/tree/user/weechat.service
> [3] https://git.sleepmap.de/config/dotfiles.git/tree/.tmux.conf
> [4] https://www.freedesktop.org/software/systemd/man/systemd.html#systemd.legacy_systemd_cgroup_controller
> [5] https://github.com/systemd/systemd/issues/3388
> [6] https://wiki.archlinux.org/index.php/Tmux#Autostart_with_systemd
> [7] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/systemd&id=e0ee38b0c90d0a0da7baad4f472f38dd451326a6
> [8] https://github.com/systemd/systemd-stable

Best,
David


--
https://sleepmap.de

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: tmux/systemd: tmux user service exits on detach/logout

David Runge
Hey all,

as follow up on this (probably not so interesting thread for too many),
I have tested systemd 234 successfully.
The weechat instance within tmux (or tmux itself) still ends
sporadically, but not the main tmux user session.
So, in conclusion, whatever has been broken in systemd 233, was probably
fixed again in 234.

Best,
David


--
https://sleepmap.de

signature.asc (849 bytes) Download Attachment
Loading...