Allowing any app unrestricted access to the input and output of any other app (like in X11) is a terrible security practice. It allows for trivially easy keyloggers and makes horizontal movement to other apps after the first has been exploited super easy.
Many people’s answer to this is “then just don’t run untrusted apps, duh”, but that is a bad take since that isn’t realistic for 99% of users. People run things like Discord or Spotify or games or Nvidia drivers all the time, not to mention random JavaScript on various websites, so the security model should be robust in the presence of that kind of behaviour. Otherwise everyone is just a single sandbox escape in the browser away from being fully compromised by malware installed with root privileges. Luckily we know better now than when X11 was designed and that is the reason for things like Bubblewrap (used in Flatpak for sandboxing), portals and the security model of Wayland.
And in the end: the people who decided this are the people actually willing to do the work to build and maintain the Linux desktop stack. If anyone knows what the right approach is, it’s them.
What do you do on your machine that requires you to completely disable a fundamental security feature of your distro? I haven’t had SELinux cause any issues for years.