Application mode setting emulation in XWayland


Most Operating Systems (OS) rely on what’s called “Windowing System” for graphical output, allowing to display and manage several applications on a single screen.

Windowing systems usually have two key components, a Display Manager (DS) and a Windowing Manager (WM). The former, among other things, is the entity programs send their graphical output to, and which manages the interaction with the hardware. It usually does so by exposing different APIs to programs, like OpenGL. The later is responsible for the logic how applications are arranged on a screen, for examples as freely movable windows or as tiles. In many cases, Display Manager and Windowing Manager are implemented as a single program, but that doesn’t need to be.

Historically, Linux and UNIX-like operating systems (OSs) like BSDs used X11 as their display server protocol. Conceptional, it was not intended to incorporate a Windowing Manager, leaving this task to third parties. Widely used WMs are Mutter (of Gnome-Shell) and Kwin (of KDE).

In recent years, some architectural shortcomings in the design of X11 lead to development of Wayland, a new protocol with a more modern and dramatically different architecture. In contrast to the X11 world, the current practice is to combine DS and WM when implementing Wayland support. So instead of having a X11-Server like Xorg as DS and e.g. Mutter as WM, now the WMs each offer their own Wayland-Server implementation, becoming a Wayland Compositor (WC). It should be noted, though, that this architectural change won’t necessarily last (see Gnome-Shell 4 plans).

Big efforts have been made to port programs and tool-kits to Wayland, each needing modifications if they used the protocol directly. To make this multi-year effort easier, the developers of Xorg, the by far most widespread X11-Server implementation, created XWayland, an extension of Xorg that allows it to be run as Wayland-client, thus making it possible to start X11-clients, whose protocol calls would then get wrapped to the Wayland equivalents.

While XWayland and the WCs succeeded to bring X11-compatibility under Wayland to a level where most programs integrate seamlessly, some functionality of the X11 protocol purposely doesn’t have a equivalent in the Wayland world. One of them is what we’ll call “application mode setting”. Mode in this context means number of pixels, vertically and horizontally, on the screen, also called resolution, together with the frequency how often they get refreshed every second. A typical mode would be 1980x1080@60Hz (FullHD), which is used by many modern displays. Since the introduction of LCD-panels, displays usually only have one “native” mode, which is always used, while being able to emulate other modes, too, for compatibility reasons. This used to be different with tube display, which usually were able to really operate in different modes.

Back in the time when tube displays were the norm, many programs were created with one or a limited list of resolutions in mind, as the techniques used back then generally made it hard to dynamically support arbitrary resolutions. For this reason, operating system usually allowed any application to set the mode it needed: application mode setting. With modern painting APIs, such as OpenGL, it is easier, cleaner and very efficient to just scale graphics to the desired size. Therefore the Wayland-protocol never specified application mode setting. In the context of this bachelor theses we want to use scaling operations to emulate application mode setting in XWayland and thus close one of the last remaining compatibility gaps adherent with using a Wayland Desktop Environment.

Problem statement

XWayland doesn’t support application mode setting, which is needed by a big number of applications. Most notably affected applications are closed-source games, which in many cases don’t receive long vendor support after release and will therefor not adapt to new protocols.

Solution statement and Project Structure

The Wayland-protocol provides a wide number of sub-protocols that XWayland can make use of. Here we want to make use of the wp_viewporter protocol, which provides scaling and cropping capabilities. Overall, we want to implement the following capabilities:

XWayland 1. Expose multiple modes to clients. The modes get obtained from the Wayland-Compositor, as XWayland doesn’t directly interact with the hardware. 2. Allow applications to set modes and remember their decision. Many applications re-request the current mode even if they set it themselves, for example after they got minimized and maximized again. 3. Set viewports on the corresponding apps accordingly, using the wp_viewporter protocol 4. As input events don’t get affected by viewports, map input events, essentially mouse events, accordingly to the viewport.

Mutter Currently only the reference Wayland-server implementation, Weston, supports wp_viewporter. To make the capability useful for end users, we want to implement everything needed in at least one major WC. Mutter is the default WC on many of the most widely used Linux distributions and generally has very mature Wayland support, making it a natural choice. 1. Implement the wp_viewport protocol 2. Implement exposure of widely used modes to XWayland. Most notably those with the sizes 640x480, 800x600 and 1024x768 pixels 3. Make Mutter properly handle X11-clients from XWayland with a viewport set. Especially, make sure fullscreen behavior is properly detected, as this includes guess-work in many cases as many legacy applications use non-standardized ways to archive that

Status quo

Most points from above have been successfully implemented and can get used already. A wide range of tested applications are presented mostly in the desired way. The implementations remain rough though and some corner cases may require cleaner approaches. It’s important to note that Xorg and Wayland are about to have major version bumps around April/May, so the current work, which was created against the current stable versions, will have to get rebased. This is important as many maintainers are currently busy finishing up that work before being free again to discuss and merge new work.

XWayland 1. Mode exposure was implemented by a Xorg developer after talking about the issue, in a ‘hold my beer’ fassion. Not upstreamed yet. 2. Mode setting so far only works in a not remembering way. This is already enough for a majority of applications but does create problems with applications using mode setting in unorthodox fassions 3. There’s a quite well working logic in place to detect fullscreen applications. But it currently relies on guess work, not being able to directly relate mode setting with window behavior. There are a number of corner cases not working with the current approach and we now aim for a clean relation method between mode setting and windows 4. Mouse wrapping works but in some cases is restricted to a box equivalent to the requested mode. This should be relatively easy to fix Mutter 1. wp_viewporter is implemented and works for reference test. Some Mutter-specific behavior needs to get discussed with the other developers, as it’s not specified how to handle it 2. Mode exposure is implemented, but currently in a static way. It might be desirable to make this more dynamic/configurable 3. Some required changes for fullscreen handling has been done but isn’t finished yet for cases where the requested mode has a different ratio than the native screen (e.g. 4:3 on 16:9)