[regression] mouse pointer support on emulator is broken

Here are actually three problem at work:
 * The input device exposed inside the emulator /dev/input/event0 only emit absolute coordinate events. We recently experienced something similar with other incarnation of qemu in kvm: https://bugs.launchpad.net/mir/+bug/1489522 ( in that bug two mouse devices are available, the one that claims to expose relative events never emits them) The actual cause might be in the emulator integration itself.
 * The input device only emits those event on mouse button down. WIthout pressing a mouse button the evdev device remains silent. Another emulator problem?
 * The android input stack used in mir does not care about absolute mouse movement events. So even on mouse down it does not handle those events. The upcomig mir will, since it will default to mir-platform-input-evdev. So this will disappear with 0.18

At the moment you can see the mouse moves when you change to the new stack:
apt-get update && apt-get install mir-platform-input-evdev3
then edit /usr/share/ubuntu-touch-session/usc-wrapper
and add
 --platform-input-lib /usr/lib/i386-linux-gnu/mir/server-platform/input-evdev.so.3
to the call to unity-system-compositor

Or is the intention of the emulator to simulate a touch screen - hence the input events only appear on press?

After some digging inside the code of the emulator.. The behaviour described above seems to be intended - at least to some degree.

To move the mouse the user has to press F6 to toggle into trackball mode and use the mouse cursor. And it should work as expected.

is this not a valid bug then? or does the default need to change?

After trying around a bit more... If we intend to use the emulator in the future we might want to tweak which input devices are exposed the guest system. RIght now there is a single odd abs-relative-touchscreen-mouse hybrid. For odd reasons it was treated as touch screen in the past (we didnt really differ with the android stack: everything not a key event is a motion event). With libinput it will be seen as a mouse that either emits absolute or relative coodinates. So we will lack the capability to simulate touch input.

Additionally the relative mouse mode works very badly. It seems to skip a lot of mouse movement or they are scaled down?

For either of the problems we might have to change the emulator window code. Which I believe we take from here:

https://android.googlesource.com/platform/external/qemu/+/master/android/skin/

Status changed to 'Confirmed' because the bug affects multiple users.

It is a valid point, I suppose we should expose a setting that would allow the user to switch between a mouse input and touch input simulation. We might need a separate bug for that though.

That being said, is the emulator working now as before? Would like QA to validate if it's at least better now and allows to continue testing the few emulator bits we do.

Emulator comes up, cursor moves however it doesn't match the position of the mouse cursor so clicks are failing and I can't get off the first page of the welcome wizard.

@Dave did you toggle the mode with F6?

FWIW do we have an idea how this all *should* work?

Having a mouse device in a windowed emulator feels wrong unless we have the cursor contained/grabbed, otherwise you won't be able to push against the edges for launcher/spread etc.

IMO by default the device should mimic touch events, with a switch to mouse events that would contain the cursor in the emulator. This way you can easily switch between windowed and touch scenarios, too.

Right - so leaving this comment here to prevent me from asking anpok again ;)

The solution believed to be correct is to modify the Android emulator code to properly emulate a touch device (which is actually a mouse). Not much we can do on our side.

I think the emulator is correct, albeit weird. Qemu provides a tablet virtual device (so you don't get two mouse pointers and can enter/leave the VM window freely), and it works well with Ubuntu and Windows. And it sounds like it worked well with Mir till recently.

The solution is not to modify the emulator, because the emulator is correctly designed to emulate a tablet. Much like that of a stylus-based tablet where the mouse cursor does follow your hovers if you're close enough, or you can jump to a new location. This is a mature established feature of Qemu that's been there for many years. So let's just fix Mir/libinput.

P.S. I have a real-life tablet device (X220t) of this sort and have tested it with Mir over the years. You can see the bugs here:
https://bugs.launchpad.net/mir/+bugs?field.tag=wacom

Although I suspect it may not work with Mir as well as it used to when I logged those bugs.

To elaborate... what you see on a "traditional" stylus-based Wacom tablet is probably what Qemu is implementing. That means:

* Stylus is near (within a few centimetres) but not touching the screen: Mouse cursor visible and follows the pen (but ideally shows something different to an arrow, like a pencil cursor).

* Stylus touches the screen: That's a touch or primary button-down event, and button up event for release.

* Stylus moves away from the screen: No cursor is visible.

I think that's what Qemu is trying to emulate and it's probably a good model for our controlling a virtual touch screen with a mouse too. So we should fix libinput and/or Mir to work in this environment. Obviously it did until recently.

Status changed to 'Confirmed' because the bug affects multiple users.

This is not a stylus emulation.. Touch screen and Mouse devices overlap in the axes they use. SInce mice with absolute coordinates are allowed too (i.e VMWare Mouse Emulation). A device that tries to mimic both will run into problems, since there are differences in the press/release behavior. If you then go on and add multitouch axes to the mix it will get even worse, because suddenly the difference between a touchscreen a touchpad and a mouse is lost. A device that changes its behavior without notifying user space is not really helping here either. The android-qemu fork could solved that in various ways. Especially not cramping everything into a single device would have helped a lot. But I will focus on getting the touch screen emulation working again. It does now locally after removing the relative mouse events, but still the edge swipes do not work. Need to look into that further.

I'm told that this bug is a promotion blocker for stable channel emulator images. With vivid going EOL next week, the security team will need to have working emulator images for at least stable and rc-proposed to properly support touch devices since everyone doesn't have devices for testing. Can we get this fixed soon?

It's not off our radar i promise

This one is more opinion than invalid for the Mir project. Indeed the bug started happening after a change in the Mir project.

I wanted to share some status update, the emulated touchscreen troubles have been resolved. Now ui elements fail to detect any gestures, the display claims its phsyical width, height and dpi to be 0.

Mir report 0x0 is the actually something to be fixed in mir to get that bug resolved.

Mir problem is solved, currently not yet integrated into upcoming 0.20.
quemu fix is on the way:
https://code-review.phablet.ubuntu.com/#/c/431/

Fix committed into lp:mir at revision 3314, scheduled for release in mir, milestone 0.21.0

We should probably consider backporting this fix to 0.20.0

> We should probably consider backporting this fix to 0.20.0
totally agree

Fix committed (backported) to lp:mir/0.20 at revision 3319. Scheduled for release in Mir 0.20.0

This bug was fixed in the package mir - 0.20.0+16.04.20160219-0ubuntu1

---------------
mir (0.20.0+16.04.20160219-0ubuntu1) xenial; urgency=medium

  [ Alan Griffiths ]
  * New upstream release 0.20.0 (https://launchpad.net/mir/+milestone/0.20.0)
    - ABI summary: Only servers need rebuilding;
      . mirclient ABI unchanged at 9
      . mirserver ABI bumped to 38
      . mircommon ABI unchanged at 5
      . mirplatform ABI unchanged at 11
      . mirprotobuf ABI unchanged at 3
      . mirplatformgraphics ABI bumped to 8
      . mirclientplatform ABI unchanged at 4
      . mirinputplatform ABI unchanged at 5
    - Enhancements:
      . Allow screencasting to create a virtual output (for Miracast)
      . Separate the protocol version number from the client API version macros.
        They're not meant to be related concepts.
      . Add UBSanitizer to the list of build types.
      . logging: Human readable timestamps in DumbConsoleLogger.
      . examples: AdorningDisplayBufferCompositor::composite() no long ignores
        output boundaries and occlusions.
      . examples: Add -a <app name> option to eglapps.
      . common, client: a more flexible way to probe modules: once we've found
        a good current platform we don't even try to load an older one.
      . Fix build and test run with CMAKE_BUILD_TYPE=ThreadSanitizer (missing
        locks).
      . Add MIR_USE_LD_GOLD build option.
    - Bug fixes:
      . unity-system-compositor crashed with std::runtime_error in
        mir::compositor::CompositingFunctor::wait_until_started() from
        usc::MirScreen::set_screen_power_mode (mir_power_mode_on)
        (LP: #1528384)
      . Phone not usable while a call comes in - followed by "restart"
        (LP: #1532607)
      . ui freezes when simultaneously moving mouse & plug/unplug hdmi
        (LP: #1538632)
      . Mir fails to build on xenial today: android_graphic_buffer_allocator.h
        fatal error: hardware/hardware.h: No such file or directory
        (LP: #1539338)
      . [mali] egl_demo_client_flicker has graphics corruption on android
        (LP: #1517205)
      . [testsfail] Intermittent failure in
        TestClientCursorAPI.cursor_passed_through_nested_server (LP: #1525003)
      . [android] External monitor slows rendering (LP: #1532202)
      . Display::create_gl_context may create context with incorrect attributes
        (LP: #1539268)
      . unity-system-compositor locked up in __libc_do_syscall() (LP: #1543594)
      . NestedServer.client_sees_set_scaling_factor intermittent failure
        (LP: #1537798)
      . [android] External monitor slows rendering - part 2 (LP: #1535894)
      . scene: make sure not to set the swapinterval to 0 when an independent
        stream is created. The default should be 1 (like the stream created as
        part of surface creation).
      . Track the displays plugged state to avoid reporting configurations in
        case they are unplugged (LP #1531503). [Cherrypicked from 0.21]
      . mouse pointer support on emulator is broken (LP: #1517597).
        [Cherrypicked from 0.21]
      . move an android-only test that ended up in tests/unit-tests/graphics.
        (LP: ##154...

This bug got separate (diverged) fixes in Mir 0.21 and 0.20. So that's why it has two separate tasks.

New release for the android package is in progress now at https://requests.ci-train.ubuntu.com/#/ticket/1081

this is now showing landed