Mouse and keyboard freezes randomly, but otherwise system seems functional

Hello!

Mouse and keyboard keeps randomly freezing while everything else keeps functioning further, e.g., music and video keeps playing, new USB devices can be found (plugged in a camera, which shows up and a pop-up about importing pictures shows up).
Sometimes it happens right after the login, sometimes randomly while I work. Only way to get the system responsive again was to restart it.

I am using Ubuntu 16.04. Installed MATE desktop environment separately.