For some reason, this makes the movement of the mouse pointer very slow and makes it jump from time to time, so I ended up adding INTERRUPT_KEY_SCANNER and disabling the timer interrupt. Also double bufferred the mouse report just like the others. Unfortunately this does not affect this issue.
Shared files
These files are shared between multiple projects of this repository.