And when I say "slow", I mean _slow_. The lag between input and output easily reached over 1 minute, which made working like that impossible.
Obviously, the first idea was that there was some kind of network issue. Especially since some of our remote locations are still connected with 2MBit DSL. But the networking department quickly dismissed the tickets with "ping works".
Therefore, it was up to me to figure out what was going on. I tried connecting from my main workstation to some of my test machines - all fine. I tried connecting to some randomly selected remote locations - all fine. I even tried all possible combinations of Windows 7 and Windows 10 that could happen.
- Windows 7 to Windows 7
- Windows 7 to Windows 10
- Windows 10 to Windows 7
- Windows 10 to Windows 10
For me everything was working fine.
So I asked the people who reported the issue for more details. Things like what software they were running at the time, what software the remote machine was running, and so on. It turned out that the remote machines all seemed to have Microsoft Office 2016 applications (Word, Excel, ...) open at the time the issues occured. And what's more, when the person doing the remote assistance pressed the "actual size" button in MSRA the issue went away...
At the time I was already using a Windows 10 system as my main workstation (to figure out during daily usage what options I still need to configure for all future users) while my co-workers were all still on Windows 7. So maybe the problem exists only with Windows 7?
So I tried to replicate the issue by using a Windows 7 machine to connect to another Windows 7 machine, while running some Office 2016 application on the remote system. But again, everything was working fine for me. So what was I missing? Right ... pressing "actual size" fixed the error for the people who were having the issue... d'oh ... the remote system only had a 19" monitor and did not support a fullhd resolution. Thus the remote window was smaller than my own monitor's resolution.
I connected a bigger monitor to my test machine and changed the resolution to 1920x1200, the same as my main workstation. And there it was, the remote assistance connection slowed down to a crawl. Pressing the "actual size" button made the cursor blink wildly (catching up to the missed frames) and within seconds input and output were pretty much instant.
After I was able to successfully reproduce the issue I started investigating what was actually happening. The problem only happened when an Office 2016 application was running ... it did not happen with Office 2010. It was also only happening between two Windows 7 machines. Windows 10 seemed to "just work"(tm). And it only happened when the remote machine was running the same or higher screen resolution as the machine I was connecting from, resulting in the remote assistance software having to shrink down the remote desktop because of task- and titlebar.
So what did I know so far?
- Remote assistance session between Windows 7 and Windows 7
- An Office 2016 application must be running
- The remote machine must have the same or a higher screen resolution as the local machine resulting in the local machine shrinking the remote desktop in order to display all of it without scrollbars (when you click "actual size")
People had the "actual size" button as a workaround but of course having scrollbars and having to scroll to get to the start menu of the remote machine was not an option, especially with the number of remote assistance session we (sometimes) have to do daily.
But what could be the underlying reason? Office updates could not be the reason because the first reports came in at the end of February. The January updates had been installed between 18.01. and 26.01. and there were no updates in February because Microsoft had pulled them last minute. So what changed between Office 2010 and Office 2016 (we skipped Office 2013 completely)? It must be something related to graphics. Hardware acceleration in Office maybe? Nope, not it. Then it dawned on me ... Office animations. Microsoft added funky animations in Office 2013.
Disabling those animations indeed fixed the issue. You can do that by either setting the DWord value "DisableAnimations" under "[HKEY_CURRENT_USER\Software\Microsoft\Office\16.0\Common\Graphics]" to "1".
Alternatively, if you have the Office group policies available you can find the option under "User Configuration > Administrative Templates > Microsoft Office 2016 > Miscellaneous > Disable Office animations".
Now that the issue had been fixed ... why was it happening so frequently all of a sudden?
Since the end of the previous year, we were actively migrating all workstations from Office 2010 to Office 2016 and in February and March the last big batches were done. But that was not all ... in February the hardware department started replacing old 19" monitors with newer models capable of fullhd resolutions.
So the increasing number of Office 2016 installations and at the same time increasing number of workstations that were actually running fullhd resolutions meant that the chance of encountering such a machine while trying to provide remote assistance to a user was increasing too.
Sadly my boss did not accept my proposal on how to fix this issue besides disabling the Office animations ... I still do not know why he did not want to buy two or three 4k monitors for every 2nd-Level-Support employee. :/
[1] https://twitter.com/BeingSysAdmin/status/892698767878754304
No comments:
Post a Comment