2020-02-10

[Hardware] Fujitsu Esprimo Q958 will not power down with USB device connected


A couple of months ago, around August-September I believe, we got a new model of Fujitsu Mini-PCs – the Esprimo Q958. We had already been using the Q957 without any troubles for a while so we did not expect to face any problems with this new iteration either. Besides a newer CPU generation (Core i5 8th Gen compared to Core i5 7th Gen) nothing had really changed, we thought.

But soon after deploying a number of those new Fujitsu Esprimo Q958 machines to our users we got weird reports of machines not fully powering down when shutting down Windows.

The machines would simply stay powered on when you pressed the “Shut down” button in Windows. The machines would keep running, with power management disabled, and thus the CPU would soon get warmer and warmer, making the fan turn on to 100%, which in turn resulted in a lot of noise thanks to the very small fan.



After some investigation we figured out that it only happened to machines with a certain USB device connected to them. Depending on the USB port the device was connected to even the keyboard and mouse LEDs would stay on when shutting down Windows.

We tried the USB device on other Fujitsu Esprimo devices, like the Q957, the P756, P757 and P758 and even Lifebook devices. But only the Q958 was affected by this issue.

The USB device in question was a “USB Button V2.0” from http://usbbutton.com/ that is being used as a kind of panic button for counters where our employees interact with customers directly. When a customer becomes aggressive or violent they can press the button to alert co-workers and even the police directly.
These USB buttons are supplied by the company that provides the alarm software - https://www.gisbo.de/.

The company had started to deliver a different kind of button a while ago but we still had dozens of the older V2.0 models in use. Since these were by no means cheap, we clearly needed a solution for the issue.

At this point the only solutions to properly turn off the affected machines with the connected USB buttons were to either shut down Windows and then hold down the power button for 10 seconds to forcefully power off the hardware, or to disconnect the USB button before shutting down Windows. Both options are obviously sub-optimal.

Crawling the internet for a solution I came up with nothing. Playing around with power saving options and hardware power options in Windows lead to no result either. I updated to the latest Windows 10 version, updated all drivers and the BIOS. I even installed older versions of the BIOS back to the version that first introduced support for the 8th Gen Intel Core CPUs. But nothing I tried worked. The machines simply refused to properly power off.

I even went as far as taping off the data pins on the USB plug. While that made the machine power off properly, it also turned the USB button itself into a worthless pile of plastic. So it was not a very useful workaround. ;)

We contacted the software company we got the buttons from and they told us that they had another customer with the same computer, the same type of USB button and ... the same problem we were having. They however had found a “solution”. To disable “Wake-On-LAN” in the BIOS. And indeed, if you disabled WOL in the BIOS the hardware would properly shut down. But we require WOL to be functional to be able to do our job since we support devices all over the city in over 150 locations. Working without WOL is simply not an option.
So I tried asking on the Fujitsu Support Forum both in German and in English but the support I received was underwhelming – if not to say none at all. Same goes for the thread I started on Reddit about this issue.

Trying to submit a ticket directly with Fujitsu failed because their contact form failed to send my request, no matter which browser I tried it with.

In the end we decided to buy the newer version USB buttons and replace the old ones whenever a problem occurred. We could still use the older buttons on non-Q958 machines after all.

With the support person from the software company I had been discussing the issue with I had an agreement that we would call each other if new information surfaced.

Fast forward to last week.

While working on a completely unrelated issue I was updating all drivers on a Q958 device and noticed that there was also a new BIOS update available.

I checked the changelog for the BIOS and found the following:

27-Jan-2020
BIOS - Change V5.0.0.13 R1.23.0 for D3613-A1x
====================================================
- Microcode updates added (Coffeelake B0/P0/R0/U0).
- New Intel(R) ME Firmware version 12.0.49.1534 implemented.
- Solved: System shutdown not working with projector connected and some BIOS Setup settings.

Reading “System shutdown not working” triggered my curiosity but what possible relation could there be between a “projector” and a “USB button”. Nevertheless, I was intrigued and decided to test the USB button again.

So I installed the BIOS update on the machine I was working on, connected the USB button and tried shutting down Windows. The system properly turned itself off. No fan spin, no lit up LEDs. Total silence!
I used three more Q958 machines I had readily available in my vicinity. All three of them would refuse to power off after shutting down Windows. So I installed the BIOS update on them and tried again. And behold! All three of them would now properly power off, even with the USB button connected.

I am still not sure what USB buttons have got to do with projectors, but at this point I am happy that we have found a solution for this problem that had cursed us for almost half a year.

Obviously, I informed the software company of the alarm software so they could forward that information to their other customers.




[4] https://twitter.com/BeingSysAdmin/status/1226873853282078730

No comments: