Application on VM runs slower then expected on few ESXi hosts while no issue on others

This is second time when I observed, during some activities on one of my database server, its responding slower than expected even though not using much system resources.  Here the server was using >50% of CPU, but was struggling to complete the task however not throwing any kind of resources contention alerts. On checking found there was no disk, RAM or network issues however CPU latency was bit high <25%.

The ESXi 6.x host where this VM resides having plenty of available CPU/memory resources so what’s wrong here.

While comparing the configuration of two hosts (one on which VM have no perf issue and the other), I recalled we might missed the changes in the Hardware Power management setting from BIOS of the target host.

“Note: for the resolution of this issue, you may directly jump to last paragraph of this post.”

Physical servers power management setting is something that can have an impact on server CPU performance as when the load is low on server it reduces the CPU clock speed or shutdown some cores to reduce the power consumption.

Ideally we should change the power management settings from BIOS during OS/ESXi installation on Server but we can also do that later from iLO console.

By default HP ProLiant servers have the power management setting set as Dynamic Power saving mode to run the server CPU with its full capacity. We need to change it to Static High performance mode, it will disable any kind of power management from server.

Like ESXi, if your server OS supports the server hardware power management then for ProLiant servers, you can also set the power management setting as OS Controlled and then can choose the required setting from within OS like in case of ESXi, from host configuration power management setting.

To Change the Power Management setting from HP ProLiant server iLO :

After logging in to iLO, from lest side menu select Power management => Power Settings => From here under Power Regulator Settings,  select Static High Performance mode.

To Change the Power Management setting from within ESXi: If you want to control this settings from within OS or when Hardware power Regulator Settings is set to OS controlled then to change the Power Management setting from within ESXi,

After connecting to vCenter via Web client, Select the intended host => Mange => Settings => under Hardware tab drop down, select Power Management => From here Select, Static High Performance

If using thick client then,

Now come back to the issue of application performance, on checking I found the Hardware Power Regulator Settings was set to default on this ESXi host and once I changed it to High performance……..Voila now there no CPU latency and the application started performing as expected.

For detailed info, please have a look at these white papers,

HP white paper on Power consumption and system performance

Host Power Management in VMware vSphere, Performance Study

Power Management settings available within ESXi 6.x host

Related KB article, kb# 1018206

That's it... :)