ERROR: Failed writing message to '/cint/share/msgbus/cfgmgmt/datastore/7287/async-msgs.db': I/O error or out of disk space.

[Rajesh Kushwaha]

Hello Team, We are facing 1 ems instance failover in every day consistently with below error logs :- ERROR: Failed writing message to '/cint/share/msgbus/cfgmgmt/datastore/7287/async-msgs.db': I/O error or out of disk space. ERROR: Exception trying to process message from store. ERROR: Exception preparing message for client send (writing appendix length): Insufficient buffer ERROR: Exception preparing message for client send (writing appendix length): Insufficient buffer Can you please check and suggest that how we can fix this issue permanently.

[Richard Flather]

Not much information to go on here. I assume this is a shared storage for EMS on a NFS mount. Is this a virtual FS (VMDK) or a physical device

If this is a vmdk, was the the FS recently expanded You need to check. Expanding the FS while EMS is running can cause issues. Second, are you running the most current version of ESXi with the latest drivers Problems have been seen when VMDKs are used for EMS shared storage, and VMWare is not kept up to date.

If this is a physical device, verify NFSv4 is being used, TCP protocol is being used, and finally there is sufficient disk space.

You should be using EMS 8.4 or greater, and shutdown_on_disk_error in the EMS is enabled. See the EMS documentation for more details.

[Rajesh Kushwaha]

Thanks Rflather for your comment.But please let me know how to check the version of ESXi and also our ems server are configured on VMDKs then how to check the proper config files are updated of VM or not.

 

Last monday there we another problem occurred that ems instance going into unknown state if we are starting the service and if service is in stopped state then ems instance are visible to Gems monitoring tool. So could you please suggest on the same what exact issue is and do we haveany controlled over failover of ems instance

[Richard Flather]

Hi,

 

Only your OPS team can asnwer your questions regarding what version of ESXi you are using. You need to reach out to them on what version and on how the VMDKs are configured.

 

Also, on the second issue. This too, could be VMWare related if there aren't enough system recources available on the host to support all of the guest VMs running.