vSpere HA agent Unreachable

http://communities.vmware.com

Hello, We have a setup where there is a cluster of 2 ESXi hosts. We upgraded ESXi from v5.0 to v5.1. ESXi Node 1 upgraded successfully without any issues. When I tried upgrading ESXi Node 2, it just wouldnt upgrade. It used to fail with the error "Cannot execute upgrade script on host". Hence, I checked for the folder state.xxxxx in /bootbank and wasnt there. Hence, I checked for the same in /altbootbank. I moved the file local.tgz one level up in parent folder /altbootbank. Once this was done, I was able to upgrade the host. However, the ESXi Node 2 shows error regarding HA Agent unreachable. I tried the option Reconfigure Host for HA  with a no-go. Also, i checked for the SSL or failed error under logs specified but the errors not visible. Following are the KB articles I already tried 2008609, 2007739 and 2006729. I tried rebooting host once and it was stuck on booting files. I had to ask DC guys to physically reboot the machine. Also, before the reboot, there was another error which is "vSphere HA Agent is in Initialized State" and currently stuck up with Unreachable error. Also, I did check with the hostd services on ESXi host. I logged in through SSH and restarted the service and then tried reconfiguring HA using vSphere Client, but still the same. I do find this log difference under fdm-installer.log file. Earlier it used to show up Message Output as Operation Finished Successfully and now Host is not changed. I believe this could be the issue because the VIB's skipped file shows file relation with bootbank and fdm. =====================================================================fdm-installer: [6876] 2013-06-12 12:23:22: Installing the VIBfdm-installer: [6876] 2013-06-12 12:23:24: Result of esxcli software vib install  -v=/tmp/vmware-root/ha-agentmgr/vpx-upgrade-installer/vmware-fdm.vib: Installation Resultfdm-installer:    Message: Host is not changed.fdm-installer:    Reboot Required: falsefdm-installer:    VIBs Installed:fdm-installer:    VIBs Removed:fdm-installer:    VIBs Skipped: VMware_bootbank_vmware-fdm_5.1.0-1123961===================================================================== As of now HA is broken between the ESXi nodes. Kindly suggest if you could something. 

Read »