Archive : VSAN

“Host cannot communicate with one or other nodes in the vSAN enabled cluster”

Highlight the host you are getting error on.

Select the Configure tab, System / Security Profile Services click on Edit:

Enable SSH on each host in the VSAN cluster:

Launch a terminal application like Putty to SSH onto your hosts.

Note: When restarting the vCenter Management Agents on the ESXi host, you will get a brief interruption to the host manageability. This is perfectly safe and will resolve itself automatically. In some cases, the host may momentarily enter a not responding state in vCenter Server, however this is rare.

Run the following command to restart the Management Agents:

/etc/init.d/vpxa restart

Run this command on all VSAN nodes. This updates the cluster information in vCenter Server and clears the message.

If you are still getting this error you can try the following:

Remove the host from the VSAN cluster and re-add. This forces all hosts to update the information about the cluster membership and clears the message.

Warning: When you attempt to remove and re-add the host, data synchronisation does not occur when the host is in maintenance mode and out of the VSAN cluster. If you select the Ensure accessibility maintenance mode option, some objects may be unprotected while executing the workaround.

To remove a host from the VSAN cluster and re-add:

  1. Select the ESXi host in the vSAN cluster that can be put to the maintenance mode temporarily
    Best Practice: VMware recommends to select the least busy/utilized host.
  2. In the vSphere Web Client, right-click the ESXi host and click Enter Maintenance Mode.
    Note: Select either the Ensure Accessibility or Full data migration options for vSAN maintenance mode.
  3. If applicable, allow powered-off virtual machines to be migrated to the remaining ESXi hosts.
  4. Remove the host from the vSAN cluster
  5. Select the host.
  6. Drag the host out of the cluster.
  7. Place the host in the datacenter object in the vCenter Server inventory.
  8. Note: After moving the host into the datacenter, wait for approximately two minutes
  9. Drag the host back into the vSAN cluster.
  10. After adding the host back in the vSAN cluster, right-click the host and click Exit Maintenance Mode.
  11. Note: The message on the remaining hosts should be clear.

Add the ESXi Host back into the VSAN cluster

Step out of your Comfort Zone? Hell Yes.

Sometimes in IT, when you are the infrastructure person in the company, you may spot a product or a solution that could be a good fit internally, or for a customer. Not only will this help you learn something new, but could bring a genuine benefit to the business or customer.

Sometimes we are too scared to build our case for why we think it should be implemented. What if it goes wrong? Luckily I work for a great company, where there is no blame culture, and we are encouraged to suggest and try new things.

I was at work one day and we were discussing storage expansion for a large customer (14x ESXi hosts across 2 sites). Internally, our expertise has always been HPE-oriented, and somebody suggested a HPE MSA as a cost-effective way of expanding the storage. My suggestion was “why not use VSAN?” I then proceeded to talk about the benefits of VSAN, and performance vs cost/GB. That was the last I heard about it!

About 3 weeks later my boss says “Right, you are going to site to install drives and deploy VSAN for our customer.” I had clearly made a decent suggestion, then!

Although we support existing customers running VSAN and have installed it in the past, this particular customer is a Blue Light Service; an NHS 111 provider. VSAN would be installed directly into their brownfield environment to extend their existing storage.

I would be lying if I said that I wasn’t a bit nervous about making this change on such a critical service. It actually forced me to brush up on VSAN, relearn everything I already knew and take a quick test drive around existing environments. But I still felt I needed a bit more help.

So, in preparation I signed into the VMware Hands-on Labs and refreshed my memory on VSAN installation specifically. It is an incredibly simple thing to do, and actually seems too good to be true.

Along came the installation day, which was a huge success and has been up and running for nearly a year now. That’s the thing with VSAN I guess, it just works!

Moral of the story is to always strive to operate outside of your comfort zone; that’s where the real fun takes place.