Clone and roll out template fail with “failed to connect to host”

Posted by Duncan Epping in January 24th, 2008

Published in Server

Recently I implemented ESX 3.5 with VirtualCenter 2.5. After I installed the ESX Hosts and the VirtualCenter server the customer told me that I had to change the ip-addresses of the service console. I removed the ESX hosts from the VirtualCenter server, removed the vswif of both hosts with “esxcfg-vswif -d” and created it again with the correct ip-address and subnet. I added both hosts to the cluster again and every thing seemed to work again… until I decided to rollout a template. It took over 10 minutes before the error “failed to connect to host” appeared. Also cloning a VM gave me this same error. Weird thing is that creating, removing and starting VM’s worked fine. So I checked if I was able to clone via vmkfstools from the service console, this also worked. Weird… checked the host files, resolv.conf, sysconfig/network… everything okay.

After a couple of nights sleep I found it: /etc/opt/vmware/vpxa/vpxa.cfg contained the old ip-address in the field “hostIp”. I tried editing the file, but when ever I initiated a clone the hostIp was changed again to the old value. There was not much time left and I suspected the VirtualCenter database so I wiped it out with “vpxd.exe -b”. (Run this command on the VMware VirtualCenter server!) I created the Datacenter and Cluster again, added the hosts and started cloning… everything worked fine again. If any knows a better solution to the problem, because wiping out the VirtualCenter database isn’t my favorite solution, let me know!

« CPU utilization increasing after VMotion in a DRS enabled cluster

Rescan all vmhba’s at once with the esxcfg-rescan command »

39 user comments or pingbacks in this post

Follow-up this post comment rss or leave a trackback

1. Remco said,

January 24th, 2008 at 15:33

Dus toch de database….hmmm gebeurt iets te vaak de laatste tijd. Vraag me af of het iets met SQL 2005 te maken heeft….

2. Rolf said,

January 25th, 2008 at 02:02

Echt niet normaal, het moest wel zoiets zijn… maar ja ga maar eens zoeken in welke bestand het oude IP adres is blijven hangen. En inderdaad, het “wipen” van de VC DB is niet wat je heel graag doet……..

3. Romeo said,

February 11th, 2008 at 17:07

Had a similliar issue with the same error message when trying storage vmotion.
wiping the virtual center was not possible.
What I did was the following:
- assured the DNS stuff was setup correctly
- removed the host fom the vc server
- removed the vpxa.cfg on the host (acctually renamed it)
- rejoined the host to the vc server (this got me a new vpxa.cfg generated with the correct IP)

4. Glenn said,

February 25th, 2008 at 18:45

Nice one Romeo!
That works a treat!! You saved me a rebuild of the ESX servers there.
Cheers!

5. BigRollTide said,

March 4th, 2008 at 15:55

I ave found this to work too:
Stop the VMware VirtualCenter Server service
Edit the /etc/opt/vmware/vpxa/vpxa.cfg
Run this script on the database:

———–
UPDATE [VCDB].[dbo].[VPX_HOST]
SET [IP_ADDRESS] = ‘w.x.y.z’
WHERE [DNS_NAME] = ‘name of esx host as it is listed in the table’
———–

Of course w.x.y.z above is the new ip address

Start the VMware VirtualCenter Server service

The vpxa.cfg file might not have to be edited as it probably would have been changed by the VirtualCenter Service anyway, but I didn’t test that. I just know the above worked for me.

6. Al said,

March 31st, 2008 at 18:53

If you check the file has has a different IP, I did the following to correct:

1. Removed the ESX host from Virtual Center.
2. Add the host again to Virtual Center.
3. Check the file, should reflect new IP.

Note: My error was “Failed to connect to Host”, occured when I tried to clone a VM.

7. David D. said,

April 1st, 2008 at 11:52

Al’s solution worked for me.

8. v4nilla said,

April 29th, 2008 at 20:25

Al, Thanks for the great tip. This worked for me as well.

1. Put the host in maintenance mode
2. Remove host from cluster
3. Add host back to cluster
4. Exit maintenance mode
5. Deploy VM from template
6. All is well.

9. eSPee said,

May 20th, 2008 at 16:58

v4nilla’s steps resolved my issue.. Thanks a lot!!

10. John P. said,

June 19th, 2008 at 16:30

Thanks to all. Al’s solution worked for me.
First VM from a template is being deployed now.

11. J said,

July 1st, 2008 at 14:49

We experienced similar, however, simply Disconnecting (not Removing), and then Connecting was sufficient. This updated the VC tables and vpxa configuration on the host.

If you Remove, you lose all VM history, permissions, etc. I’d prefer not to discard all the history.

12. Tucker said,

July 3rd, 2008 at 06:40

I just stopped the service(service vmware-vpxa stop)
edited vpxa.cfg
then restarted the service (service vmware-vpxa start)

and everyting worked great after that.

Hope this helps

13. tadda said,

July 3rd, 2008 at 17:38

BigRollTide… thank you. Works like a charm.

14. Br4zilNutz said,

August 2nd, 2008 at 08:44

BigRollTide fix is the way to go. Moving hosts out of cluster are not ideal when you have 10+ vms running. They all need to migrate elsewhere before you can evict the host.

15. C0lP4nic said,

August 18th, 2008 at 10:48

Similar issue here – no cloning allowed since changing the IP address of the service console (done everything with the vmware kb guide).

Done this to resolve the cloning issue:
- Disconnect + Removed the Host from the cluster
- moved the vpxa.cfg
- Connect + Added the thost to the DRS/HA Cluster again
- vpxa.cfg was generated as expected

Cloning works fine now.

16. Walter said,

August 21st, 2008 at 21:19

The best way to fix it is remove the esx from the cluster.

First put the esx in maint. mode. (optional)
Disconnect the esx from the cluster (vms will show disconnected but will still be running)
remove the esx from the cluster (the vms will remain running) ( this removes the vpxa.cfg (host agent file))
add the esx back to the cluster

check the vpxa.cfg file (cat /etc/opt/vmware/vpxa/vpxa.cfg)

the new numbers should be there.

simply editing the file will only temporarily change it. it WILL revert back.

17. Richard Marsh said,

September 15th, 2008 at 19:52

romeo, thanks very much, this helped me immensely…..appreciate your post.

rich

18. Stephane said,

September 18th, 2008 at 16:31

Hi,

I have the same issue.
I tried to put my esx in maint.mode, disconnect and remove, but now I can’t add the host.

I have this error : “Unable to access the specified host. It either does not exist, the server software is not responding, or there is a network problem.”

Any idea to help me ? thanks

19. Duncan said,

September 18th, 2008 at 18:23

Can you ping your server on name and ip?

20. Stephane said,

September 19th, 2008 at 09:21

I already try to ping both esx and virtual center with ip address and hostname, I have no problem.

I put the oldconfiguration file, but have still the same issue, can add my esx to the cluster.

21. Duncan Epping said,

September 19th, 2008 at 09:35

Your best bet would be the following:
go to the console to see if the file /etc/opt/vmware/vpxa/vpxa.cfg is still there, remove it if it’s there. and check if the host is disconnected and removed if the vpxa agent is still installed, if so check this kb article to remove it: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1002531&sliceId=2&docTypeID=DT_KB_1_1&dialogID=30678559&stateId=1%200%2030684123

22. Rubeck said,

September 19th, 2008 at 12:06

Guys… On what VC and ESX builds did you experience this issue?
Not that I have this issue, or ever even seen it….. Just want to know on which shelf in my brain I need to store this info on

/Rubeck

23. Duncan Epping said,

September 19th, 2008 at 12:22

esx 3.5 and vc 2.5 pre update 1

24. Stephane said,

September 19th, 2008 at 14:11

I have this issue with VI 3.5u1 and VC2.5.

the problem was with the firewall, the guy who manage the firewall do some modification after the first installation I think, and when i want to re add the esx, i have my issue.

now i’m looking if this tips works

25. Russ said,

September 25th, 2008 at 18:13

I had to follow Duncans version.

1) Disconnect and remove the ESX servers from the cluster.
2) Connect to the ESX console and rpm -e VMware-vpxa-
(to find the version you are running do rpm -qa | grep VMware-vpxa, when you re-add the ESX server to the cluster, the vpxa rpm is installed automatically if it is missing)
3) Delete the vpxa.cfg from /etc/opt/vmware/vpxa (it is recreated when the host is added to the cluster)
4) Add your ESX server back to the cluster and check the vpxa.cfg to ensure the IP address has been changed.

My deploy from template started working properly after this was completed.

26. chell said,

October 27th, 2008 at 18:19

After I have done the steps mentioned from above I did have the correct IP.
However, all of my templates I made were blown away. Thank goodness the VM’s were still intact. How do I recover the templates?

27. Duncan said,

October 27th, 2008 at 21:46

Are the templates gone? If so, just browse the data store, go to the template folder and click “import” om the vmxt file…

28. John said,

October 29th, 2008 at 02:41

I’m having the same issue where i can’t clone
Can i find out what the vmware-vpxa service does?
When you restart this service would it affect the vm clients running on the ESX server?

29. Duncan Epping said,

October 29th, 2008 at 12:23

vmware-vpxa = vmware virtualcenter agent… you should be able to restart is without any affect on the vm guests. but this depends on the version of esx your using, old (3.0.1) versions had the bad habbit of restarting the vm when the vmware-vpxa was restarted.

“service vmware-mgmt restart” should do the trick.

30. samehere said,

November 10th, 2008 at 20:47

somewhat of same issue…support had me remove one host from cluster re-add now I can’t get to it or another that is in cluster. I am running 3.5i on host and 2.5 Virt. Cent. each time I try to “remove” the host from the list that is not in the cluster it shuts down my virtual center service, when I try and reconnect the one that is still in cluster that lists that it is disconnected I get the error: “Unable to access the specified host. It either does not exist, the server software is not responding, or there is a network problem.” I can ping the host just fine it resolves forward and reverse. Any help would be greatly appreciated….

31. Alec said,

November 14th, 2008 at 16:21

Hi thanks guys this halped me a lot. I can confirm that removing teh HOSTS from the cluster and re-adding them updated the /etc/opt/vmware/vpxa/vpxa.cfg file with the correct IP address.

Cheers!

32. Dolly said,

November 19th, 2008 at 18:29

Thanks chill..I tried to follow your method to recover my templates.but still i get this error :-
“Unable to access virtual machine configuration:- invalid datastore path:’sanfs://uuid:48d3da56-b1e008cc-465c-001a6407a231/xxxx(vmname/xxxx.vmtx.’
All my datastores are valid and exists.
Any help is appreciated.

33. Eric said,

November 24th, 2008 at 11:54

Stephane,
Try restarting the VMware VirtualCenter service on the VC server, then try adding the host again.

34. Dolly said,

November 24th, 2008 at 19:24

Thanks Eric that helped

35. stoz said,

February 11th, 2009 at 22:08

guys and girls,

server one has ip from server to and vice versa
in file /etc/opt/vmware/vpxa/vpxa.cfg in my situation

remove and rejoin host helps alot

36. rexit1982 said,

March 3rd, 2009 at 14:33

I ran into the same issue attempting SVMotion after changing the SC addresses.
Stopped the the virtual center service, updated the DB and started it back up.

Thanks to post number 5 for the sample, my mssql needed something like
UPDATE [VIM_VCDB].[dbo].[VPX_HOST]
SET IP_ADDRESS =”
WHERE DNS_NAME = N”

Considering I only have 3 hosts I just opened the table in sql manager and made sure my like statement did not allow for any ambiguity.

37. Luuk G said,

March 25th, 2009 at 06:50

3. Romeo said, February 11th, 2008 at 17:07 Had a similliar issue with the same error message when trying storage vmotion.
wiping the virtual center was not possible.
What I did was the following:
- assured the DNS stuff was setup correctly
- removed the host fom the vc server
- removed the vpxa.cfg on the host (acctually renamed it)
- rejoined the host to the vc server (this got me a new vpxa.cfg generated with the correct IP)

Above did the job!! Thanks Romeo!

Greet Luuk

38. sst said,

May 5th, 2009 at 15:34

Cudos to post 5 and 36…

To preserve performance data, this method is preferred:
Stopping the Virtual Center Service
Editing the Virtual Center Database Table to correct the specific “VPX_HOST” IP Address.
Restarting the Virtual Center Service.

The /etc/opt/vmware/vpxa/vpxa.cfg need not be edited on the host.
Once you restart the Virtual Center Service, the vpxa.cfg is updates properly.

Leave A Reply Below

Currently browsing Clone and roll out template fail with “failed to connect to host”

Pingback and trackback

Link trail from various blogs:

Confluence: Product

Pingback on October 5th, 2008 at 19:07