ESXi 5.1 : Fixing ‘Failed to deploy OVF package: The task was canceled by a user.

Posted on by David Saliba

Where I work, we love using OVA templates to speed up our deployment of virtual machines. I recently upgraded one of my servers to ESXi 5.1 (which also required an update to vSphere). ESXi 5.1 provides support for Windows 8 and Server 2012, which is incredibly useful. However, whilst building OVA templates for these operating systems, I stumbled across an issue.

I ran through the ‘New Virtual Machine’ wizard, selecting Windows 8 (or Server 2012), leaving all settings default. Installed my operating system, and made the required customisations, shutdown the machine and exported an OVA template through vSphere – excellent, how easy!

However, whilst trying to re-deploy the OVA to the ESXi 5.1 host, through the ‘Deploy OVA template’ wizard, it failed immediately after completing the wizard (right before it shows the deployment progress bar). Now, I have a particular hate for misleading error messages, and this one seems to fall right in-to that category –

Failed to deploy OVF package: The task was canceled by a user.

How misleading. I, or any other user, certainly didn’t cancel the task. So what happened? I took a look through the (horrendous) hostd.log on the ESXi box and found absolutely nothing of any value.

Frustrated by the inability to redeploy a template I spent so long preparing, I broke open the OVA template and took a look inside. There were three files with different extensions,

  • .ova – OVF descriptor, written in XML, which describes the hardware requirements
  • .mf – contains SHA1 checksums of the .OVA and .VMDK
  • .vmdk – the virtual hard disk for the virtual machine.

I immediately discarded (renaming to .mfx will do the trick) the .mf. If you modify the .ova and don’t update the .mf, it’ll complain that the checksum is invalid. Removing this file seems to prevent vSphere from checking the checksums, which is useful, seeing as we want to poke around the .ova. After fiddling around inside the .ova, I stumbled across the following line…

<rasd:ResourceSubType>vmware.cdrom.iso</rasd:ResourceSubType>

Changing the above line, to read…

<rasd:ResourceSubType>vmware.cdrom.atapi</rasd:ResourceSubType>

…appears to have fixed my deployment issues. Perhaps changing the ‘CD Drive Device type’ in the virtual machine’s settings would’ve fixed it. But by that point, I had already exported the OVA and deleted the source virtual machine.

Hopefully someone will stumble across this one day, and it’ll save them a few hours!

SVN checkout fails to authenticate

Posted on by David Saliba

When running:

svn checkout https://mysvn.mydomain.com:8443/svn/Linux/trunk/scripts/ –username *** –password ‘****’

I got :

WARNING: gnome-keyring:: couldn’t connect to: /tmp/keyring-fTq763/pkcs11: No such file or directory
Password for ‘default’ GNOME keyring: *****
svn: OPTIONS of ‘https://mysvn.mydomain.com:8443/svn/Linux/trunk/scripts’: authorization failed: Could not authenticate to server: rejected Basic challenge (mysvn.mydomain.com:8443)

Problem could be damaged keyring or wrong keyring password.

To fix quickly :

pkill keyring

re-run the checkout.

Voila ! you get a new keyring and the process works. Checkout goes through.

Njoy 🙂

p.s.

As a desparate measure :

mv /usr/bin/mate-keyring-daemon /usr/bin/mate-keyring-daemon.rubbish

SSH through HTTP proxy

Posted on by David Saliba

This article explains how to connect to a ssh server located on the internet from a local network protected by a firewall through a HTTPS proxy.

Requirement are :

  • Your firewall has to allow HTTPS connections through a proxy
  • You need to have root access to the server where ssh is listening

Configure the ssh server

The ssh daemon need to listen on 443 port. To accomplish this, just edit this file (on debian system) /etc/ssh/sshd_config and add this line :

Port 443

Then restart the daemon :

sudo /etc/init.d/ssh restart

Configure the client

I suppose you are on a Linux system (debian for example). First you have to compile the connect binary which will help your ssh client to use proxies (HTTPS in our case). Then you have to configure your ssh client to tell him to use HTTPS proxy when he tries to connect to your ssh server.

  1. Install the connect software :
    • On debian system, just install the connect-proxy package :

      sudo apt-get install connect-proxy
    • On other Linux systems, you have to compile it :

      cd /tmp/
wget http://www.meadowy.org/~gotoh/ssh/connect.c
gcc connect.c -o connect
sudo cp connect /usr/local/bin/ ; chmod +x /usr/local/bin/connect
  2. Configure your ssh client. Open or create your ~/.ssh/config file and add these lines :

    ## Outside of the firewall, with HTTPS proxy
Host my-ssh-server-host.net
  ProxyCommand connect -H proxy.free.fr:3128 %h 443
## Inside the firewall (do not use proxy)
Host *
   ProxyCommand connect %h %p
  3. Then pray and test the connection :

    ssh my-ssh-server-host.net

SSH to another server through the tunnel

For example to connect to in ssh github.com :

Host github.com
  ProxyCommand=ssh my-ssh-server-host.net "/bin/nc -w1 %h %p"

List VMs in ESX 5.1

Posted on by David Saliba

To get a description of all the vms on an ESX 5.1 box use the following:

~ # vim-cmd vmsvc/getallvms | grep vmx | awk '{ print $2 }'

Moving a machine from one VM or physical box to another

Posted on by David Saliba

This is the simple case where all we have is one disk that needs cloning.

Assumptions for the following example

  1. Tools including backtrack distro available and permissible by company policy ( some c**ts get all agitated when sysadmins use a ‘knife’ to cut the ‘bread’ so be warned)
  2. Networking in place for the transfer preferable pre-organized IPs, gatway ,  DNS for resolving the updates like ssh etc.. , also verify the level of traffic your network guys are willing to tolerate for a long time this should be your –rate-limit value (remember this value is in Mega bytes so 9-10x the Megabit bandwidth.
  3. Cloning one disk is enough for the volumes to move. Compex LVM / software raids and concats or stripes need further steps.
  4. For the sake of example i assume this is a P2V but it’s just as good an approach in a V2V.

 

Points to perform :

  1. Download / Burn Backtrack or Knoppix
  2. Create a VM with large enough a disk and closely supported disk subsystem and NICs eg. SCSI and e1000
  3. Boot the two machines e.g. physical (source) and VM (target) with BT
  4. Enable ssh on both machines for a third person point of view even for monitoring the transfer
  5. Connect to the receiving VM booting into the live CD.
  6. Setup a screen session to avoid your disconnection from the session affecting the transit. Using screen -S transfer.
  7. run nc -l -p 19000 | bzip2 -d | dd bs=16M of =/dev/sda  replacing the 19000 with the port you want to use and /dev/sda with the disk you want to clone.
  8. Connect to the transmitting side.
  9. Setup a screen session to avoid your disconnection from the session affecting the transit. Using screen -S transfer.
  10. run dd bs=16M if=/dev/sda | pv –rate-limit 1M | bzip2 -c | nc 192.168.1.24 19000
    replacing the ip with the ip of the listener  connected in point 5, and you can skip the rate limit thingy (–rate-limit 1M) if you want full throttle..
  11. Once finished you can boot the target VM and reconfigure it as you please.

 

An extra help might be connecting to the target box in a target session and run iftop to see the transfers.

Njoy.

 

How to install MySQL on CentOS

Posted on by David Saliba

Here are the steps of what you need to do in order to install and setup MySQL on a new server.

We’ll prepare a fresh CentOS 6 system (64 bit) for use as a database server. All you need is access to an SSH client and your server root credentials.

 

Preparing the System

The first thing I’d like to do is make sure I have all available updates. So once I log in I run

yum update

This can take some time so grab a coffee while you wait. Once yum has finished, let’s check if MySQL is already installed on your system:

mysql

If you get “command not found” then you know you need to install MySQL.

Installing MySQL and MySQL Server

MySQL consists of two parts: the client and the server. In order for our system to run the daemon which will process external requests we need both on our system. Here’s how we get those:

yum install mysql mysql-server

This will take a minute or two. No need to restart your server, all you need to to is start MySQL with

service mysql start

and it will be available for use. You’ll see the following message:

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h yourdomain.com password 'new-password'

Alternatively you can run:
/usr/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /usr/mysql-test ; perl mysql-test-run.pl

Please report any problems with the /usr/bin/mysqlbug script!

Securing the installation

Looks like we need to do two things: set a root password and secure the installation. You can do these steps manually, but MySQL is rather nice in that it provides a script which you can use to secure your installation. Note that this path may be different on your system:

/usr/bin/mysql_secure_installation

The script will ask you the following questions:

  • current root password (in our case it’s not set so hit enter)
  • remove anonymous users (say yes)
  • disallow remote login (in our case we want remote login active so we say no here, but if you’re using MySQL on a system which will not need this then say yes here)
  • remove test database and access to it (say yes)
  • reload all privieleges (say yes)

Now you can access MySQL with the following command:

mysql -p

Starting MySQL at boot time

You will likely need to make sure MySQL is running when you reboot the server, it’s cumbersome to start it manually every time you do that. This will take care of it:

chkconfig --levels 235 mysqld on

We’re done – MySQL is now running on your server and yours to populate.

Have fun ;-)

Sharing a screen SSH session

Posted on by David Saliba

Sharing your Session

Assuming you start a screen session using

screen -S david

Ask your partner to connect using (assuming they are logged in using the same user account):

screen -x david

Now it’s simply magical. Multiple persons can type and work on the same terminal – it works best when you’re coordinating over the phone. Note that the dimensions of your terminal output will be the same for every user – to change it, press Ctrl–A and then capital F. This will make the screen output fit your current terminal size, and change it to that size for every connected user.

Detaching from a Screen Session

Important: To detach from the screen session so that you can resume later, simply close the window. If you typeexit, you’ll end up terminating the screen session and the processes running within.

Moving SSL Certs from IIS to Apache

Posted on by David Saliba

Some instructions for converting SSL certificates generated for IIS to private key, and cert files you can use on unix, or Apache for windows.

First Export your IIS certificate into a pfx file (this is something you should do anyways for backup)

  • Run mmc.exe
  • Click the ‘Console’ menu and then click ‘Add/Remove Snap-in’.
  • Click the ‘Add’ button and then choose the ‘certificates’ snap-in and click on ‘Add’.
  • Select ‘Computer Account’ then click ‘Next’.
  • Select ‘Local Computer’ and then click ‘OK’.
  • Click ‘Close’ and then click ‘OK’.
  • Expand the menu for ‘Certificates’ and click on the ‘Personal’ folder.
  • Right click on the certificate that you want to export and select ‘All tasks’ -> ‘Export’.
  • A wizard will appear. Make sure you check the box to include the private key and continue through with this wizard until you have a .PFX file.

Next run openssl to extract the private key, and the cert file.

# Export the private key file from the pfx file
openssl pkcs12 -in filename.pfx -nocerts -out key.pem
# Export the certificate file from the pfx file
openssl pkcs12 -in filename.pfx -clcerts -nokeys -out cert.pem
# This removes the passphrase from the private key so Apache won't
# prompt you for your passphase when it starts
openssl rsa -in key.pem -out server.key