Rescue an instance

In case of a system or access failure, it is possible to rescue an instance. Due to limitations of OpenStack, the Rescue Instance feature is only available on instances that do not have an volume attached. Nevertheless we present an option for instances that have their root disk as volume.

If you don’t know if the instance’s root disk is a volume or not, then read the I want to know if the instance disk is a persistent volume topic on the Storage page.

Rescue an instance without a volume (ephemeral disk)

  1. On the VMCloud portal, select the Rescue Instance option on the desired instance. This will create a temporary instance with the original disk mounted as secondary.

  2. The instance will go to the Rescue status and when the Task property goes from Rescuing to None, the instance is ready for login via SSH. Depending on the selected image, the login user may vary from the original. Check the Images page to confirm the login user for the temporary instance. Also, a SSH warning may appear due to the fact that the fingerprint has changed for the same IP address, since it is a new instance. Fix this with the indicated ssh-keygen command, this will be done again after the instance is rescued.

  3. After logging into the instance, check if the original disk is detected, typically as vdb:

    debian@baba:~$ sudo su
root@baba:/home/debian# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda     254:0    0  1.7G  0 disk 
└─vda1  254:1    0  1.7G  0 part /
vdb     254:16   0   10G  0 disk 
├─vdb1  254:17   0   10G  0 part 
└─vdb15 254:31   0    8M  0 part

  4. Mount the original disk and then perform the needed changes:

    root@baba:~# mkdir /mnt/disk
root@baba:~# mount /dev/vdb1 /mnt/disk/
root@baba:~# ls /mnt/disk/
bin  boot  dev	etc  home  init  initrd.img  lib  lib64  linuxrc  lost+found  media  mnt  old-root  opt  proc  root  run  sbin	sys  tmp  usr  var  vmlinuz

  5. After all the necessary changes are done, unmount the original disk and exit:

    root@baba:~# umount /mnt/disk
root@baba:~# exit

  6. Finally, on the VMCloud portal, select the Unrescue Instance option, the Status property will go to Active and the original disk will be launched. After a few seconds, the original instance will be available and hopefully fixed.

Rescue an instance with a volume (persistent disk)

Warning

NOTE: As these instructions are an alternative to the Rescue Instance feature for instances with volumes, please read very carefully and proceed with caution. This procedure involves changing Instance, Volume and Network settings.

  1. Go to the instance’s page on VMCloud, then go to the Interfaces tab and click on the name of the interface. Then copy all the information on the Overview tab to a text editor, so that we will have all the instance’s network information saved, such as IP addresses, MAC address and so on.
Name
    None
ID
    0dc93acf-cxx3-4aa0-a2a5-1daxxxxxx
Network Name
    n-xxxx
Network ID
    xxxx432-8xx02-46xx0-864e-9bbxxxxx5
Project ID
    xxx542084xxxxxxxxxxxxxxxxxxxx
MAC Address
    fa:16:3e:xx:xx:xx
Status
    Active
Admin State
    UP
Port Security Enabled
    True
DNS Name
    vbn

DNS Assignment
(...)
  1. Additionally on the instance’s page, note the volume ID on the Volumes Attached section, as this volume will be used again later on, and also other relevant information, such as flavor, security groups, etc.
Volumes Attached

Attached To
    4a7xxxxb66-e0b3-4x44-b429-396xxxx2c on /dev/vda

  1. If you do not know if the instance was created with the Delete Volume on Instance Delete option, then read the I don't know if I created the instance volume with the Delete Volume on Instance Delete option topic on the Storage page. This is essential, as we do not want to delete the original volume. Otherwise, if you know the option was not set, then just delete the original instance.

  2. After the original instance is deleted, go to the Volumes -> Volumes page to confirm that the volume previously attached to the original instance becomes available.

  3. Create a new instance. The specifications of this instance are not relevant, as this instance is temporary. Check the Images page to confirm the login user for the temporary instance.

  4. Select the Attach Volume option on the new instance, select the volume with the ID that was indicated in the original instance’s page and click on Attach Volume.

  5. Log into the new instance via SSH and check if the original disk is detected, typically as vdb:

    debian@baba:~$ sudo su
root@baba:/home/debian# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda     254:0    0  1.7G  0 disk 
└─vda1  254:1    0  1.7G  0 part /
vdb     254:16   0   10G  0 disk 
├─vdb1  254:17   0   10G  0 part 
└─vdb15 254:31   0    8M  0 part

  6. Mount the original disk:

    root@baba:~# mkdir /mnt/disk
root@baba:~# mount /dev/vdb1 /mnt/disk/
root@baba:~# ls /mnt/disk/
bin  boot  dev	etc  home  init  initrd.img  lib  lib64  linuxrc  lost+found  media  mnt  old-root  opt  proc  root  run  sbin	sys  tmp  usr  var  vmlinuz

  7. Now perform the necessary changes and fixes on the disk.

  8. After all the necessary changes are done, unmount the original disk and exit:

    root@baba:/mnt/disk# cd
root@baba:~# umount /mnt/disk
root@baba:~# exit

  9. Delete the temporary instance and go to the Volumes -> Volumes page to confirm that the original volume becomes available.

  10. Go to the Networks -> Network page, click on the project network (n-[project name]), select the Ports tab and click on Create Port. Insert the following information and click on Create:

    • Name: Does not matter, as long it is something identifiable, instance name for example.
    • Enable Admin State: Checked/Yes/True
    • Device ID: Leave blank
    • Device Owner: Leave blank
    • Specify IP address or subnet: FIXED IP ADDRESS
    • Fixed IP Address: The IP address, in IPv4 form, indicated on the original instance’s network information. Format: xxx.xxx.xxx.xxx
    • MAC Address: The MAC address indicated on the original instance’s network information. Format: xx:xx:xx:xx:xx:xx
    • Port Security: Checked/Yes/True
    • VNIC Type: NORMAL

  11. Go to the Instances page and launch an instance with the following specifications:

    • Details tab:
      • Name: Same name as the original instance
    • Source tab:
      • Select Boot Source: Volume
      • On the Available section, click on the upwards arrow of the volume with the original ID as Name. It should go to the Allocated section.
    • Flavor tab:
      • Select the flavor of the original instance
    • Networks tab:
      • On the Allocated section, click on the downwards arrow of the project network. It should go to the Available section. The Allocated section should be empty.
    • Network Ports tab:
      • On the Available section, click on the upwards arrow of the created port. It should go to the Allocated section.
    • On the remaining tabs, if the original instance had additional configurations, such as security groups, then add them.

  12. A new instance will be created and will be ready for login. IT should be the same as the original with the additional fixes and changes. When logging in with the original user, a SSH warning may appear due to the fact that the fingerprint has changed for the same IP address, since it is a new instance. Fix this with the ssh-keygen command indicated in the warning.

1.0