Baptiste Dauphin's doc

# Virtualization

Virtualization (OS-level)
OS-level virtualization refers to an operating system paradigm in which the kernel allows the existence of multiple isolated user-space instances. Such instances, called

  • containers (Solaris, Docker)
  • Zones (Solaris)
  • virtual private servers (OpenVZ)
  • partitions
  • virtual environments (VEs)
  • virtual kernel (DragonFly BSD)
  • jails (FreeBSD jail or chroot jail)

Those instance may look like real computers from the point of view of programs running in them.

  • A computer program running on an ordinary operating system can see all resources (connected devices, files and folders, network shares, CPU power, quantifiable hardware capabilities) of that computer.
  • However, programs running inside of a container can only see the container's contents and devices assigned to the container.

# Overhead

Operating-system-level virtualization usually imposes less overhead than full virtualization because programs in virtual partitions use the operating system's normal system call interface and do not need to be subjected to emulation or be run in an intermediate virtual machine, as is the case with full virtualization (such as VMware ESXi, QEMU or Hyper-V) and paravirtualization (such as Xen or User-mode Linux). This form of virtualization also does not require hardware support for efficient performance.
Wikipedia of Virtualization OS-levelopen in new window

# VMWare

# Definitions

Datastore files :

  • .vmx : Virtual Machine

  • .vmdk : Virtual disk

  • *-ctk.vmdk : Changed Tracking Block

  • .nvram : Non-volatile memory file

  • .vswp : Swap file

  • .vmsd : Database file

  • .log : vm log file

  • .vmsn : RAM file (generaly not used)

  • RDM : Raw Device Mappingopen in new window

  • VMFS : ESX hosts store all files necessary for virtual machine operation in a VMware Virtual Machine File System (VMFS). Proper capacity management is critical because this file system is required for any operations related to a virtual machine

# Changed Tracking Block

If you don't see the CTK file, then certainly the CBT just isn't activated. The CBT functionality is available for VMs with the virtual hardware version isn't 7 and higher. The CBT in fact appeared first in vSphere 4. The CBT is usually activated by backup products, like Veeam or VDP automatically during the first backup. The CBT can also be activated manually, through the vSphere web client OR editing directly the configuration file of a particular VM (VMX file).

The CBT enabled allows up to 10 times faster backups.

The file can be several Mb in size in total. The size of this file is fixed and does not grow over its initial size. Only if you grow the size of a virtual disk, than the size of CTK file changes. The real size of this CTK file depends on the size of a virtual disk, but it's about .5MB for every 10 GB of virtual disk size. The CTK's file content stores the state of each block, for tracking purposes, and is using sequence numbers. Those sequence numbers are used by backup application, to see if a block has changed its state or not.

# Snapshot

# Snapshot Files

official docopen in new window A Take Snapshot operation creates .vmdk, -delta.vmdk, vmsd, and vmsn files.

A snapshot consists of files that are stored on a supported storage device. A Take Snapshot operation creates .vmdk, -delta.vmdk, .vmsd, and .vmsn files. By default, the first and all delta disks are stored with the base .vmdk file. The .vmsd and .vmsn files are stored in the virtual machine directory.

Delta disk files A .vmdk file to which the guest operating system can write. The delta disk represents the difference between the current state of the virtual disk and the state that existed at the time that the previous snapshot was taken. When you take a snapshot, the state of the virtual disk is preserved, which prevents the guest operating system from writing to it, and a delta or child disk is created. A delta disk has two files, including a descriptor file that is small and contains information about the virtual disk, such as geometry and child-parent relationship information, and a corresponding file that contains the raw data.

The files that make up the delta disk are referred to as child disks or redo logs. A child disk is a sparse disk. Sparse disks use the copy-on-write mechanism, in which the virtual disk contains no data in places, until copied there by a write operation. This optimization saves storage space. A grain is the unit of measure in which the sparse disk uses the copy-on-write mechanism. Each grain is a block of sectors that contain virtual disk data. The default size is 128 sectors or 64KB. Flat file A -flat.vmdk file that is one of two files that comprises the base disk. The flat disk contains the raw data for the base disk. This file does not appear as a separate file in the Datastore Browser. Database file A .vmsd file that contains the virtual machine's snapshot information and is the primary source of information for the Snapshot Manager. This file contains line entries, which define the relationships between snapshots and between child disks for each snapshot. Memory file A .vmsn file that includes the active state of the virtual machine. Capturing the memory state of the virtual machine lets you revert to a turned on virtual machine state. With nonmemory snapshots, you can only revert to a turned off virtual machine state. Memory snapshots take longer to create than nonmemory snapshots. The time the ESX host takes to write the memory onto the disk is relative to the amount of memory the virtual machine is configured to use.

# Deleting Snapshots

official docopen in new window

Deleting a snapshot removes the snapshot from the Snapshot Manager. The snapshot files are consolidated and written to the parent snapshot disk and merge with the virtual machine base disk.

Deleting a snapshot does not change the virtual machine or other snapshots. Deleting a snapshot consolidates the changes between snapshots and previous disk states and writes all the data from the delta disk that contains the information about the deleted snapshot to the parent disk. When you delete the base parent snapshot, all changes merge with the base virtual machine disk.

To delete a snapshot, a large amount of information needs to be read and written to a disk. This process can reduce virtual machine performance until consolidation is complete. Consolidating snapshots removes redundant disks, which improves virtual machine performance and saves storage space. The time it takes to delete snapshots and consolidate the snapshot files depends on the amount of data that the guest operating system writes to the virtual disks after you take the last snapshot. The required time is proportional to the amount of data the virtual machine is writing during consolidation if the virtual machine is powered on.

Consolidation = merge snapshot(s) with vmdk(s)