VMware 2 commands and some explanation of ESX See also: commands in Unix _by TheBen
|Item||Description||if you print thiscomplete page, you can find it on page|
& Stopping ESX Server
|6||Basic File Commands||13|
& Disk Management
|16||Backup/recovery + DR||34|
is the boot loader (LILO=Linux Loader) used for ESX server (remember the
service console is based on a modified version of Red Hat Linux 7.2).
Lilo.conf is the text file to describe the OS to boot. This text file is
compiled into a binary file that is actually used by LILO. If you are
troubleshooting the APPEND line, then use vmkpcidivy tool. You should
not have to revert to manually editing this file. If you ever do edit
this file, then you need to compile it with lilo –c to write the
The pci device mask specified in the append line of lilo.conf is actually an include, not a mask out. The important thing to remember is the append line defines the hardware visible to the service console. Don’t replace LILO with another boot loader, e.g. GRUB. ESX manages allocation of PCI devices between service console and VM kernel with the expectation of LILO.
You can also view PCI device allocation using the MUI, found under Startup Options
Alternatively, you can use the legacy web interface using the URL
loads the service console kernel, e.g. /boot/initrd-2.4.9-vmnix2.img
This kernel obeys what is stored in the file /etc/inittab
file is read by the init process and specifies the run level to be used
by the service console. The line that states the run level will look
something like this.
The init process then works through the start up scripts in the appropriate diectory. For run level 3, this directory would be
The file also starts up the virtual
terminals on the ESX server mingetty tty2 through mingetty tty5. The
mingetty process is minimum getty.
text file used most commonly when we need to share the service console
NIC with the VM kernel. This may be required in say a blade server which
has only 2 NICs, and we need to dedicate 1 NIC to Vmotion.
insmod vmxnet_console devName=vmnic0
start-up scripts for run level 2
start-up scripts (logical links) for run level 3. Run level 3 is used
most of the time as it is command line full multi-user mode. The ones we
are interested in are shown below:
service console tool displays a table showing which daemons are enabled
for the run levels for the vmnix operating system.
ntpd 0:off 1:off 2:on
3:on 4:on 5:off 6:off
If we wanted to change a service so that it is enabled for a particular run level, then we can use chkconfig –level.
chkconfig --level 1 ntpd on
The above command would turn on ntpd for run level 1, this would not affect the run levels that ntpd was already set for. So in this example, the ntpd run levels would be
ntpd 0:off 1:on 2:on 3:on 4:on
lists all the service daemons and their status. We can find running
services by looking for the running status
service –-status-all |grep running
would produce an output similar to the following:
crond (pid 1423) is running
To avoid unnecessarily rebooting an ESX server after making certain configuration changes, we can frequently just restart the appropriate daemon. For example we could restart the Apache web server for the MUI with the command:
service httpd.vmware restart
and we can also check a named service running status with
service httpd.vmware status
centralised logging system. When ESX is running, both the service
console and VMkernel log messages through it.
Quick way to restart the ssh daemon. Use full path to do this.
Text file with the configuration of SSH client.
Text file with the configuration file for SSH daemon. An important setting in this file is PermitRootLogin=Yes/No. You can quickly check this with a grep on the file.
grep permit /etc/ssh/sshd_config
A text file that stores the RSA keys
for known hosts. If we rebuild a host and then reconnect to it over SSH
we may be prevented if the known_hosts file has cached the old key. I
think SCP (secure copy) uses RSA keys as well as SSH sessions. Note this
file is maintained on a per-user basis.
config file for Xinetd, the network services daemon? The daemon itself
is stored in /usr/sbin/xinetd This launches the daemons that are bound
to it on demand
is the authentication daemon. This daemon authenticates users of the
management interface (MUI) and remote consoles using the
username/password database defined in /etc/passwd. This service binds
via the xinetd daemon and so the configuration file that specifies the
listening port is in
This text file contains the settings for the VMware remote access authentication daemon. This file specifies the TCP:902 port used by remote console. If this port was changed here, it must also be changed in the file /etc/vmware/config.
If we wanted to add Kerberos off-box authentication for MUI access, then its in the pluggable authentication module configuration file that corresponds to this daemon that we would make a change. This file is found at
We would need to change the current
"auth required" to "auth sufficient" and add a last
line of "auth required" using the Kerberos authentication
script performs the following actions:
vmnix driver Loaded by
daemon runs on demand in the service console to provide information to
any VMware service that needs it. This performs actions in the service
console on behalf of the VMware Remote Console and the web based MUI. It
is started at boot time to do any VM autostarts. This process is
replaced with vmware-ccagent if Virtual Center is installed.
script starts the Apache web server which provides the ESX Server MUI.
Configuration is stored in
This process communicates with
vmware-serverd for backend data. Remember a refresh in the browser is
only a refresh to Apache, to get new data, click on the refresh button
to get new kernel data. Remember if the httpd.vmware service starts and
then stops immediately, check your service console disk space.
is a perl script which runs every x seconds on tty1.
drivers loaded for the service console linux. Remember this command
differs from the vmkload_mod utility which shows the modules loaded for
Size Used by Tainted: PF
The same information can be found by
Starting & Stopping ESX Server
a halt, hard way to stop ESX server
run level 1
this is entered at the LILO boot prompt we can get a root shell. We are
instructing the vmnix kernel to execute at run level 1 (single user
at the LILO boot prompt, this time, we are instructing the vmnix kernel
to execute at run level 3 (the default). If we suspected the run level
was wrong we can use this to get back up and running without having to
revert to booting Linux on its own.
when you boot the ESX server with a Red Hat Linux CD
root directory to new directory specified as a parameter
down the system in a secure way.
shutdown -h now
Halt after shutdown
the LILO boot loader, the default options are
Normal ESX boot
If we use the cursor key at the LILO screen to select one of the three default choices, the boot prompt (displayed below the menu) changes to reflect this. This allows us to augment the boot command with an option switch.
boot: linux –s
In this case, the –s instructs Linux
to boot in single user mode. A critical security point here is that in
single user mode, Linux automatically logs on as root! Once in single
user mode if we wish to continue into multi-user mode then we type
either exit or CTRL-D
|vmware-cmd||shutting down or restarting a VM
from the command line
vmware-cmd <vmx file location> stop <mode>
vmware-cmd <vmx file location> reset <mode>
ESX service console is based on modified Red Hat Linux, we can use the
RPM package installation method.
comes with a list of all installed
|vmware -v||displays version and build information of the ESX system|
interface eth0 and aliases
alias eth0 e1000
interface “eth0” up
interface “eth0” down.
a list of the network interfaces. A quick way of getting IP and MAC of
interfaces Can also be used to bring interfaces up or down
ifconfig eth0 up
Doesn’t work correctly with some network cards, including Intel 1000
Pro copper NICs.
|service network restart||Great
way to do orderly restart of vmnix network configuration. Another way
would be to do
ifdown eth0; ifup eth0
lookup file, just like windows. The ESX server MUST have an entry for
itself in its own HOSTS file.
DNS name server settings, This file does not need to be present if you
are not using DNS.
Text/config file containing
hostname and default gateway.
mask and device name. The text file contains IP configuration for
interface eth0, which is reserved for the service console.
or print routing table
Prints routing table
configuration setup program
which shows ESX and vmnix version
VMware ESX Server 2.1.2
details of the server as shown
2.4.9-vmnix2 #1 Fri Aug 6 04:38:44 PDT 2004 i686
mii-toll -F 100BaseTX-FD
and remote file systems which are mounted at boot. This file is
explained in more detail in the mount section.
# cat /proc/net/pro_lan_adapters/eth0.ino
Cable_Status Cable OK
to force the Service Console's ethernet NIC to 100Mbs full duplex
text file that can be checked to see what the service console NIC is
doing. The speed of the console NIC specified in the file modules.conf
can be confirmed by this file.
|user account information files||/etc/passwd - contains
a list of configured users
/etc/group - contains a list of groups
/etc/shadow - password file, encrypted of course
Displays the UID for the currently logged on user or if supplied with a parameter can be used to display the UID of a named user.
to change the logged on users password or for a named account
Adds a user to /etc/passwd
By default a user added with default options has a group created of the same name. We can add the user to another group, e.g. have a group membership of say ESXUsers
useradd alistair –G vmsqladmins
The above command adds a user called alistair whose primary group (-g) is alistair and other group (-G) membership is vmsqladmins
We can add additional parameters to more fully specify the account.
useradd financeflagshipuser –g Finance –s /bin/false
In the above example the users’
primary group is Finance and the shell is specified. In this case the
shell is /bin/false which is a bogus shell which would prevent logon as
this user. By default in the service console vmnix, this would be
/bin/bash for the Bourne-again shell. It appears the only other shell
shipped with vmnix is csh (the C shell).
adduser -c "Jane Doe" jdoe
adds a new user to Service Console
changes the password for jdoe
usermod -G esxadmins jdoe
a new group to Service Console group
In the above example, a new group
called esxadmins is created and therefore a new line appears in
best tool for modifying the /etc/group file. The following command adds
greg to the ESXUsers secondary group.
gpasswd –a greg ESXUsers
Group removal is simple with the –d switch:
gpasswd –d tony ESXUsers
command is used to create/modify a user.
Example adduser theben -p test123 -d /home/theben -c " The Ben Testaccount"
-p = password
-d = home directory
-c = comment
Be very careful with this command as when used with –G to set the users group membership, it is not adding the user to a group but is actually setting the list of secondary groups a user belongs to. Therefore in the following example if bill had secondary group list of ESXUsers and SQLadmin, then after entering:
usermod –G ali bill
then bill would only have a secondary
group of ali and nothing else!
user, super user and used without parameters expects root. If we
restrict root from logging in over SSH, then we force remote users to
authenticate as themselves and then SU to run privileged commands if
need be, thus leaving a decent audit trail.
delegation of administration in terms of certain commands that normally
only a particular user can execute (usually root).
is just the "vi" text editor, but it automatically opens the
/etc/sudoers file. The point of visudo is to ensure we always edit the
right file as the location of the sudoers file differs between nix
distributions, but this command is constant and will utilise the right
sudoers file for the distro being used.
text file that contains the sudo users and the rules that apply to them.
of security groups.
This may look like a list of users, but it is groups. As the service console (vmnix) is a modified version of Red Hat Linux, the Linux security configuration is the same as Red Hat. One feature of Red Hat no found in all Linux distributions is that of the user private group (UPG). Whenever you create a user, a group of the same name is created also and the user is made a member. The format of the file is:
so when we see groups like
JohnSmith:x:513 we can assume the 513 is the UID for the user JohnSmith
and this is his UPG.
of security users. When we add a user with either the MUI or a command
line tool such as useradd, we are adding to this text file.
the format of the file is
Normally the group ID will match the user ID.
directory – new home directories take copies of the files stored here.
directory stores key log files for both the service console and the
VMkernel. Of note are the vmkernel, vmkwarning & messages file logs.
These logs can be viewed with the more, cat, head and tail command line
We can also access these logs via the
MUI via the following link in the Options tab.
Check the file /etc/syslog.conf
for logging settings.
currently active network connections
open files Pipe the results into grep to check for open ports
lsof |grep IPv4.\*LISTEN
free memory in the service console. The "-m" switch specifies
to display the results in megabytes.
Disk partitioning tool
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty dos partition table
p print the partition table
q quit without saving changes
s create a new empty SUN disklabel
t change a partitions system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only !!)
usage. Great for finding out which folders are using disk space
du –h /home/ali/vmare
disk partitions (with human readable switch)
vmkfstools -l volume label
vmkfstools -N <label name>
vmkfstools -N vmhba0:4:60:0 /dev/vsd60
vmkfstools -l vmhbac:t:l:p
disk partitions with knowledge of VMFS partitions and space on disks(type FB) (with human
With vdf we can also see de vmfs partitions and de df of the Linux partitions.
to display the content of a VMFS volume
puts a label on the file system
to display the content of a VMFS volume, c is for controller number, t is for scsi target number, i for logical unit number (LUN) and the p for partition number.
dd floppy command
dump utility common to Linux. Copy a file while converting and
formatting. This can be a quick and dirty way of making an ISO. This
could be done in the service console with
dd if=/dev/cdrom of=/vmimages/new.iso
We can also make iso to floppy:
dd if=/dev/fd0 of=/vmimages/floppy.flp
This tool can be used to create an additional swap file. For example, if we did not allocate a big enough swap partition for the service console during ESX installation, we can create one now in a file of 64MB.
dd if=/dev/zero of=/swapfile bs=1M count=64
If we did add a swap file, we would
need to make sure it is started when ESX starts. Therefore, an entry in
/etc/fstab would be needed as this file describes the local and remote
file systems to mount at boot. The total amount of service console swap
space is the sum of the swap partition and any swap files that are
A command that must be run against a newly created swap file in order to activate it. Think of creating a swap file with dd like creating a partition, then mkswap is like formatting that partition. The swapon command then turns it on when needed.
swap file for service console
swap file for service console
text file that can be checked to see what swap the service console is
using. The output contains a priority which shows which swap device will
be used first before the other(s). Useful to determine if swap space is
getting used and if there is more than 1 swap. Remember this is vmnix
(service console) swap, not VMkernel. The VMkernel swap is in one or
more files on a VMFS volume (hence the strong recommendation that even
when using a SAN, a vmfs volume is created on direct attached storage to
allow local swap).
Type Size Used Priority
Basic File Commands
|rights management||-rw------ 1
root root 8654 Nov 8
-rw-rwx--- 1 theben root 1234 Nov 8 11:00 mftest.vmx
-rw-r--r-- 1 root root 28654 Nov 8 11:00 vmware-0.log
-rw-r--r-- 1 root root 28654 Nov 8 11:00 vmware-1.log
The first 10 characters in the line for mftest.vmx.
First (-) normal file or (d) for directory.
The following 9 characters are the permissions for the file. They are broken into 3 sets.
The first set is for the owner of the file.
The second set is for the group for the file.
The third set contains the others
R = read
Example for permissions:
used with a non-existent filename, this tool creates an empty file of
name filename. However, this can be used to touch an existing file and
update its last modified or last accessed attributes. This could be
scripted if required. Watch for running touch against any file stored on
a VMFS as there appears to be a problem. Remember that not all Linux
tools are modified for VMFS awareness. The VMFS is not an ext3 partition
but is a mount point, it’s the VMkernel that accesses any VMFS
scp stands for Secure CoPy.
With this command we can securly copy files between ESX servers.
Example syntax: scp /tmp/sourcefile.iso root@theben-on-remotehost: /vmimages/destinationfilename.iso
Let op:!! tussen remotehost:/vmimages geen spatie !!!
cat > file
|echo blah > file||Writes
the text following echo command to file. This could be good for quickly
echo modprobe usb-uhci > S92usb
Another great use of this technique is to make changes to the ESX server configuration via the /proc hierarchy, e.g. changing the number of shares for a VM
echo 2500 > /proc/vmware/vm/nnn/cpu/shares
would change the VM CPU shares to 2500. However such a change would only exist for the duration of the world created for that VM. After the VM is powered off this in memory structure is lost. To make such a change persistent, we would need to add the line
sched.cpu.shares = "2476"
to the VMX file of the virtual machine.
default, the head command prints the first 10 lines of the specified
file. We can choose how many lines we want instead of 10 by specifying
the –n switch. This is good for looking at the file /proc/vmware/vmhba:x:x:x/0:0
with the –n 22 switch. Also good for using with the file command to
determine whether a virtual disk is in ESX format or COW format.
head server.dsk | file -
The “–“ is crucial to making the above command work. For an ESX virtual disk we would expect to see something like
standard input: x86 boot sector
the last 10 lines of the specified file. Just like the head command,
there is a –n switch that can be specified to list the last n lines of
the named file.
alphabetically or numerically sort redirected command output or files.
or to sort a basic score sheet
sort –g –k 2 scores.txt
regular expression, used to string search the files or command outputs.
You can use grep –i to indicate search with case insensitive.
find utility is used much in the same way as many Windows people used
the DIR command. If you know roughly what files you are looking for,
then this is the tool. The ls tool simply lists, whereas the find tool
will find according to one or more criteria, a common one being find
files modified in the last day using the –mtime switch as shown in the
–mount used to
ensure it doesn't traverse to remote file systems
find –mmin -30
files modified in last 30 minutes
mount -o loop -t iso9660 (path to iso) /mnt/cdrom
mount alle devices
mount your cd-rom
list all files of cdrom
opposite of mount
to mount an ISO image in the service console, as if it were a cd-rom
mount -o loop -t iso9660 /vmimages/w2k-server.iso /mnt/cdrom
tar cvf archive_file directory_to_archive
tar xvf archive_file
tar xzvf archive_file
tar xjvf archive_file
tar tvf archive_file
to bundle a group of files into an archive
for example: tar cvf /root/etc_backup.tar .
Notice the trailing dot. It means that we wish to archive the current directory.
To use gzip, first make a tar file. The extention for gzip is: .gz
After we tar-ed the file, we can gzip it with: gzip filename.tar
To unbundle an archive into the current directory
To unbundle a compressed gzip archive into the current directory
To unbundle a compressed bzip2 archive into the current directory
list the content of an archive, not compressed
list the content of an archive compressed with gzip
list the content of an archive compressed with bzip2
to compress a file
the result will be: /root/etc_backup.tar.gz
A new compress algorithm, the result will be: /root/etc_backup.tar.bz2
to uncompress a file with gunzip
to uncompress a file with bzip2
UNIX text editor
text editor, more friendly but you should use –w to avoid word wrap.
changing of NIC, region, firewall, mouse, keyboard
files in a directory including hidden (also known as dot files due to
their prefix) files.
|ls -dl */||List
directories in long format (does not display files). Could add as a
shell alias, say lsd.
as ls –al
command line, great for piping large output into
the same as DOS and Windows, also great for piping large output into.
file ownership. If only 1 user name is specified then the user ownership
is set only and the group ownership if left unchanged as shown in the
solaris.vmx example below. However rather than having to use chown and
then chgrp straight after it, you can set user and group ownership in
one operation by specifying the username(s) separated by a colon as in
the netware5.vmx example shown.
#chown ali solaris.vmx
the group owner for a file, leaving the user owner unchanged.
#chgrp ali file.txt
file permission. We can use either letters or numeric equivalency when
setting permissions. We set permission for 3 principals, the user, the
group and others (ugo). If you are from a Windows background then
don’t confuse “o” with owner.
#chmod u+rx,g+r,o+r file.txt
Note that using + or – indicates we are adding to or removing from existing permission. If we wish to reset the permission we use “=” to explicitly set the object permissions.
#chmod u=rx,g=r,o=r file.txt yields r-xr—-r--
Sometimes you will see a chmod using “a” to specify all (user, group & other), so we could quickly set read permissions by
#chmod a-wx,a+r file.txt yields r—-r-—r--
A more common way to set permission is using chmod is using numeric equivalent values (4,2,1 for r,w,x) and permutations thereof.
chmod 777 windows2k.vmx
set perms to rwxrwxrwx
Watch for chmod commands with 4 digits, e.g. chmod 0754. This refers to additional attributes
Sticky bit SUID (Set User ID) SGID (Set Group ID)
The sticky bit can be set on executables which tells Linux to keep the application in memory. The reason for this is to improve load times for other users who wish to run the same executable. This relates to the multi-user nature of UNIX/Linux. Given the speed of memory and disk access nowadays the need to keep applications in memory is much less important and so the sticky bit needed so much.
Set User ID is used on an executable so that when it is run, it is run under the security context of the file owner not the current user. That means in case I have an application whose owner is ' root ' and it has its SUID bit set, then when I run this application as a normal user, that application would still run as root. Since the SUID bit tells Linux that the the User ID root is set for this application and whenever this application executes it must execute as if root was executing it (since root owns this file).
Set Group ID. Just like SUID, setting the SGID bit for a file sets your group ID to the file's group while the file is executing
To set any of these 3 attributes, we use a 4th digit preceding the usual 3 used with chmod. That digit is set using the following:
4 Set user ID (s) 2 Set group ID 1 Set sticky bit (t)
So if we want to set a file with permission rwxr-xr-x and set the user ID bit we could use
permissions feature you may encounter is that of umask.
VMware tool. Used to divide up the PCI resources between the service
console and VMs or if they are shared. E.g. 1 NIC to SC, 1 SCSI adapter
to share between SC and VMs, 1 SCSI adapter to VMs. This tool is stored
This is a command line tool which asks a series of questions. Use with the –i switch for interactive. Use the [ctrl] + [c] to leave vmkpcidivy without changes
This is also used to refreshnames and –q vmhba_devs
For example, if I had a SAN LUN of vmhba1:0:25 and lets say I removed the VMFS from this LUN and wished to use it from the service console, I’d refresh names and then query the vmhba_devs to find out what device name the service console was going to use for this LUN, e.g. /dev/sda.
A very useful feature of this tool is
the ability to create a new profile. This adds a new boot option to the
LILO boot menu that will have its own allocation of memory and PCI
devices. If you are unsure about the changes you are making, then create
a new profile e.g. esx (modified)
vmkfstools utility is the tool for managing virtual disks. Remember that
to copy into a VMFS could have an adverse affect on other VMs with
virtual disks on the same LUN. We always want to avoid using copy to
populate a VMFS. Copy operations will update the volume in 16k blocks
causing unnecessary SCSI reservations to update vmfs metadata.
Default syntax: vmkfstools <options> <path>
The switches that can be used with the command are listed below:
to import a virtual disk to VMFS
Remember that the vmfs parameter always goes last on this command parameter set for vmkfstools. This can be confusing for the beginner as the source and target order is different for imports and exports.
If we want to simply list the files on a vmfs volumes we use the -l switch.
vmkfstools –l /vmfs/vmhba0:0:0:8
or if we wish to use the volume label;
vmkfstools –l <vmfs-metadatalabel>
would produce an output similar to the following
Name: VMFS2-VOL1 (public)
vmkfstools –c 2048M /vmfs/vmhba0:0:0:8:newdisk.dsk
This command would create a new virtual disk (monolithic) on the specified vmfs volume. Remember it is always better to use the VMFS name as this will not change even if your hardware does.
To import a virtual disk into the VMFS we use vmkfstools with the -i switch. This will take a virtual disk in COW format into monolithic format without causing excessive SCSI reservations on the LUN holding the target VMFS
vmkfstools –i /vmimages/file.vmdk /vmfs/vmhba1:0:25:1/virtdisk.dsk
As always with this command, the parameter
specifying the VMFS location is always the last parameter.
command line tool to perform VM operations. This tool is found in /usr/bin
vmware-cmd –s register /home/ali/vmware/newVM/newVM.vmx
vmware-cmd –s unregister /<path to VM>
vmware-cmd /home/user/vmware/server/server.vmx stop soft ,,,stops a server by command softly
vmware-cmd /home/user/vmware/server/server.vmx getheartbeat
vmware-cmd /home/user/vmware/server/server.vmx getstate
vmware-cmd /home/user/vmware/server/server.vmx getid
vmware-cmd /home/user/vmware/server/server.vmx getpid
The stop type can be soft, try soft or hard.
is used to manage the VM kernel core dump partition. We can change the
partition used if required. This tool is also needed if the core dump
partition had been removed because ESX expects it to be there when
starting up, so we need to tell ESX that it has gone.
Query the VM kernel for which
partition it will use
Remember the vmkcore partition does not have a mount point in the
service console and is not specified as ext3. We can use the fdisk
-l command to view where the core dump partition is in relation
to the disk layout.
viewing with the –l switch, loaded and unloaded VMkernel modules. This
command differs from lsmod
which lists the modules loaded for the service console. This is a very
good way of differentiating what modules the kernel is using versus the
ones used by vmnix.
R/O Addr Length R/W Addr Length ID Loaded
Note, the -l parameter can also be
specified as --list
only lists eth0 (as discussed in network section) but in fact this text
file sometimes describes the devices that are assigned to the service
console by vmkpcidivy.
tool takes a service console driver and displays the options it
supports. For example
Would produce a list of flow control settings for the Intel gigabit NIC.
Would produce the file details and version of the HP Smart Array controller.
tool for listing installed pci devices. Could be used to demo what the VM is
presenting to the guest OS.
lspci –v verbose
You may wish to examine /proc/pci also
in order to correctly identify PCI devices and their slot
configurations. One point to note is that when you are faced with PCI
slot numbers is that not all hardware vendors number their slots in a
straight forward left to right configuration. Make sure you know your
slot numbers and their layout!
tool to list USB devices. Gives out way more info that is actually
required. Remember that USB devices cannot be presented to virtual
machines in ESX Server. If you wish to use a USB device in ESX, then you
will have to use a USB over IP device and install the appropriate driver
software into your guest OS for this. The most common USB over IP device
Red Hat tool to detect and configure hardware, can be dangerous with
text file maps ESX virtual switch names to device names. It is a network
map config file as opposed to the more generic devices map config file (devnames.conf).
network0.name = "SecuredGigabit"
Remember that if a virtual switch has no physical adapters, then it is vmnet_x. If a virtual switch has only 1 physical adapter is is vmnic_x. If there are two or more physical adapters assigned to a virtual switch, then the device is bondx.
A bond can be in one of three modes, out-mac (default), out-ip and standby
out-mac A VM virtual NIC is assigned to
a pNIC in the bond and it uses only that
text file maps device names (example above) to modules and their PCI
addresses. Note that the devnames.conf file contains SCSI devices and
002:14.0 megaraid vmhba0
to be like a hardware compatibility list. Watch out for creating your
own device map, devices.local The /etc/vmware/vmware-devices.map file
contains a list of devices supported by ESX Server. This release
includes support for a local version of this file, /etc/vmware/vmware-devices.map.local.
Modify the vmware-devices.map.local to select different device drivers.
This file is not modified during an ESX Server upgrade, preserving your
customizations. The vmware-devices.map.local is read when the VMkernel
Any changes to the vmware-devices.map.local file require a reboot, or at least an unload/reload of the VMkernel to take effect.
Entries in the vmware-devices.map.local files are used in addition to the entries in the vmware-devices.map file. The vmware-devices.map.local file does not need to mirror the vmware-devices.map file.
Any vmware-devices.map.local file entries that correspond to the vmware-devices.map file entries supercede the vmware-devices.map file entries.
of information in this text file. Useful for finding which nic is in
which team. If using alongside devnames.conf and netmap.conf use the
grep –i vmnic /etc/vmware/hwconfig
to address the floppy disk drive
cd /proc/scsi/<driver name>/<adapter number>
directory of scsi SAN card
list of the config file
The /proc/vmware/scsi directory will present 1 directory vmhba<N> where <N> is the host adapter. The directory will contain 1 entry per device, with a format <scsi id> : <LUN number>
file with discovered PCI devices
Bus:Sl.F Vend:Dvid Subv:Subd Type
Vendor ISA/irq/Vec P M Module Name Spawned bus
tool is also known as “The VMkernel Network Card Locator”. It
locates a physical NIC in an ESX server by using pings. It is left to
the operator to check which NIC is sending the echo requests by either
unplugging network cables or inspecting flickering lights on the NIC or
the switch. The useful feature of this tool is we can ping based on the
device name that the VMkernel uses to access the NIC. We should remember
that each physical NIC in an ESX server that is allocated to the
VMkernel does not itself have an IP address, therefore in to perform an
ICMP echo request, we need to temporarily give that NIC an IP; this is
specified as the first IP parameter in the command, the second IP
parameter being the ICMP destination.
findnic –i 5 vmnic2 10.0.0.1 192.168.1.3
The above command will send ICMP echo requests to 192.168.1.3 every 5 seconds. We could also use the –f switch which would flood ping.
|cat /proc/vmware/version||Provides an list of the kernel
List all services
/etc/init.d/sshd restart or
service sshd restart
Restart a service
volatile /proc directory hierarchy can be treated as a file system but
is held in RAM.
text file snapshot of cpu scheduling. You could increase CPU shares
using this method
echo 10000 > /proc/vmware/vm/<number>/cpu/shares
is a fantastic utility that polls whatever command you supply it with an
displays a running changing status. For example, we could use the
watch cat /proc/vmware/mem
to obtain a dynamic view of memory usage by VMware. If you are viewing lots of output but can't see what is actually changing we use the -d parameter to specify differences, thus highlighting changes between refreshes.
watch –d cat /proc/vmware/mem
To specify a different refresh rate, we use the --interval=n parameter
rqm -qa | sort | more
rqm - qi -p /mnt/cdrom/vmware/rpms/*1*
rqm -Uvh /mnt/cdrom/vmware/rpms/*1*
rqm -e package name
to list all installed software packages
to list all packages in sorted order with screendisplay pause
to get a discription of an installed package
to get a list of the files that make up a package
to mount de ESX cdrom
see the packages /*1* = package file name
to install a package /*1* = package file name
to uninstall a package
is a redirector type tool to view and connect to SMB (Microsoft
networking) hosts. Before updating the /etc/fstab file with remote file
system information, check first using smbclient that the share is
visible. The following was produced with
smbclient –I 192.168.1150 –U username –L computername
Sharename Type Comment
To create a mount point to a Microsoft share is very straightforward. Remember, we are allowing the service console to access a remote file system. This is not related to what virtual machines are doing. Further, we need to be careful if we are attempting to do any file operations due to potential limits with 2GB file sizes.
1. Add entry to local hosts
(/etc/hosts) for MS host
//server/share /mountdir smbfs ip=ipaddress,username=user,password=pass,noauto 0 0
5. Mount the remote file system with
Alternatively, if you just want to map temporarily to a Microsoft host and not have to modify fstab, then use smbclient interactively as shown:
smbclient //ipaddress/share –U NThost\NTuser Password: ****** Domain=[TAUPO] OS=[Windows 5.0] Server=[Windows 2000 LAN Manager] smb: \>
File System (NFS) is provided by rpc.nfsd and would normally be launched
by an nfs script in /etc/rc.d. To start using NFS to mount directories
on other ESX or Linux servers, we can use the following steps:
1. Change the ESX Server which is to be
the NFS server to use medium security (using the MUI is easiest for
command is used by a NFS client to see what directories are being
exported by a NFS server.
showmount –e nfsserver
This command can be specified with the hostname name or IP address of the NFS server holding the exported directories.
exportfs command allows you to selectively export or unexport
directories without restarting the various NFS services.
stands for Network Information Service. Ensures numeric owner IDs are
unique across the organisation. This is because numeric owner IDs are
used in NFS, so we are often going to have mistaken identity as used 515
on the nfs client will not be the same as user 515 on the nfs server.
like the Windows utility to mount virtual disks when they are powered
off so you can check what’s in them. This will mount ext3 and vfat as
read/write but NTFS as read only. To find out what file systems are in
the virtual disk, use the –p switch.
vmware-mount.pl –p /vmfs/VMFS-VOL1/win2k3.dsk
Start Size Type Id Sytem
If we actually want to mount a partition then we need to be specific and create a directory (or use an existing) to be our mount point.
This command will tie up the console window hence you’ll need to spawn a new window first to navigate to /myntfs to view the contents.
file system This command can be used with the –t switch to specify
file system type, e.g. nfs, smbfs or iso9660
mount -t nfs [-o options] esx4:/vmimages /root/other_server_vmimages
file system. Note it is u-mount and not unmount!
mount –t smbfs passes control across to this utility. We can use this
utility directly if we prefer for mounting SMB host file systems.
we have smb mount points permanently, then this file could end up with
user credentials in it. This file is readable by everyone so this is not
good. We can place the credentials for the smbmount in a hidden secured
file in our home folder eg. /root/.smbcreds
echo username=user > .smbcreds echo password=pass >> .smbcreds chmod 600 .smbcreds
Then in the /etc/fstab file we substitute the username and password for credentials=/root/.smbcreds. Therefore the whole line in the fstab would be
//server/share /mountdir smbfs ip=ipaddress,credentials=/root/.smcreds,noauto 0 0
hidden file that extends shell script for the BASH shell. This is found
in the users home folder.
bash shell settings
of CLS command in MS-DOS and in Windows command prompt.
the 16-bit checksum and size of the specified file.
the MD5 checksum for the file supplied as a command parameter. VMware
publish md5 checksums for all downloads to allow the end user to confirm
that the file had not be corrupted in download.
calendar for current month -3 last, current and next month 200x display
year cal 2005 | less to get scrollable calendar
generated text file listing the VMs on the server. This file should not
be manually edited, should match up with output of vmware-cmd –l.
stands for Process ID. Every running process has a process ID that is
valid for the length of process execution.
stands for Parent Process ID, which is the PID of the process that
launched that process.
running processes in the service console.
ps –A ps –eaf
is useful as the “w” indicates wide format,
so we can see the full directory path to the vmx file.
to kill a process:
kill pid (process ID)
example: kill -9 12345 -9 parameter is for the kill level, 9 is highest level.
to list all running servie console processes, organized to show their parent child relationchips.
to list all running servie console processes, organized to show their parent child relationchips.
Show process tree with PIDs and ancestors
version of process viewer top
process priority. If there were many VMs running concurrently, it may be
necessary to temporarily raise the priority of the MUI in order to
complete administrative tasks
#ps –eaf |grep vmware
To reset the PID of these processes back to their defaults, use renice again to set the priority to zero.
the PID of a named process.
can configure processes to run in the background by adding a “&”
suffix when launching the program.
#sleep 10 &
the process jobs running in the background.
SAN & Disk Management
is the default setting, which scans 0 to 7 on start up. Remember if we
what to scan up to LUN N we must set Disk.MaxLUN to N+1. This setting is
found in the /etc/vmware/vmkconfig file but should be modified via the
MUI (Options tab, Advanced Settings).
setting Disk.SupportSparseLUN should =1 as LUNs may be discontiguous.
For example if there are disk volumes at LUNs 0,1,2 and 6 then we want
to be sure that after LUN 2, the VMkernel storage driver does not stop
scanning. We want the LUN scanning to reach the last LUN specified in
DiskMaxLUN parameter, regardless of whether the visible LUN numbers are
contiguous or not.
parameter controls LUN visibility, again this is in the /etc/vmware/vmkconfig
file. LUN masking is only supported on fibre channel HBAs. This
overrides the Disk.MaxLUN setting.
vmhba0:0:4,6-255 would scan 0,1,2,3,5 i.e. skip 4 and skip 6 through 255 vmhba0:0:3,4,9-255 would scan 0,1,2,5,6,7,8 i.e. skip 3 & 4 and skip 9 through 255
is a text file which stores vmkernel configuration, like an INI file or
registry key. The important fact to note is that this file does not
exist until you make an edit away from the default setting.
|vmkfstools –s vmhba0||
Used to re-scan for new LUNs on specified host bus adapter.
devices are identified by a world wide name, a unique 64-bit address.
Remember we can use the perl script wwpn.pl to determine quickly what
the WWN is for the installed FC hba.
LUN identifies individual units of storage behind a SCSI ID. A LUN could
be a single disk or a RAID5 volume.
is either hard (switch port) or soft (WWN controlled)
Masking is a disk array feature that controls which LUNs are presented
to which WWNs
WWN for an adapter would be found in this file.
setting tells ESX server to retry SCSI commands as vendor specific
status codes may have been received and ESX may think the volume is
present but not accessible when in fact its just a message that cache
has been upgraded.
|Rules of Thumb||Processor rule, 4VM's per
processor. (3Ghz processors of course)
Hyper-threading and large cache of the processor is a big benefit.
Do not forget that the service console of ESX also uses processor time !!
Don't start making VM's with default 2 processors.
Only servers as SQL and Exchange can use VSMP, multi processor.
HDD rule, use 15k SCSI disks.
Network rules, install at least 3 NIC's. One for console, one for VMotion and one for VM.
simple utility to display free memory in the service console.
is the VMware version of top and provides CPU, memory and disk
configuration realtime information just like top does, but this time we
only see the data relating to the VMkernel so we see worlds instead of
processes as resource consumers.
PCPU is a comma delimited field that has the current utilization of ech processor followed by the average utilization of all processors.
LCPU is the Logical CPU line. Only available with Hyper Threading processors.
MEM line, shows the amount of physical memory managed by the system.
SWAP line, this one should be 0 !! This is the VMware swap file stored on VMFS and should not be confused with the Console's swap space.
We can config the esxtop command, just type esxtop -f Only by placing a *, the selected fields will be displayed.
If we want to create a logfile with esxtop, just run this syntax:
Shows the running processes in the service console and lists the top consumers of CPU time.
lists all programs running in the background
lots of cpu time means system is busy
little idleness means system is busy
get confused by this command, it is a Linux command, not a VMware
command. This is meant to view Linux processes, memory and paging. We
generally use vmstat with 2 numeric parameters, the first parameter is
how frequently the tool should run (specified in seconds). The second
parameter specifies how many times the tool should run e.g.
vmstat 3 10
would run the tool every 3 seconds for 10 times
and then exit.
info on using the /proc/vmware file system to change share allocation
and processor affinity.
a subdirectory called /vm there are subdirectories for each vm labelled
by number How do you find out what number corresponds to what VM?
grep –i worldid= ~ali/vmware/ISAserver/vmware.log
Some other examples of great info that can be extracted are outlined in the following table:
/proc/vmware/vm/139/cpu affinity 0,1,2,3
|vmkusagectl install||run this
install first to use vmkusagectl
After installation we can you a webinterface to look at the performance. Use http://ipaddress-server/vmkusage
Generates web page usage reports. Was not automatically installed in previous versions of ESX
vmkusagectl install to install the utilisation web pages & setup a cron job vmkusage –graph to generate graph images vmkusage –regroove to wipe the db and start stats again
is a web page like the MUI accessible via
Not sure if we need to be logged in for this to
work, but vmkusage does appear to be required. We get loads of output on
this page, similar to running command line tools. This is part of what
looks like the legacy interface to ESX server, i.e. it doesn’t look as
cool as the MUI of ESX 2.x.
the cron jobs scheduled for the user when used with the –l (list)
|troubleshoot, what to do when randomly processes being killed on the ESX server||there will be insufficient Service Console swap space|
|troubleshoot, i am unable to start new virtual machines||perhaps you have insufficient Service Console Swap space|
to walkthrough SNMP mibs. -M – use MIBSDIR -m all use mibs list
instead of default mibs list
utility to display SNMP traps. MIBs can be loaded into this.
Backup/recovery + Disaster Recovery
things to be discussed
Difference between backing up the ESX host/Service Console or the VM's
files to backup - console Operating System
virtual machine configuration
/etc/profile - /etc/ssh/sshd_config - /etc/pam.d/system_auth
/etc/passwd - /etc/group - /etc/sudoers - /etc/shadow
/etc/vmware = all ESX config files + kernel
are the vmx files, stored in /home/vmware or /root/vmware
backup options for the Service Console
1. over the network
.2. to a local tape device
.3. via local device used by a VM
Add SCSI controller in the ESX server, install the tape drive on the ESX server.
Using vmkpcidivy, assign the SCSI controller and the tape drive to the VM.
Then, using the Web Interface, assign the device to your specific backup VM.
Install the device drivers within the VM and install any backup software.
Verify that the VM has network connectivity since the backup will work like a backup over the network. Traffic will flow from the VM's VMNIC to eth0 over the network.
Ensure a backup agent is installed on the Console and Schedule the backups.
The most simple way to make a backup, stop the VM and make a backup
of the vmdk file.
soft stopping a VM by commands: vmware-cmd <configfile> stop soft
example of hard stopping : vmware-cmd <configfile> stop hard
to power on a VM from the command line: vmware-cmd vmconfig_file start soft
example: vmware-cmd /home/ted/vmware/w2k/w2k.vmx start soft
not ready jet
|Use this command to change the
date and time of the ESX server.
Syntax = date MMDDHHMMYYY = Month-Day-Hour-Minute-Year
We can also use the setup command to change the system time settings.
View of all time servers = http://ntp.isc.org
chkconfig ntpd on
to synchronize the Service Console's clock over the network
The service Console clock can be client of a NTP or SNTP server
arrange for NTP client daemon to be started at boot
create /etc/ntp/step-tickers for rapid clock adjustment at boot. Into this file place the DNS name or IP address of your NTP server, for example:
create /etc/ntpd.conf for normal operation. Into this file place the DNS name or IP address of your NTP server, preceded with the word server, for example:
launch the NTP daemon with: service ntpd start
gives a list of used, executed commands
history |tail shows only the last used, executed commands
Clear this History database = history -c
with this command you can generate
an output file which you can use for troubleshooting with Vmware of for
documentation of your ESX farm. It collects all config files and makes
one big zipped file of it.
disabling root account over ssh
Because of security, we can permit
the use of the root account. In the /etc/ssh/sshd_config we have to
change the line #PermitRootLogin yes in PermitRootLogin no, note:
see that the # has gone and we changed yes into no. Donot forget to
restart the sshd service.
You can also find a version of the guide at: www.b2v.co.uk/b2vguide2vmware.htm, but the input of TheBen is not included in that page!