SysAdmin TnT

System administration tips ‘n’ tricks

Kickstart

Blog, SysAdmin TnT
Posted on

References

Pre-Installation Script
The important bits:
  1. The pre-install script is not run in the change root environment.
  2. You can access the network in the %pre section; however, name service has not been configured at this point, so only IP addresses work.
Post-Installation Script

The important bits:

  1. If you configured the network with static IP information, including a nameserver, you can access the network and resolve IP addresses in the %post section. If you configured the network for DHCP, the /etc/resolv.conf file has not been completed when the installation executes the %post section. You can access the network, but you can not resolve IP addresses. Thus, if you are using DHCP, you must specify IP addresses in the %post section
  2. The post-install script is run in a chroot environment; therefore, performing tasks such as copying scripts or RPMs from the installation media do not work.

Clonezilla Use Cases

Blog, SysAdmin TnT
Posted on

01 – PXE boot PXELinux into Clonezilla live mode

PXE server ---> PXELinux ---> pxelinux.0

The kernel boot parameters control the behavior of the Clonezilla operating system that is being told to boot.

This is the most helpful and concise description of what our goal is, taken from clonezilla.org’s boot parameter doc.

…..when Clonezilla live boots, the file assigned by ocs_preload will be downloaded and extracted to /opt. Its mode will be set automatically, too. i.e. set as mode 755 and Unix format script.

 

Besides, if /opt/{overwrite-all-boot-param,overwrite-part-boot-param} exists (Downloaded from the file assigned by ocs_preload), it can be used to overwrite the boot parameters. File overwrite-all-boot-param is to overwrite the whole /proc/cmdline, while overwrite-part-boot-param only overwrites part of the variables in /proc/cmdline. Especially those “ocs_*” parameters. This is useful when you want to customize the boot parameters while you do not want to or can not modify the boot parameter file (isolinux.cfg, for example).

For example, the contents of “overwrite-all-boot-param” can be like: boot=live union=overlay username=user config components quiet noswap edd=on nomodeset nodmraid locales=en_US.UTF-8 keyboard-layouts=en ocs_live_run="ocs-sr -x -um beginner --batch -p reboot -scs -scr -sfsck -senc" ocs_live_extra_param="" ocs_live_batch=no vga=788 ip= net.ifnames=0 nosplash i915.blacklist=yes radeonhd.blacklist=yes nouveau.blacklist=yes vmwgfx.enable_fbdev=1

 

The content of “overwrite-part-boot-param” can be like: locales=ja_JP.UTF-8 keyboard-layouts=fr ocs_live_run="ocs-sr -x -scr -scs" ocs_live_batch="no"

 

//NOTE//

  • Remember to set the network configuration either by the option “ocs_prerun” to run “dhclient” or the option “ip” if your files are on the network repository.
  • You can also use “ocs_prerun” to use your own command to fetch your files and put in Clonezilla live environment if you like, and that’s more flexible if you want to assign more parameters for your own command.
Example Configuration
DEFAULT linux
LABEL linux
KERNEL /iso/clonezilla/vmlinuz

 APPEND initrd=/iso/clonezilla/initrd.img boot=live username=user \
union=overlay config components quiet noswap edd=on nomodeset nodmraid \
locales=en_US.UTF-8 keyboard-layouts=NONE nicif="00:00:00:fd:55:14" \
ip=eth0:172.16.31.16:255.255.255.0:172.16.31.1:172.16.31.13, \
eth0:172.16.31.16:255.255.255.0:172.16.31.1:172.16.31.13:: \
ocs_live_netdev="eth0" \
ocs_repository="nfs://files.pjnet.local/data/disk-images/raw/" \
ocs_preload="http://files.pjnet.local/data/pxe-images/clonezilla/restore.sh" \
ocs_live_run="ocs-sr --batch -g auto -e1 auto -e2 -r -j2 -k1 -icds -p true restoredisk ubuntu18_4 sda" \
ocs_postrun="wget -q -O /dev/null --no-check-certificate http://iftt.pjnet.local/built && sudo reboot" \
ocs_live_batch=yes net.ifnames=0 nosplash nopromp fetch=tftp://172.16.31.211/iso/clonezilla/filesystem.squashfs

IPAPPEND 2

ocs_preload – To fetch tarball/zip/sh files from http(s), ftp, tftp, and local URL then exdtract to /opt/ in the Clonezilla live environment, use ocs_preload

ocs_live_run – For saving or restoring an image or cloning disk/partition. It is the main program to be run in Clonezilla live.

The programs that are available to us at run-time are:

ocs-btsrv ocs-chkimg ocs-chnthn ocs-clean-part-fs ocs-cnvt-usb-zip-to-dsk ocs-cvt-dev
ocs-cvtimg-comp ocs-decrypt-img ocs-encrypt-img ocs-expand-gpt-pt ocs-expand-mbr-pt
ocs-gen-bt-slices ocs-gen-grub2-efi-bldr ocs-get-part-info ocs-img-2-vdk ocs-install-grub
ocs-iso ocs-iso-2-onie ocs-label-dev ocs-lang-kbd-conf ocs-langkbdconf-bterm ocs-live
ocs-live-bind-mount ocs-live-boot-menu ocs-live-bug-report ocs-live-dev ocs-live-feed-img
ocs-live-final-action ocs-live-general ocs-live-get-img ocs-live-netcfg ocs-live-nicbonding
ocs-live-preload ocs-live-repository ocs-live-restore ocs-live-run-menu ocs-live-save
ocs-lvm2-start ocs-lvm2-stop ocs-makeboot ocs-match-checksum ocs-onthefly ocs-prep-home
ocs-put-signed-grub2-efi-bldr ocs-related-srv ocs-resize-part ocs-restore-ebr ocs-restore-mbr
ocs-restore-mdisks ocs-rm-win-swap-hib ocs-run-boot-param ocs-sr ocs-srv-live ocs-tune-conf-for-s3-swift
ocs-tune-conf-for-webdav ocs-tux-postprocess ocs-update-initrd ocs-update-syslinux ocsmgrd

In our case, we want to save or restore an image, so we’ll want to use ocs-sr (“sr” means “save or restore”).  I can’t find the help text for ocs-sr online so I’ll post the entire thing here.  The only way I know to get it is to boot into Clonezilla and grab it from there.

Usage:
To save or restore image
ocs-sr [OPTION] {savedisk|saveparts|restoredisk|restoreparts} IMAGE_NAME DEVICE

Options for saving:
-noabo, --not-only-access-by-owner Make the image of partition can be accessed by others, not only by owner. By default the image of partition will be changed as 600. With this option, it will be 644.
-senc, --skip-enc-ocs-img Skip encrypting the image with passphrase.
-enc, --enc-ocs-img To encrypt the image with passphrase.
-sfsck, --skip-fsck-src-part Skip running fsck on the source file system before saving it.
-fsck, -fsck-src-part, --fsck-src-part Run fsck interactively on the source file system before saving it.
-fsck-y, -fsck-src-part-y, --fsck-src-part-y Run fsck automatically on the source file system before saving it. This option will always attempt to fix any detected filesystem corruption automatically. //NOTE// Use this option in caution.
-gm, --gen-md5sum Generate the MD5 checksum for the image. Later you can use -cm|--check-md5sum option to check the image when restoring the image. Note! It might take a lot of time to generate if the image size is large.
-gs, --gen-sha1sum Generate the SHA1 checksum for the image. Later you can use -cs|--check-sha1sum option to check the image when restoring the image. Note! It might take a lot of time to generate if the image size is large.
-gmf, --gen-chksum-for-files-in-dev Generate the checksum for files in the source device. Later you can use -cmf|--chk-chksum-for-files-in-dev to check the files in the destination device after they are restored. Note! It might take a lot of time to inspect the checksum if there are many files in the destination device.
-i, --image-size SIZE Set the size in MB to split the partition image file into multiple volumes files. For the FAT32 image repository, the SIZE should not be larger than 4096.
-j2, --clone-hidden-data Use dd to clone the image of the data between MBR (1st sector, i.e. 512 bytes) and 1st partition, which might be useful for some recovery tool.
-ntfs-ok, --ntfs-ok Assume the NTFS integrity is OK, do NOT check again (for ntfsclone only)
-rm-win-swap-hib, --rm-win-swap-hib Try to remove the MS windows swap file in the source partition.
-q, --use-ntfsclone If the partition to be saved is NTFS, use program ntfsclone instead of partimage (i.e. Priority: ntfsclone > partimage > dd)
-q1, --force-to-use-dd Force to use dd to save partition(s) (inefficient method, very slow, but works for all the file system).
-q2, --use-partclone Use partclone to save partition(s) (i.e. partclone > partimage > dd).
-rescue, --rescue Turn on rescue mode, i.e. try to skip bad sectors.
-sc, -scs, --skip-check-restorable, --skip-check-restorable-s By default Clonezilla will check the image if restorable after it is created. This option allows you to skip that.
-z0, --no-compress Don't compress when saving: very fast but very big image file (NOT compatible with multicast restoring!!!)
-z1, --gzip-compress Compress using gzip when saving: fast and small image file (default)
-z1p, --smp-gzip-compress Compress using parallel gzip program (pigz) when saving: fast and small image file, good for multi-core or multi-CPU machine
-z2, --bz2-compress Compress using bzip2 when saving: slow but smallest image file
-z2p, --smp-bzip2-compress Compress using parallel bzip2 program (lbzip2) when saving: faster and smallest image file, good for multi-core or multi-CPU machine
-z3, --lzo-compress Compress using lzop when saving: similar to the size by gzip, but faster than gzip.
-z4, --lzma-compress Compress using lzma when saving: slow but smallest image file, faster decompression than bzip2.
-z5, --xz-compress Compress using xz when saving: slow but smallest image file, faster decompression than bzip2.
-z5p, --smp-xz-compress Compress using parallel xz when saving: slow but smallest image file, faster decompression than bzip2.
-z6, --lzip-compress Compress using lzip when saving: slow but smallest image file, faster decompression than bzip2.
-z6p, --smp-lzip-compress Compress using parallel lzip when saving: slow but smallest image file, faster decompression than bzip2.
-z7, --lrzip-compress Compress using lrzip when saving.
-z8, --lz4-compress Compress using lz4 when saving.
-z8p, --lz4mt-compress Compress using lz4mt when saving.
-z9, --zstd-compress Compress using zstd when saving.
-z9p, --pzstd-compress Compress using pzstd when saving.
-i, --image-size SIZE Set the split image file volume size SIZE (MB). When ocs-sr is run with -x, the default SIZE is set as 4096, if without -x, we will not split it.
-pe, --passwd-ecryptfs PASSWD Set the password to encrypt the image. Use with option "-enc" when saving, or restoring when the image is encrypted. //NOTE// This is not a safe way to use it because the password is shown in the command line or system processes.
-pfe, --passwd-file-ecryptfs FILE Set the password to encrypt the image in the FILE. Use with option -enc, or restoring when the image is encrypted. The file contains the password to encrypt the image. Its format is like: passphrase_passwd=YOURPASSWORD.
Some words are reserved for IMAGE_NAME, "ask_user" is used to let user to input a name when saving an image. "autoname" is used to automatically generate the image name based on network card MAC address and time. "autohostname" is used to automatically generate the image name based on hostname. "autoproductname" is used to automatically generate the image name based on hardware product model gotten from dmidecode.
A word is reserved for DEVICE, "ask_user" could be used to let user to select the source device when saving an image.

Options for restoring:
-f, --from-part-in-img PARTITION Restore the partition from image. This is especially for "restoreparts" to restore the image of partition (only works for one) to different partition, e.g. sda1 of image to sdb6.
-g, --grub-install GRUB_PARTITION Install grub in the MBR of the disk containing partition GRUB_PARTITION with root grub directory in the same GRUB_PARTITION when restoration finishes, GRUB_PARTITION can be one of "/dev/sda1", "/dev/sda2"... or "auto" ("auto" will let clonezilla detect the grub root partition automatically). If "auto" is assigned, it will work if grub partition and root partition are not in the same partition.
-r, --resize-partition Resize the partition when restoration finishes, this will resize the file system size to fit the partition size. It is normally used when when a small partition image is restored to a larger partition.
-k, --no-fdisk, --no-create-partition Do NOT create partition in target harddisk. If this option is set, you must make sure there is an existing partition table in the current restored harddisk. Default is to create the partition table.
-icrc, --icrc Skip Partclone CRC checking.
-irhr, --irhr Skip removing the Linux udev hardware records on the restored GNU/Linux.
-irvd, --irvd Skip removing the NTFS volume dirty flag after the file system is restored.
-ius, --ius Skip updating syslinux-related files on the restored GNU/Linux.
-icds, --ignore-chk-dsk-size-pt Skip checking destination disk size before creating the partition table on it. By default it will be checked and if the size is smaller than the source disk, quit.
-iefi, --ignore-update-efi-nvram Skip updating boot entries in EFI NVRAM after restoring.
-k1, Create partition table in the target disk proportionally.
-k2, Enter command line prompt to create partition table manually before restoring image.
-scr, --skip-check-restorable-r By default Clonezilla will check the image if restorable before restoring. This option allows you to skip that.
-t, --no-restore-mbr Do NOT restore the MBR (Mater Boot Record) when restoring image. If this option is set, you must make sure there is an existing MBR in the current restored harddisk. Default is Yes
-u, --select-img-in-client Input the image name in clients
-e, --load-geometry Force to use the saved CHS (cylinders, heads, sectors) when using sfdisk
-e1, --change-geometry NTFS-BOOT-PARTITION Force to change the CHS (cylinders, heads, sectors) value of NTFS boot partition after image is restored. NTFS-BOOT-PARTITION can be one of "/dev/sda1", "/dev/sda2"... or "auto" ("auto" will let clonezilla detect the NTFS boot partition automatically)
-e2, --load-geometry-from-edd Force to use the CHS (cylinders, heads, sectors) from EDD (Enhanced Disk Device) when creating partition table by sfdisk
-j, --create-part-by-sfdisk Use sfdisk to create partition table instead of using dd to dump the partition table from saved image (This is default)
-j0, --create-part-by-dd Use dd to dump the partition table from saved image instead of sfdisk. ///Note/// This does NOT work when logical drives exist.
-j1, --dump-mbr-in-the-end Use dd to dump the MBR (total 512 bytes, i.e. 446 bytes (executable code area) + 64 bytes (table of primary partitions) + 2 bytes (MBR signature; # 0xAA55) = 512 bytes) after disk image was restored. This is an insurance for some hard drive has different numbers of cylinder, head and sector between image was saved and restored.
-j2, --clone-hidden-data Use dd to clone the image of the data between MBR (1st sector, i.e. 512 bytes) and 1st partition, which might be useful for some recovery tool.
-hn0 PREFIX Change the hostname of MS Windows based on the combination of hostname prefix and IP address, i.e. PREFIX-IP
-hn1 PREFIX Change the hostname of MS Windows based on the combination of hostname prefix and NIC MAC address, i.e. PREFIX-MAC
--max-time-to-wait TIME When not enough clients have connected (but at least one), start anyways when TIME seconds since first client connection have pased. This option is used with --clients-to-wait
-cm, --check-md5sum Check the MD5 checksum for the image. To use this option, you must enable -gm|--gen-md5sum option when the image is saved. Note! It might take a lot of time to check if the image size is large.
-cs, --check-sha1sum Check the SHA1 checksum for the image. To use this option, you must enable -gs|--gen-sha1sum option when the image is saved. Note! It might take a lot of time to check if the image size is large.
-cmf, --chk-chksum-for-files-in-dev Check the checksum for the files in the device. To use this option, you must enable -gmf|--gen-chksum-for-files-in-dev when the image is saved. Note! (1) The file system must be supported by Linux kernel so that it can be mounted as read-only to check the files. (2) It might take a lot of time to check if there are many files in the source device.
-srel, --save-restore-error-log Save the error log file in the image dir. By default the log file won't be saved when error occurs.
--mcast-port NO Assign the udp port number for multicast restore. This is used by clonezilla server. Normally it's not necessary to manually assign this option.
Some words are reserved for IMAGE_NAME, "ask_user" is used to let user to input a name when saving an image. "autoproductname" is used to automatically get the image name based on hardware product model from dmidecode.
A word is reserved for DEVICE, "ask_user" could be used to let user to select the source device when saving an image.

General options:
-l, --language INDEX Set the language to be shown by index number:
[0|en_US.UTF-8]: English,
[2|zh_TW.UTF-8]: Traditional Chinese (UTF-8, Unicode) - Taiwan
[a|ask]: Prompt to ask the language index
-b, -batch, --batch (DANGEROUS!) Run program in batch mode, i.e. without any prompt or wait for pressing enter key. //NOTE// You have to use '-batch' instead of '-b' when you want to use it in the boot parameters. Otherwise the program init on system will honor '-b', too.
-c, --confirm Wait for confirmation before saving or restoring
-d, --debug-mode Enter command mode to debug before saving/restoring
--debug=LEVEL Output the partimage debug log in directory /var/log/ with debug LEVEL (0,1,2... default=0)
-m, --module MODULE Force to load kernel module MODULE, this is useful when some SCSI device is not detected. NOTE! Use only one module, more than one may cause parsing problem.
-o0, --run-prerun-dir Run the script in the directory /usr/share/drbl/postrun/ocs/ before clone is started. The command will be run before MBR is created or saved.
-o1, -o, --run-postrun-dir Run the script in the directory /usr/share/drbl/postrun/ocs/ when clone is finished. The command will be run before that assigned in -p or --postaction.
-w, --wait-time TIME Wait for TIME secs before saving/restoring
-nogui, --nogui Do not show GUI (TUI) of Partclone or Partimage, use text only
-a, --no-force-dma-on Do not force to turn on HD DMA
-mp, --mount-point MOUNT_POINT Use NFS to mount MOUNT_POINT as directory ocsroot (ocsroot is assigned in drbl.conf)
-or, --ocsroot DIR Specify DIR (absolute path) as directory ocsroot (i.e. overwrite the ocsroot assigned in drbl.conf)
-p, --postaction [choose|poweroff|reboot|command|CMD] When save/restoration finishes, choose action in the client, poweroff, reboot (default), in command prompt or run CMD
-ns, --ntfs-progress-in-image-dir Save the ntfsclone progress tmp file in the image dir so that if cloning is in DRBL client, the progress can be check in the server (Default in to be put in local /tmp/, which is local tmpfs).
-um, --user-mode [beginner|expert] Specify the mode to use. If not specified, default mode is for a beginner.
-v, --verbose Prints verbose information
-d0, --dialog Use dialog
-d1, --Xdialog Use Xdialog
-d2, --whiptail Use whiptail
-d3, --gdialog Use gdialog
-d4, --kdialog Use kdialog
-x, --interactive Interactive mode to save or restore.
-os, --ocs-server SRV Assign the OCS server as SRV.

Example:
To save or restore image in client (Only that DRBL client will join, and its local partitions is NOT mounted). NOTE!!! You should run the command in DRBL client or you have to make sure the target device is NOT busy!.
To save all the data in local first IDE harddrive 'hda' as image 'IMAGE1', use ntfsclone instead of partimage, and lzop compression (NOTE!!! You should run the command in DRBL client or make sure hda is NOT busy/mounted!):
ocs-sr --use-ntfsclone -z3 savedisk IMAGE1 hda

To save the data in first and second partitions in local first IDE harddrive 'hda' as image 'IMAGE2', use ntfsclone instead of partimage, and lzop compression (NOTE!!! You should run the command in DRBL client, or make sure hda is NOT busy/mounted!):
ocs-sr --use-ntfsclone -z3 saveparts IMAGE2 "hda1 hda2"

To restore image IMAGE1 to local hda. grub-install will be run after cloning (image IMAGE1 is already in DRBL server. NOTE!!! You should run the command in DRBL client or make sure hda is NOT busy/mounted!):
ocs-sr -g auto restoredisk IMAGE1 hda

To restore image first and second partitions from IMAGE2 to local hda1 and hda2. grub-install will be run after cloning (image IMAGE2 is already in DRBL server. NOTE!!! You should run the command in DRBL client or make sure hda is NOT busy/mounted!):
ocs-sr -g auto restoreparts IMAGE2 "hda1 hda2"

To save disk(s)/partitition(s) as an image or restore an image to disk(s)/partitition(s) interactively, use:
ocs-sr -x

ocs_postrun, ocs_postrun1, ocs_postrun2, etc… – Intended for running a command after an operation of saving/restoring an image or cloning disk/partition has been completed.

Mount Disk Images (Partitions)

sudo zcat sda2.ext4-ptcl-img.gz.aa | partclone.restore --restore_raw_file -C -s - -o pjakey.img
sudo mkdir /mnt/partclone
sudo mount -o loop pjakey.img /mnt/partclone -t ext4 -o r

Yellowdog Updater, Modified (YUM)

Blog, SysAdmin TnT
Posted on

Basic Commands

List packages with pending updates
yum list updates
Show all versions of a package installed on the system
yum --showduplicates list dpdk | expand
Install a specific package
yum install dpdk 19.02-2

If the version you wish to install is older, you may need to…

Remove an installed package
yum remove dpdk
To lock packages
yum install yum-versionlock
yum versionlock dpdk

Stock Repos

/etc/yum.repos.d/

Method 1: Boot to live CD and copy repos from there to your system

Method 2: Create a “base” set of repo files by hand.

Create the following file:

vi /etc/yum.repos.d/Centos-Base.repo

Put the following info inside of the file:

[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=
$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
priority=1

Save your changes, run yum clean all and yum makecache

Method 3: Using rpm, manually reinstall centos-release-rpm

The first step is to see if the centos-release package is installed, by typing rpm -q centos-release

[root@smartproxy-iad ~]# rpm -q centos-release
centos-release-7-6.1810.2.el7.centos.x86_64

If it’s not installed, then the --replacepkgs --replacefiles part of the command can be omitted.

yum remove centos-release
# Option A
rpm -ivh --replacepkgs --replacefiles centos-release*.rpm
# Option B
rpm -Uvh centos-release.*.rpm

Debian Networking

Blog, Linux Admin, NetEng TnT, SysAdmin TnT
Posted on

Using a Bridge

$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
  address 172.16.16.111
  netmask 255.255.255.0
  gateway 172.16.16.1
  dns-domain mydomain.local
  dns-nameservers 8.8.4.4 8.8.8.8
  bridge_ports eno1
  bridge_stp off
  bridge_fd 0

NFS

Blog, SysAdmin TnT
Posted on

Setup

Install
$ apt-get update
$ apt-get install nfs-common nfs-kernel-server
Configure exports
$ vi /etc/exports

Example: /data 172.16.16.209/255.255.255.0(rw,insecure,root_squash)

/mnt/backupvm is your directory
172.16.16.209/255.255.255.0 is your network
rw are the permissions
secure – This option requires that requests originate on an Internet port less than IPPORT_RESERVED (1024). This option is on by default. To turn it off, specify insecure

root_squash – Map requests from uid/gid 0 to the anonymous uid/gid. Note that this does not apply to any other uids or gids that might be equally sensitive, such as user bin or group staff.

no_root_squash – Turn off root squashing. This option is mainly useful for diskless clients.

Start services
$ systemctl enable nfs-common
$ systemctl start nfs-common
$ systemctl enable nfs-kernel-server
$ systemctl start nfs-kernel-server

Command Line

List the NFS shares exported by the server: showmount -a <nfsserver>

Components

Exports

man exports – subtree_check root_squash

Located at /etc/exports

exportfs -a – Update list of shared folders

insecure – Helpful when connecting from macOS (resolves Operation Not Permitted message)

Example: /data/documents 172.16.16.0/24(rw,insecure,root_squash)

nfs-common
nfs-kernel-server

fstab Entry

 

Mounting

From macOS client

Finder > Go > Connect to Server > nfs://<nfsservername>:/<sharename>

Terminal > sudo mount_nfs -o resvport <nfsservername>:/<sharename> <mountpoint>

Troubleshooting

Linux NFS-HOWTO Troubleshooting NFS

Error Messages Seen and Resolved (During SetUp of my NFS Server)

RPC: Program Not Registered

Taking Inventory of Your Hardware

Blog, SysAdmin TnT
Posted on

CPUs

How many does your system have?
Linux
# lscpu | egrep 'CPU\(s\)'
CPU(s): 32
On-line CPU(s) list: 0-31
NUMA node0 CPU(s): 0-7,16-23
NUMA node1 CPU(s): 8-15,24-31

Extending KVM Disk

Blog, SysAdmin TnT
Posted on

I’ve been a fan of virtual machines from early on, like KVM for instance. A greatly respected ex-coworker recommended Proxmox, which was the front end to a Type [1.5] hypervisor. He nonchalantly try to sell a few us on it from time to time and I think I failed to make note of how awesome it really was.

I use it daily and am sometimes asked to extend a user’s VM host’s disk. Usually, it to support development (storing test data) and not because of laziness or ignorance (needing more space for crap because they’re not sure how to keep it in check).

The process for increasing the size of your KVM disk is as follows.

Prerequisites

  • KVM host
  • KVM guest
  • root access via command line
  • VM ID
  • Know your LVM
    • Group name; use vgdisplay
    • Logical volume name: use lvdisplay
    • Is it ext4 or xfs

Procedure

  1. Shutdown your guest
    shutdown -hP now
  2. Make a copy of your VM’s current disk
    cp /var/lib/images/vm-117.qcow2 /var/lib/images/vm-117.qcow2.orig
  3. Increase the size of the disk image (CLI or GUI)
    qm resize <vmid> <disk> +<size>G
    qemu-img resize ...
    Example:
    qm resize 117 /dev/sda +100%FREE
  4. Convert VMDK to raw, if applicable
    qemu-img convert -O raw vm-117.qcow2 vm-117.raw
  5. Start guest
  6. On guest, enlarge the partition using a partition tool (Proxmox already has parted installed, you can use that if you prefer). If your disk already has a swap (id 82) partition, you will want to delete it and add it back to the end of the disk.  See this guide or the last step in this guide.
    fdisk /dev/sda
    Create a new partition to add to the new volume group
    type: primary
    start:<default value>
    end:<default value>
    type:8e
    write changes: w
  7. Reboot or rescan partitions on guest
  8. Initialize a disk or partition for use by LVM
    pvcreate /dev/sdX#
  9. Verify your physical volumes
    pvdisplay
  10. Add the partition to the partition map.  Confirm your partition is present using
    cat /proc/partitions
  11. If your partition isn’t there, reboot or rescan
    partx /dev/sdX#
  12. Get the name of your volume group
    vgdisplay
  13. Extend your volume group by adding your new partition to it
    vgextend <vg_name> <partition>
  14. Get the name of your logical volume root
    lvscan
  15. Extend the logical volume (-r means you don’t have to run resize2fs separately)
    lvextend -r -l +100%FREE /dev/<vg_name>/<lv_root>
    Example:
    lvextend -l +100%FREE /dev/mapper/centos-root
  16. Resize the filesystem (if you did not do a -r above).
    Determine your filesystem:
    mount | column -t
    for ext4, use resize2fs /dev/mapper/centos-root (or value from mount command above)
    for xfs, use xfs_growfs /dev/mapper/centos-root (or value from mount command above)
  17. Confirm larger size

 

The Boot Process

Blog, SysAdmin TnT
Posted on

I boot systems multiple times a day. Sometimes, I have to pay close attention to how they are booting or why they are not booting up. Being familiar with the boot process is rather important. Here it is in a nutshell.

  • Power supply sends signal to the motherboard and other system components
  • Processor is hard-coded to know where to find BIOS (in system BIOS ROM – normally at location FFFF0h, right at the end of system memory)
  • BIOS runs tests, called POST (Power On Self Test).
  • After the test, results of POST are compared with data in CMOS chip. Typically one beep means “all is well” while other sequences of beeps indicate “all is not well”. The sequences of beeps are used for troubleshooting and are typically specific to the motherboard in question. Check with the manufacturer for POST beep codes.
  • Next, the video card’s BIOS is loaded and run. Other BIOSes are found and executed. The order depends on timing, mostly?
  • BIOS displays its startup screen and performs additional tests such as memory count-up tests and an inventory of hardware. Settings for memory and hard drives are set at this time. Messages are typically displayed to the screen for logical devices it finds as well as Plug and Play devices.
  • BIOS looks for boot devices according to boot order and a master boot record (at cylinder 0, head 0, sector 1)
  • The master boot record points to the boot sector where the kernel can be found
  • Entry point for kernel is located and the kernel is loaded into RAM. BIOS hands off to the kernel at this point.
  • Operating system continues to load according to instructions and parameters passed to kernel

Thanks to flint.cs.yale.edu for their wonderful write-up with links to more in-depth articles.


Modifying initramfs

Blog, SysAdmin TnT
Posted on

BACKGROUND

Booting Linux involves various components at several different stages.  The following summary was taken from here.

First is BIOS.  After loading date, time, and important peripherals from CMOS, then the storage devices are probed.  When the first hard disk and its geometry are recognized, the system control passes from BIOS to the boot loader.

The Master Boot Record (first 512 bytes) on the disk is where you will find the boot loader.  The commands executed by the boot loader determine the rest of the boot process.  Control is passed to the actual operating system, like the Linux kernel.

The kernel and an initial RAM-based file system is loaded into memory.  The initramfs contains a small executable called init that handles the mounting of the real root file system.

The init program handles mounting the proper root file system.  If it is successful, initramfs is cleaned and the init program on the root file system is executed.

The init process handles the actual booting of the system through different levels.  I don’t want to summarize the following information because every bit of it is important for the purpose of the document (again, taken directly from here):

initramfs

 

initramfs is a small cpio archive that the kernel can load to a RAM disk. It provides a minimal Linux environment that enables the execution of programs before the actual root file system is mounted. This minimal Linux environment is loaded into memory by BIOS routines and does not have specific hardware requirements other than sufficient memory. initramfs must always provide an executable named init that should execute the actual init program on the root file system for the boot process to proceed.

 

Before the actual root file system can be mounted and the actual operating system can be started, the kernel needs the corresponding drivers to access the device on which the root file system is located. These drivers may include special drivers for certain kinds of hard drives or even network drivers to access a network file system. The needed modules for the root file system may be loaded by init on initramfs. After the modules are loaded, udev provides the initramfs with the needed devices. initramfs is available during the entire boot process. This makes it possible to handle all device events generated during boot.

Problem

So, my issue was trying to get a system image onto a system using Clonezilla.  This is fairly straight-forward as long as you don’t have any devices present that the Clonezilla’s Linux kernel isn’t prepared to recognize.  Otherwise, during boot, you will be dropped to a shell in the initramfs itself.  The system I was attempting to restore had a QLogic card which Clonezilla wasn’t prepared to handle.

Solution

Add the required firmware to the initramfs so it’s available for the module to load.

Process

Since Initramfs is essentially a concatenation of gzipped cpio archives which are extracted into a ramdisk and used as an early userspace by the Linux kernel, the only thing we need to do is to concatenate another file onto the archive.

The Q&A presented here opened my eyes to how easy this can be done.  This quote specifically cleared up any confusion I had:

Debian Installer’s initrd.gz is in fact a single gzipped cpio archive containing all the files the installer needs at boot time. By simply appending another gzipped cpio archive – containing the firmware files we are missing – we get the show on the road!

This means, all I had to do was:

    1. Extract the Clonezilla ISO 
      mount -t iso9660 -o loop ./clonezilla-live.iso myiso
# copy contents from read-only directory to a writable one
cp -a myiso myiso_write
cd myiso_write
    2. Concatenate my firmware onto the initramfs contained within the ISO 
      echo '/lib/firmware/ql2500_fw.bin' | cpio -H newc -o | gzip >> live/initrd.img
    3. Repackage the ISO 
      genisoimage -A 'My Clonezilla Installer' -f -r -hide-rr-moved -hide-joliet-trans-tbl \
     -J -l -allow-limited-size -b syslinux/isolinux.bin -c syslinux/boot.cat -no-emul-boot \
     -boot-load-size 4 -boot-info-table -eltorito-alt-boot -efi-boot boot/grub/efiboot.img \
     -no-emul-boot ./ > ../clonezilla-live-PJA.iso
    4. Boot from ISO
    5. Restore system

I had the easy task of just adding another file to the cpio archive.  If you need to modify existing files or structure or just unpack it in full for some other reason, you’ll need to consult this link – Modifying the Clonezilla initrd.

The file initrd.img from the Clonezilla live is not a ext2 file system, it’s cpio format. Therefore you can not mount it, instead you have to do something like this:

The initrd.img maybe in gzip format, or in xz format. You can use command “file initrd.img” to know the foramt.

(1) mkdir ~/tmp/initrd; cd ~/tmp/initrd

(2) for gzip format, run: zcat initrd.img | cpio -idm
    for xz format, run: xzcat initrd.img | cpio -idm

Then you can edit the files in ~/tmp/initrd. After that, you can use the following command to pack it as new initrd.img:

(3) cd ~/tmp/initrd

(4) For gzip format, run: find . | cpio --quiet -o -H newc | gzip -9 > ../initrd.img
    For xz format, run: find . | cpio --quiet -o -H newc | xz -c -9 --check=crc32 >  ../initrd.img

Then the new one is in ~/tmp/initrd.img

Setting Up a New Service in SystemD on CentOS 7

SysAdmin TnT
Posted on

I have a custom service that I need to start on reboot on a CentOS system running systemd.  Following the guidelines posted here, I was able to do it myself in my environment.

  1. Create the following file: /etc/systemd/system/tcp-server.service
  2. Put the following contents in the file:
  3. [Unit]
    Description=tcp-server for hhreplay Service
    After=network.target
    [Service]
    Type=simple
    User=root
    Environment="OPTIONS=--logfile /tmp/tcp-server.log"
    ExecStart=/home/jenkins/tcp-server.py $OPTIONS
    Restart=on-abort[Install]
    WantedBy=multi-user.target
  4. Chown your log file: chown root:root /tmp/tcp-server.log
  5. Reload systemd: sudo systemctl daemon-reload
  6. Start your service to ensure it’s a functional service file: systemctl start tcp-server
  7. Check status: systemctl status tcp-server
  8. If it looks good, then set it to start at boot: systemctl enable tcp-server --now

MAN PAGE

In-depth information can be found in the systemd.service man page.  Read up on options such as Type, RemainAfterExit, ExecStart, WantedBy, ExecStop , Environment, etc.

Special Considerations

  1. In some distributions you are required to have a ExecStart in your service file.  This can be set to /bin/true if you don’t explicitly need something there.
  2. RemainAfterExit=true can be added to trick systemd into believing that the service is running.  This is helpful when you don’t have a ExecStart and still need a ExecStop to run after the service, or another service, shutsdown.
  3. DefaultDependencies=no means ignore all dependencies and run “first” on start and “last” on stop.  If you have Before or After clauses, they will still be honored.

Additional References

0pointer.net – SystemD for Administrators Part XXI
freedesktop.org – systemd System and Service Manager

Copyright © 2024 Pete Akey. All rights reserved.