Mayhem for Code can help you find and fix vulnerabilities in your code.
Mayhem for Code can run on the cloud, but, when you're testing critical/sensitive/confidential code, you can make it more difficult for malicious actors to access Mayhem's findings: you can deploy Mayhem on-prem.
In earlier days, to install Mayhem for Code on-prem, we co-ordinated with the IT staff of our customers to ensure that they have the hardware, OS, and networking set up for Mayhem.
These days, to get Mayhem for Code on your premises:
In our experience, RedHat Enterprise Linux - and its derivatives like Oracle Linux - are quite welcome in the IT environment of our customers. So, we looked into bundling Oracle Linux (OL) as we distribute Mayhem. (Okay, we initially started with CentOS, but have since moved on to Oracle).
This "OS family" can be installed using DVD ISOs that include the Anaconda-installer.
Anaconda-installer can be automated using kickstart.
We can kickstart the OS and networking set-up.
We can also (after adding Mayhem bits into the ISO image) use kickstart to automatically install Mayhem for Code.
The resulting ISO image was 15 GB. 10 GB comes from the "full ISO" to install Oracle Linux (8.5 at the time of this writing), which we use as a base. This "full ISO" carries enough packages to install LibreOffice, video conferencing software, etc. While these are useful software, we don't need to install these on machines set apart for Mayhem.
Can we trim the size of the ISO image?
In the "full ISO" for OL 8.5, we see that the AppStream
directory takes up 8.4 GB. If these were "normal" files & directories, we would "normally" try rm [-r]
. Can we do that with ISO images?
We can, with xorriso
("X/Open Rock Ridge ISO" tool). We can tell it to -rm $PATH ...
and/or -rm_r $PATH ...
.
At the time of this writing, we can install xorriso
using yum
in OL and apt-get
in Debian/Ubuntu without adding additional package archives/repositories. We can also install it in macOS with Home-brew
.
We can also use xorriso
in a Docker container (using, for example, oraclelinux
or ubuntu
images), but we noticed that disk I/O performance is significantly slower than using xorriso
outside the container.
A basic xorriso
invocation would be:
xorriso \
-indev OracleLinux-R8-U5-x86_64-dvd.iso \
-outdev out.iso \
-boot_image any replay
xorriso
to use OracleLinux-R8-U5-x86_64-dvd.iso
as "input drive" and sets "output drive" to out.iso
.-boot_image any replay
tells xorriso
to also copy the bits that make the "full ISO" boot-able.If you're following along, at this point, we should have an out.iso
that is basically equal to the input.
Before telling xorriso
to start removing files/folders, we first need to figure out what we need.
We can do this by manually going through the installation process. We should, however, hold off from setting up the network connection: going through an "off-line" installation will force Anaconda-installer to only use RPM files from the DVD (instead of fetching new/updated ones from the internet).
After a successful installation of the OS, Anaconda-installer creates a /root/anaconda-ks.cfg
file containing kickstart configuration. We can use this file to automate what we just manually installed. More about this file in a bit.
We'll also need the list of RPM packages that we have just installed. We can get this list from yum list --installed
. Here's a sample output:
Installed Packages
. . .
kernel.x86_64 4.18.0-348.el8 @anaconda
. . .
rsyslog.x86_64 8.2102.0-5.el8 @AppStream
. . .
rsyslog
package from the @AppStream
repository in the ISO image at /AppStream/Packages/rsyslog-8.2102.0-5.el8.x86_64.rpm
.@anaconda
repository in the above output points to /BaseOS/Packages/
in the ISO image, so the kernel
package can be found at /BaseOS/Packages/kernel-core-4.18.0-348.el8.x86_64.rpm
.If, after installing the OS, we need to install additional packages from the DVD, we can have yum
install packages from the DVD, instead of fetching new/updated packages from the internet:
# assuming we:
mount /dev/cdrom /mnt
# we can invoke `yum` like so:
yum \
--disablerepo=* \
--repofrompath BaseOS,/mnt/BaseOS \
--repofrompath AppStream,/mnt/AppStream \
install $PACKAGE ...
Now we know what we need. Following our previous example:
/root/anaconda-ks.cfg
/mnt/BaseOS/Packages/kernel-core-4.18.0-348.el8.x86_64.rpm
/mnt/AppStream/Packages/rsyslog-8.2102.0-5.el8.x86_64.rpm
We can copy the RPM files from the DVD to separate folders for BaseOS and AppStream, for example local_BaseOS_Packages
and local_AppStream_Packages
.
We can then proceed with the fun stuff: removing files/folders.
It first appears that we don't need /BaseOS/repodata/
and /AppStream/repodata/
from the ISO image. But when we tried removing them, it broke the "Software Selection" section of Anaconda-installer. 😅
So, we settled to only remove /BaseOS/Packages/
and /AppStream/Packages/
from the ISO image, and add only the necessary RPM files from our local folders back in to the ISO image:
rm -f out.iso \
&& xorriso \
-indev OracleLinux-R8-U5-x86_64-dvd.iso \
-outdev out.iso \
-boot_image any replay \
-rm_r /BaseOS/Packages /AppStream/Packages -- \
-map local_BaseOS_Packages /BaseOS/Packages \
-map local_AppStream_Packages /AppStream/Packages
Some notes about the snippet above:
rm -f out.iso
to remove any existing out.iso
, or else xorriso
will "append" to the ISO image, instead of creating a new ISO image.-rm_r
takes a list of folders. We tell xorriso
that we're done with the list with --
, before giving additional command (-map
in the snippet above).-map
to copy local files/folders into the ISO image.At this point, we should have a much smaller ISO image, but we have yet to automate the installation. We have /root/anaconda-ks.cfg
. We can edit this file to further adjust how we want to automate Anaconda.
We then need to:
The "full ISO" can be used to boot (older) BIOS and (newer) EFI systems. We'll need to edit the boot menu for both. Assuming we add the kickstart config file into the ISO image at /anaconda-ks.cfg
, we'll need append inst.ks=hd:LABEL=OL-8-5-0-BaseOS-x86_64:/anaconda-ks.cfg
to the "kernel command line" (which the Linux kernel will eventually pass to Anaconda-installer).
The edited BIOS boot menu at /isolinux/isolinux.cfg
should end up looking like:
. . .
label linux
menu label ^Install Oracle Linux 8.5.0
kernel vmlinuz
append initrd=initrd.img inst.stage2=hd:LABEL=OL-8-5-0-BaseOS-x86_64 quiet inst.ks=hd:LABEL=OL-8-5-0-BaseOS-x86_64:/anaconda-ks.cfg
. . .
The edited EFI boot menu at /EFI/BOOT/grub.cfg
should end up looking like:
. . .
menuentry 'Install Oracle Linux 8.5.0' . . . {
linuxefi /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=OL-8-5-0-BaseOS-x86_64 quiet inst.ks=hd:LABEL=OL-8-5-0-BaseOS-x86_64:/anaconda-ks.cfg
. . .
Our xorriso
invocation is now:
rm -f out.iso \
&& xorriso \
-indev OracleLinux-R8-U5-x86_64-dvd.iso \
-outdev out.iso \
-boot_image any replay \
-rm_r /BaseOS/Packages /AppStream/Packages -- \
-map local_BaseOS_Packages /BaseOS/Packages \
-map local_AppStream_Packages /AppStream/Packages \
-map /root/anaconda-ks.cfg /anaconda-ks.cfg \
-update edited_isolinux.cfg /isolinux/isolinux.cfg \
-update edited_grub.cfg /EFI/BOOT/grub.cfg \
-volid MAYHEMDRIVE
-update edited_isolinux.cfg /isolinux/isolinux.cfg
will replace/overwrite /isolinux/isolinux.cfg
in the ISO image with the edited_isolinux.cfg
we supplied. We similarly update grub.cfg
.-volid
) of the ISO image from OL-8-5-0-BaseOS-x86_64
MAYHEMDRIVE
. And, yes, we'll have to update the BIOS and EFI boot menu *.cfg
files accordingly.We use xorriso
to do a few more things for us:
createrepo
to create additional yum
repositories and had xorriso -map
it to the ISO image.Some readers might notice that some of what we've done so far is quite similar to mkksiso
. And that's partly true: mkksiso
uses xorriso
under the hood (in combination with other tools) to:
But mkksiso
doesn't trim the ISO image. Also, mkksiso
(part of lorax
) is not as easy to install as xorriso
in Debian/Ubuntu or macOS.
There are still other ways to create Anaconda-kickstart bootable-DVDs. Notably, Fedora (on which RHEL is based) uses pungi
.
We would understand if some readers consider what we're doing with xorriso
as a "hack". But it works for us. We started with a 15 GB ISO image. We were able to trim it down to ~4 GB for Mayhem for Code 1.17. So, these days, we've got an ISO image that can automatically install and configure OS and Mayhem, so our customers can start finding and fixing vulnerabilities on-prem, making it that more difficult for malicious actors.
By submitting this form, you agree to our Terms of Use and acknowledge our Privacy Statement.