Enhanced PXE booting with iPXE
While doing more reading on PXE as a followup to Setting up a PXE environment for OS installations, I ran into iPXE, which has some interesting features which can simplify PXE booting. The main feature that caught my interest was HTTP support - meaning it can fetch various modules, kernels, etc from a web server, which in general is much more customizable and configurable than stock TFTP servers, and also much faster.
It took a while to figure out where it all fits into a boot stack, I
thought I'd share what I've roughly figured out and gotten to work.
Stock PXE - a review
This diagram shows where we ended previously previously with installing Ubuntu over a network:
The PXE stack would obtain from a DHCP server the IP address of a TFTP
server and the name of a Network Boot Program (NBP) such as pxelinux.0
pxelinux.0 would then fetch a config file, and the config file fetched
a menu module. Depending on what was selected from the menu, a Linux
kernel and initrd could be fetched and booted, and then the Ubuntu
installer presumable in the initrd would fetch additional install
packages from an HTTP server.
Let's see what iPXE can do for us.
iPXE - replacement ROM
I didn't actually try this because it doesn't seem practical on a large scale, but you can on many NICs replace the stock PXE ROM with iPXE, giving you an arrangement like this:
A few things to note here:
-
We can now use HTTP to load our NBPs - it's just a matter of changing the DHCP config to say something like:
filename "http://10.0.0.1/whatever";(thenext-serverclause in the DHCP config doesn't really matter now) -
We can use the same NBPs such as PXELINUX that worked in the stock PXE setup.
-
There's now an interactive console that lets you manually change settings and load modules including Linux kernels and ramdisks directly - so iPXE is also a bootloader itself and you don't necessarily need something like PXELINUX.
-
The same commands you can use interactively can be saved in script files, so essentially it's like having a NBP that's a text file instead of having to be a compiled binary.
As I mentioned, I don't think it's practical to go around flashing various NICs and motherboards, but there's another way to use iPXE....
Booting iPXE with PXE
There's a version of iPXE that can be chainloaded, so that you're using the stock PXE on your machine to bring in the enhanced iPXE stack. Here's the big picture first:
One of the APIs a PXE stack makes available is UNDI
(Universal Network Device Interface), giving a simple device driver
for the NIC that an NBP can use. We can configure DHCP to load
undionly.kpxe as our initial NBP, it will use the UNDI part
of the stock PXE ROM (so it doesn't need to be configured for any
particular NIC), and it will start the PXE cycle again (querying the DHCP
server for an IP address, the name of an NBP, etc...), but with all the
extra features of iPXE available.
We need to use a bit of logic in the DHCP config so that the iPXE stack
isn't also told to load undionly.kpxe (basically causing a loop). This
can be done in ISC dhcpd with an if statement:
if exists user-class and option user-class = "iPXE" {
filename "http://10.0.0.1/pxelinux.0";
}
else {
filename "undionly.kpxe";
}
So that plain PXE is told to (TFTP) load undionly.kpxe and iPXE is told
to load http://10.0.0.1/pxelinux.0
One cool thing about this setup is that we only need to make one file
available through TFTP, undionly.kpxe. Everything else can be served up
by an HTTP server like Nginx. This is great because HTTP servers are
generally more configurable than a stock TFTP server, plus you aren't
limited to serving up static files - some of these requests from iPXE could
be handled by CGI scripts or other webapps for more of a dynamic behavior.
PXELINUX on iPXE
If iPXE is the active PXE stack, then PXELINUX gains the ability to use HTTP urls for module names, for example, an Ubuntu install might point directly into a mounted ISO image shared on the web like:
LABEL ubuntu-11.04-server-amd64-install
MENU LABEL Ubuntu 11.04 Server AMD64 Install
kernel http://10.0.0.1/ubuntu-11.04-server-amd64/install/netboot/ubuntu-installer/amd64/linux
append vga=788 initrd=http://10.0.0.1/ubuntu-11.04-server-amd64/install/netboot/ubuntu-installer/amd64/initrd.gz url=http://10.0.0.1/ubuntu-11.04-server-amd64.txt
Conclusion
So, drop iPXE's undionly.kpxe into your TFTP server,
configure DHCP to give it out as an NBP only if a non-iPXE stack is asking,
and you can use HTTP for everything else.
