DISKLESS(8) | System Manager's Manual | DISKLESS(8)

# NAME

`diskless` —
booting a system over the network

# DESCRIPTION

The ability to boot a system over the network is useful for two kinds of systems:

diskless
  a system with no attached mass storage media to boot or run from (e.g. a network computer).
dataless
  a system with a hard drive that only contains system and application software, and user data is mounted over the network from a central server.

It can also be done as a temporary measure while repairing or re-installing file systems on a local disk. This capability is necessarily platform dependent because of its dependence on system firmware support; not all platforms supported by NetBSD are capable of being network booted.

The protocols used to obtain a network address (e.g. an IP host address), include, but are not limited to:

RARP
  Reverse Address Resolution Protocol (ARP)
DHCP
  Dynamic Host Configuration Protocol
BOOTP
  Bootstrap Protocol

This information can also be derived from non-volatile RAM or by a transform of a network interface (e.g. Ethernet) MAC address.

The protocols used to load a NetBSD kernel over a network include, but are not limited to:

TFTP
  Trivial File Transfer Protocol
NFS
  Sun Network File System
RMP
  HP Remote Maintenance Protocol
MOP
  DEC Maintenance Operations Protocol

Derivation of the filename of the secondary bootstrap program can be done by a transform of a network interface MAC address (or other protocol address), or provided by a server as with BOOTP, and DHCP. How this is done is platform dependent; see boot(8).

The NetBSD kernel doesn't care how it gets loaded and started. The protocols used to boot NetBSD can be completely different than the ones that NetBSD uses operationally, i.e. you can netboot the system using HP RMP and the NetBSD kernel can use IP to communicate after bootstrap.

There is no standard way to pass all the required information from a boot loader to an operating system kernel, so the NetBSD kernel usually has to recapitulate the same (or similar) protocol exchanges over the network to obtain a network address, determine which servers to use, and so on. NetBSD supports obtaining this information from RARP, BOOTP, DHCP, and Sun RPC “bootparams”. See options(4) for a list of methods that can be compiled into a NetBSD kernel.

NetBSD only supports the Sun Network File System (NFS) for mounting its root file system over a network. NetBSD can use any local mass storage device for which it has a driver, after bootstrap, even if that device is not supported by the system's firmware for booting.

N.B. DHCP is essentially a series of extensions to BOOTP; the NetBSD dhcpd(8) is capable of responding to both kinds of protocol requests.

In the majority of configurations, network boot servers and clients are attached to the same LAN so that broadcast queries from the clients can be heard by the servers. Unless specially configured, routers block broadcasts from propagating from LAN to LAN; some routers can be configured to “forward” broadcast BOOTP packets to another LAN attached to that router, which permits a server on that remote LAN to respond to the client's broadcast query.

# OPERATION

When booting a system over the network, there are three phases of interaction between client and server:

1. The system firmware (or stage-1 bootstrap) loads a boot program.
2. The boot program loads a NetBSD kernel.
3. The NetBSD kernel performs an NFS mount of the root file system.

Each of these phases are described in further detail below.

## 1. loading a boot program

In phase 1, the system firmware loads a boot program. Firmware designs vary widely, so this phase is inherently machine-specific. Some examples:

DEC Alpha systems use BOOTP to determine the client's IP address and then use TFTP load a secondary bootstrap program from the server and filename specified in the BOOTP reply. DEC Alpha systems can also use MOP to load a program to run the system.

Sun systems use RARP to determine the client's IP address, transform that address to a hexadecimal string to form the filename of the secondary boot program, and then use TFTP to download the boot program from the server that sent the RARP reply.

HP 300-series systems use the HP RMP to download a boot program.

Typical personal computers may load a network boot program either from diskette or from a PROM on a Network Interface Card (NIC). Some BIOSes support booting from a network interface.

## 2. loading a kernel

In phase 2, the secondary boot program loads a kernel. Operation in this phase depends on the design of the boot program (the design described here is the one used by Sun and NetBSD/hp300). The boot program:

1. gets the client IP address using RARP.
2. gets the client name and server IP address by broadcasting an RPC / BOOTPARAMS / WHOAMI request with the client IP address.
3. gets the server path for this client's root using an RPC / BOOTPARAMS / GETFILE request with the client name.
4. gets the root file handle by calling mountd(8) with the server path for the client root file system.
5. gets the kernel file handle by calling NFS `lookup`() on the root file handle.
6. loads the kernel using NFS read calls on the kernel file handle.
7. transfers control to the kernel entry point.

A BOOTP and/or DHCP secondary bootstrap program will do the following:

1. query for the client's bootstrap parameters. The response must include the client's IP address, and a TFTP server to load the NetBSD kernel from.
2. loads the NetBSD kernel from the TFTP server.
3. transfers control to the kernel entry point.

## 3. NFS mounting the root file system

In phase 3, the kernel performs an NFS mount of the root file system. The kernel repeats much of the work done by the boot program because there is no standard way for the boot program to pass the information it gathered on to the kernel.

In general, the GENERIC kernel config(1) file for any particular architecture will specify compile-time options to use the same protocol used by the secondary boot program for that architecture. A NetBSD kernel can be compiled to use any of BOOTP, DHCP, or Sun RPC BOOTPARAMS; see options(4).

The procedure typically used by the kernel is as follows:

1. The kernel finds a boot server using the same procedures as described above to determine the client's IP address, an NFS server, etc.
2. The kernel gets the NFS file handle for root using the same procedure as described above.
3. The kernel calls the NFS `getattr`() function to get the last-modified time of the root directory, and uses it to check the system clock.

# SERVER CONFIGURATION

Before a client can bootstrap over the network, its server must be configured. Each daemon that implements these protocols must be set up so that it can answer queries from the clients. Some of these daemons are invoked as packets come in, by inetd(8), and some must run independently, started from /etc/rc; see rc.conf(5).

Protocol | Program | Startup
RARP | rarpd | rc.conf(5)
DHCP | dhcpd | rc.conf(5)
BOOTP | bootpd | inetd.conf(5)
TFTP | tftpd | inetd.conf(5)
Sun RPC | rpcbind | rc.conf(5)
Sun RPC | rpc.bootparamd | rc.conf(5)
Sun NFS | mountd | rc.conf(5)
Sun NFS | nfsiod | rc.conf(5)
HP RMP | rbootd | rc.conf(5)

N.B. DHCP is essentially a series of extensions to BOOTP; the NetBSD dhcpd(8) is capable of responding to both kinds of protocol requests. Since they both bind to the same UDP port, only one may be run on a given server.

In the following examples, the client's hostname is myclient; the server is myserver, and the addresses are all fictional. In these examples the hostnames may be Fully Qualified Domain Names (FQDN, e.g. “myclient.mydomain.com”) provided that they are used consistently.

## RARP

For clients that use RARP to obtain their IP address, an entry must be added for each client to /etc/ethers with the client's Ethernet MAC address and Internet hostname:

    8:0:20:7:c5:c7          myclient

This will be used by rarpd(8) to reply to queries from the clients. There must be one entry per client system.

A client system's Ethernet MAC address is often printed on the system case, or on a chip on its motherboard, or on the NIC. If not, “sniffing” the network with tcpdump(8) when the client is powered-on should reveal its Ethernet MAC address.

Each client system that uses RARP must have its own, unique IP address assigned to it. Assign an IP address for myclient in your /etc/hosts file, or in the master file for your DNS zone. For /etc/hosts the entry should look like:

    192.197.96.12           myclient

## DHCP/BOOTP

The NetBSD DHCP server dhcpd(8) was developed by the Internet Software Consortium (ISC).

DHCP can provide a wide range of information to a requesting client; the key data for bootstrapping a diskless client are:

1. an IP address
2. a subnet mask
3. a TFTP server address for loading the secondary bootstrap and the NetBSD kernel
4. a filename of the secondary bootstrap
5. an NFS server address for the client's file system
6. the client's root file system path, to be NFS mounted.

An example for /etc/dhcpd.conf

    host myclient {
    	hardware ethernet 8:0:20:7:c5:c7;
    	fixed-address myclient;		# client's assigned IP address
    	filename "myclient.netboot";	# secondary bootstrap
    	next-server myserver;		# TFTP server for secondary bootstrap
    	option swap-server myserver;	# NFS server for root filesystem
    	option root-path "/export/myclient/root";
    }

That host declaration goes inside a subnet declaration, which gives parameters for all hosts on the subnet that will be using DHCP, such as the “routers” (the default route), “subnet-mask”, “broadcast-address”, “domain-name-servers”, etc. See dhcpd.conf(5) for details. In that example, myclient has an assigned IP address.

The DHCP parameters required for network bootstrapping a system will vary from platform to platform, as dictated by each system's firmware. In particular, because the DHCP is extensible, some hardware vendors have specified DHCP options to return information to requesting clients that are specific to that platform. Please see your platform's boot(8) for details.

## TFTP

If booting a Sun system, or other system that expects to use TFTP, ensure that inetd(8) is configured to run tftpd(8). The tftpd(8) server should be set up to serve the directory /tftpboot.

If booting a SPARC system, install a copy of the appropriate diskless secondary boot loader (such as /usr/mdec/boot or ofwboot.net) in the /tftpboot directory. Make a link such that the boot program is accessible by a filename composed of the client's IP address in hexadecimal, a dot, and the architecture name (all upper case). For example:

    # cd /tftpboot
    # ln -s boot C0C5600C.SUN4

For a Sun-3 or UltraSPARC system, the filename would be just C0C5600C (these systems' firmware does not append the architecture name). The name used is architecture dependent, it simply has to match what the booting client's system firmware wishes to it to be.

If the client's system firmware fails to fetch the expected file, tcpdump(8) can be used to discover which filename the client is being requested. Also, examination of tftpd(8) log entries (typically in /var/log/messages) should show whether the server is hearing the client system, and what filename the client is asking for.

## HP RMP

If booting an HP 300-series system, ensure that /etc/rbootd.conf is configured properly to transfer the boot program to the client. An entry might look like this:

    08:00:09:01:23:E6	SYS_UBOOT	# myclient

The secondary bootstrap program for an HP 300-series system SYS_UBOOT (which may be called uboot.lif before installation) must be installed in the directory /usr/mdec/rbootd.

See the rbootd(8) manual page for more information.

## Sun RPC BOOTPARAMS

Add myclient to the bootparams database in /etc/bootparams:

    myclient  root=myserver:/export/myclient/root \
              swap=myserver:/export/myclient/root/swap \
              dump=myserver:/export/myclient/root/swap

and ensure that rpc.bootparamd(8) and rpcbind(8) are running. Both myclient and myserver must have IP addresses in the DNS or /etc/hosts.

## Diskless Client File Systems

Build the swap file for myclient on the NFS server:

    # cd /export/myclient/root
    # dd if=/dev/zero of=swap bs=16k count=1024

This creates a 16 megabyte swap file.

Populate myclient's root file system on the NFS server. How this is done depends on the client architecture and the version of the NetBSD distribution. It can be as simple as copying and modifying the server's root file system, or unpack a complete NetBSD binary distribution for the appropriate platform.

If the NFS server is going to support multiple different architectures (e.g. Alpha, PowerPC, SPARC, MIPS), then it is important to think carefully about how to lay out the NFS server's exported file systems, to share what can be shared (e.g. text files, configuration files, user home directories), and separate that which is distinct to each architecture (e.g. binary executables, libraries).

## NFS

Export the client-populated file systems on the NFS server in /etc/exports:

    /usr -ro myclient
    # for SunOS:
    # /export/myclient -rw=myclient,root=myclient
    # for NetBSD:
    /export/myclient -maproot=root -alldirs myclient

If the server and client are of the same architecture, then the client can share the server's /usr file system (as is done above). If not, you must build a properly fleshed out /usr partition for the client in some other part of the server's file system, to serve to the client.

If your server is a SPARC, and your client a Sun-3, you might create and fill /export/usr.sun3 and then use the following /etc/exports lines:

    /export/usr.sun3 -ro myclient
    /export/myclient -rw=myclient,root=myclient

Of course, in either case you will have to have an NFS server running on the server side.

# CLIENT CONFIGURATION

Copy and customize at least the following files in /export/myclient/root:

    # cd /export/myclient/root/etc
    # vi fstab
    # cp /etc/hosts hosts
    # echo 'hostname="myclient"' >> rc.conf
    # echo "inet 192.197.96.12" > ifconfig.le0

Note that "le0" above should be replaced with the name of the network interface that the client will use for booting; the network interface name is device dependent in NetBSD.

Correct the critical mount points and the swap file in the client's /etc/fstab (which will be /export/myclient/root/etc/fstab) i.e.

    myserver:/export/myclient/root  /    nfs  rw 0 0
    myserver:/usr                   /usr nfs  rw 0 0
    /swap                           none swap sw 0 0

Note, you must specify the swap file in /etc/fstab or it will not be used! See swapctl(8).

It may be useful to set “flushroutes=NO” in /etc/rc.conf to avoid the default route supplied by the boot setup disappearing mid-boot.

# FILES

/etc/hosts
  table of associated IP addresses and IP host names; see hosts(5)
/etc/ethers
  table of associated Ethernet MAC addresses and IP host names used by rarpd(8); see ethers(5)
/etc/bootparams
  client root pathname and swap pathname; see bootparams(5)
/etc/exports
  exported NFS mount points; see exports(5)
/etc/rbootd.conf
  configuration file for HP RMP; see rbootd(8)
/usr/mdec/rbootd
  location of boot programs offered by rbootd(8)
/tftpboot
  location of boot programs offered by tftpd(8)

# SEE ALSO

bootparams(5), dhcpd.conf(5), ethers(5), exports(5), fstab(5), hosts(5), networks(5), boot(8), dhcpd(8), mopd(8), mountd(8), nfsd(8), rarpd(8), rbootd(8), reboot(8), rpc.bootparamd(8), tftpd(8)

Reverse Address Resolution Protocol, RFC, 903, June 1984.

Bootstrap Loading using TFTP, RFC, 906, June 1984.

Bootstrap Protocol, RFC, 951, September 1985.

The TFTP Protocol (Revision 2), RFC, 1350, July 1992.

Dynamic Host Configuration Protocol, RFC, 2131, March 1997.

DHCP Options and BOOTP Vendor Extensions, RFC, 2132, March 1997.

RFC Editor

October 23, 2018 | NetBSD 9.2