Good practices for building packages from source

Many people install their software packages from vendor-supplied methods (RPM, up2date, apt-get, and the like), and this works for the most. But some of us are adamant about installing our internet-facing software from source obtained from the project mirrors, and over time we've developed what we believe are best practices in managing them.

"Use the source, Luke"

The point of this Tech Tip is not to defend the practice of building everthing from source - we presume you've already decided that you want to, and are looking for advice - but we will touch on some of our reasoning;

It insures complete control over all the build options, including the pathnames where things are installed. Using a prebuilt package with options you don't need (say, an LDAP client) adds to bloat.
It insures that cooperating packages are built with compatible options: if Postfix and Courier-IMAP both need to link with MySQL libraries, building them all yourself guarantees that the same compiler was used for all of them, with flags you can see.
You develop a reusable skill that applies anywhere you install the package: apt-get is great for Debian but it won't be too helpful on Solaris or HP-UX.

Overall directory structure

We typicall find a place - any place - on a filesystem with sufficient room for our package building, and /home/source is a common parking place. But because this varies from machine to machine, we always make a symbolic link to it so it's easy to find. We create the tarballs directory at the same time: this is where we park the packages we've fetched from the distribution servers:

# mkdir /home/source

# mkdir /home/source/tarballs

# ln -s home/source /source

We use the excellent wget tool to fetch each package, and from there we extract it to its own area. This example fetches Postfix:

# cd /source/tarballs

# wget ftp://postfix.cloud9.net/official/postfix-2.0.12.tar.gz

# cd ..

# gtar -xzvf tarballs/postfix-2.0.12.tar.gz

# cd postfix-2.0.12

continue building

Saving "./configure" parameters

Just finding a place to park packages is not terribly difficult, but remembering how to build them is another story. Most packages these days use the configure tool, but these can take many parameters that often have to be adjusted as the package is explored.

Our practice has been to create a small shell script in the /source directory that contains the methods required to build the package. For instance, in the case of Postfix above, we had

# cat /source/configure-postfix

make makefiles CCARGS='-DDEF_COMMAND_DIR=\"/usr/local/sbin\" \
        -DDEF_DAEMON_DIR=\"/usr/local/libexec/postfix\" \
        -DDEF_MAILQ_PATH=\"/usr/local/bin/mailq\" \
        -DDEF_NEWALIAS_PATH=\"/usr/local/bin/newaliases\" \
        -DDEF_SAMPLE_DIR=\"/etc/postfix/samples\" \
        -DDEF_SENDMAIL_PATH=\"/usr/local/sbin/sendmail\" \
        -DHAS_MYSQL -I/home/mysql/include/mysql' \
        AUXLIBS='-L/home/mysql/lib/mysql -lmysqlclient'

Here we provided non-default installation directories that are to our liking, plus pointed Postfix at the location of the MySQL client libraries. To use these, we do

# cd /source/postfix-2.0.12

# sh ../configure-postfix

# make
... continue to build/test/install

It's important that the configure-postfix script be kept in the parent of the Postfix source area, for two reasons.

Sometimes it's necessary to simply blow away the whole extracted source and start fresh: it would be a drag to keep creating the configure front-end script over and over each time. When installing a new package for the first time, we usually go through many rounds of trash-everything-and-start-over before we settle on things that work for us.
When the next version of the package comes out, it may be possible to simply use this script with minimal or no changes. But even if tuning is required, important things like "installation paths" will be the same: getting these wrong from build to build could lead to two competing installations of the package (no good ever comes of this).

Though Postfix here did not use the configure program, many do, and we can take one small additional step to make configuration a bit easier. When a new package is installed, go to that directory and populate the configure front-end with the configure --help output:

# cd /source/mysql-4.0.13

# ./configure --help > ../configure-mysql

# vi ../configure-mysql
edit the file to add the actual configure code

# more ../configure-mysql
exec ./configure \
        --prefix=/home/mysql \
        --enable-assembler  \
        --enable-local-infile \
        --without-innodb \
        --with-mysqld-user=mysql \
        --with-unix-socket-path=/var/run/mysql/mysqld.sock

`configure' configures this package to adapt to many kinds of systems.

Usage: ./configure [OPTION]... [VAR=VALUE]...

To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE. See below for descriptions of some of the useful variables.

Defaults for the options are specified in brackets.
.... the rest of configure help output

The help output itself is not executable - it's just a quick reference - and the exec just before the configure script name insures that the shell won't execute any of the documentation. The configure front-end script can also contain additional commentary, such as important pre-installation steps (such as creating specific system users or groups).

These front-end scripts show their further utility when a build need be done on some other system: copy the configure-whatever script to the new system and known-to-work parameters will be an excellent starting point even if machine-specific changes need be made.

Navigate: More Tech Tips