Skip to main content

building packages with poudriere

Source

Poudriere is a simple way to create FreeBSD binary packages. https://pkg.freebsd.org does this, and those packages are sufficient for most use cases. Building your own packages has some advantages:

  • Work on ports, and have a quick method for testing them
  • Compiling ports at a different frequency than upstream
  • Compiling custom port options that upstream will not take
  • Compiling restricted ports
  • Compiling custom ports trees

Please keep in mind, the version of FreeBSD you want to run poudriere on must match or be greater than the target systems you are building packages for. This is because the jail binaries that are being executed must be compatible with the kernels ABI. I'm building packages for FreeBSD-10.3-RELEASE and FreeBSD-11.0-RELEASE in this example.

Poudriere installation

If you want a more bleeding edge version of poudriere, with newer features try ports-mgmt/poudriere-devel. Otherwise ports-mgmt/poudriere.

Via ports

cd /usr/ports/ports-mgmt/poudriere[-devel] make install

Via pkg

pkg install poudriere[-devel]

Configuration

poudriere.conf

You should go through /usr/local/etc/poudriere.conf and make sure everything in there matches your use case, and suits you. I changed the following options:

ZPOOL=zroot
PKG_REPO_SIGNING_KEY=/usr/local/poudriere/keys/pkg.key

If you want to do signing, leave the PKG_REPO_SIGNING_KEY parameter in the configuration and hit the next section. Otherwise, skip the next section.

Signing

Signing is easy. There isn't any reason you shouldn't do it!

mkdir -p /usr/local/poudriere/keys
chmod 600 /usr/local/poudriere/keys
openssl genrsa -out /usr/local/poudriere/keys/pkg.key 8192
openssl rsa -in /usr/local/poudriere/keys/pkg.key -pubout > /usr/local/poudriere/keys/pkg.cert

Create distfile directory

This is referenced in the poudriere.conf file. mkdir -p /usr/ports/distfiles

Prepare jails

Create a jail for each version of FreeBSD you want to build packages for:

poudriere jail -c -j FreeBSD:10:amd64 -v 11.0-RELEASE -a amd64
poudriere jail -c -j FreeBSD:11:amd64 -v 10.3-RELEASE -a amd64

Prepare ports

poudriere ports -c -p upstream

Building

I wrote this all into a script:

PORTS_TREES='upstream'
JAILS='103-RELEASE 110-RELEASE'
PYTHON_VERSION='3.6'
for PORTS_TREE in ${PORTS_TREES}; do

        # Update poudriere ports tree
        poudriere ports -u -p ${PORTS_TREE}

        for JAIL in $JAILS; do
                MAKEFILE="/usr/local/etc/poudriere.d/${JAIL}-make.conf"
                # Ensure the jail is up to date
                poudriere jail -u -j ${JAIL}

                # Build the ports
                echo "DEFAULT_VERSIONS+= python=${PYTHON_VERSION}" > ${MAKEFILE}
                poudriere bulk -f /usr/local/poudriere/pkg-list.txt -j ${JAIL} -p ${PORTS_TREE}
                rm -f ${MAKEFILE}
        done

done

Just run the script:

Serving built packages

To serve built packages, I'm going to use nginx.

pkg install -y nginx

Simple nginx configuration. Just points the location blocks at the packages and logs. /usr/local/etc/nginx/nginx.conf

load_module /usr/local/libexec/nginx/ngx_mail_module.so;
load_module /usr/local/libexec/nginx/ngx_stream_module.so;

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location /logs {
            alias   /usr/local/poudriere/data/logs/bulk;
            autoindex on;
        }
        location / {
            root   /usr/local/poudriere/data/packages/;
            index  index.html index.htm;
            autoindex on;
        }
    }
}

Enable it at boot time:

sysrc nginx_enable="YES"

Start it on the running system:

service nginx start

You should be able to see your packages by going to your poudriere server in your web browser. If you want to see the logs, simply apend /logs to the end of your URL.

Configuration on target system

I suggest writing the pkg.conf and placing it in the webroot of your nginx server: /usr/local/poudriere/data/packages/pkg.douglas-enterprises.com.conf

# Douglas Enterprises repo configuration 
pkg.douglas-enterprises.com: {
  url: "pkg+http://pkg.douglas-enterprises.com/${ABI}-upstream",
  mirror_type: "srv",
  signature_type: "pubkey",
  pubkey: "/usr/local/etc/pkg/certs/pkg.douglas-enterprises.com.cert",
  enabled: yes
}

I also suggest copying the cert to the webroot for easy fetching:

cp /usr/local/poudriere/keys/pkg.cert /usr/local/poudriere/data/packages/pkg.douglas-enterprises.com.cert

Now installation of this repo is super easy on the target system:

mkdir -p /usr/local/etc/pkg/certs /usr/local/etc/pkg/repos
fetch -o /usr/local/etc/pkg/certs http://pkg.douglas-enterprises.com/pkg.douglas-enterprises.com.cert
fetch -o /usr/local/etc/pkg/repos http://pkg.douglas-enterprises.com/pkg.douglas-enterprises.com.conf
pkg update -f

Automation

Whenever I want to build, I simply run that script. It keeps everything up to date, and is perfect to get added to cron:

0 2 * * * /usr/local/poudriere/run_poudriere.sh

I also checked a few files into a github repo https://github.com/elstevi/poudriere-config and set up an auto deploy mechanism with cron. This allows me to change what packages poudriere is building from anywhere without having to ssh into the machine.