RPM How-To for packagers

Basic informations for packagers: from writing a spec file, to testing a package, to releasing a package.

Pre Req

yum install rpm-build

in the below description the %HOME% is used for the top rpm directory. If you want to have your rpm topdirectory in another location you have to do the following:

• create a file called .rpmmacros in your %HOME% dir
• add the following to the file "%_topdir x:/rpmbuild" (w/o the quotes)

Introduction

RPM uses a special file to compile source code, prepare installation and create .rpm files: it is a .spec file, which is just a bunch of shell commands to prepare, build, install, check and clean the setup.

The Fedora project has a very good ​manual on creating RPM packages. It is recommended for reading in addition to these brief instructions to get deeper understanding of the RPM creation process.

Also, there is a Fedora ​RPM Guide that may be used as an RPM reference documentation.

Instead of writing a .spec file from scratch, you can get an existing one: a good source for them is the RPM find service like ​rpm.pbone.net or ​www.rpmfind.net. Enter the name of your package, and choose a distribution: most core packages have been built using Fedora 13 as template; when Fedora 13 is not available, Fedora 14/15 or OpenSUSE 11.3 are good. Select your package from the left link, you will get a page with all details; one of them is the .src.rpm package, it contains the spec file, platform specific patches, package sources (in .tar.gz, .tar.bz2, .tar.xz format). Use unrpm.cmd (check bootstrap directory) to extract all files to a temporary location.

RPM packages are created from .spec files using the rpmbuild command. The first time you try to build a package (see Building package below), rpmbuild will create a set of directories in your %HOME% directory that it uses for the package generation process:

• %HOME%\rpmbuild\BUILD (temporary storage for compiled files)
• %HOME%\rpmbuild\BUILDROOT (temporary storage for installed files)
• %HOME%\rpmbuild\RPMS (destinations of built .rpm files)
• %HOME%\rpmbuild\SOURCES (source files and patches)
• %HOME%\rpmbuild\SPECS (spec files)
• %HOME%\rpmbuild\SRPMS (destination of built .src.rpm files)

Basic RPM tags:

• summary: a brief (usually one line) description of the package
• name: the name of the package
• version: the package version
• release: the release of the package, e.g., an initial release is 1%{?dist} a next release of the same version is +1. When the version changes, the release starts with 1 again.
• license: the license under which the package is being released (GPLv3, etc.)
• url: the main url to the original project or maintainer
• group: see more on the group tag, below
• source: the url (or filename) of the source package used
• source1: the url (or filename) of additional source package, etc.
• patch0: the url (or filename) of the first patch file applied
• patch1: the url (or filename) of the next patch file applied, etc.

For more information on tags, see ​this link.

Package groups

Groups help to organize packages according to different functionality which they provide. In order to be useful, certain guidelines should be followed when specifying a group tag. In general, the following groups should be used (specified verbatim):

• Applications/Archiving
• Applications/Databases
• Applications/File
• Applications/Internet
• Applications/Publishing
• Applications/System
• Applications/Text
• Development/C
• Development/Languages
• Development/Libraries
• Development/Libraries/C and C++
• Development/Other
• Development/Tools
• Documentation
• Libraries
• Productivity/Archiving/Compression
• System Environment/Base
• System Environment/Daemons
• System Environment/Libraries
• System Environment/Shells
• System/Libraries
• User Interface/X

If left empty, the group tag will be interpreted as "Unspecified" which may not be very helpful.

It is also interesting to note that while Fedora 18 has apparently done away with the requirement to utilize the group tag, we continue to use this for OS/2.

Prepare source code

This is done in the %prep section. Here the %setup macro will expand your source package (the one in the Source: line) and write the files by default in the %HOME%\rpmbuild\BUILD\{name}-{version} directory. You can change this with -n parameter for %setup macro.

If you need to extract more source files, use the -a parameter.

%prep
# -D Do not delete the directory before unpacking.
# -T Disable the automatic unpacking of the archives.
%setup -q -n %{name}-%{srcver} %{?with_int_bdb:-a 1} -a 2

If your files are packed in the root directory of the zip file, use -c to tell rpmbuild to create a new directory using %{name}-%{version} scheme and to put files there (preserving zip directory layout):

%prep
%setup -q -c

To apply first patch add patch0 -p1 -b .my_suffix; the second will use patch1... and so on. Example:

%prep
# -D Do not delete the directory before unpacking.
# -T Disable the automatic unpacking of the archives.
%setup -q -n %{name}-%{srcver} %{?with_int_bdb:-a 1} -a 2
%patch001 -p1 -b .base

Note that rpmbuild defaults expects to find source tarball and patches in the SOURCES directory.

Build source code

Done in the %build section: common steps are execution of the configure script and use of gnu make to build code (this may be different for other programs). Example:

%build
CONFIG_SHELL="/bin/sh" ; export CONFIG_SHELL ; \
LDFLAGS="-Zbin-files -Zhigh-mem -Zomf -Zargs-wild -Zargs-resp" ; export LDFLAGS ; \
LIBS="-lurpo -lmmap -lpthread" ; export LIBS ; \
%configure \
    --enable-shared --disable-static --without-lua \
    %{!?with_int_bdb: --with-external-db} \
    %{?with_sqlite: --enable-sqlite3} \
    --enable-python \
    "--cache-file=%{_topdir}/cache/%{name}.cache"

make %{?_smp_mflags}

In this case, %configure is another macro, and it will be expanded to add proper paths for common variables, like prefix, exec_prefix and others. Keep in mind that rpmbuild will use /@unixroot/usr as prefix, so all built software will go to the same tree layout. Also programs will be able to be executed from other drives.

Most of the times, the CONFIG_SHELL variable is required, otherwise bash shell will be selected if available: bash does not work very well, ash is more compatible. CFLAGS will be added by rpm.

Install binaries

Done in the %install section: most of the times, make install is the required step,

%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR="$RPM_BUILD_ROOT" install

DESTDIR is usually a macro supported in most recent build environments.

File packaging

Done in the %files section: here all required package files must be listed. While /* could be fine since it will include all files, it is better to have a detailed list, so it is easier to catch make install errors.

%files
%defattr(-,root,root,-)
%{_bindir}/rpm.exe
%{_mandir}/man8/rpm.8*
%{_libdir}/rpm*.dll

Building package

When the .spec file is in place, packaging of binary files and source code can be started with

rpmbuild -ba myfile.spec

Special parameters can be used instead of -ba:

• -bb build only binaries
• -bs build only sources
• -bc compile only code and stop.
• -bi install only code and stop.
• -bl check file list only and stop.

Since -bc, -bi, and -bl are also normally performing the previous steps, you can add the special option --short-circuit - this will skip all previous steps (rpmbuild will assume they are already okay), and do only the selected task.

All output files will be written in the %HOME%\rpmbuild tree, regardless of current directory and drive.

Taking sources from source repositories

It's a common practice that the program sources to build the RPM packages are taken from a tarball (referred to by the Source: keyword in the .spec file) which is provided by the program vendor and then stored in the source RPM. It works fine when the packages are built from the vendor sources w/o any modifications but this is rarely a case on OS/2. On OS/2, we don't push our patches upstream (for many reasons) and instead host our own repositories for programs that we support where we manage our patches (many of the programs are located here: ​http://trac.netlabs.org/ports/).

Given this, using the tarball approach on OS/2 requires to pull a set of patches from our own source repository and apply it to the vendor's tarball to make the program work on OS/2 each time a new change is made. This requires a lot of manual work and is very inconvenient when dealing with a large number of packages. For this reason, it is strongly recommended to write .spec files that download the sources directly from the source repositories rather than use tarballs and patch files. This may be achieved with a very simple block of commands.

Note that one should not worry too much about the network speed and traffic issues when using this approach since the sources are downloaded only once for each given repository revision; any subsequent attempt to build a package form the same revision will simply use the source ZIP created when the revision is fetched for the first time w/o involving any network connection. There is also a possibility to use local working copies to fetch the sources from instead of online repositories (see below).

SVN

The snippets below assume our ​http://trac.netlabs.org/ports/ source repository layout. They are taken from this real-life ​spec file.

Paste this block after the main keyword block and before the %description field for the main package:

%define svn_url     http://svn.netlabs.org/repos/ports/libtool/trunk
%define svn_rev     891

Source: %{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip

BuildRequires: gcc make subversion zip

Paste this block instead of the usual %prep / %setup / %patch sequence:

%prep
%if %{?svn_rev:%(sh -c 'if test -f "%{_sourcedir}/%{name}-%{version}-r%{svn_rev}.zip" ; then echo 1 ; else echo 0 ; fi')}%{!?svn_rev):0}
%setup -q
%else
%setup -n "%{name}-%{version}" -Tc
svn export %{?svn_rev:-r %{svn_rev}} %{svn_url} . --force
rm -f "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip"
(cd .. && zip -SrX9 "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip" "%{name}-%{version}")
%endif

In order to use the working copy instead of the URL (useful for creating test packages during development) one modifies the first snippet to something like this:

%define svn_url     D:/Coding/ports/libtool/trunk
#%define svn_rev     891

Note the commented out svn_rev variable — this causes the local changes from the working copy to be picked up. If svn_rev is not commented out then the specified revision w/o any local modifications will be used (the same way as when giving an URL; the only difference is that no network connection is required given that the specified revision is already contained in the working copy).

Handling documentation in Info format

Many packages provide extended documentation in Info format (.info files in /@unixroot/usr/share/info). These .info files need special processing when the package is installed (and removed) so that the Info reader could properly find them. Many .spec files already contain the required commands but they need tailoring to OS/2. This tailoring is rather easy and consists of 3 steps.

1. Fix the Requires: statements. You will usually find this in a .spec file:

   Requires(post): /sbin/install-info
   Requires(preun): /sbin/install-info
   

   Replace this block with these lines:

   # @todo Replace with `%info_requires` when it's available. 
   Requires(post): %{_sbindir}/install-info.exe 
   Requires(preun): %{_sbindir}/install-info.exe 
   

Note also that the package that provides Info files should normally build-depend on the texinfo package. This will make sure that all tools necessary to process the documentation are installed. So please add this line

BuildRequires: texinfo

to the .spec file if it's not already there.

2. Fix the post-install script. Replace lines looking like this:

   %post
   /sbin/install-info %{_infodir}/foobar.info %{_infodir}/dir 2>/dev/null || :
   

   with this:

   %post 
   # @todo Replace with `%info_post foobar.info` when it's available. 
   if [ -f %{_infodir}/foobar.info ]; then 
       %{_sbindir}/install-info.exe %{_infodir}/foobar.info %{_infodir}/dir || : 
   fi
   

3. Fix the pre-uninstall script. Replace lines looking like this:

   %preun 
   if [ $1 -eq 0 ]; then 
     /sbin/install-info --delete %{_infodir}/help2man.info \ 
       %{_infodir}/dir 2>/dev/null || : 
   fi
   

   with this:

   %preun 
   # @todo Replace with `%info_preun foobar.info` when it's available. 
   if [ $1 -eq 0 ]; then 
       if [ -f %{_infodir}/foobar.info ]; then 
           %{_sbindir}/install-info.exe --delete %{_infodir}/foobar.info %{_infodir}/dir || : 
       fi 
   fi
   

Note that the lines you insert contain a @todo block which you should leave in. It is to remind to replace these lines with simpler constructs when #119 is fixed.

OS/2 specific notes

While the syntax of .spec files is the same used under Unix, not everything is currently working in the OS/2 port. Also many packages are already built and manually installed: this means RPM is not aware of their presence.

Many Unix .spec files have a (or multiple) BuildRequires: rules: if the package is not already built with RPM, the line should be commented until the required package will have a RPM install. The same applies to Requires: rules.

Another missing feature is the automatic detection of required packages: while OS/2 rpm can analyze the LX binaries to detect the required DLLs on the fly, the same is not possible for python or perl scripts (unix rpm detects the required abi level by parsing scripts and querying python/perl modules).
Under OS/2 these requirements must be manually added using a BuildRequires: or Requires: rule. Example:

Requires: python
Requires: python(abi) = 2.6

Repacking binary software archives

When you create RPM packages for software that you don't build from sources (like old OS/2 programs or any other software that is only available in the binary form), you need to follow a special procedure in order to preserve the sizes and timestamps of the original files (which may be important for their identification in case someone needs that information).

1. Add the following to the .spec file (somewhere before the %install section):

   # disable lxlite compression
   %define __os_install_post	%{nil}
   

2. Use

   cp -dp <src> <dst>
   

   in the %install section when you copy files from BUILD to BUILDROOT for getting packed by rpmbuild; the naked cp command will kill the original timestamp (the <dst> file will get the current time). The -p flag causes cp to preserve timestamps and the -d flag preserves symlinks (just in case a file you copy is a LIBC symlink). Another cp flag you may find useful is -R which causes it to copy directories recursively.

Note that it doesn't make much sense to use cp -dp in packages which you build from sources, since the original timestamp in this case is the build time which won't differ too much from the install time (as the %install stage happens right after the %build stage).

WPS object creation

RPM provides a number of macros to create WPS objects for your package from %post and %postun scriplets. These macros start with %wps_object_. All WPS objects created using these macros are associated with the package and recorded in a special database. Each object also has a reference count (which is increased when the object is created and decreased when it is deleted). The physical object deletion only happens when the reference count drops to zero. This is useful for objects (e.g. application folders) shared between several packages since it allows to keep them alive as long as there is one or more packages which reference these objects installed and automatically remove them once all of the referencing packages are removed.

Here is an example:

%post
if [ "$1" -ge 1 ]; then # (upon update)
    %wps_object_delete_all
fi
%wps_object_create_begin
MYAPP_EXE:WPProgram|My App|<WP_DESKTOP>|EXENAME=((%{_bindir}/myapp.exe))
%wps_object_create_end

%postun
if [ "$1" -eq 0 ]; then # (upon removal)
    %wps_object_delete_all
fi

For a sub-package named "extras", the scriptlets will look as follows:

%post extras
if [ "$1" -ge 1 ]; then # (upon update)
    %wps_object_delete_all -n %{name}-extras
fi
%wps_object_create_begin -n %{name}-extras
MYAPP_EXTRAS_FOLDER:WPFolder|My App Extras|<WP_DESKTOP>
MYAPP_FOOBAR_EXE:WPProgram|My App FooBar|<MYAPP_EXTRAS_FOLDER>|EXENAME=((%{_bindir}/myapp-foobar.exe))
%wps_object_create_end

%postun
if [ "$1" -eq 0 ]; then # (upon removal)
    %wps_object_delete_all -n %{name}-extras
fi

CONFIG.SYS changes

It is sometimes necessary to modify the CONFIG.SYS file during installation, even when using RPM. One of the examples is when your package installs a device driver that needs to be loaded at boot time or when a global environment variable needs to be set. This can be done from the %post section using the special macro %cube which is based on the CUBE tool.

For example, this snippet will add a DEVICE= line to CONFIG.SYS after the first DEVICE= line (or to the bottom if there are no DEVICE= statements), add an environment variable (using the similar logic).

%post 
%cube {ADDLINE "DEVICE=%UNIXROOT%\usr\%{_lib}\mydrv.sys" (AFTER "DEVICE="} %{os2_config_sys} >nul
%cube {ADDLINE "SET MYENVVAR=myvalue" (AFTER "SET "} %{os2_config_sys} >nul
echo; echo "NOTE:"
echo; echo "The file '%{os2_config_sys}' has been changed. You need to reboot your"
echo "computer in order to activate these changes."
echo

Here is the corresponding uninstallation snippet that will undo changes made by the above code. You should always perform symmetric uninstallation steps to make sure that all changes your package does to CONFIG.SYS are rolled back when the package is uninstalled from the system.

%postun 
%cube {DELLINE "SET MYENVVAR="} %{os2_config_sys} >nul
%cube {DELLINE "DEVICE=%UNIXROOT%\usr\%{_lib}\mydrv.sys"} %{os2_config_sys} >nul
echo; echo "NOTE:"
echo; echo "The file '%{os2_config_sys}' has been changed. You need to reboot your"
echo "computer in order to activate these changes."
echo

The complete CUBE documentation is available here: ​http://www3.sympatico.ca/gjarviseng/cube/cube.html