= RPM How-To for packagers =
[[PageOutline]]
Basic informations for packagers: from writing a spec file, to testing a package, to releasing a package.

== Pre Req ==
yum install rpm-build[[br]]

in the below description the %HOME% is used for the top rpm directory. If you want to have your rpm topdirectory in another location you have to do the following:[[br]]
 * create a file called .rpmmacros in your %HOME% dir
 * add the following to the file "%_topdir   x:/rpmbuild" (w/o the quotes)

== Introduction ==
RPM uses a special file to compile source code, prepare installation and create .rpm files: it is a .spec file, which is just a bunch of shell commands to prepare, build, install, check and clean the setup.

The Fedora project has a very good [http://fedoraproject.org/wiki/How_to_create_an_RPM_package manual on creating RPM packages]. It is recommended for reading in addition to these brief instructions to get deeper understanding of the RPM creation process.

Also, there is a Fedora [http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html-single/RPM_Guide/ RPM Guide] that may be used as an RPM reference documentation.

Instead of writing a .spec file from scratch, you can get an existing one: a good source for them is the RPM find service like [http://rpm.pbone.net/ rpm.pbone.net] or [http://www.rpmfind.net www.rpmfind.net]. Enter the name of your package, and choose a distribution: most core packages have been built using Fedora 13 as template; when Fedora 13 is not available, Fedora 14/15 or OpenSUSE 11.3 are good. Select your package from the left link, you will get a page with all details; one of them is the .src.rpm package, it contains the spec file, platform specific patches, package sources (in .tar.gz, .tar.bz2, .tar.xz format). Use  unrpm.cmd (check bootstrap directory) to extract all files to a temporary location.

RPM packages are created from .spec files using the ''rpmbuild'' command. The first time you try to build a package (see '''Building package''' below), rpmbuild will create a set of directories in your %HOME% directory that it uses for the package generation process:
 * %HOME%\rpmbuild\BUILD (temporary storage for compiled files)
 * %HOME%\rpmbuild\BUILDROOT (temporary storage for installed files)
 * %HOME%\rpmbuild\RPMS (destinations of built .rpm files)
 * %HOME%\rpmbuild\SOURCES (source files and patches)
 * %HOME%\rpmbuild\SPECS (spec files)
 * %HOME%\rpmbuild\SRPMS (destination of built .src.rpm files)

Basic RPM tags:
 * ''summary:'' a brief (usually one line) description of the package
 * ''name:'' the name of the package
 * ''version:'' the package version
 * ''release:'' the release of the package, e.g., an initial release is 1%{?dist} a next release of the same version is +1. When the version changes, the release starts with 1 again.
 * ''license:'' the license under which the package is being released (GPLv3, etc.)
 * ''url:'' the main url to the original project or maintainer
 * ''group:'' see more on the group tag, below
 * ''source:'' the url (or filename) of the source package used
 * ''source1:'' the url (or filename) of additional source package, etc.
 * ''patch0:'' the url (or filename) of the first patch file applied
 * ''patch1:'' the url (or filename) of the next patch file applied, etc.

For more information on tags, see [http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html this link].

== Package groups ==

Groups help to organize packages according to different functionality which they provide. In order to be useful, certain guidelines should be followed when specifying a group tag. In general, the following groups should be used (specified verbatim):

 * !Applications/Archiving
 * !Applications/Databases
 * !Applications/File
 * !Applications/Internet
 * !Applications/Publishing
 * !Applications/System
 * !Applications/Text
 * Development/C
 * !Development/Languages
 * !Development/Libraries
 * !Development/Libraries/C and C++
 * !Development/Other
 * !Development/Tools
 * Documentation
 * Libraries
 * !Productivity/Archiving/Compression
 * System !Environment/Base
 * System !Environment/Daemons
 * System !Environment/Libraries
 * System !Environment/Shells
 * !System/Libraries
 * User Interface/X

If left empty, the group tag will be interpreted as "Unspecified" which may not be very helpful.

It is also interesting to note that while Fedora 18 has apparently done away with the requirement to utilize the group tag, we continue to use this for OS/2.

== Prepare source code ==
This is done in the ''%prep'' section. Here the ''%setup'' macro will expand your source package (the one in the ''Source:'' line) and write the files by default in the %HOME%\rpmbuild\BUILD\{name}-{version} directory. You can change this with ''-n'' parameter for %setup macro.

If you need to extract more source files, use the ''-a'' parameter.

{{{
%prep
# -D Do not delete the directory before unpacking.
# -T Disable the automatic unpacking of the archives.
%setup -q -n %{name}-%{srcver} %{?with_int_bdb:-a 1} -a 2
}}}

If your files are packed in the root directory of the zip file, use -c to tell rpmbuild to create a new directory using %{name}-%{version} scheme and to put files there (preserving zip directory layout):

{{{
%prep
%setup -q -c
}}}

To apply first patch add ''patch0 -p1 -b .my_suffix''; the second will use ''patch1...'' and so on.
Example:
{{{
%prep
# -D Do not delete the directory before unpacking.
# -T Disable the automatic unpacking of the archives.
%setup -q -n %{name}-%{srcver} %{?with_int_bdb:-a 1} -a 2
%patch001 -p1 -b .base
}}}
Note that rpmbuild defaults expects to find source tarball and patches in the SOURCES directory.

== Build source code ==
Done in the %build section: common steps are execution of the configure script and use of gnu make to build code (this may be different for other programs).
Example:
{{{
%build
CONFIG_SHELL="/bin/sh" ; export CONFIG_SHELL ; \
LDFLAGS="-Zbin-files -Zhigh-mem -Zomf -Zargs-wild -Zargs-resp" ; export LDFLAGS ; \
LIBS="-lurpo -lmmap -lpthread" ; export LIBS ; \
%configure \
    --enable-shared --disable-static --without-lua \
    %{!?with_int_bdb: --with-external-db} \
    %{?with_sqlite: --enable-sqlite3} \
    --enable-python \
    "--cache-file=%{_topdir}/cache/%{name}.cache"

make %{?_smp_mflags}
}}}

In this case, ''%configure'' is another macro, and it will be expanded to add proper paths for common variables, like prefix, exec_prefix and others. Keep in mind that rpmbuild will use /@unixroot/usr as prefix, so all built software will go to the same tree layout. Also programs will be able to be executed from other drives.

Most of the times, the CONFIG_SHELL variable is required, otherwise bash shell will be selected if available: bash does not work very well, ash is more compatible.
CFLAGS will be added by rpm.

== Install binaries ==
Done in the %install section: most of the times, make install is the required step,

{{{
%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR="$RPM_BUILD_ROOT" install
}}}

DESTDIR is usually a macro supported in most recent build environments.

== File packaging ==
Done in the %files section: here all required package files must be listed. While /* could be fine since it will include all files, it is better to have a detailed list, so it is easier to catch make install errors.

{{{
%files
%defattr(-,root,root,-)
%{_bindir}/rpm.exe
%{_mandir}/man8/rpm.8*
%{_libdir}/rpm*.dll
}}}

== Building package ==
When the .spec file is in place, packaging of binary files and source code can be started with
{{{
rpmbuild -ba myfile.spec
}}}

Special parameters can be used instead of -ba:
 * -bb build only binaries
 * -bs build only sources
 * -bc compile only code and stop.
 * -bi install only code and stop.
 * -bl check file list only and stop.

Since -bc, -bi, and -bl are also normally performing the previous steps, you can add the special option ''--short-circuit'' - this will skip all previous steps (rpmbuild will assume they are already okay), and do only the selected task.

All output files will be written in the %HOME%\rpmbuild tree, regardless of current directory and drive.

== Taking sources from source repositories ==

It's a common practice that the program sources to build the RPM packages are taken from a tarball (referred to by the `Source:` keyword in the `.spec` file) which is provided by the program vendor and then stored in the source RPM. It works fine when the packages are built from the vendor sources w/o any modifications but this is rarely a case on OS/2. On OS/2, we don't push our patches upstream (for many reasons) and instead host our own repositories for programs that we support where we manage our patches (many of the programs are located here: http://trac.netlabs.org/ports/).

Given this, using the tarball approach on OS/2 requires to pull a set of patches from our own source repository and apply it to the vendor's tarball to make the program work on OS/2 each time a new change is made. This requires a lot of manual work and is very inconvenient when dealing with a large number of packages. For this reason, it is strongly recommended to write `.spec` files that download the sources directly from the source repositories rather than use tarballs and patch files. This may be achieved with a very simple block of commands.

Note that one should not worry too much about the network speed and traffic issues when using this approach since the sources are downloaded only once for each given repository revision; any subsequent attempt to build a package form the same revision will simply use the source ZIP created when the revision is fetched for the first time w/o involving any network connection. There is also a possibility to use local working copies to fetch the sources from instead of online repositories (see below).

=== SVN ===

The snippets below assume our http://trac.netlabs.org/ports/ source repository layout. They are taken from  this real-life [http://trac.netlabs.org/rpm/browser/spec/trunk/SPECS/libtool.spec?rev=471 spec file].

Paste this block after the main keyword block and before the `%description` field for the main package:
{{{
%define svn_url     http://svn.netlabs.org/repos/ports/libtool/trunk
%define svn_rev     891

Source: %{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip

BuildRequires: gcc make subversion zip
}}}

Paste this block instead of the usual `%prep / %setup / %patch` sequence:
{{{
%prep
%if %{?svn_rev:%(sh -c 'if test -f "%{_sourcedir}/%{name}-%{version}-r%{svn_rev}.zip" ; then echo 1 ; else echo 0 ; fi')}%{!?svn_rev):0}
%setup -q
%else
%setup -n "%{name}-%{version}" -Tc
svn export %{?svn_rev:-r %{svn_rev}} %{svn_url} . --force
rm -f "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip"
(cd .. && zip -SrX9 "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip" "%{name}-%{version}")
%endif
}}}

In order to use the working copy instead of the URL (useful for creating test packages during development) modify the first snippet to something like this (note `%` replaced with `#`, not with `#%` — this is because rpmbuild evaluates macros even if they are in comment lines):
{{{
%define svn_url     D:/Coding/ports/libtool/trunk
#define svn_rev     891
}}}

Note the commented out `svn_rev` variable — this causes the local changes from the working copy to be picked up. If `svn_rev` is not commented out then the specified revision w/o any local modifications will be used (the same way as when giving an URL; the only difference is that no network connection is required given that the specified revision is already contained in the working copy).

=== Git ===

Projects that use git may be handled similarly but they have their own specifics. There is currently no real-life example of such a project in our repositories but let's assume there is a project called `foobar` with a remote repository at  `git://mygitserver.com`.

Paste this block after the main keyword block and before the `%description` field for the main package:
{{{
%define git_url     git://mygitserver.com/foobar.git
%define git_rev     mytag

Source: %{name}-%{git_rev}.zip

BuildRequires: gcc make git zip
}}}

Paste this block instead of the usual `%prep / %setup / %patch` sequence:
{{{
%prep
%if %(sh -c 'if test -f "%{_sourcedir}/%{name}-%{git_rev}.zip" ; then echo 1 ; else echo 0 ; fi')
%setup -q
%else
%setup -n "%{name}-%{version}" -Tc
rm -f "%{_sourcedir}/%{name}-%{git_rev}.zip"
git archive --format zip --output "%{_sourcedir}/%{name}-%{git_rev}.zip" --prefix "%{name}-%{version}/" --remote "%{git_url}" "%{git_rev}"
unzip "%{name}-%{git_rev}.zip" -d ..
%endif
}}}

Note that you may use your local clone instead of the remote repository as well as with SVN to save some traffic. The URL in this case should be something like `file://D:/Coding/foobar`.

=== Github ===

If a project is hosted at [http://github.com github], we will need to use `curl` instead of `git archive` because github does not support the `archive` command and instead provides a special URL for downloading ZIPs of the tree at any given commit or tag. The snippets below are based on this real-life [http://trac.netlabs.org/rpm/browser/spec/trunk/SPECS/libkai.spec?rev=576 spec file].

Paste this block after the main keyword block and before the `%description` field for the main package:
{{{
%define github_name kai
%define github_url  https://github.com/komh/%{github_name}/archive
%define github_rev  kai-%{version}

Source: %{github_name}-%{github_rev}.zip

BuildRequires: gcc make curl zip
}}}

Paste this block instead of the usual `%prep / %setup / %patch` sequence:
{{{
%prep
%if %(sh -c 'if test -f "%{_sourcedir}/%{github_name}-%{github_rev}.zip" ; then echo 1 ; else echo 0 ; fi')
%setup -n "%{github_name}-%{github_rev}" -q
%else
%setup -n "%{github_name}-%{github_rev}" -Tc
rm -f "%{_sourcedir}/%{github_name}-%{github_rev}.zip"
curl -sSL "%{github_url}/%{github_rev}.zip" -o "%{_sourcedir}/%{github_name}-%{github_rev}.zip"
unzip "%{_sourcedir}/%{github_name}-%{github_rev}.zip" -d ..
%endif
}}}

Note that due to github naming specifics (and especially if the github's project name doesn't match the RPM package name, (like `kai` vs `libkai` in the example above), the source ZIP will have a name that differs from the usual `%{name}-%{version}` scheme (in the example above it will be `kai-kai-1.1.4.zip` rather than `libkai-1.1.4.zip`).

You could use your local clone instead of the remote repository here as well but in this case magic github URLs for downloading ZIPs won't work so you will have to use the `git archive` command as in the [#Git] section above which is not very practical because you will have to make a lot of temporary changes to your `.spec` file which you will have to revert before committing it to the SPEC repo.

== Handling documentation in Info format ==

Many packages provide extended documentation in Info format (`.info` files in `/@unixroot/usr/share/info`). These `.info` files need special processing when the package is installed (and removed) so that the Info reader could properly find them. Many `.spec` files already contain the required commands but they need tailoring to OS/2. This tailoring is rather easy and consists of 3 steps.

1. Fix the `Requires:` statements. You will usually find this in a `.spec` file:
{{{
Requires(post): /sbin/install-info
Requires(preun): /sbin/install-info
}}}
 Replace this block with these lines:
{{{
# @todo Replace with `%info_requires` when it's available. 
Requires(post): %{_sbindir}/install-info.exe 
Requires(preun): %{_sbindir}/install-info.exe 
}}}

 Note also that the package that provides Info files should normally build-depend on the `texinfo` package. This will make sure that all tools necessary to process the documentation are installed. So please add this line 
{{{
BuildRequires: texinfo
}}}
 to the `.spec` file if it's not already there.

2. Fix the post-install script. Replace lines looking like this:
{{{
%post
/sbin/install-info %{_infodir}/foobar.info %{_infodir}/dir 2>/dev/null || :
}}}
 with this:
{{{
%post 
# @todo Replace with `%info_post foobar.info` when it's available. 
if [ -f %{_infodir}/foobar.info ]; then 
    %{_sbindir}/install-info.exe %{_infodir}/foobar.info %{_infodir}/dir || : 
fi
}}}

3. Fix the pre-uninstall script. Replace lines looking like this:
{{{
%preun 
if [ $1 -eq 0 ]; then 
  /sbin/install-info --delete %{_infodir}/help2man.info \ 
    %{_infodir}/dir 2>/dev/null || : 
fi
}}}
 with this:
{{{
%preun 
# @todo Replace with `%info_preun foobar.info` when it's available. 
if [ $1 -eq 0 ]; then 
    if [ -f %{_infodir}/foobar.info ]; then 
        %{_sbindir}/install-info.exe --delete %{_infodir}/foobar.info %{_infodir}/dir || : 
    fi 
fi
}}}

Note that the lines you insert contain a `@todo` block which you should leave in. It is to remind to replace these lines with simpler constructs when #119 is fixed.

== Generating debug packages ==

RPM can automatically extract debug info from the EXEs and DLLs built by the .spec file and put this info into a separate sub-package named `PACKAGE-debug` where PACKAGE is the main .spec package name. In order to do that, the following directive needs to be added to the .spec file (usually, after all package and sub-package definitions and before the `%prep` section, surrounded by empty lines):
{{{
%debug_package
}}}

Normally nothing else needs to be done, RPM will automatically handle all `*.exe`, `*.dll` and `*.pyd` files generated by your .spec. However, sometimes it's desirable to disable debug info generation for certain files. This may be done with the `%_strip_opts` definition placed at the beginning of the .spec file, e.g. as follows:
{{{
# Exclude myfile*.dll from the processed files
%define _strip_opts --debuginfo -x "myfile*.dll"

# Add *.mymod files to the default list of files to be processed
%define _strip_opts --debuginfo -i "*.mymod"

# Process only *.mymod files (do not process default extensions at all)
%define _strip_opts --debuginfo -n "*.mymod"
}}}

Note that if you don't need any debug info generation at all, simply remove the `%debug_package` macro from your .spec file and debug file generation will be automatically suppressed (all the debug info will be silently discarded from the executable files during the compression phase in such case).

== Setting up executable compression ==

By default, RPM performs compression for the following executable files built by the .spec file before putting them to `.rpm` bundles: `*.exe`, `*.dll` and `*.pyd`. Compression is done using the `lxlite` tool by optimizing LX object layout and discarding debug info. Sometimes it is desirable to change the list of files to be compressed (or to disable compression at all). Here are the examples of how to do it:
{{{
# Disable compression completely
%define _strip_no_compress 1

# Exclude myfile*.dll from the compressed files
%define _strip_opts --compress -x "myfile*.dll"

# Add *.mymod files to the default list of files to be compressed
%define _strip_opts --compress -i "*.mymod"

# Compress only *.mymod files (do not compress default extensions at all)
%define _strip_opts --compress -n "*.mymod"
}}}

== OS/2 specific notes ==
While the syntax of .spec files is the same used under Unix, not everything is currently working in the OS/2 port. Also many packages are already built and manually installed: this means RPM is not aware of their presence.

Many Unix .spec files have a (or multiple) '' !BuildRequires: '' rules: if the package is not already built with RPM, the line should be commented until the required package will have a RPM install. The same applies to ''Requires:'' rules.

Another missing feature is the automatic detection of required packages: while OS/2 rpm can analyze the LX binaries to detect the required DLLs on the fly, the same is not possible for python or perl scripts (unix rpm detects the required abi level by parsing scripts and querying python/perl modules).[[BR]]
Under OS/2 these requirements must be manually added using a '' !BuildRequires: '' or ''Requires:'' rule.
Example:
{{{
Requires: python
Requires: python(abi) = 2.6
}}}

=== Repacking binary software archives ===
When you create RPM packages for software that you don't build from sources (like old OS/2 programs or any other software that is only available in the binary form), you need to follow a special procedure in order to preserve the sizes and timestamps of the original files (which may be important for their identification in case someone needs that information).

 1. Add the following to the .spec file (somewhere before the %install section):
{{{
# disable lxlite compression
%define __os_install_post	%{nil}
}}}
 2. Use
{{{
cp -dp <src> <dst>
}}}
  in the %install section when you copy files from BUILD to BUILDROOT for getting packed by rpmbuild; the naked cp command will kill the original timestamp (the <dst> file will get the current time). The -p flag causes cp to preserve timestamps and the -d flag preserves symlinks (just in case a file you copy is a LIBC symlink). Another cp flag you may find useful is -R which causes it to copy directories recursively.

Note that it doesn't make much sense to use cp -dp in packages which you build from sources, since the original timestamp in this case is the build time which won't differ too much from the install time (as the %install stage happens right after the %build stage).

=== WPS object creation ===

RPM provides a number of macros to create WPS objects for your package from %post and %postun scriplets. These macros start with {{{%wps_object_}}}. All WPS objects created using these macros are associated with the package and recorded in a special database. Each object also has a reference count (which is increased when the object is created and decreased when it is deleted). The physical object deletion only happens when the reference count drops to zero. This is useful for objects (e.g. application folders) shared between several packages since it allows to keep them alive as long as there is one or more packages which reference these objects installed and automatically remove them once all of the referencing packages are removed.

Here is an example:
{{{
%post
if [ "$1" -ge 1 ]; then # (upon update)
    %wps_object_delete_all
fi
%wps_object_create_begin
MYAPP_EXE:WPProgram|My App|<WP_DESKTOP>|EXENAME=((%{_bindir}/myapp.exe))
%wps_object_create_end

%postun
if [ "$1" -eq 0 ]; then # (upon removal)
    %wps_object_delete_all
fi
}}}

For a sub-package named "extras", the scriptlets will look as follows:
{{{
%post extras
if [ "$1" -ge 1 ]; then # (upon update)
    %wps_object_delete_all -n %{name}-extras
fi
%wps_object_create_begin -n %{name}-extras
MYAPP_EXTRAS_FOLDER:WPFolder|My App Extras|<WP_DESKTOP>
MYAPP_FOOBAR_EXE:WPProgram|My App FooBar|<MYAPP_EXTRAS_FOLDER>|EXENAME=((%{_bindir}/myapp-foobar.exe))
%wps_object_create_end

%postun
if [ "$1" -eq 0 ]; then # (upon removal)
    %wps_object_delete_all -n %{name}-extras
fi
}}}

=== CONFIG.SYS changes ===
It is sometimes necessary to modify the CONFIG.SYS file during installation, even when using RPM. One of the examples is when your package installs a device driver that needs to be loaded at boot time or when a global environment variable needs to be set. This can be done from the {{{%post}}} section using the special macro {{{%cube}}} which is based on the CUBE tool.

For example, this snippet will add a DEVICE= line to CONFIG.SYS after the first DEVICE= line (or to the bottom if there are no DEVICE= statements), add an environment variable (using the similar logic).
{{{
%post 
%cube {ADDLINE "DEVICE=%UNIXROOT%\usr\%{_lib}\mydrv.sys" (AFTER "DEVICE="} %{os2_config_sys} >nul
%cube {ADDLINE "SET MYENVVAR=myvalue" (AFTER "SET "} %{os2_config_sys} >nul
echo; echo "NOTE:"
echo; echo "The file '%{os2_config_sys}' has been changed. You need to reboot your"
echo "computer in order to activate these changes."
echo
}}}

Here is the corresponding uninstallation snippet that will undo changes made by the above code. You should always perform symmetric uninstallation steps to make sure that all changes your package does to CONFIG.SYS are rolled back when the package is uninstalled from the system.
{{{
%postun 
%cube {DELLINE "SET MYENVVAR="} %{os2_config_sys} >nul
%cube {DELLINE "DEVICE=%UNIXROOT%\usr\%{_lib}\mydrv.sys"} %{os2_config_sys} >nul
echo; echo "NOTE:"
echo; echo "The file '%{os2_config_sys}' has been changed. You need to reboot your"
echo "computer in order to activate these changes."
echo
}}}

The complete CUBE documentation is available here: http://www3.sympatico.ca/gjarviseng/cube/cube.html