Changes between Version 59 and Version 60 of RpmHowToPackagers

Timestamp:

Feb 7, 2017, 11:28:09 PM (8 years ago)

Author:

dmik

Comment:

--

RpmHowToPackagers

@@ -177 +177 @@
 == Taking sources from source repositories ==
 
-It's a common practice that the program sources to build the RPM packages are taken from a tarball (referred to by the `Source:` keyword in the `.spec` file) which is provided by the program vendor and then stored in the source RPM. It works fine when the packages are built from the vendor sources w/o any modifications but this is rarely a case on OS/2. On OS/2, we don't push our patches upstream (for many reasons) and instead host our own repositories for programs that we support where we manage our patches (many of the programs are located here: http://trac.netlabs.org/ports/).
-
-Given this, using the tarball approach on OS/2 requires to pull a set of patches from our own source repository and apply it to the vendor's tarball to make the program work on OS/2 each time a new change is made. This requires a lot of manual work and is very inconvenient when dealing with a large number of packages. For this reason, it is strongly recommended to write `.spec` files that download the sources directly from the source repositories rather than use tarballs and patch files. This may be achieved with a very simple block of commands.
-
-Note that one should not worry too much about the network speed and traffic issues when using this approach since the sources are downloaded only once for each given repository revision; any subsequent attempt to build a package form the same revision will simply use the source ZIP created when the revision is fetched for the first time w/o involving any network connection. There is also a possibility to use local working copies to fetch the sources from instead of online repositories (see below).
+It's a common practice to take the source code for building a set of RPM packages for a program from a tarball file (referred to by the `Source:` keyword in the `.spec` file) which is provided by the program vendor. This works fine when the packages are built from unmodified vendor sources but this is rarely a case on OS/2. This is mainly because we don't push our patches upstream (for many reasons) and instead host our own repositories for software that we support (many of the software is located here: http://trac.netlabs.org/ports/).
+
+Given this, using the tarball approach on OS/2 requires to pull a set of patches from our own source repository and apply it to the vendor's tarball to make the program work on OS/2 each time a new change is made and a new RPM version is to be released. This requires a lot of manual work which very inconvenient and time consuming, especially when dealing with a large number of packages. For this reason, it is strongly recommended to write `.spec` files that download the sources directly from the source repositories rather than use tarballs and patch files. This may be easily achieved with a couple of RPM macros that we added to RPM for OS/2 specifically for this purpose. The following instructions apply to both porting the existing `.spec` files to OS/2 and creating new ones.
+
+Note that you should not worry too much about the network speed and traffic issues when using these macros since the sources are downloaded only once for each given repository revision; any subsequent attempt to build a package form the same revision will simply use the source ZIP created when the revision is fetched for the first time w/o involving any network connection. This ZIP is also stored in a SRPM file as usual, so a rebuild of the package may be done without accessing the repository as long as the SRPM file for the given set of packages is available. There is also a possibility to use local working copies to fetch the sources from instead of online repositories (see below).
+
+==== %scm_source ====
+
+This macro is intended as a replacement for the standard `Source:` tag. The format is as follows:
+{{{
+%scm_source SCM URL REV
+}}}
+where SCM is the type of the source code management system (currently supported ones are `svn`, `git` and `github`), URL is the full URL of the source code repository for the given program and REV is the revision to use.
+
+==== %scm_setup ====
+
+This macro is intended as a replacement for the standard `%setup` macro that starts a `%prep` section.
+
+The format is as simple as:
+{{{
+%scm_setup
+}}}
+
+Further sections describe the supported SCM types in detail. 
 
 === SVN ===
 
-The snippets below assume our http://trac.netlabs.org/ports/ source repository layout. They are taken from  this real-life [http://trac.netlabs.org/rpm/browser/spec/trunk/SPECS/libtool.spec?rev=471 spec file].
-
-Paste this block after the main keyword block and before the `%description` field for the main package:
-{{{
-%define svn_url     http://svn.netlabs.org/repos/ports/libtool/trunk
-%define svn_rev     891
-
-Source: %{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip
-
-BuildRequires: gcc make subversion zip
-}}}
-
-Paste this block instead of the usual `%prep / %setup / %patch` sequence:
-{{{
-%prep
-%if %{?svn_rev:%(sh -c 'if test -f "%{_sourcedir}/%{name}-%{version}-r%{svn_rev}.zip" ; then echo 1 ; else echo 0 ; fi')}%{!?svn_rev):0}
-%setup -q
-%else
-%setup -n "%{name}-%{version}" -Tc
-svn export %{?svn_rev:-r %{svn_rev}} %{svn_url} . --force
-rm -f "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip"
-(cd .. && zip -SrX9 "%{_sourcedir}/%{name}-%{version}%{?svn_rev:-r%{svn_rev}}.zip" "%{name}-%{version}")
-%endif
-}}}
-
-In order to use the working copy instead of the URL (useful for creating test packages during development) modify the first snippet to something like this (note `%` replaced with `#`, not with `#%` — this is because rpmbuild evaluates macros even if they are in comment lines):
-{{{
-%define svn_url     D:/Coding/ports/libtool/trunk
-#define svn_rev     891
-}}}
-
-Note the commented out `svn_rev` variable — this causes the local changes from the working copy to be picked up. If `svn_rev` is not commented out then the specified revision w/o any local modifications will be used (the same way as when giving an URL; the only difference is that no network connection is required given that the specified revision is already contained in the working copy).
+An example:
+{{{
+%scm_source svn http://svn.netlabs.org/repos/ports/which/trunk 733
+}}}
+This will check out revision 733 of program `which` from the trunk of the our main OS/2 repository.
+
+Note that for testing purposes you may use a local working copy of the SVN repository by specifying something like ` D:/Coding/ports/which/trunk` instead of the URL. You may also omit the revision number if you want your local changes to be used instead of a clean revision. But please never distribute RPM packages built this way as it's generally impossible to rebuild them on another machine which breaks RPM repository consistency and our deployment rules. Also note that if you omit the revision number, no source ZIP file will be created by `%scm_source` so the source code will be checked out each time you build an RPM and an attempt to run the full build cycle that creates an SRPM file will fail.
 
 === Git ===
 
-Projects that use git may be handled similarly but they have their own specifics. There is currently no real-life example of such a project in our repositories but let's assume there is a project called `foobar` with a remote repository at  `git://mygitserver.com`.
-
-Paste this block after the main keyword block and before the `%description` field for the main package:
-{{{
-%define git_url     git://mygitserver.com/foobar.git
-%define git_rev     mytag
-
-Source: %{name}-%{git_rev}.zip
-
-BuildRequires: gcc make git zip
-}}}
-
-Paste this block instead of the usual `%prep / %setup / %patch` sequence:
-{{{
-%prep
-%if %(sh -c 'if test -f "%{_sourcedir}/%{name}-%{git_rev}.zip" ; then echo 1 ; else echo 0 ; fi')
-%setup -q
-%else
-%setup -n "%{name}-%{version}" -Tc
-rm -f "%{_sourcedir}/%{name}-%{git_rev}.zip"
-git archive --format zip --output "%{_sourcedir}/%{name}-%{git_rev}.zip" --prefix "%{name}-%{version}/" --remote "%{git_url}" "%{git_rev}"
-unzip "%{name}-%{git_rev}.zip" -d ..
-%endif
-}}}
-
-Note that you may use your local clone instead of the remote repository as well as with SVN to save some traffic. The URL in this case should be something like `file://D:/Coding/foobar`.
-
-=== Github ===
-
-If a project is hosted at [http://github.com github], we will need to use `curl` instead of `git archive` because github does not support the `archive` command and instead provides a special URL for downloading ZIPs of the tree at any given commit or tag. The snippets below are based on this real-life [http://trac.netlabs.org/rpm/browser/spec/trunk/SPECS/libkai.spec?rev=576 spec file].
-
-Paste this block after the main keyword block and before the `%description` field for the main package:
-{{{
-%define github_name kai
-%define github_url  https://github.com/komh/%{github_name}/archive
-%define github_rev  kai-%{version}
-
-Source: %{github_name}-%{github_rev}.zip
-
-BuildRequires: gcc make curl zip
-}}}
-
-Paste this block instead of the usual `%prep / %setup / %patch` sequence:
-{{{
-%prep
-%if %(sh -c 'if test -f "%{_sourcedir}/%{github_name}-%{github_rev}.zip" ; then echo 1 ; else echo 0 ; fi')
-%setup -n "%{github_name}-%{github_rev}" -q
-%else
-%setup -n "%{github_name}-%{github_rev}" -Tc
-rm -f "%{_sourcedir}/%{github_name}-%{github_rev}.zip"
-curl -sSL "%{github_url}/%{github_rev}.zip" -o "%{_sourcedir}/%{github_name}-%{github_rev}.zip"
-unzip "%{_sourcedir}/%{github_name}-%{github_rev}.zip" -d ..
-%endif
-}}}
-
-Note that due to github naming specifics (and especially if the github's project name doesn't match the RPM package name, (like `kai` vs `libkai` in the example above), the source ZIP will have a name that differs from the usual `%{name}-%{version}` scheme (in the example above it will be `kai-kai-1.1.4.zip` rather than `libkai-1.1.4.zip`).
-
-You could use your local clone instead of the remote repository here as well but in this case magic github URLs for downloading ZIPs won't work so you will have to use the `git archive` command as in the [#Git] section above which is not very practical because you will have to make a lot of temporary changes to your `.spec` file which you will have to revert before committing it to the SPEC repo.
+An example:
+{{{
+%scm_source git git://mygitserver.com/foobar.git mytag
+}}}
+This will check out a commit named `mytag` from the `foobar`'s repository (you may use any supported Git commit identifier instead of tags, of course).
+
+You may also omit the revision specification at all in which case it will checkout the HEAD commit. Note, however, that as well as in case of SVN, packages with a missing revision specification must not be distributed No ZIP files are created for them, so a clean checkout will happen each time and SRPM creation will fail.
+
+Using local clones of git repositories for test builds is also possible, the URL in this case will be something like `file://D:/Coding/foobar`.
+
+=== !GitHub ===
+
+An example:
+{{{
+%scm_source github https://github.com/komh/kai kai-%{version}
+}}}
+This will checkout a commit named `kai-%{version}` from the GitHub-hosted project `kai` which is owned the user named `komh`. Any valid Git commit identifier may be used. Note that you cannot omit the commit specification in case of GitHub.
+
+You may use local clones of GitHub projects for test builds but in this case you should use the `git` SCM type as described in the [#Git] section above.
 
 == Handling documentation in Info format ==