SVN Tutorial

This is a simple guide for developers, to allow a common tree structure and commit messages for every project stored into the main repository.

Preparation

While there is a great SVN GUI called ​SmartSVN which is highly recommended to use (version 6.x.x and 7.0.7 work greatly under OS/2 due to our nice ​OpenJDK 1.6 port), sometimes working with the SVN tool from the command line is more desirable. In particular, the examples in this tutorial show the usage of the SVN command line client. Note that these command line examples imply using RPM ash as the current shell (unless stated otherwise).

In order to make the command line SVN client more comfortable, there are several useful tips and tricks.

1. Set the EDITOR environment variable to a text editor that can be started form the command line by passing a file name to edit as the first (and only) argument, e.g.:

   set EDITOR=e.exe
   

   This will make SVN call this editor whenever it needs you to edit some multi-line text (e.g. when writing a commit message).

2. Set up an external diff viewer (to be described later).

Creating New Project From Existing Sources

Let's assume that there is a project is named pixman that you want to import to this repository. First, you need to create the vendor tree on the SVN server where you will import the original sources:

svn mkdir --parents http://svn.netlabs.org/repos/ports/pixman/vendor/current -m "pixman: Create vendor tree."

The real-life result of this step is in r1228.

Then you need to check out the vendor tree to your file system:

mkdir pixman
svn co http://svn.netlabs.org/repos/ports/pixman/vendor/current pixman/vendor-current

The next step is to import original sources. There are two possible cases: using a tarball from the vendor or taking the sources right from the vendor's source repository (SVN, GIT, HG, etc). There is one essential difference between these cases: the tarball will usually contain pre-generated sources necessary for the build (and sometimes also the documentation) while the repository doesn't contain these files assuming they will be regenerated by the developer. The vendor's repository is a preferred source for importing (because it makes no sense to store generated files in a version control system). But sometimes it's not possible, either because the vendor repository is not accessible or because not all files can be re-generated on OS/2 due to the lack of some Linux tools that hasn't been ported to OS/2 yet.

Importing from a vendor's repository

Fortunately, the pixman project has a public repository which is located here: ​http://cgit.freedesktop.org/pixman/. It's a GIT repository, so you need the git package to check it out. You also need to define a version you want to import and a tag corresponding to it. In our case version is 0.32.8 (the latest stable one at the time of writing) and the respective tag is pixman-0.32.8. To check out this tag to vendor-current, run the following command:

git clone --depth=1 http://anongit.freedesktop.org/git/pixman.git -b pixman-0.32.8 pixman/tmp
rm -rf pixman/tmp/.git
mv `find pixman/tmp/ -maxdepth 1 -mindepth 1` pixman/vendor-current/
rmdir pixman/tmp

Importing from a mercurial repository would look like this (an example for the NSPR project):

hg clone https://hg.mozilla.org/projects/nspr -r NSPR_4_12_RTM nspr/tmp
rm -rf nspr/tmp/.hg nspr/tmp/.hgtags
mv `find nspr/tmp/ -maxdepth 1 -mindepth 1` nspr/vendor-current/
rmdir nspr/tmp

If it were a SVN repository, the command would look like this (an example for the ICU project):

svn export http://source.icu-project.org/repos/icu/icu/tags/release-56-1 --force icu/vendor-current

Importing from a tarball

Let's assume we downloaded a tarball of the version we want to import, e.g.pixman-0.32.8.tar.gz for the pixman library version 0.32.8. Now unpack it to pixman/vendor-current:

tar xvf pixman-0.32.8.tar.gz --strip-components=1 -C pixman/vendor-current

Committing the tree

Now commit all the original files with these commands:

cd pixman/vendor-current
svn add . --force
svn commit

This should open your favorite editor (set by the EDITOR environment variable) where you enter the commit message. Note that using the -m switch is not enough in this case as your commit message at this step needs to be multi-line, like this one:

pixman: Import version 0.32.8 from vendor.

Source URL: http://cgit.freedesktop.org/pixman/tag/?id=pixman-0.32.8

Note that you should specify the exact URL of the corresponding repository tag (or the download URL of the tarball, like ​http://cairographics.org/releases/pixman-0.32.8.tar.gz) you used for the import operation in this commit — it may be needed to get the original vendor source later to check that everything was properly imported etc. Check r1229 for the real-life example.

Tagging vendor release

The next step is to mark the result of the new vendor import with a version tag for further reference, like this (see r1230):

svn copy http://svn.netlabs.org/repos/ports/pixman/vendor/current http://svn.netlabs.org/repos/ports/pixman/vendor/0.32.8 -m "pixman: Tag version 0.32.8 from vendor."

Creating trunk

Now it's time to start off the project trunk where OS/2 development will actually take place (r1231):

svn copy http://svn.netlabs.org/repos/ports/pixman/vendor/0.32.8 http://svn.netlabs.org/repos/ports/pixman/trunk -m "pixman: Start off trunk from version 0.32.8 from vendor."

Checking out working copy

Now you can check out the trunk locally with:

cd ..
svn co http://svn.netlabs.org/repos/ports/pixman/trunk

and start working on patches to make it build and work on OS/2.

Removing generated files

This step is applicable only if the sources were imported from a tarball. In this case it is necessary to detect which files can be re-generated and should be removed. If the project you are importing is autotools-based, then the best way to do it is to unpack the tarball into a temporary directory and try to run the following command from there (unless there is a special script like autogen.sh, bootstrap and similar in which case you should use that script instead). This also assumes you have autoconf, automake and libtool packages installed:

autoreconf -fvi

The changed timestamps will tell you which files were re-generated. You may then try to remove these files and run autoreconf again to see if they can be actually re-generated from scratch. If they do (and look similar to the originals), you may safely remove them at this point. You may even try to build the project to double check that the build files were correctly re-generated. For our pixman project, such files are these:

aclocal.m4
compile
config.guess
config.h.in
config.sub
configure
depcomp
install-sh
ltmain.sh
Makefile.in
missing
test-driver
demos/Makefile.in
pixman/Makefile.in
test/Makefile.in

Now you first remove these files from under SVN control with svn rm FILENAME or with svn rm --targets FILE where FILE contains all the files to remove, one per line (be careful not to remove necessary files!), then commit the result with this command:

svn commit -m "pixman: Remove generated files from trunk."

Note that another important reason to re-generate as much files as possible on the developer's machine (besides saving space and history in the SVN repository) is to make sure the generated files will get OS/2-specific changes necessary for the build to work correctly. This is needed since not all OS/2 patches to the toolchain reach upstream repositories in time and hence the generated file provided in the tar ball may not know about OS/2 at all and will simply not work. Besides this, a recent version of autotools may have some important fix to apply and so on. So the developer should always re-generate these files locally before performing a build.

Note. It is important to remove generated files on the trunk rather than on the vendor tree because:

1. The vendor tree is meant as an exact copy of what is imported into our repository.
2. If the files are removed from the vendor tree and the vendor tree is then updated to a newer upstream version, the removed files will appear as NEW and it will be hard to distinguish if they are really new and needed for the new version or if we already removed them in the past and need to remove them again (a check of the commit history is required for this). If we remove these files from the trunk instead, then there will be a merge conflict (a change to a file that we deleted) which is easy to spot and fix (just svn rm these files before the final commit).

Working on trunk

Changes should be committed to the trunk grouped into logically connected chunks, one commit per each logical modification using this commit message as a template:

pixman: Describe your modification here in few words.

Provide a more detailed description of this logical modification here, if necessary.
You may also give references to related tickets and to other commits here.

An example of such a commit is r1232.

And the last but not the least: Please double-check the diff of every changed file, as well as the commit message, before you actually commit it! In SVN, it's impossible to invisibly undo a failed commit, you will have to apply a new commit with fixes for your typos, forgotten files and so on which messes up history and not good for further analysis.

Updating Sources to a New Vendor Version

[to be written later]

Connection Timed Out Error

If getting errors such as

svn co http://svn.netlabs.org/repos/ports/pixman/trunk
svn: Server sent unexpected return value (504 Connection Timed Out) in response
to OPTIONS request for 'http://svn.netlabs.org/repos/ports/pixman/trunk'

Try using https instead of http. Note that you may have to accept a certificate.

svn co https://svn.netlabs.org/repos/ports/pixman/trunk

Old Instructions (to be removed)

First you need to checkout the current ports tree or at least the root directory:

svn checkout http://svn.netlabs.org/repos/ports

Here is the commit sequence for storing a new port (named xx here):

[ports] 
	mkdir xx\vendor\current.
	unpack sources into current.

[ports] 
	svn add xx
	svn commit -m "xx: initial vendor import of xx 8.12."

[ports\xx\vendor] 
	svn copy current 8.12
	svn commit -m "xx: tag current vendor version."

[ports\xx] 
	svn copy vendor/current trunk
	svn commit -m "xx: import current vendor version into trunk."

[ports\xx\trunk]
	add os2 patches, new files, etc.
	svn commit -m "xx: applied os2 patches, added new files, etc etc."

Once you have to upgrade then vendor tree to a newer version, first put it in current tree and tag the version release. Please take note of svn revision for tag operation.

To merge changes into the trunk, supposed 318 is the latest revision, use

[ports\xx] 
	svn merge vendor/8.12@318 vendor/8.13@318 trunk
	svn commit -m "xx: update trunk to 8.13".