Updating packages in TeX Live

This page describes the current process of updating packages in TeX Live. Additional volunteers to help with this job would be most welcome.

The basic premise is to take TL updates from CTAN. At present, on the machine where updates will be done, you need to have:

Usual updates

Given the above, running the script tlpkg-ctan-check --check compares a number of TL packages against a local CTAN tree and reports any that are out of date; each line starting with # in the output is one such package. If all packages are up to date, there will be no output.

Let's use poemscol as an example - it is a standard LaTeX package with no funny tricks.

  1. Run the command ctan2tl poemscol and inspect the output:
  2. Check the block of output labeled cooked. This is the package as it will be installed; it's available on the filesystem in Build/tmp.cooked/poemscol. In particular, there should be some newly-created files.
  3. Check the block of (comm) output labeled new vs. present tpm. Any changes here should make sense, e.g., a new file being added, old ones being removed. Huge masses of output probably indicate that ctan2tds (see next item) isn't set up to deal with this package. The original source as copied from CTAN is on the filesystem in Build/tmp.raw/poemscol.
  4. If there are any irregularities or questionable changes, they should be resolved before anything is checked in. The first block of output is the rather cryptic logging from ctan2tds, which does the actual CTAN-to-TDS conversion. It is not unusual for changes in a package to imply changes in ctan2tds.
  5. The last block of output is the equally cryptic logging from place, which takes the cooked tree and "places" it in the TL source hierarchy, eventually including updating the package database and doing repository adds and deletes as necessary. However, all that is a no-op in this run; so far, nothing we have done alters anything outside of the temp directories Build/tmp.*.
  6. Usually, all will look fine. Once it does, the next command to run is: ctan2tl --place. This performs the repository commands, but does not commit anything. The most likely thing to fail here is the Subversion binary/text heuristic. Just rerunning ctan2tl --place usually suffices.
  7. Occasionally, the first block of output will contain special instructions labeled YOU-MUST-DO. You must do them :). These involve copying executable scripts to the right place in the Build/source subtree. (Maybe one day ctan2tl will be smart enough to handle this automatically.)
  8. The last command commits the changes to the repository: svn commit -m'poemscol newversion' `cat /tmp/tlplace.dirs`. Explanation: with subversion, it is very inefficient to run a commit from a high-level directory, such as texmf-dist. Therefore, place writes the modified directories to the temporary file /tmp/tlplace.dirs (the environment variable TMPDIR will be used instead of /tmp if it is set). So we pass the contents of that file to svn. Naturally, if any other files were modified in the process, e.g., ctan2tds.pl, they should be included in the commit also. (The version of the updated package is not always readily available; feel free to omit it from the log message.)
  9. After this is done, tlpkg-ctan-check poemscol should yield no output.
  10. \relax. Then \repeat.

Adding more packages to the automated checks

The tlpkg-ctan-check script used above currently knows about only a fraction of the packages in TL. Please help by adding more packages to the semi-automated system. Here's the story:

  1. Choose a package, i.e., a tlpsrc (for now, choose one from the texmf-dist tree), which currently is not checked. You can get a list of all such packages with tlpkg-ctan-check --list-not-treated.
  2. You can find the package on CTAN by running CTAN=/path/to/CTAN tlpkginfo --ctan-dir package, or generally searching CTAN. If the tlpkginfo script does not return the right directory, something needs to be fixed. Please look into it (or at least report it).
  3. Check the licensing. TeX Live is free software. That's free as in freedom, in case you didn't know. If it's nonfree, it should be removed from TL and a note left in ctan2tds.
  4. In particular, check that the source to the documentation (typically a pdf) is available. Often the documentation is generated from a dtx, and that is fine. But not infrequently it is written separately, and we must have sources (available under a free license) to include it in TL. If it's not there, please politely write the author - usually it is simply an oversight, not a conscious exclusion. Without sources, the processed documentation must be excluded.
  5. Optional (but would be much appreciated): check the TeX Catalogue entry for the package, and if there are any updates, send email with:
  6. Run ctan2tl and inspect the output, following the procedure and information above.
    1. If everything looks good, including files being installed in the correct directories:
      • add the package name to tlpkg-ctan-check.
      • call ctan2tl --place package, as above.
      • commit, as above.
    2. If anything looks wrong: hack ctan2tds.pl as necessary, which can mean anything from merely adding the package to the directory hashes at the top of the program, to a simple use of MAKEflatten, to a completely new routine that does arbitrary things. Or email the list so others can help.
  7. \relax. Then continue to \repeat.

Removing a package from TeX Live

To remove a package from the TeX Live sources, use the tlprm script. It doesn't actually remove anything (or commit any changes), just tells you the actions to take.

Adding a new package on CTAN to TeX Live

If you want to add a new package that is not presently in TL, do more or less the same as above, only at the "if everything looks good" stage, also add it to one of the collections (Master/tlpkg/tlpsrc/collection-*). If you don't know which collection it belongs in, ask.

Also, a few packages need a special execute directive; the most common case is font packages, which should have an addMap or addMixedMap line. See the main infrastructure documentation for all the possibilities, and/or the existing *.tlpsrc files.

Incidentally, if you come across a TeX package in TL that is not on CTAN, please encourage the authors to upload it to CTAN. Our ultimate goal is for everything in TL to come from CTAN.

Updating programs

The description above is only for the usual TeX-world packages - LaTeX styles, fonts, etc. Packages which contain executables, especially compiled executables, are updated differently. Maybe someday we will write something about that too. If you get this far, and want to help on this score, please write.

Mirroring CTAN

First, before doing any mirroring, please email ctan_at_dante_de, so that they are not surprised by a new daily load.

An easy and efficient way to mirror CTAN is to use rsync. Here is a skeleton of the command used on tug.org (of course you'll probably want to redirect the output, etc.): 

cd /my/ctan/mirror/directory
nice /usr/local/bin/rsync -avz --delete --delete-excluded \
  --exclude='.in.*' --exclude=.htaccess \
  rsync://$CTAN_MIRROR/ \
  $CTAN_LOCAL_DIR

If the CTAN mirror is only for TL update purposes, much more can be excluded: 

nice rsync -avz --delete --delete-excluded \
  --exclude='.in.*' --exclude=.htaccess \
  --include=/digests/tugboat/hyphenex --exclude='/digests/*' \
  --include=/dviware/dvipsconfig --exclude='/dviware/*' \
  --exclude=/nonfree \
  --include='/obsolete/{biblio,macros/latex,fonts/wasy}' --exclude=/obsolete \
  --exclude=/systems --exclude=/tools --exclude=/usergrps --exclude=/web \
  rsync://$CTAN_MIRROR/ \
  $CTAN_LOCAL_DIR

By the way, although the ctan-ann list is of course a good source for discovering new updates to be propagated, it is not complete. Updates to CTAN holdings are not infrequently made without an announcement. This is one reason that tlpkg-ctan-check exists.

$Date: 2008/06/10 18:07:35 $; TeX Live;