Updating packages in TeX Live

This page describes the current process of updating packages in TeX Live. Additional volunteers to help with this job would be most welcome.

The basic premise is to take TL updates from CTAN. At present, on the machine where updates will be done, you need to have:

Usual updates

Given the above, running the (Perl) script Build/tools/tpm-ctan-check --check compares a number of TL packages against a local CTAN tree and reports any that are out of date; each line starting with # in the output is one such package. If all packages are up to date, there will be no output. (Suggestion: put /your/tex/live/root/Build/tools in your PATH.)

Let's use poemscol as an example - it is a standard LaTeX package with no funny tricks (and happens to be frequently updated).

  1. Run the command Build/tools/ctan2tl poemscol and inspect the output:
  2. Check the block of output labeled cooked. This is the package as it will be installed; it's available on the filesystem in Build/cdbuild/cooked/poemscol. In particular, there should be some just-created files.
  3. Check the block of (comm) output labeled new vs. present tpm. Any changes here should make sense, e.g., a new file being added, old ones being removed. Huge masses of output probably indicate that ctan2tds.pl (see next item) isn't set up to deal with this package. The original source as copied from CTAN is on the filesystem in Build/cdbuild/raw/poemscol.
  4. If there are any irregularities or questionable changes, they should be resolved before anything is checked in. The first block of output is the rather cryptic logging from Build/cdbuild/ctan2tds.pl, which does the actual CTAN-to-TDS conversion. It is not unusual that changes in a package might imply changes in ctan2tds.pl.
  5. The last block of output is the equally cryptic logging from Build/cdbuild/place, which takes the cooked tree and "places" it in the TL source hierarchy, eventually including updating the tpm and doing repository adds and deletes as necessary. However, all that is a no-op in this run - so far, nothing we have done alters anything outside of the temp directories Build/cdbuild/raw and Build/cdbuild/cooked.
  6. Usually, all will look fine. Once it does, the next command is: ctan2tl --place. This performs the repository commands, but does not commit anything. The most likely thing to fail here is the Subversion binary/text heuristic. Just rerunning ctan2tl --place usually suffices.
  7. Occasionally, the first block of output will contain special instructions labeled YOU-MUST-DO. You must do them :). These involve copying executable scripts to the right place in the Build subtree. (Maybe one day ctan2tl will be smart enough to handle this automatically.)
  8. The last command commits the changes to the repository: svn commit -m'poemscol newversion' `cat /tmp/tlplace.dirs`. Explanation: with subversion, it is very inefficient to run a commit from a high-level directory, like texmf-dist. Therefore, place writes the modified directories to the temporary file /tmp/tlplace.dirs (the environment variable TMPDIR will be used instead of /tmp if it is set). So we pass the contents of that file to svn. Naturally, if any other files were modified in the process, e.g., ctan2tds.pl, they should be included in the commit also. (The version of the updated package is not always readily available; feel free to omit it from the log message.)
  9. After this is done, tpm-ctan-check poemscol should yield no output.
  10. \relax. Then \repeat.

Adding more packages to the automated checks

The tpm-ctan-check script used above currently knows about only a fraction of the packages in TL. Please help by adding more packages to the semi-automated system. Here's the story:

  1. Even more strongly recommended: add the Build/tools directory to your path.
  2. Choose a package, i.e., a tpm (for now, choose one from the texmf-dist tree), which currently is not checked. You can get a list of all such packages with tpm-ctan-check --list-not-treated-tpm.
  3. You can find the package on CTAN by running CTAN=/path/to/CTAN tlpkginfo --ctan-dir package, or generally searching CTAN. If the tlpkginfo script does not return the right directory, something needs to be fixed. Please look into it or report it.
  4. Check the licensing. TeX Live is free software. That's free as in freedom, in case you didn't know.
  5. In particular, check that the source to the documentation (typically a pdf) is available. Often the documentation is generated from a dtx, and that is fine. But not infrequently it is written separately, and we must have sources (available under a free license) to include it in TL. If it's not there, please politely write the author - usually it is simply an oversight, not a conscious exclusion.
  6. Optional (but would be great): check the TeX Catalogue entry for the package, and if there are any updates, send email with:
  7. Run ctan2tl and inspect the output, following the procedure and information above.
    1. If everything looks good, including files being installed in the correct directories:
      • add the package name to tpm-ctan-check (yeah, one more!)
      • call ctan2tl --place package, as above.
      • commit, as above.
    2. If anything looks wrong: hack ctan2tds.pl as necessary, which can mean anything from merely adding the package to the directory hashes at the top of the program, to a simple use of MAKEflatten, to a completely new routine that does arbitrary things. Or email the list so others can help.
  8. \relax. Then \repeat once more.

Adding a new package on CTAN to TeX Live

If you want to add a NEW package without a tpm, do more or less the same as above, only at the "if everything looks good" stage it must also be add to one of the collections (texmf/tpm/collection-*). If you don't know which collection it belongs in, ask.

If you come across a TeX package not on CTAN, please encourage the authors to upload it to CTAN. Our ultimate goal is for everything in TL to come from CTAN.

Updating programs

The description above is only for the usual TeX-world packages - LaTeX styles, fonts, etc. Packages which contain executables, especially compiled executables, are updated differently. Maybe someday we will write something about that too. If you get this far, and want to help on this score, please write.

Mirroring CTAN

First, before doing any mirroring, please email ctan_at_dante_de, so that they are not surprised by a new daily load.

An easy and efficient way to mirror CTAN is to use rsync. Here is a skeleton of the command used on tug.org: 

cd /my/ctan/mirror/directory
nice /usr/local/bin/rsync -avz --delete \
  --exclude='.in.*' --exclude=.htaccess \
  rsync://rsync.whatever.ctan.node \
  . \
  >/var/tmp/mirror-ctan.log 2>&1

If the CTAN mirror is only for TL update purposes, much more can be excluded:
systems usergrps digests dviware tools web *.zip
Then it needs only 3-4gb.

(It would be possible to enhance the various scripts to download a package over the network instead of assuming it is on local disk, but this was easier, and tug.org is a CTAN mirror, so there has been no need.)

By the way, although the ctan-ann list is of course a good source for discovering new updates to be propagated, it is not complete. Updates to CTAN holdings are not infrequently made without an announcement.

$Date: 2006/10/19 22:26:33 $; TeX Live;