<!-- vim: set tw=78 sts=2 ai expandtab: -->
# Contributing to debian-reference on SALSA

2026-08-17T03:06:31+00:00

Osamu Aoki, <osamu at debian dot org>

See [README.md](README.md) for general guide of the `debian-reference` source.

If you just want to translate without technical complication, please use the
weblate.org service. See [README_weblate.md](README_weblate.md) .

You can see translation progress listed in [po_completed.md](po_completed.md).
This can be updated by `make po`.

This page is for people with write access to the `salsa` git repository.

See also [README_weblate.md](README_weblate.md) since it is applicable to the
`salsa` git repository..

## Setup for the `salsa` git repository

Please check following web page and download the source tree using the `git`
command from https://salsa.debian.org/debian/debian-reference

For read-only access:
```sh
 $ git clone https://salsa.debian.org/debian/debian-reference.git
```

For read-write access with SSH:
```sh
 $ git clone git@salsa.debian.org:debian/debian-reference.git
  ```

In order to enable smooth merge operation, I configure my checked out
repository as:

```sh
 $ git config merge.merge-po-files.driver "./bin/merge-po-files %A %O %B %P"
 $ git config merge.dpkg-mergechangelogs.driver "dpkg-mergechangelogs -m %O %A %B %A"
 $ git config pull.rebase true
 $ git remote add weblate https://hosted.weblate.org/git/debian-reference/translations/
 $ git remote update weblate
```

There are three branches now:

  - `latest` is local branch which uses salsa's `latest` as upstream
  - `origin/latest` is remote tracking branch for the `salsa`'s `latest`
  - `weblate/latest` is remote tracking branch for the `weblate`'s `latest`

## Setup for the `weblate` service

For the smooth managing of contents of `weblate` and the `salsa` GIT repositoriess, install its CLI:

```sh
$ sudo apt install wlc
```

Register your account in weblate and get your API key from:

* https://hosted.weblate.org/accounts/profile/#api

Put your API key in your personal weblate conf file: `~/.config/weblate`

```
[keys]
https://hosted.weblate.org/api/ = ***your API key strings***
```

## Preparing the `debian-reference` source for the Debian package upload

The following is the workflow of preparing the `debian-reference` source for
the Debian package upload.

1. Merge the `weblate` service contents: `git merge weblate latest` ...
2. Update data in the source: `make entity` .. (This must be run in sid/unstable chroot/VM)
3. Generate `debian/changelog` template: `gbp dch`
4. Touch up `debian/changelog`: `dch -r` ...
5. Test build: `sbuild`
6. Build the source package: `dgit -wgf build-source`
7. Upload  the source package: `dgit -wgf push-source`

In the above workflow, you must commit changes to the `salsa` GIT repository
as needed with `git commit -a` or similar.  If you encounter errors, try debug
builds and fix them. If you used `LANGPO` for debug builds etc., please be
careful not to commit some files such as `po_completed.md`.

## Cleaning the checked out GIT repository.

Here is a quick tip:

```sh
 $ git reset --hard HEAD
 $ git clean -dfx
```

## Building flow of the `debian-reference` source

* The English content source is kept as `rawxml/*_*.rawxml` files under `rawxml/`.
* The translation source is kept as PO files under `po/`.

```text
  >>> rawxml/*_*.rawxml                          remotely fetched data files
      $(SRC_XML)                                          $(REMOTE_DATA)
      =========                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        |                                                         :
        |                                                         :
      debian-reference.rawxml                                     :
        |                                                         :
        |                                                         :
        +---> package_list ---------------------------> remote.ent, common.ent
        |                                                     $(SRC_ENT)    (+)
        |                                                      ========
        |                                                         |
        +---> $(DRAW)/header.rawxml                               |
        |         |                                               |
        +<--------+                                               |
        |                                                         |
        v                                                         |
        +---------------------------+                             |
        |                           |                             |
        v                           v                             |
        debian-reference.en.xmlt    debian-reference.en.xmlf      |
        |                   (**)    |                  |          |
        v                           |                  |          |
        po/templates.pot (old)      |                  |          |
        po/ja.po (old)              |                  |          |
        |                           |                  |          |
        |  Generated from           |                  |          |
        |  po4a.cfg.in              |                  |          |
        |    |                      |                  |          |
        V    V                      |                  |          |
  [po4a with po4a.cfg]              |                  |          |
        |                           |                  |          |
        po/templates.pot (new)      |                  |          |
        po/ja.po (new)              |                  |          |
        |                           |                  |          |
        | (debian-reference.ja.xmlt)|                  |          |
        |  ^^unsed data generated^^ |                  |          |
        |                           |                  |          |
  [cp -r po po-tmp]                 |                  |          |
        |  (For Chinese, cp ...     |                  |          |
        |   is replaced with a      |                  |          |
        |   script using OpenCC)    |                  |          |
        |                           |                  |          |
        po-tml/templates.pot        |                  |          |
        po-tmp/ja.po                |                  |          |
        |                           |                  |          |
        |  Generated from           |                  |          |
        |  po4a-imp.cfg.in          |                  |          |
        |    |                      |                  |          |
        V    V                      |                  |          |
  [po4a with po4a-tmp.cfg] <--------+                  |          |
        |                                              |          |
        debian-reference.ja.xmlf  generated            |          |
        | (fuzzy.log)  (++)                            |          |
        |                                              |          |
        v                                              v          v
        +----------------------------------------------+----------+
        |
        v
        debian-reference.*.xml from debian-reference.*.xmlf
        |   * expanding marker to exact versioned package names
        |   * adding manpage URL references
        |
        v
        |(make all), (make test_html),
        |(make html), (make txt), (make pdf), ...
        v
      all documents (*.*.html *.*.txt *.*.pdf)
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Here, the reason behind the 2 stage application of po4a is generation of
non-optimal-but-useful translation strings for `zh-cn`/`zh-tw` and
`pt`/`pt-br` using their counterpart strings.

Use of `debian-reference.en.xmlt` reduce clutter in PO files by excluding
not-for-translation strings.

Use of `debian-reference.en.xmlf` replaces fixed place markers with variable
strings to keep PO files unaffected.


The `LANGPO` variable value sets languages configured for the translation
using po4a-system.

* A language code is added to `LANGPO` in `Makefile` when its translation
  requested to start in the source tree.
  * This setting can be overridden, e.g., to Japanese (ja) only as: `make LANGPO=ja`
  * The translation can be, e.g., skipped as: `make LANGPO=`
* A language code is added to `LANGPO` in `debian/rules` when its translation
  is published as a part of official Debian packages.
  * This override setting of `LANGPO` in `Makefile` for building of the Debian
    package..

Setting the `NOPDF` variable value in `Makefile` can accommodate skipping of
the corresponding PDF generation.  This is useful for working around frequent
TeX tool chain bugs.

Here are some examples for using this `Makefile`:

```sh
# Generate remote.ent etc. from updated remote data and
# listed Debian packages in rawxml/*.rawxml files
 $ make entity
# Update PO update for LANGPO=XX in line with updated rawxml/*.rawxml
 $ make po LANGPO=XX       # update all PO in sync with RAWXML and RAWENT
# clean source for commit  (except ===== data)
 $ make clean
# clean source for commit  (except ===== and ^^^^^^^ data) (Keep remote data)
 $ make localclean
# clean source including entity (except ===== and ^^^^^^^ data) (Keep remote data)
 $ make entityclean
# build package
 $ make all                # build documents (html, text, pdf)
# test build English html
 $ make test_html LANGPO=
# test build LANGPO=XX html
 $ make test_html LANGPO=XX
# no wrap po file for LANGPO=XX
 $ make nowrap LANGPO=XX
# wrap po file for LANGPO=XX
 $ make wrap LANGPO=XX
```

## Package information table in rawxml/*.rawxml

You see quite a bit of `@-@` in `rawxml/*_*.rawxml` and its derivative
`rawxml/debian-reference.rawxml` source.  That is placed there when
auto-generated data replaces them as entity in the final form (`*.xml`,
`*.po`, `*.html`, ...).

For example:

 * `@-@foo@-@`     is converted to the entity definition &foo;
 * `@-@popcon1@-@` is converted to popcon value and URL link of package in column 1.
 * `@-@psize1@-@`  is converted to size value and URL link of package in column 1.
 * `@-@popcon2@-@` is converted to popcon value and URL link of package in column 2.
 * `@-@psize2@-@`  is converted to size value and URL link of package in column 2.

These are transformed into XML when conversion from
`rawxml/debian-reference.rawxml` to `en.xml` and `en.xmlt` happens.

## Tools to edit the XML files (English)

Text editor. (Vim, Emacs, ...)

## Tools to edit the PO files (Translation)

Translator are encouraged to use the dedicated PO file editors such as
`poedit`, `gtranslator`, `gettext.el` (with `emacs`), and `lokalize` on the PO
files `po/*.po` after obtaining the source tree with the `git` command.

The use of the dedicated PO file editor helps translator to find missing and
fuzzy translation data.

## Debugging PO file contents

###  Sanity check of a PO file:

If you wish to check only "`ja`", you can try following:

```sh
 $ make LANGPO=ja po-check
```

This looks for:

* PO file syntax
  * unquoted `"`
  * accidental folding of lines (missing trailing `"`)
* URL style
  * `%` in URL string (use un-escaped UTF-8 characters, instead)
* Test irregular PO file structures using `bin/xmlpo`


This check is very useful for identifying the exact root cause of build
failures.

Here, `bin/xmlpo` not only finds invalid XML but also find unmatched XML tags
which may be caused by careless unfuzzy.

```sh
 $ bin/xmlpo  po/ro.po
I: Process pofile='po/ro.po'

W: unmatched XML tag at 40464
   msgid_tags  = Counter({'ulink': 1, 'literal': 1})
   msgstr_tags = Counter({'ulink': 1})
   msgid "<ulink url=\"&lzma;\">LZMA</ulink> compression utility with higher compression ratio than <literal>gzip</literal>(1)  (deprecated)"
   msgstr "instrument de comprimare <ulink url=\"&lzma;\">LZMA</ulink> cu o raport de comprimare mai mare decât gzip(1) (depreciat)"

E: msgstr XML error at 41057
   \"<literal>-print0</literal>\" returnează întotdeauna <emphasis role=\"strong\">ADEVĂRAT</emphasis> logic și afișează numele complet al fișierului (ulink url=\"&nullterminated;\">terminat cu caracterul null</"<<< ERROR
   msgid "\"<literal>-print0</literal>\" always returns logical <emphasis role=\"strong\">TRUE</emphasis> and print the full filename (<ulink url=\"&nullterminated;\">null terminated</ulink>) on the standard output."
   msgstr "\"<literal>-print0</literal>\" returnează întotdeauna <emphasis role=\"strong\">ADEVĂRAT</emphasis> logic și afișează numele complet al fișierului (ulink url=\"&nullterminated;\">terminat cu caracterul null</ulink>) la ieșirea standard."

E: msgstr XML error at 41083
   Caută toate fișierele ce încep cu „<literal>/path/to<literal>”"<<< ERROR
   msgid "Search all files starting from \"<literal>/path/to</literal>\""
   msgstr "Caută toate fișierele ce încep cu „<literal>/path/to<literal>”"

E: XML markup error counts  = 2
W: XML unmatched tag counts = 1
```

## Test build tips

### Full build

You test build the whole tree in `testing`/`sid` (chroot/VM) environment as:

```
 $ sudo apt-get update
 $ sudo apt-get install devscripts dctrl-tools wget git gitk
 $ sudo apt-get build-dep debian-reference
 $ cd /path/to/debian-reference
 $ debuild # or sbuild
  ...
```

Please note this takes very long time.

### Partial build after translation

If you just translated a PO file, e.g. `po/it.po`, please
reformat and test it first by generating HTML files only:

```
 $ make wrap; make po
 $ make test_html "LANGPO=it"
```

You have English and Italian HTML files under the `build/` directory.

Specifying "`LANGPO`" variable as above saves build time by skipping other
languages.

## Hint for debugging PO file.

If "`make test_html LANGPO=your_language`" fails while processing PO file but it is
not obvious where is the sorce of problem from he error message, execute
"`make wrap LANGPO=your_language`".  Then you may get better error message to
identify the source of problem.

If the above works and "`make test_html LANGPO=`" to build English HTML works
without problem, then try "`make test_html LANGPO=your_language`".  If it fails
while making translated XML file, you have a bit more complicated problem.
    You may see "Error 255" message like:
```
 $ make test_html LANGPO=de
...
No candidate position for the addendum /home/osamu/salsa/debian-reference/debian-reference/po-tmp/de.add.
Addendum po-tmp/de.add does NOT apply to debian-reference.de.xml (translation discarded).
make: *** [Makefile:462: xml] Error 255
```

(Most likely, your `debian-reference.de.xml` is much smaller than `debian-reference.en.xml`.)

If this happens, remove offending `po/*.add` file and build the source again.

Then you see different error message.

```sh
 $ make test_html LANGPO=de
...
mkdir -p /home/osamu/salsa/debian-reference/debian-reference/build
xsltproc --novalid --nonet --xinclude   --stringparam base.dir /home/osamu/salsa/debian-reference/debian-reference/build/ \
                --stringparam html.ext .de.html \
                xslt/style-html.xsl debian-reference.de.xml
debian-reference.de.xml:1940: parser error : Premature end of data in tag section line 1917
...
unable to parse debian-reference.de.xml
make: *** [Makefile:495: /home/osamu/salsa/debian-reference/debian-reference/build/index.de.html] Error 6
```
This means:

* `debian-reference.de.xml` has 1940 lines and stopped due to error.
* Some error happened somewhere between 1917 and 1940 in
  `debian-reference.de.xml`
* Look for matching content of line near 1917 and 1940 in `po/de.po`
* Error is most likely, missing `<placeholder type=\"footnote\" id=\"0\"/>`
  etc.


### Build failure due to po/*.add

If `po/foo.add` can't locate a matching string, build fails.   Since no
`debian-reference.foo.xml` for the problematic language is build, it's a bit
difficult to debug situation.  If you face such problem, please rename such
`po/foo.add` to something like `po/foo.add-keep`, and execute
"`make LANGPO=foo html`" to make `debian-reference.foo.xml`.  Then you can
look for the location of interest and make it the matching string.  Then
rename `po/foo.add-keep` back to `po/foo.add` and update the matching string.

It's tricky.  (This should be less problematic after recent po4a updates)

## Publication to the www.debian.org servers

The contents of `debian-reference` are published to the www.debian.org servers
and they are linked from:

 * https://www.debian.org/doc/user-manuals#quick-reference

These are generated from the latest released packages by the `cron` script run
on the master www.debian.org server and they are mirrored to all the
www.debian.org servers.

 * https://salsa.debian.org/webmaster-team/cron
   * `parts/1ftpfiles`
   * `parts/7doc`

Osamu
