# Usage How to use the translation infrastructure with your `mdbook` project. ## Installation Run ```shell cargo install mdbook-i18n-helpers ``` to install three binaries: - `mdbook-xgettext`: This program extracts the source text. It is an [`mdbook` renderer]. - `mdbook-gettext`: This program translates the book into a target language. It is an [`mdbook` preprocessor]. - `mdbook-i18n-normalize`: This program normalizs a PO file. Use it after breaking changes. [`mdbook` renderer]: https://rust-lang.github.io/mdBook/format/configuration/renderers.html [`mdbook` preprocessor]: https://rust-lang.github.io/mdBook/format/configuration/preprocessors.html Together, the two programs makes it possible to do i18n for `mdbook` in a standard and maintainable way. ## Gettext Overview We use the [Gettext] system for translations. This system is widely used for translations of open source software and it also works reasonably well for documentation. The advantage of Gettext is that you get a structured way to approach the translations. Instead of copying Markdown files and tracking changes by hand, you modify `.po` files in a `po/` directory. The `.po` files are small text-based translation databases. You update the `.po` files using tools (described below) and you can see at a glance how much text still needs to be translated. > **Tip:** You should never edit the `.po` files by hand. Instead use a PO > editor, such as [Poedit](https://poedit.net/). There are also several online > editors available. This will ensure that the file is encoded correctly. There is a `.po` file for each language. They are named after the [ISO 639] language codes: Danish would go into `po/da.po`, Korean would go into `po/ko.po`, etc. The `.po` files contain all the source text plus the translations. They are initialized from a `messages.pot` file (a PO template) which contains the extracted source text from your `mdbook` project. If your source files are in English, then the `messages.pot` file will contain the English text and your translators will be translating from English into their target language. We will show how to update and manipulate the `.po` and `.pot` files using the GNU Gettext utilities below. [Gettext]: https://www.gnu.org/software/gettext/manual/html_node/index.html [ISO 639]: https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes ## Creating and Updating Translations First, you need to know how to update the `.pot` and `.po` files. As a general rule, you should never touch the auto-generated `po/messages.pot` file. You should not even check it into your repository since it can be fully generated from your source Markdown files. You should also never edit the `msgid` entries in a `po/xx.po` file. If you find mistakes, you need to update the original text instead. The fixes to the original text will flow into the `.po` files the next time the translators update them. ### Generating the PO Template To extract the original text and generate a `messages.pot` file, you run `mdbook` with the `mdbook-xgettext` renderer: ```shell MDBOOK_OUTPUT='{"xgettext": {}}' \ mdbook build -d po ``` You will find the generated POT file as `po/messages.pot`. To extract the text into smaller `.pot` files based on the text's Markdown outline, use the `depth` parameter. For a `depth` of `1`, the `.pot` lines will be separated into a file for each section or chapter title. Use greater values to split the `.pot` file further. ```shell MDBOOK_OUTPUT='{"xgettext": {"depth": "1"}}' \ mdbook build -d po/messages ``` ### Initialize a New Translation To start a new translation for a fictional `xx` locale, first generate the `po/messages.pot` file. Then use `msginit` to create a `xx.po` file: ```shell msginit -i po/messages.pot -l xx -o po/xx.po ``` You can also simply copy `po/messages.pot` to `po/xx.po` if you don't have `msginit` from the GNU Gettext tools available. If you do that, then you have to update the header (the first entry with `msgid ""`) manually to the correct language. > **Tip:** You can use the > [`cloud-translate`](https://github.com/mgeisler/cloud-translate) tool to > quickly machine-translate a new translation. Untranslated entries will be sent > through GCP Cloud Translate. Some of the translations will be wrong after > this, so you must inspect them by hand afterwards. ### Updating an Existing Translation As the source text changes, translations gradually become outdated. To update the `po/xx.po` file with new messages, first extract the source text into a `po/messages.pot` template file. Then run ```shell msgmerge --update po/xx.po po/messages.pot ``` Unchanged messages will stay intact, deleted messages are marked as old, and updated messages are marked "fuzzy". A fuzzy entry will reuse the previous translation: you should then go over it and update it as necessary before you remove the fuzzy marker. ## Using Translations This will show you how to use the translations to generate localized HTML output. > **Note:** `mdbook-gettext` will use the original untranslated text for all > entries marked as "fuzzy" (visible as "Needs work" in Poedit). This is > especially important when using > [`cloud-translate`](https://github.com/mgeisler/cloud-translate) for initial > translation as all entries will be marked as "fuzzy". > > If your text isn't translated, double-check that you have removed all "fuzzy" > flags from your `xx.po` file. ### Building a Translated Book The translation is done using the `mdbook-gettext` preprocessor. Enable it in your project by adding this snippet to your `book.toml` file: ```toml [preprocessor.gettext] after = ["links"] ``` This will run `mdbook-gettext` on the source after things like `{{ #include }}` has been executed. This makes it possible to translate included source code. You can leave `mdbook-gettext` enabled: if no language is set or if it cannot find the `.po` file corresponding to the language (e.g., it cannot find `po/en.po` for English), then it will return the book untranslated. To use the `po/xx.po` file for your output, you simply set `book.language` to `xx`. You can do this on the command line: ```shell MDBOOK_BOOK__LANGUAGE=xx mdbook build -d book/xx ``` This will set the book's language to `xx` and store the generated files in `book/xx`. ### Serving a Translated Book Like normal, you can use `mdbook serve` to view your translation as you work on it. You use the same command as with `mdbook build` above: ```shell MDBOOK_BOOK__LANGUAGE=xx mdbook serve -d book/xx ``` To automatically reload the book when you change the `po/xx.po` file, add this to your `book.toml` file: ```toml [build] extra-watch-dirs = ["po"] ``` ### Publishing Translations with GitHub Actions Please see the [`publish.yml`] workflow in the Comprehensive Rust 🦀 repository. [`publish.yml`]: https://github.com/google/comprehensive-rust/blob/main/.github/workflows/publish.yml ## Marking Sections with a comment A block can be marked with a comment for translation by prepending a special HTML comment `` in front of it. Consecutive HTML comments will be collected into a single translation comment. For example: ```markdown The following will have a comment attached to the message. But what is a man, what has he got. If not himself, then he has naught. ``` ## Marking Sections to be Skipped for Translation A block can be marked to be skipped for translation by prepending a special HTML comment `` in front of it. For example: ````markdown The following code block should not be translated. ``` fn hello() { println!("Hello world!"); } ``` Itemized list: - A should be translated. - B should be skipped. - C should be translated. ```` Note that we don't extract the full text of code blocks. Only text that is recognized as comments and literal strings is extracted. ## Normalizing Existing PO Files When mdbook-i18n-helpers change, the generated PO files change as well. This can result in a situation where the messages in a `xx.po` file are no longer exactly like the ones expected by `mdbook-gettext`. An example is the change from version 0.1.0 to 0.2.0: `mdbook-xgettext` from version 0.1.0 will output a list as a whole: ```markdown - foo - bar ``` becomes ```gettext msgid "" "- foo\n" "- bar\n" msgstr "" ``` in the PO file. However, `mdbook-xgettext` version 0.2.0 will produce two messages instead: ```gettext msgid "foo" msgstr "" msgid "bar" msgstr "" ``` Use `mdbook-i18n-normalize` version 0.2.0 to convert the old PO file to the new format. Importantly, existing translations are kept intact! If the old PO file is translated like this ```gettext msgid "" "- foo\n" "- bar\n" msgstr "" "- FOO\n" "- BAR\n" ``` then the new PO file generated with `mdbook-i18n-normalize` will contain two messages: ```gettext msgid "foo" msgstr "FOO" msgid "bar" msgstr "BAR" ``` You will only need to run `mdbook-i18n-normalize` once after upgrading mdbook-i18n-helpers.