Skip to content

Port PDF generation to Hugo #972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MuhammadAashirAslam wants to merge 1 commit into
base: hugo
Choose a base branch
from
Open

Port PDF generation to Hugo #972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions config/pdf/hugo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
[params]
output = "pdf"
print_title = "preCICE"
print_subtitle = "The coupling library for partitioned multi-physics simulations"

[outputs]
home = ["PDFTitle", "PDFTOC", "PDFDocsPart", "PDFTutorialsPart", "PDFAppendixPart", "PrinceList"]
page = ["HTML"]
section = ["HTML"]

[outputFormats.PDFTitle]
mediaType = "text/html"
baseName = "title"
path = "pdf"
isHTML = true
notAlternative = true

[outputFormats.PDFTOC]
mediaType = "text/html"
baseName = "toc"
path = "pdf"
isHTML = true
notAlternative = true

[outputFormats.PDFDocsPart]
mediaType = "text/html"
baseName = "docs-part"
path = "pdf"
isHTML = true
notAlternative = true

[outputFormats.PDFTutorialsPart]
mediaType = "text/html"
baseName = "tutorials-part"
path = "pdf"
isHTML = true
notAlternative = true

[outputFormats.PDFAppendixPart]
mediaType = "text/html"
baseName = "appendix-part"
path = "pdf"
isHTML = true
notAlternative = true

[outputFormats.PrinceList]
mediaType = "text/plain"
baseName = "prince-list"
path = "pdf"
isPlainText = true
notAlternative = true

[segments.pdf]
[[segments.pdf.includes]]
output = "{pdftitle,pdftoc,pdfdocspart,pdftutorialspart,pdfappendixpart,princelist}"

[[segments.pdf.includes]]
output = "html"
path = "{/docs,/docs/**,/tutorials,/tutorials/**}"
246 changes: 110 additions & 136 deletions content/docs/docs-meta/publish-to-pdf.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,189 +1,163 @@
---
title: Generate PDF Documentation
permalink: docs-meta-publish-to-pdf.html
aliases:
- /docs-meta-publish-to-pdf.html
keywords: pdf, publish to pdf, generate pdf
summary:
summary: "Build the complete documentation PDF with Hugo and Prince."
---

## Overview and tools needed

At the core of PDF generation process is [Prince](https://www.princexml.com/), which converts HTML/CSS (and even some JS) to print quality PDF with bookmarks, links, page numbers etc. Unfortunately there are no open-source alternatives that operate at the same level as Prince (yet) - however, Prince offers a [non-commercial license](https://www.princexml.com/purchase/license_faq/#non-commercial) that is perfectly suited for our use-case. As a requisite a water-mark is displayed on the first page of the PDF and we must place prominent links to the Prince website at the places we intend to serve the PDF.
The full-document PDF is generated by rendering the Hugo site and converting
the resulting HTML with [Prince](https://www.princexml.com/). Prince supports
print-specific CSS, bookmarks, cross-reference page numbers, links, and the
JavaScript needed for equation rendering. Its [non-commercial
license](https://www.princexml.com/purchase/license_faq/#non-commercial) is
sufficient for local and project use, but it adds a watermark to generated
documents. A license suitable for public distribution is required before
publishing an unwatermarked PDF.

[Download Prince](https://www.princexml.com/download/): The minimum version for MathJax 3 support is `20210624`, so get the [latest release](https://www.princexml.com/latest/).

On a Windows environment it makes sense to add the location of the Prince executable, e.g. `C:\Program Files (x86)\Prince\engine\bin`, to the `PATH`.
Install Hugo Extended, Go, Python 3, `curl`, and Prince before running the
generator. The minimum Prince version for the MathJax compatibility script is
`20210624`; the latest Prince release is recommended.

## Build process

The bash script 'pdf-docs.sh' in root can be used to build the PDF. It goes through the following steps

### Build a web target with jekyll
The repository-root `pdf-docs.sh` script performs the complete build directly
with Hugo. It performs four steps:

First, let's kill all running instances of jekyll:
1. Hugo renders the PDF outputs and selected documentation pages into a
temporary directory.
2. Hugo generates a Prince input list from the documentation, tutorials, and
appendix sidebars.
3. A temporary local HTTP server serves the rendered HTML and static assets.
4. Prince reads the input list and writes the final PDF.

```bash
kill -9 $(ps aux | grep '[j]ekyll' | awk '{print $2}')
```
### Render the PDF target with Hugo

Then build the site locally with jekyll:
Run the script from the repository root:

```bash
bundle exec jekyll serve --detach --config _config.yml,pdfconfigs/config_docs_pdf.yml
```sh
./pdf-docs.sh
```

The `--detach` option is not strictly necessary, it detaches jekyll from the terminal. For debugging reasons it might actually be preferable to use two separate shells in parallel.

Note that in a Windows environment the `--detach` option will likely fail and that the `--config` option, which specifies a list of `config.yml`s to be used, needs to be enclosed in `"`:
Internally, the Hugo step is equivalent to:

```bash
bundle exec jekyll serve --config "_config.yml,pdfconfigs/config_docs_pdf.yml"
```sh
hugo --gc --minify --cleanDestinationDir \
--environment pdf \
--renderSegments pdf \
--destination public-pdf \
--baseURL http://127.0.0.1:4173/
```

At this point the site is up and can be accessed at `http://localhost:4000/`.

In the directory `pdfconfigs` there is a file called `prince-list.txt` that looks like this:

```liquid
{%raw%}{% for entry in sidebar %}
{% for folder in entry.folders %}
{% if folder.output contains "pdf" %}
{% for folderitem in folder.folderitems %}
{% if folderitem.output contains "pdf" %}

{{folderitem.url}}

{% endif %}
{% endfor %}
{% endif %}
{% endfor %}
{% endfor %}{%endraw%}
```
The `pdf` environment is defined in `config/pdf/hugo.toml`. It adds the title
page, table of contents, part-title pages, and Prince input-list output. The
`pdf` segment limits normal HTML output to the documentation and tutorial
sections included in the PDF.

I.e. we loop through every item in the appropriate `sidebar.yml`, check whether `output` is `pdf`, and if so print the item's URI. As a result, after running jekyll, `prince-list.txt` contains a list of URIs for all items in `sidebar.yml`:
### Generate the Prince input list

```txt
http://localhost:4000/titlepage.html
http://localhost:4000/tocpage.html
http://localhost:4000/docs.html
[...]
```
The list is generated at `public-pdf/pdf/prince-list.txt`; it must not be
edited manually. `layouts/partials/pdf/prince-list.txt` traverses the Hugo
sidebar data and includes only entries whose sidebar `output` contains `pdf`.
The current order is:

### Convert to PDF
1. title page;
2. table of contents;
3. documentation part page and documentation pages;
4. tutorials part page and tutorial pages; and
5. appendix part page and appendix pages.

This list is consumed by Prince and converted into PDF:
The generated list contains local absolute URLs, for example:

```bash
prince --javascript --raster-dpi=150 --input-list=_site/pdfconfigs/prince-list.txt -o pdf/docs.pdf
```text
http://127.0.0.1:4173/pdf/title.html
http://127.0.0.1:4173/pdf/toc.html
http://127.0.0.1:4173/docs/fundamentals/overview/
```

The final PDF can be found in `pdf/docs.pdf` as specified. The `--javascript` option enables JavaScript support.

## Styling
`layouts/partials/pdf/page-for-url.html` resolves sidebar URLs and aliases to
the corresponding Hugo page, so renamed pages can still be included.

The web target built by
### Convert the rendered pages to PDF

```bash
bundle exec jekyll serve --config "_config.yml,pdfconfigs/config_docs_pdf.yml"
```
After the temporary server is ready, the script invokes Prince with the
generated list:

is different from a usual build only in the way it specifies a second `config.yml` namely `pdfconfigs/config_docs_pdf.yml`. From a styling perspective the latter includes a different default layout of type `page_print`:

```yml
defaults:
-
scope:
path: ""
type: "pages"
values:
layout: "page_print"
comments: true
search: true
```sh
prince --javascript \
--input-list=public-pdf/pdf/prince-list.txt \
-o pdf/docs.pdf
```

This layout type includes a separate `<head></head>` in the form of `_includes/head_print.html`. Apart from some minor changes, e.g. by referencing resources such as stylesheets, scripts and fonts locally, it also points to `css/printstyles.css`. This is where styles specific to the PDF are stored.
The final document is written to `pdf/docs.pdf`. The server is stopped and its
temporary log is removed automatically when the script exits.

For further reference consult the [documentation of documentation-theme-jekyll](https://idratherbewriting.com/documentation-theme-jekyll/mydoc_generating_pdfs.html).
The output directory, output file, and local server port can be changed using
environment variables:

## Contents, title page and table of contents

The PDF will contain every page of type `pdf` that is referenced in the sidebars configured in `pdfconfigs/config_docs_pdf.yml`.
You can configure a single sidebar via `pdf_sidebar` or multiple sidebars via `pdf_sidebars`. E.g. if

```yml

pdf_sidebars:
- docs_sidebar
- tutorials_sidebar
```sh
PDF_BUILD_DIR=/tmp/precice-pdf \
PDF_OUTPUT=/tmp/precice-docs.pdf \
PDF_PORT=4174 \
./pdf-docs.sh
```

and `docs_sidebar.yml` is

```yml
- title: "A selection of fruits"
output: web, pdf
folderitems:

- title: Apples
url: /apples.html
output: web, pdf

- title: Oranges
url: /oranges.html
output: web
```

the PDF will contain `apples.html` but not `oranges.html`.

Furthermore two more pages have to be included in `prince-list.txt`:
## Styling

```html
{{site.url}}{{site.baseurl}}/titlepage.html
{{site.url}}{{site.baseurl}}/tocpage.html
```
When `params.output` is `pdf`, the base and content layouts use the print
markup instead of the normal navigation and sidebar wrappers. The print
templates and PDF-specific partials are in `layouts/partials/pdf/`, while
`layouts/partials/head_print.html` loads the local print resources.

These two pages are located in `pdfconfigs/` and govern the layout of the title page as well as the table of contents.
PDF-only styles are in `static/css/printstyles.css`. They define page headers
and footers, page counters, table-of-contents leaders, page breaks, image
limits, and print-friendly alert boxes. Do not edit minified vendor CSS for a
PDF-only adjustment; add a scoped rule to `printstyles.css` instead.

For further information, see the [Jekyll theme documentation](https://idratherbewriting.com/documentation-theme-jekyll/mydoc_generating_pdfs.html).
Prince compatibility scripts and print assets are served locally from
`static/js/`, `static/fonts/`, and `static/webfonts/`. Keeping these resources
in the generated site prevents the PDF build from depending on a browser or
on CDN availability.

## Troubleshooting and common issues
## Contents, title page, and table of contents

### Warnings by prince
The Hugo sidebars in `data/sidebars/` define the PDF contents. Add `pdf` to a
sidebar entry's `output` value when a page belongs in the document, for example
`output: web, pdf`. Entries without `pdf` remain available on the website but
are excluded from the PDF. The appendix uses
`data/sidebars/pdf_appendix_sidebar.yaml`.

Prince XML is different from a browser in the way it handles HTML/CSS more rigorously and will warn about every CSS property that are not 100% W3C compliant. Bootstrap, say, on the other hand, does make use of CSS hacks deliberately.
The title page, table of contents, and part-title pages are separate Hugo
outputs. The table of contents is built recursively from the sidebar folders,
preserving nested sections and the order defined in the data files.

Prince also consumes one page at a time and doesn't cache common resources (stylesheets, scripts, etc) like a browser.
## Troubleshooting

Due to these two points, taken together, Prince can output a long number of warnings in the conversion process.
### Prince warnings

### Make resources available locally
Prince validates CSS more strictly than a browser. Warnings about unsupported
Bootstrap properties, modern pseudo-classes, or browser-only media features
are expected when they do not affect the printed output. A missing resource,
failed URL, or non-zero Prince exit status requires investigation.

Because Prince consumes HTML pages one at a time, it is convenient to make resources (stylesheets, scripts, fonts etc.) available locally and not have them fetched from a CDN. For this purpose a separate `_includes/head_print.html` exists.
### Missing resources

The print head references the local Bootstrap stylesheet in `css/bootstrap.min.css`. Keep the matching `css/bootstrap.min.css.map` next to it, since the minified stylesheet references that source map. The regular web head may still load Bootstrap from a CDN, but the print head should stay self-contained so that PDF generation does not depend on network access.
Prince reads one URL at a time from the local server. Run the script rather
than opening generated HTML with a `file://` URL, and check that referenced
stylesheets, images, scripts, and fonts exist below `public-pdf/`.

### Missing glyphs or fonts

If Prince complains about missing glyphs or fonts make sure that the specified fonts are either available as resources (as a `*.ttf`, `*.woff` etc) or installed on the local machine. In our case, at the time of writing, this includes

* Fira Sans Light, Regular, Medium, Bold, Italic (in `./fonts`)
* Fira Mono Regular (in `./fonts`)
* Font Awesome 6.7.2 (in `./webfonts`)
* KaTeX fonts (in `./css/fonts`)

The location of these fonts has to be relative to where they are referenced, e.g. `css/fontawesome.6.7.2.all.min.css` mentions `url(../webfonts/fa-brands-400.woff2)`.

For further information see the [Prince documentation on missing fonts](https://www.princexml.com/doc/troubleshooting/).

### Troubleshooting

A useful hack is to modify `prince-list.txt` to only contain reference to a single html page, that can then be easily troubleshooted.

For further information see the [Prince documentation on troubleshooting](https://www.princexml.com/doc/troubleshooting/).

### Overriding Bootstrap print styles

Do not edit the minified Bootstrap vendor file directly. Bootstrap is vendored as `css/bootstrap.min.css` for PDF builds, and project-specific print overrides belong in `css/printstyles.css`. If a Bootstrap update changes PDF output, add the required scoped overrides there and replace `css/bootstrap.min.css` and `css/bootstrap.min.css.map` together.
If Prince reports missing glyphs, verify that the required Fira Sans, Fira
Mono, Font Awesome, and KaTeX resources are present in `static/fonts/`,
`static/webfonts/`, and `static/css/fonts/`. See the [Prince troubleshooting
documentation](https://www.princexml.com/doc/troubleshooting/) for details.

### KaTeX
### Debugging one page

We are using KaTeX for the rendering of LaTeX formulas in the documentation. In order for KaTeX to work with prince, a rendering script has been added in '_includes/head_print.html'.
To debug a specific page, copy its local URL into a temporary input list and
run Prince against that list. This avoids rebuilding the entire document while
investigating CSS, images, or equations.
13 changes: 13 additions & 0 deletions layouts/_default/baseof.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,18 @@ <h1>Redirecting&hellip;</h1>
<a href="{{ . | safeURL }}">Click here if you are not redirected.</a>
</body>
{{- else }}
{{- if eq (site.Params.output | default "web") "pdf" }}
<head>
{{ partial "head_print.html" . }}
</head>
<body class="{{ with .Params.type }}{{ . }} {{ end }}print">
<div class="container print-container">
<main class="print-content">
{{ block "main" . }}{{ end }}
</main>
</div>
</body>
{{- else }}
<head>
{{ partial "head.html" . }}
</head>
Expand Down Expand Up @@ -77,4 +89,5 @@ <h1>Redirecting&hellip;</h1>
</script>
</body>
{{- end }}
{{- end }}
</html>
2 changes: 1 addition & 1 deletion layouts/_default/default_print.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<body class="{{ with .Params.type }}{{ . }} {{ end }}print">
<div class="container print-container">
<main class="print-content">
{{ .Content }}
{{ partial "compat_content.html" . }}
</main>
</div>
</body>
Expand Down
Loading
Loading