Path: blob/main/documentation/content/en/books/fdp-primer/doc-build/_index.adoc
18099 views
---
title: Chapter 5. The FreeBSD Documentation Build Process
prev: books/fdp-primer/structure
next: books/fdp-primer/asciidoctor-primer
description: Describes the FreeBSD Documentation Build Process
tags: ["build", "process", "make"]
showBookMenu: true
weight: 6
params:
path: "/books/fdp-primer/doc-build/"
---
[[doc-build]]
= The FreeBSD Documentation Build Process
:doctype: book
:toc: macro
:toclevels: 1
:icons: font
:sectnums:
:sectnumlevels: 6
:sectnumoffset: 5
:partnums:
:source-highlighter: rouge
:experimental:
:images-path: books/fdp-primer/
ifdef::env-beastie[]
ifdef::backend-html5[]
:imagesdir: ../../../../images/{images-path}
endif::[]
ifndef::book[]
include::shared/authors.adoc[]
include::shared/mirrors.adoc[]
include::shared/releases.adoc[]
include::shared/attributes/attributes-{{% lang %}}.adoc[]
include::shared/{{% lang %}}/teams.adoc[]
include::shared/{{% lang %}}/mailing-lists.adoc[]
include::shared/{{% lang %}}/urls.adoc[]
toc::[]
endif::[]
ifdef::backend-pdf,backend-epub3[]
include::../../../../../shared/asciidoctor.adoc[]
endif::[]
endif::[]
ifndef::env-beastie[]
toc::[]
include::../../../../../shared/asciidoctor.adoc[]
endif::[]
This chapter covers organization of the documentation build process and how man:make[1] is used to control it.
[[doc-build-rendering]]
== Rendering AsciiDoc into Output
Different types of output can be produced from a single AsciiDoc source file.
[cols="20%,20%,60%", frame="none", options="header"]
|===
| Formats
| File Type
| Description
|`html`
|HTML
|An `article` or `book` chapter.
|`pdf`
|PDF
|Portable Document Format.
|`epub`
|EPUB
|Electronic Publication.
ePub file format.
|===
[[doc-build-rendering-html]]
=== Rendering to html
To render the documentation and the website to `html` use one of the following examples.
[[documentation-build-example]]
.Build the documentation
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make
....
====
[[website-build-example]]
.Build the website
[example]
====
[source,shell]
....
% cd ~/doc/website
% make
....
====
[[project-build-example]]
.Build the entire documentation project
[example]
====
[source,shell]
....
% cd ~/doc
% make -j2
....
====
Advanced build examples are given below:
[[documentation-build-example-verbose]]
.Build English and Spanish documentation with verbose and debug messages
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en es" HUGO_ARGS="--verbose --debug"
....
====
[[documentation-build-example-server]]
.Build and serve the content with Hugo’s internal webserver
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make run
....
This webserver runs on `localhost`, port `1313` by default.
To serve the content with Hugo’s internal webserver binding a specific IP address and port:
[source,shell]
....
% make run BIND=192.168.15.10 HUGO_ARGS="-p 8080"
....
A `hostname` can also be set as base url to Hugo’s internal webserver:
[source,shell]
....
% make run BIND=192.168.15.10 HOSTNAME=example.com
....
====
[[documentation-build-example-offline]]
.Build documentation in html for offline usage
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make html
....
To compress the html output, add `DOC_HTML_ARCHIVE=1`:
[source,shell]
....
% cd ~/doc/documentation
% DOC_HTML_ARCHIVE=1 make html
....
====
[[doc-build-rendering-pdf]]
=== Rendering to pdf
To render the documentation to `pdf`, use one of the following examples.
[[document-pdf-example]]
.Build all documents in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf
....
====
[[document-pdf-articles-example]]
.Build all articles in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf-articles
....
====
[[document-pdf-books-example]]
.Build all books in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf-books
....
====
[[document-pdf-language-example]]
.Build documents in pdf for specific languages
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en" pdf
....
This will build all English documents in pdf.
[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en fr" pdf-books
....
This will build all English and French books in pdf.
====
[[doc-build-toolset]]
== The FreeBSD Documentation Build Toolset
These are the tools used to build and install the FDP documentation.
* The primary build tool is man:make[1], specifically Berkeley Make.
* Hugo
* AsciiDoctor
* Git
[[doc-build-makefile]]
== Understanding the Makefile in the Documentation Tree
There are three [.filename]#Makefile# files for building some or all of the documentation project.
* The [.filename]#Makefile# in the [.filename]#documentation# directory will build only the documentation.
* The [.filename]#Makefile# in the [.filename]#website# directory will build only the website.
* The [.filename]#Makefile# at the top of the tree will build both the documentation and the website.
The [.filename]#Makefile# appearing in subdirectories also support `make run` to serve built content with Hugo's internal webserver.
This webserver runs on port 1313 by default.
[[documentation-makefile]]
=== Documentation Makefile
This [.filename]#Makefile# takes the following form:
[source,shell]
....
# Generate the FreeBSD documentation
#
# Copyright (c) 2020-2026, The FreeBSD Documentation Project
# Copyright (c) 2020-2026, Sergio Carlavilla <[email protected]>
#
# Targets intended for use on the command line
#
# all (default) - generate the books TOC and compile all the documentation
# clean - removes generated files
# run - serves the built documentation site for local browsing
# pdf - build PDF versions of the articles and books.
# html - build HTML versions of the articles and books for
# offline use.
# If variable DOC_HTML_ARCHIVE is set, all documents will be
# archived/compressed, and only these files will be kept in the public
# directory.
# epub - build EPUB versions of the articles and books (Experimental).
#
# The run target uses hugo's built-in webserver to make the documentation site
# available for local browsing. The documentation should have been built prior
# to attempting to use the `run` target. By default, hugo will start its
# webserver on port 1313.
[email protected] <.>
# List of languages without book translations
ARTICLEONLY_LANGS= bn-bd da ko tr
# List of languages without article translations
BOOKONLY_LANGS= mn
# List of all languages we have content for
ALL_LANGUAGES= bn-bd da de el en es fr hu it ja ko mn nl pl pt-br ru tr zh-cn zh-tw <.>
LOCALBASE?= /usr/local
RUBY_CMD = ${LOCALBASE}/bin/ruby <.>
HUGO_CMD = ${LOCALBASE}/bin/hugo <.>
HUGO_ARGS?= --verbose --minify
HUGO_OFFLINE_ARGS?= --environment offline --verbose --minify
ASCIIDOCTOR_CMD= ${LOCALBASE}/bin/asciidoctor
ASCIIDOCTORPDF_CMD= ${LOCALBASE}/bin/asciidoctor-pdf
.if defined(DOC_LANG) && !empty(DOC_LANG)
LANGUAGES= ${DOC_LANG:S/,/ /g}
.if ${LANGUAGES:Men} == "" && ${.TARGETS:Mpdf*} == "" && ${.TARGETS:Mhtml*} == ""
.warning "Warning: cannot skip 'en'; adding it back"
LANGUAGES+= en
.endif
.else
LANGUAGES= ${ALL_LANGUAGES}
.endif
RUBYLIB = ../shared/lib
.export RUBYLIB
RUN_DEPENDS= ${HUGO_CMD} \
${LOCALBASE}/bin/asciidoctor \
${LOCALBASE}/bin/rougify
.ifndef HOSTNAME
. ifdef BIND
.HOST=$(BIND)
. else
.HOST=localhost
. endif
.else
.HOST=$(HOSTNAME)
.endif
# Strip the languages with only articles from the list of languages we
# will use to build books.
BOOK_LANGS= ${LANGUAGES}
.for a in ${ARTICLEONLY_LANGS}
BOOK_LANGS:= ${BOOK_LANGS:N${a}}
.endfor
# Strip the languages with only books from the list of languages we
# will use to build articles.
ARTICLE_LANGS= ${LANGUAGES}
.for a in ${BOOKONLY_LANGS}
ARTICLE_LANGS:= ${ARTICLE_LANGS:N${a}}
.endfor
# Take the list of all languages, and take out the ones we have been
# asked for. We'll feed this to hugo.
SKIP_LANGS=
.for a in ${ALL_LANGUAGES}
.if ${LANGUAGES:M${a}} == ""
SKIP_LANGS+= ${a}
.endif
.endfor
.ORDER: all run <.>
.ORDER: requirements <.>
.ORDER: starting-message
.ORDER: starting-message build
.ORDER: build
all: requirements starting-message generate-pgpkeys-txt build
run: requirements starting-message generate-pgpkeys-txt run-local
# clean does not call pdf-clean as that is a subset of hugo-clean
clean: hugo-clean pgp-clean
requirements:
.for dep in ${RUN_DEPENDS}
.if !exists(${dep})
@(echo ${dep} not found, please run 'pkg install docproj'; exit 1)
.endif
.endfor
requirements-pdf:
.if !exists(${LOCALBASE}/bin/asciidoctor-pdf)
@(echo ${LOCALBASE}/bin/asciidoctor-pdf not found, please run 'pkg install rubygem-asciidoctor-pdf'; exit 1)
.endif
requirements-epub:
.if !exists(${LOCALBASE}/bin/asciidoctor-epub3)
@(echo ${LOCALBASE}/bin/asciidoctor-epub3 not found, please run 'pkg install rubygem-asciidoctor-epub3'; exit 1)
.endif
starting-message: .PHONY <.>
@echo ---------------------------------------------------------------
@echo Building the documentation
@echo included languages: ${LANGUAGES}
@echo excluded languages: ${SKIP_LANGS}
@echo ---------------------------------------------------------------
generate-pgpkeys-txt: static/pgpkeys/pgpkeys.txt
static/pgpkeys/pgpkeys.txt: static/pgpkeys/*key
${RUBY_CMD} ./tools/global-pgpkeys-creator.rb
run-local: .PHONY <.>
HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"
build: .PHONY <.>
HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}
build-offline: .PHONY
HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_OFFLINE_ARGS}
pgp-clean: .PHONY
rm -f static/pgpkeys/pgpkeys.txt
hugo-clean: .PHONY
rm -rf resources public
#
# PDF targets
# Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" pdf-books
#
pdf: pdf-articles pdf-books
pdf-books: requirements-pdf
.for _lang in ${BOOK_LANGS}
./tools/asciidoctor.sh books ${_lang} pdf
.endfor
pdf-articles: requirements-pdf
.for _lang in ${ARTICLE_LANGS}
./tools/asciidoctor.sh articles ${_lang} pdf
.endfor
pdf-clean: pdf-articles-clean pdf-books-clean
pdf-books-clean:
.for _lang in ${BOOK_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/books
-rmdir ${.CURDIR}/public/${_lang}
.endfor
-rmdir ${.CURDIR}/public/
pdf-articles-clean:
.for _lang in ${ARTICLE_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/articles
.if !exists(${.CURDIR}/public/${_lang}/books)
rm -fr ${.CURDIR}/public/${_lang}
.endif
.endfor
-rmdir ${.CURDIR}/public
#
# HTML targets
#
html: build-offline html-clean-global html-clean-articles html-clean-books html-archive html-archive-clean-files
html-clean: hugo-clean
html-clean-global:
rm -fr ${.CURDIR}/public/index.html
rm -rf pgpkeys js
html-clean-articles:
.for _lang in ${ARTICLE_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/index.html
rm -fr ${.CURDIR}/public/${_lang}/articles/index.html
.endfor
html-clean-books:
.for _lang in ${BOOK_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/books/index.html
.endfor
html-archive:
.if defined(DOC_HTML_ARCHIVE)
.for _lang in ${ARTICLE_LANGS}
./tools/asciidoctor.sh articles ${_lang} archive
.endfor
.for _lang in ${BOOK_LANGS}
./tools/asciidoctor.sh books ${_lang} archive
.endfor
.endif
html-archive-clean-files:
.if defined(DOC_HTML_ARCHIVE)
find ${.CURDIR}/public/ ! -name '*.pdf' ! -name '*.tar.gz' -type f -delete
find ${.CURDIR}/public/ -type d -empty -delete
.endif
#
# EPUB targets
# Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" epub-books
#
epub: epub-articles epub-books
epub-books: requirements-epub
@echo ---------------------------------------------------------------
@echo !!! EPUB output is experimental !!!
@echo
@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
@echo gaps where conversion is incomplete or unstyled.
@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
@echo ---------------------------------------------------------------
.for _lang in ${BOOK_LANGS}
./tools/asciidoctor.sh books ${_lang} epub
.endfor
epub-articles: requirements-epub
@echo ---------------------------------------------------------------
@echo !!! EPUB output is experimental !!!
@echo
@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
@echo gaps where conversion is incomplete or unstyled.
@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
@echo ---------------------------------------------------------------
.for _lang in ${ARTICLE_LANGS}
./tools/asciidoctor.sh articles ${_lang} epub
.endfor
epub-clean: epub-articles-clean epub-books-clean
epub-books-clean:
.for _lang in ${BOOK_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/books
-rmdir ${.CURDIR}/public/${_lang}
.endfor
-rmdir ${.CURDIR}/public/
epub-articles-clean:
.for _lang in ${ARTICLE_LANGS}
rm -fr ${.CURDIR}/public/${_lang}/articles
.if !exists(${.CURDIR}/public/${_lang}/books)
rm -fr ${.CURDIR}/public/${_lang}
.endif
.endfor
-rmdir ${.CURDIR}/public
....
<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
<.> `ALL_LANGUAGES` flag specifies in which languages the table of contents has to be generated.
<.> `RUBY_CMD` flag specifies the location of the Ruby binary.
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
<.> `.ORDER` directives are used to ensure multiple make jobs may run without problem.
<.> `all` target builds the documentation and puts the result in *~/doc/documentation/public*.
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
<.> `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
<.> `build` builds the documentation and puts the result in the *~/doc/documentation/public*.
[[website-makefile]]
=== Website Makefile
This [.filename]#Makefile# takes the form of:
[source,shell]
....
# Generate the FreeBSD website
#
# Copyright (c) 2020-2026, The FreeBSD Documentation Project
# Copyright (c) 2020-2026, Sergio Carlavilla <[email protected]>
#
# Targets intended for use on the command line
#
# all (default) - generate the releases.toml and compile all the website
# run - serves the built website for local browsing
#
# The run target uses hugo's built-in webserver to make the built website
# available for local browsing. The website should have been built prior
# to attempting to use the `run` target. By default, hugo will start its
# webserver on port 1313.
[email protected] <.>
# List of all languages we have content for
ALL_LANGUAGES= de el en es fr hu it ja nl ru tr zh-cn zh-tw
LOCALBASE?= /usr/local
RUBY_CMD = ${LOCALBASE}/bin/ruby <.>
HUGO_CMD = ${LOCALBASE}/bin/hugo <.>
HUGO_ARGS?= --verbose
RUBYLIB = ../shared/lib
.export RUBYLIB
.ifndef HOSTNAME
. ifdef BIND
.HOST=$(BIND)
. else
.HOST=localhost
. endif
.else
.HOST=$(HOSTNAME)
.endif
.if defined(DOC_LANG) && !empty(DOC_LANG)
LANGUAGES= ${DOC_LANG:S/,/ /g}
.if ${LANGUAGES:Men} == ""
.warning "Warning: cannot skip 'en'; adding it back"
LANGUAGES+= en
.endif
.else
LANGUAGES= ${ALL_LANGUAGES}
.endif
# Take the list of all languages, and take out the ones we have been
# asked for via DOC_LANG. We'll feed this to hugo.
SKIP_LANGS=
.for a in ${ALL_LANGUAGES}
.if ${LANGUAGES:M${a}} == ""
SKIP_LANGS+= ${a}
.endif
.endfor
.ORDER: all run <.>
.ORDER: starting-message generate-releases
.ORDER: starting-message build
.ORDER: generate-releases build
.ORDER: build post-build
.ORDER: post-build end-message
all: starting-message generate-releases build post-build end-message <.>
run: starting-message generate-releases run-local
clean: hugo-clean releases-clean
starting-message: .PHONY <.>
@echo "---------------------------------------------------------------"
@echo "Building the website started on $$(date)"
@echo " included languages: ${LANGUAGES}"
@echo " excluded languages: ${SKIP_LANGS}"
@echo "---------------------------------------------------------------"
end-message: .PHONY
@echo "---------------------------------------------------------------"
@echo "Building the website completed on $$(date)"
@echo "---------------------------------------------------------------"
generate-releases: data/releases.toml <.>
data/releases.toml:
${RUBY_CMD} ./tools/releases-toml.rb
run-local: .PHONY <.>
HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"
build: .PHONY <.>
HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}
post-build: cgi-permissions
cgi-permissions:
@chmod 555 ./public/cgi/*.cgi
hugo-clean:
rm -fr public resources
releases-clean:
rm -f data/releases.toml
....
<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
<.> `RUBY_CMD` flag specifies the location of the Ruby binary.
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
<.> `.ORDER` directives are used to ensure multiple make jobs may run without problem.
<.> `all` target builds the website and puts the result in *~/doc/website/public*.
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
<.> `generate-releases` calls the script used to convert from AsciiDoc variables to TOML variables.
With this conversion, the releases variables can be used in AsciiDoc and in the Hugo custom templates.
<.> `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
<.> `build` builds the website and puts the result in the *~/doc/website/public*.