blob: e8aed86eb43e1c6787fdc442a90be6a58eda38bf [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001documentation
2=============
3
4This is the directory that contains the Yocto Project documentation. The Yocto
Andrew Geisslerd1e89492021-02-12 15:35:20 -06005Project source repositories at https://git.yoctoproject.org/cgit.cgi have two
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006instances of the "documentation" directory. You should understand each of
7these instances.
8
9 poky/documentation - The directory within the poky Git repository containing
10 the set of Yocto Project manuals. When you clone the
11 poky Git repository, the documentation directory
12 contains the manuals. The state of the manuals in this
13 directory is guaranteed to reflect the latest Yocto
14 Project release. The manuals at the tip of this
15 directory will also likely contain most manual
16 development changes.
17
18 yocto-docs/documentation - The Git repository for the Yocto Project manuals.
19 This repository is where manual development
20 occurs. If you plan on contributing back to the
21 Yocto Project documentation, you should set up
22 a local Git repository based on this upstream
23 repository as follows:
24
25 git clone git://git.yoctoproject.org/yocto-docs
26
27 Changes and patches are first pushed to the
28 yocto-docs Git repository. Later, they make it
29 into the poky Git repository found at
30 git://git.yoctoproject.org/poky.
31
32Manual Organization
33===================
34
William A. Kennington IIIac69b482021-06-02 12:28:27 -070035Here the folders corresponding to individual manuals:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050036
Andrew Geissler5199d832021-09-24 16:47:35 -050037* overview-manual - Yocto Project Overview and Concepts Manual
38* sdk-manual - Yocto Project Software Development Kit (SDK) Developer's Guide.
39* bsp-guide - Yocto Project Board Support Package (BSP) Developer's Guide
40* dev-manual - Yocto Project Development Tasks Manual
41* kernel-dev - Yocto Project Linux Kernel Development Manual
42* ref-manual - Yocto Project Reference Manual
43* brief-yoctoprojectqs - Yocto Project Quick Start
44* profile-manual - Yocto Project Profiling and Tracing Manual
45* toaster-manual - Toaster User Manual
46* test-manual - Yocto Project Test Environment Manual
Patrick Williamsc124f4f2015-09-15 14:41:29 -050047
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050048Each folder is self-contained regarding content and figures.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050049
50If you want to find HTML versions of the Yocto Project manuals on the web,
Andrew Geissler5199d832021-09-24 16:47:35 -050051the current versions reside at https://docs.yoctoproject.org.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050052
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050053poky.yaml
54=========
Patrick Williamsc124f4f2015-09-15 14:41:29 -050055
56This file defines variables used for documentation production. The variables
57are used to define release pathnames, URLs for the published manuals, etc.
58
Andrew Geissler595f6302022-01-24 19:11:47 +000059standards.md
60============
61
62This file specifies some rules to follow when contributing to the documentation.
63
64template/
65=========
66
67Contains a template.svg, to make it easier to create consistent
68SVG diagrams.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050069
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050070Sphinx
71======
72
73The Yocto Project documentation was migrated from the original DocBook
74format to Sphinx based documentation for the Yocto Project 3.2
75release. This section will provide additional information related to
76the Sphinx migration, and guidelines for developers willing to
77contribute to the Yocto Project documentation.
78
79 Sphinx is a tool that makes it easy to create intelligent and
80 beautiful documentation, written by Georg Brandl and licensed under
81 the BSD license. It was originally created for the Python
82 documentation.
83
84Extensive documentation is available on the Sphinx website:
85https://www.sphinx-doc.org/en/master/. Sphinx is designed to be
86extensible thanks to the ability to write our own custom extensions,
87as Python modules, which will be executed during the generation of the
88documentation.
89
90Yocto Project documentation website
91===================================
92
Andrew Geissler09036742021-06-25 14:25:14 -050093The website hosting the Yocto Project documentation, can be found
94at: https://docs.yoctoproject.org/.
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050095
Andrew Geissler95ac1b82021-03-31 14:34:31 -050096The entire Yocto Project documentation, as well as the BitBake manual,
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050097is published on this website, including all previously released
98versions. A version switcher was added, as a drop-down menu on the top
99of the page to switch back and forth between the various versions of
100the current active Yocto Project releases.
101
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500102Transition pages have been added (as rst files) to show links to old
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500103versions of the Yocto Project documentation with links to each manual
104generated with DocBook.
105
106How to build the Yocto Project documentation
107============================================
108
109Sphinx is written in Python. While it might work with Python2, for
110obvious reasons, we will only support building the Yocto Project
111documentation with Python3.
112
113Sphinx might be available in your Linux distro packages repositories,
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500114however it is not recommended to use distro packages, as they might be
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500115old versions, especially if you are using an LTS version of your
Andrew Geissler9aee5002022-03-30 16:27:02 +0000116distro. The recommended method to install the latest versions of Sphinx
117and of its required dependencies is to use the Python Package Index (pip).
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500118
119To install all required packages run:
120
121 $ pip3 install sphinx sphinx_rtd_theme pyyaml
122
Andrew Geissler9aee5002022-03-30 16:27:02 +0000123To make sure you always have the latest versions of such packages, you
124should regularly run the same command with an added "--upgrade" option:
125
126 $ pip3 install --upgrade sphinx sphinx_rtd_theme pyyaml
127
Andrew Geisslereff27472021-10-29 15:35:00 -0500128Also install the "inkscape" package from your distribution.
129Inkscape is need to convert SVG graphics to PNG (for EPUB
130export) and to PDF (for PDF export).
131
Patrick Williams92b42cb2022-09-03 06:53:57 -0500132Additionally install "fncychap.sty" TeX font if you want to build PDFs. Debian
133and Ubuntu have it in "texlive-latex-extra" package while RedHat distributions
134and OpenSUSE have it in "texlive-fncychap" package for example.
135
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500136To build the documentation locally, run:
137
138 $ cd documentation
Andrew Geisslerf0343792020-11-18 10:42:21 -0600139 $ make html
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500140
141The resulting HTML index page will be _build/html/index.html, and you
142can browse your own copy of the locally generated documentation with
143your browser.
144
Andrew Geissler6ce62a22020-11-30 19:58:47 -0600145Alternatively, you can use Pipenv to automatically install all required
146dependencies in a virtual environment:
147
148 $ cd documentation
149 $ pipenv install
150 $ pipenv run make html
151
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500152Sphinx theme and CSS customization
153==================================
154
155The Yocto Project documentation is currently based on the "Read the
156Docs" Sphinx theme, with a few changes to make sure the look and feel
157of the project documentation is preserved.
158
159Most of the theme changes can be done using the file
160'sphinx-static/theme_overrides.css'. Most CSS changes in this file
161were inherited from the DocBook CSS stylesheets.
162
163Sphinx design guidelines and principles
164=======================================
165
166The initial Docbook to Sphinx migration was done with an automated
167tool called Pandoc (https://pandoc.org/). The tool produced some clean
168output markdown text files. After the initial automated conversion
169additional changes were done to fix up headings, images and links. In
170addition Sphinx has built in mechanisms (directives) which were used
171to replace similar functions implemented in Docbook such as glossary,
172variables substitutions, notes and references.
173
174Headings
175========
176
177The layout of the Yocto Project manuals is organized as follows
178
179 Book
180 Chapter
181 Section
Andrew Geissler9aee5002022-03-30 16:27:02 +0000182 Subsection
183 Subsubsection
184 Subsubsubsection
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500185
Andrew Geissler9aee5002022-03-30 16:27:02 +0000186Here are the heading styles that we use in the manuals:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500187
Andrew Geissler9aee5002022-03-30 16:27:02 +0000188 Book => overline ===
189 Chapter => overline ***
190 Section => ====
191 Subsection => ----
192 Subsubsection => ~~~~
193 Subsubsubsection => ^^^^
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500194
195With this proposal, we preserve the same TOCs between Sphinx and Docbook.
196
197Built-in glossary
198=================
199
200Sphinx has a glossary directive. From
201https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:
202
203 This directive must contain a reST definition list with terms and
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500204 definitions. It's then possible to refer to each definition through the
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500205 [https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
206 'term' role].
207
208So anywhere in any of the Yocto Project manuals, :term:`VAR` can be
209used to refer to an item from the glossary, and a link is created
210automatically. A general index of terms is also generated by Sphinx
211automatically.
212
213Global substitutions
214====================
215
216The Yocto Project documentation makes heavy use of global
217variables. In Docbook these variables are stored in the file
218poky.ent. This Docbook feature is not handled automatically with
219Pandoc. Sphinx has builtin support for substitutions
220(https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions),
221however there are important shortcomings. For example they cannot be
222used/nested inside code-block sections.
223
224A Sphinx extension was implemented to support variable substitutions
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500225to mimic the DocBook based documentation behavior. Variable
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500226substitutions are done while reading/parsing the .rst files. The
227pattern for variables substitutions is the same as with DocBook,
228e.g. `&VAR;`.
229
230The implementation of the extension can be found here in the file
231documentation/sphinx/yocto-vars.py, this extension is enabled by
232default when building the Yocto Project documentation. All variables
233are set in a file call poky.yaml, which was initially generated from
234poky.ent. The file was converted into YAML so that it is easier to
235process by the custom Sphinx extension (which is a Python module).
236
237For example, the following .rst content will produce the 'expected'
238content:
239
240 .. code-block::
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500241 $ mkdir poky-&DISTRO;
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500242 or
243 $ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;
244
245Variables can be nested, like it was the case for DocBook:
246
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600247 YOCTO_HOME_URL : "https://www.yoctoproject.org"
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500248 YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
249
250Note directive
251==============
252
253Sphinx has a builtin 'note' directive that produces clean Note section
254in the output file. There are various types of directives such as
255"attention", "caution", "danger", "error", "hint", "important", "tip",
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500256"warning", "admonition" that are supported, and additional directives
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500257can be added as Sphinx extension if needed.
258
259Figures
260=======
261
262The Yocto Project documentation has many figures/images. Sphinx has a
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500263'figure' directive which is straightforward to use. To include a
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500264figure in the body of the documentation:
265
266 .. image:: figures/YP-flow-diagram.png
267
268Links and References
269====================
270
271The following types of links can be used: links to other locations in
272the same document, to locations in other documents and to external
273websites.
274
275More information can be found here:
276https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.
277
Patrick Williams2390b1b2022-11-03 13:47:49 -0500278For external links, we use this syntax:
279`link text <link URL>`__
280
281instead of:
282`link text <link URL>`_
283
284Both syntaxes work, but the latter also creates a "link text" reference
285target which could conflict with other references with the same name.
286So, only use this variant when you wish to make multiple references
287to this link, reusing only the target name.
288
289See https://stackoverflow.com/questions/27420317/restructured-text-rst-http-links-underscore-vs-use
290
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500291Anchor (<#link>) links are forbidden as they are not checked by Sphinx during
292the build and may be broken without knowing about it.
293
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500294References
295==========
296
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500297The following extension is enabled by default:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500298sphinx.ext.autosectionlabel
299(https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).
300
301This extension allows you to refer sections by their titles. Note that
302autosectionlabel_prefix_document is enabled by default, so that we can
303insert references from any document.
304
305For example, to insert an HTML link to a section from
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500306documentation/manual/intro.rst, use:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500307
308 Please check this :ref:`manual/intro:Cross-References to Locations in the Same Document`
309
310Alternatively a custom text can be used instead of using the section
311text:
312
313 Please check this :ref:`section <manual/intro:Cross-References to Locations in the Same Document>`
314
315TIP: The following command can be used to dump all the references that
316 are defined in the project documentation:
317
318 python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv
319
320This dump contains all links and for each link it shows the default
321"Link Text" that Sphinx would use. If the default link text is not
322appropriate, a custom link text can be used in the ':ref:' directive.
323
324Extlinks
325========
326
327The sphinx.ext.extlinks extension is enabled by default
328(https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension),
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500329and it is configured with the 'extlinks' definitions in
330the 'documentation/conf.py' file:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500331
332 'yocto_home': ('https://yoctoproject.org%s', None),
333 'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
334 'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
335 'yocto_lists': ('https://lists.yoctoproject.org%s', None),
336 'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
337 'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
338 'yocto_docs': ('https://docs.yoctoproject.org%s', None),
339 'yocto_git': ('https://git.yoctoproject.org%s', None),
340 'oe_home': ('https://www.openembedded.org%s', None),
341 'oe_lists': ('https://lists.openembedded.org%s', None),
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500342 'oe_git': ('https://git.openembedded.org%s', None),
343 'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
344 'oe_layerindex': ('https://layers.openembedded.org%s', None),
345 'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500346
347It creates convenient shortcuts which can be used throughout the
348documentation rst files, as:
349
350 Please check this :yocto_wiki:`wiki page </Weekly_Status>`
351
352Intersphinx links
353=================
354
355The sphinx.ext.intersphinx extension is enabled by default
356(https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html),
357so that we can cross reference content from other Sphinx based
358documentation projects, such as the BitBake manual.
359
Andrew Geisslerfc113ea2023-03-31 09:59:46 -0500360References to the BitBake manual can directly be done:
Patrick Williams213cb262021-08-07 19:21:33 -0500361 - With a specific description instead of the section name:
Andrew Geisslerfc113ea2023-03-31 09:59:46 -0500362 :ref:`Azure Storage fetcher (az://) <bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`
Patrick Williams213cb262021-08-07 19:21:33 -0500363 - With the section name:
Andrew Geisslerfc113ea2023-03-31 09:59:46 -0500364 :ref:`bitbake-user-manual/bitbake-user-manual-intro:usage and syntax` option
365
366If you want to refer to an entire document (or chapter) in the BitBake manual,
367you have to use the ":doc:" macro with the "bitbake:" prefix:
368 - :doc:`BitBake User Manual <bitbake:index>`
369 - :doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata`" chapter
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500370
Patrick Williams213cb262021-08-07 19:21:33 -0500371Note that a reference to a variable (:term:`VARIABLE`) automatically points to
372the BitBake manual if the variable is not described in the Reference Manual's Variable Glossary.
373However, if you need to bypass this, you can explicitely refer to a description in the
374BitBake manual as follows:
375
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500376 :term:`bitbake:BB_NUMBER_PARSE_THREADS`
Andrew Geissler6ce62a22020-11-30 19:58:47 -0600377
Andrew Geisslerfc113ea2023-03-31 09:59:46 -0500378This would be the same if we had identical document filenames in
379both the Yocto Project and BitBake manuals:
380
381 :ref:`bitbake:directory/file:section title`
382
Andrew Geissler6ce62a22020-11-30 19:58:47 -0600383Submitting documentation changes
384================================
385
386Please see the top level README file in this repository for details of where
387to send patches.