blob: 2d9eb887b28152d3d16421c05687279558c4a577 [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001documentation
2=============
3
4This is the directory that contains the Yocto Project documentation. The Yocto
Andrew Geisslerd1e89492021-02-12 15:35:20 -06005Project source repositories at https://git.yoctoproject.org/cgit.cgi have two
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006instances of the "documentation" directory. You should understand each of
7these instances.
8
9 poky/documentation - The directory within the poky Git repository containing
10 the set of Yocto Project manuals. When you clone the
11 poky Git repository, the documentation directory
12 contains the manuals. The state of the manuals in this
13 directory is guaranteed to reflect the latest Yocto
14 Project release. The manuals at the tip of this
15 directory will also likely contain most manual
16 development changes.
17
18 yocto-docs/documentation - The Git repository for the Yocto Project manuals.
19 This repository is where manual development
20 occurs. If you plan on contributing back to the
21 Yocto Project documentation, you should set up
22 a local Git repository based on this upstream
23 repository as follows:
24
25 git clone git://git.yoctoproject.org/yocto-docs
26
27 Changes and patches are first pushed to the
28 yocto-docs Git repository. Later, they make it
29 into the poky Git repository found at
30 git://git.yoctoproject.org/poky.
31
32Manual Organization
33===================
34
William A. Kennington IIIac69b482021-06-02 12:28:27 -070035Here the folders corresponding to individual manuals:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050036
Andrew Geissler5199d832021-09-24 16:47:35 -050037* overview-manual - Yocto Project Overview and Concepts Manual
38* sdk-manual - Yocto Project Software Development Kit (SDK) Developer's Guide.
39* bsp-guide - Yocto Project Board Support Package (BSP) Developer's Guide
40* dev-manual - Yocto Project Development Tasks Manual
41* kernel-dev - Yocto Project Linux Kernel Development Manual
42* ref-manual - Yocto Project Reference Manual
43* brief-yoctoprojectqs - Yocto Project Quick Start
44* profile-manual - Yocto Project Profiling and Tracing Manual
45* toaster-manual - Toaster User Manual
46* test-manual - Yocto Project Test Environment Manual
Patrick Williamsc124f4f2015-09-15 14:41:29 -050047
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050048Each folder is self-contained regarding content and figures.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050049
50If you want to find HTML versions of the Yocto Project manuals on the web,
Andrew Geissler5199d832021-09-24 16:47:35 -050051the current versions reside at https://docs.yoctoproject.org.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050052
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050053poky.yaml
54=========
Patrick Williamsc124f4f2015-09-15 14:41:29 -050055
56This file defines variables used for documentation production. The variables
57are used to define release pathnames, URLs for the published manuals, etc.
58
59template
60========
61Contains various templates, fonts, and some old PNG files.
62
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050063Sphinx
64======
65
66The Yocto Project documentation was migrated from the original DocBook
67format to Sphinx based documentation for the Yocto Project 3.2
68release. This section will provide additional information related to
69the Sphinx migration, and guidelines for developers willing to
70contribute to the Yocto Project documentation.
71
72 Sphinx is a tool that makes it easy to create intelligent and
73 beautiful documentation, written by Georg Brandl and licensed under
74 the BSD license. It was originally created for the Python
75 documentation.
76
77Extensive documentation is available on the Sphinx website:
78https://www.sphinx-doc.org/en/master/. Sphinx is designed to be
79extensible thanks to the ability to write our own custom extensions,
80as Python modules, which will be executed during the generation of the
81documentation.
82
83Yocto Project documentation website
84===================================
85
Andrew Geissler09036742021-06-25 14:25:14 -050086The website hosting the Yocto Project documentation, can be found
87at: https://docs.yoctoproject.org/.
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050088
Andrew Geissler95ac1b82021-03-31 14:34:31 -050089The entire Yocto Project documentation, as well as the BitBake manual,
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050090is published on this website, including all previously released
91versions. A version switcher was added, as a drop-down menu on the top
92of the page to switch back and forth between the various versions of
93the current active Yocto Project releases.
94
Andrew Geissler95ac1b82021-03-31 14:34:31 -050095Transition pages have been added (as rst files) to show links to old
Andrew Geisslerc3d88e42020-10-02 09:45:00 -050096versions of the Yocto Project documentation with links to each manual
97generated with DocBook.
98
99How to build the Yocto Project documentation
100============================================
101
102Sphinx is written in Python. While it might work with Python2, for
103obvious reasons, we will only support building the Yocto Project
104documentation with Python3.
105
106Sphinx might be available in your Linux distro packages repositories,
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500107however it is not recommended to use distro packages, as they might be
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500108old versions, especially if you are using an LTS version of your
109distro. The recommended method to install Sphinx and all required
110dependencies is to use the Python Package Index (pip).
111
112To install all required packages run:
113
114 $ pip3 install sphinx sphinx_rtd_theme pyyaml
115
Andrew Geisslereff27472021-10-29 15:35:00 -0500116Also install the "inkscape" package from your distribution.
117Inkscape is need to convert SVG graphics to PNG (for EPUB
118export) and to PDF (for PDF export).
119
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500120To build the documentation locally, run:
121
122 $ cd documentation
Andrew Geisslerf0343792020-11-18 10:42:21 -0600123 $ make html
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500124
125The resulting HTML index page will be _build/html/index.html, and you
126can browse your own copy of the locally generated documentation with
127your browser.
128
Andrew Geissler6ce62a22020-11-30 19:58:47 -0600129Alternatively, you can use Pipenv to automatically install all required
130dependencies in a virtual environment:
131
132 $ cd documentation
133 $ pipenv install
134 $ pipenv run make html
135
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500136Sphinx theme and CSS customization
137==================================
138
139The Yocto Project documentation is currently based on the "Read the
140Docs" Sphinx theme, with a few changes to make sure the look and feel
141of the project documentation is preserved.
142
143Most of the theme changes can be done using the file
144'sphinx-static/theme_overrides.css'. Most CSS changes in this file
145were inherited from the DocBook CSS stylesheets.
146
147Sphinx design guidelines and principles
148=======================================
149
150The initial Docbook to Sphinx migration was done with an automated
151tool called Pandoc (https://pandoc.org/). The tool produced some clean
152output markdown text files. After the initial automated conversion
153additional changes were done to fix up headings, images and links. In
154addition Sphinx has built in mechanisms (directives) which were used
155to replace similar functions implemented in Docbook such as glossary,
156variables substitutions, notes and references.
157
158Headings
159========
160
161The layout of the Yocto Project manuals is organized as follows
162
163 Book
164 Chapter
165 Section
166 Section
167 Section
168
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700169Here are the heading styles defined in Sphinx:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500170
171 Book => overline ===
172 Chapter => overline ***
173 Section => ====
174 Section => ----
175 Section => ^^^^
176 Section => """" or ~~~~
177
178With this proposal, we preserve the same TOCs between Sphinx and Docbook.
179
180Built-in glossary
181=================
182
183Sphinx has a glossary directive. From
184https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:
185
186 This directive must contain a reST definition list with terms and
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500187 definitions. It's then possible to refer to each definition through the
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500188 [https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
189 'term' role].
190
191So anywhere in any of the Yocto Project manuals, :term:`VAR` can be
192used to refer to an item from the glossary, and a link is created
193automatically. A general index of terms is also generated by Sphinx
194automatically.
195
196Global substitutions
197====================
198
199The Yocto Project documentation makes heavy use of global
200variables. In Docbook these variables are stored in the file
201poky.ent. This Docbook feature is not handled automatically with
202Pandoc. Sphinx has builtin support for substitutions
203(https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions),
204however there are important shortcomings. For example they cannot be
205used/nested inside code-block sections.
206
207A Sphinx extension was implemented to support variable substitutions
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500208to mimic the DocBook based documentation behavior. Variable
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500209substitutions are done while reading/parsing the .rst files. The
210pattern for variables substitutions is the same as with DocBook,
211e.g. `&VAR;`.
212
213The implementation of the extension can be found here in the file
214documentation/sphinx/yocto-vars.py, this extension is enabled by
215default when building the Yocto Project documentation. All variables
216are set in a file call poky.yaml, which was initially generated from
217poky.ent. The file was converted into YAML so that it is easier to
218process by the custom Sphinx extension (which is a Python module).
219
220For example, the following .rst content will produce the 'expected'
221content:
222
223 .. code-block::
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500224 $ mkdir poky-&DISTRO;
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500225 or
226 $ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;
227
228Variables can be nested, like it was the case for DocBook:
229
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600230 YOCTO_HOME_URL : "https://www.yoctoproject.org"
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500231 YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
232
233Note directive
234==============
235
236Sphinx has a builtin 'note' directive that produces clean Note section
237in the output file. There are various types of directives such as
238"attention", "caution", "danger", "error", "hint", "important", "tip",
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500239"warning", "admonition" that are supported, and additional directives
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500240can be added as Sphinx extension if needed.
241
242Figures
243=======
244
245The Yocto Project documentation has many figures/images. Sphinx has a
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500246'figure' directive which is straightforward to use. To include a
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500247figure in the body of the documentation:
248
249 .. image:: figures/YP-flow-diagram.png
250
251Links and References
252====================
253
254The following types of links can be used: links to other locations in
255the same document, to locations in other documents and to external
256websites.
257
258More information can be found here:
259https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.
260
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500261Anchor (<#link>) links are forbidden as they are not checked by Sphinx during
262the build and may be broken without knowing about it.
263
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500264References
265==========
266
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500267The following extension is enabled by default:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500268sphinx.ext.autosectionlabel
269(https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).
270
271This extension allows you to refer sections by their titles. Note that
272autosectionlabel_prefix_document is enabled by default, so that we can
273insert references from any document.
274
275For example, to insert an HTML link to a section from
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500276documentation/manual/intro.rst, use:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500277
278 Please check this :ref:`manual/intro:Cross-References to Locations in the Same Document`
279
280Alternatively a custom text can be used instead of using the section
281text:
282
283 Please check this :ref:`section <manual/intro:Cross-References to Locations in the Same Document>`
284
285TIP: The following command can be used to dump all the references that
286 are defined in the project documentation:
287
288 python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv
289
290This dump contains all links and for each link it shows the default
291"Link Text" that Sphinx would use. If the default link text is not
292appropriate, a custom link text can be used in the ':ref:' directive.
293
294Extlinks
295========
296
297The sphinx.ext.extlinks extension is enabled by default
298(https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension),
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500299and it is configured with the 'extlinks' definitions in
300the 'documentation/conf.py' file:
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500301
302 'yocto_home': ('https://yoctoproject.org%s', None),
303 'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
304 'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
305 'yocto_lists': ('https://lists.yoctoproject.org%s', None),
306 'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
307 'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
308 'yocto_docs': ('https://docs.yoctoproject.org%s', None),
309 'yocto_git': ('https://git.yoctoproject.org%s', None),
310 'oe_home': ('https://www.openembedded.org%s', None),
311 'oe_lists': ('https://lists.openembedded.org%s', None),
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500312 'oe_git': ('https://git.openembedded.org%s', None),
313 'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
314 'oe_layerindex': ('https://layers.openembedded.org%s', None),
315 'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500316
317It creates convenient shortcuts which can be used throughout the
318documentation rst files, as:
319
320 Please check this :yocto_wiki:`wiki page </Weekly_Status>`
321
322Intersphinx links
323=================
324
325The sphinx.ext.intersphinx extension is enabled by default
326(https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html),
327so that we can cross reference content from other Sphinx based
328documentation projects, such as the BitBake manual.
329
Patrick Williams213cb262021-08-07 19:21:33 -0500330References to the BitBake manual can be done:
331 - With a specific description instead of the section name:
332 :ref:`Azure Storage fetcher (az://) <bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`
333 - With the section name:
334 ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax` option
335 - Linking to the entire BitBake manual:
336 :doc:`BitBake User Manual <bitbake:index>`
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500337
Patrick Williams213cb262021-08-07 19:21:33 -0500338Note that a reference to a variable (:term:`VARIABLE`) automatically points to
339the BitBake manual if the variable is not described in the Reference Manual's Variable Glossary.
340However, if you need to bypass this, you can explicitely refer to a description in the
341BitBake manual as follows:
342
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500343 :term:`bitbake:BB_NUMBER_PARSE_THREADS`
Andrew Geissler6ce62a22020-11-30 19:58:47 -0600344
345Submitting documentation changes
346================================
347
348Please see the top level README file in this repository for details of where
349to send patches.