blob: a0fbb4bfd153e90192552654e7a9ba2301f2e779 [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='dev-manual-newbie'>
6
7<title>The Yocto Project Open Source Development Environment</title>
8
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009<section id="usingpoky-changes-collaborate">
Brad Bishopd7bf8c12018-02-25 22:55:05 -050010 <title>Setting Up a Team Yocto Project Development Environment</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011
12 <para>
13 It might not be immediately clear how you can use the Yocto
Brad Bishopd7bf8c12018-02-25 22:55:05 -050014 Project in a team development environment, or scale it for a large
15 team of developers.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050016 One of the strengths of the Yocto Project is that it is extremely
17 flexible.
18 Thus, you can adapt it to many different use cases and scenarios.
19 However, these characteristics can cause a struggle if you are trying
20 to create a working setup that scales across a large team.
21 </para>
22
23 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -050024 To help you understand how to set up this type of environment,
25 this section presents a procedure that gives you the information
26 to learn how to get the results you want.
27 The procedure is high-level and presents some of the project's most
28 successful experiences, practices, solutions, and available
29 technologies that work well.
30 Keep in mind, the procedure here is a starting point.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050031 You can build off it and customize it to fit any
32 particular working environment and set of practices.
Brad Bishopd7bf8c12018-02-25 22:55:05 -050033 <orderedlist>
34 <listitem><para>
35 <emphasis>Determine Who is Going to be Developing:</emphasis>
36 You need to understand who is going to be doing anything
37 related to the Yocto Project and what their roles would be.
38 Making this determination is essential to completing the
39 steps two and three, which are to get your equipment together
40 and set up your development environment's hardware topology.
41 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050042
Brad Bishopd7bf8c12018-02-25 22:55:05 -050043 <para>The following roles exist:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050044 <itemizedlist>
Brad Bishopd7bf8c12018-02-25 22:55:05 -050045 <listitem><para>
46 <emphasis>Application Development:</emphasis>
47 These types of developers do application level work
48 on top of an existing software stack.
49 </para></listitem>
50 <listitem><para>
51 <emphasis>Core System Development:</emphasis>
52 These types of developers work on the contents of the
53 operating system image itself.
54 </para></listitem>
55 <listitem><para>
56 <emphasis>Build Engineer:</emphasis>
57 This type of developer manages Autobuilders and
58 releases.
59 Not all environments need a Build Engineer.
60 </para></listitem>
61 <listitem><para>
62 <emphasis>Test Engineer:</emphasis>
63 This type of developer creates and manages automated
64 tests needed to ensure all application and core
65 system development meets desired quality standards.
66 </para></listitem>
67 </itemizedlist>
68 </para></listitem>
69 <listitem><para>
70 <emphasis>Gather the Hardware:</emphasis>
71 Based on the size and make-up of the team, get the hardware
72 together.
73 Any development, build, or test engineer should be using
74 a system that is running a supported Linux distribution.
75 Systems, in general, should be high performance (e.g. dual,
76 six-core Xeons with 24 Gbytes of RAM and plenty of disk space).
77 You can help ensure efficiency by having any machines used
78 for testing or that run Autobuilders be as high performance
79 as possible.
80 </para></listitem>
81 <listitem><para>
82 <emphasis>Understand the Hardware Topology of the Environment:</emphasis>
83 Now that you know how many developers and support engineers
84 are required, you can understand the topology of the
85 hardware environment.
86 The following figure shows a moderately sized Yocto Project
87 development environment.
88
89 <para role="writernotes">
90 Need figure.</para>
91
92 </para></listitem>
93 <listitem><para>
94 <emphasis>Use Git as Your Source Control Manager (SCM):</emphasis>
95 Keeping your
96 <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>
97 and any software you are developing under the
98 control of an SCM system that is compatible
99 with the OpenEmbedded build system is advisable.
100 Of the SCMs BitBake supports, the
101 Yocto Project team strongly recommends using
102 <ulink url='&YOCTO_DOCS_REF_URL;#git'>Git</ulink>.
103 Git is a distributed system that is easy to backup,
104 allows you to work remotely, and then connects back to the
105 infrastructure.
106 <note>
107 For information about BitBake, see the
108 <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
109 </note></para>
110
111 <para>It is relatively easy to set up Git services and create
112 infrastructure like
113 <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>,
114 which is based on server software called
115 <filename>gitolite</filename> with <filename>cgit</filename>
116 being used to generate the web interface that lets you view the
117 repositories.
118 The <filename>gitolite</filename> software identifies users
119 using SSH keys and allows branch-based
120 access controls to repositories that you can control as little
121 or as much as necessary.
122
123 <note>
124 The setup of these services is beyond the scope of this
125 manual.
126 However, sites such as these exist that describe how to
127 perform setup:
128 <itemizedlist>
129 <listitem><para>
130 <ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>:
131 Describes how to install <filename>gitolite</filename>
132 on the server.
133 </para></listitem>
134 <listitem><para>
135 <ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
136 All topics for <filename>gitolite</filename>.
137 </para></listitem>
138 <listitem><para>
139 <ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
140 Documentation on how to create interfaces and frontends
141 for Git.
142 </para></listitem>
143 </itemizedlist>
144 </note>
145 </para></listitem>
146 <listitem><para>
147 <emphasis>Set up the Application Development Machines:</emphasis>
148 As mentioned earlier, application developers are creating
149 applications on top of existing software stacks.
150 Following are some best practices for setting up machines
151 that do application development:
152 <itemizedlist>
153 <listitem><para>
154 Use a pre-built toolchain that
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500155 contains the software stack itself.
156 Then, develop the application code on top of the
157 stack.
158 This method works well for small numbers of relatively
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500159 isolated applications.
160 </para></listitem>
161 <listitem><para>
162 When possible, use the Yocto Project
163 plug-in for the
164 <trademark class='trade'>Eclipse</trademark> IDE
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500165 and SDK development practices.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500166 For more information, see the
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500167 "<ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>"
168 manual.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500169 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500170 <listitem><para>
171 Keep your cross-development toolchains updated.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500172 You can do this through provisioning either as new
173 toolchain downloads or as updates through a package
174 update mechanism using <filename>opkg</filename>
175 to provide updates to an existing toolchain.
176 The exact mechanics of how and when to do this are a
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500177 question for local policy.
178 </para></listitem>
179 <listitem><para>
180 Use multiple toolchains installed locally
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500181 into different locations to allow development across
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500182 versions.
183 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500184 </itemizedlist>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500185 </para></listitem>
186 <listitem><para>
187 <emphasis>Set up the Core Development Machines:</emphasis>
188 As mentioned earlier, these types of developers work on the
189 contents of the operating system itself.
190 Following are some best practices for setting up machines
191 used for developing images:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500192 <itemizedlist>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500193 <listitem><para>
194 Have the Yocto Project build system itself available on
195 the developer workstations so developers can run their own
196 builds and directly rebuild the software stack.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500197 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500198 <listitem><para>
199 Keep the core system unchanged as much as
200 possible and do your work in layers on top of the
201 core system.
202 Doing so gives you a greater level of portability when
203 upgrading to new versions of the core system or Board
204 Support Packages (BSPs).
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500205 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500206 <listitem><para>
207 Share layers amongst the developers of a
208 particular project and contain the policy configuration
209 that defines the project.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500210 </para></listitem>
211 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500212 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500213 <listitem><para>
214 <emphasis>Set up an Autobuilder:</emphasis>
215 Autobuilders are often the core of the development
216 environment.
217 It is here that changes from individual developers are brought
218 together and centrally tested and subsequent decisions about
219 releases can be made.
220 Autobuilders also allow for "continuous integration" style
221 testing of software components and regression identification
222 and tracking.</para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500223
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500224 <para>See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
225 for more information and links to buildbot.
226 The Yocto Project team has found this implementation
227 works well in this role.
228 A public example of this is the Yocto Project
229 Autobuilders, which we use to test the overall health of the
230 project.</para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500231
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500232 <para>The features of this system are:
233 <itemizedlist>
234 <listitem><para>
235 Highlights when commits break the build.
236 </para></listitem>
237 <listitem><para>
238 Populates an sstate cache from which
239 developers can pull rather than requiring local
240 builds.
241 </para></listitem>
242 <listitem><para>
243 Allows commit hook triggers,
244 which trigger builds when commits are made.
245 </para></listitem>
246 <listitem><para>
247 Allows triggering of automated image booting
248 and testing under the QuickEMUlator (QEMU).
249 </para></listitem>
250 <listitem><para>
251 Supports incremental build testing and
252 from-scratch builds.
253 </para></listitem>
254 <listitem><para>
255 Shares output that allows developer
256 testing and historical regression investigation.
257 </para></listitem>
258 <listitem><para>
259 Creates output that can be used for releases.
260 </para></listitem>
261 <listitem><para>
262 Allows scheduling of builds so that resources
263 can be used efficiently.
264 </para></listitem>
265 </itemizedlist>
266 </para></listitem>
267 <listitem><para>
268 <emphasis>Set up Test Machines:</emphasis>
269 Use a small number of shared, high performance systems
270 for testing purposes.
271 Developers can use these systems for wider, more
272 extensive testing while they continue to develop
273 locally using their primary development system.
274 </para></listitem>
275 <listitem><para>
276 <emphasis>Document Policies and Change Flow:</emphasis>
277 The Yocto Project itself uses a hierarchical structure and a
278 pull model.
279 Scripts exist to create and send pull requests
280 (i.e. <filename>create-pull-request</filename> and
281 <filename>send-pull-request</filename>).
282 This model is in line with other open source projects where
283 maintainers are responsible for specific areas of the project
284 and a single maintainer handles the final "top-of-tree" merges.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500285 <note>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500286 You can also use a more collective push model.
287 The <filename>gitolite</filename> software supports both the
288 push and pull models quite easily.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500289 </note></para>
290
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500291 <para>As with any development environment, it is important
292 to document the policy used as well as any main project
293 guidelines so they are understood by everyone.
294 It is also a good idea to have well structured
295 commit messages, which are usually a part of a project's
296 guidelines.
297 Good commit messages are essential when looking back in time and
298 trying to understand why changes were made.</para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500299
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500300 <para>If you discover that changes are needed to the core
301 layer of the project, it is worth sharing those with the
302 community as soon as possible.
303 Chances are if you have discovered the need for changes,
304 someone else in the community needs them also.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500305 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500306 <listitem><para>
307 <emphasis>Development Environment Summary:</emphasis>
308 Aside from the previous steps, some best practices exist
309 within the Yocto Project development environment.
310 Consider the following:
311 <itemizedlist>
312 <listitem><para>
313 Use <ulink url='&YOCTO_DOCS_REF_URL;#git'>Git</ulink>
314 as the source control system.
315 </para></listitem>
316 <listitem><para>
317 Maintain your Metadata in layers that make sense
318 for your situation.
319 See the "<link linkend='understanding-and-creating-layers'>Understanding
320 and Creating Layers</link>" section for more information on
321 layers.
322 </para></listitem>
323 <listitem><para>
324 Separate the project's Metadata and code by using
325 separate Git repositories.
326 See the
327 "<ulink url='&YOCTO_DOCS_REF_URL;#yocto-project-repositories'>Yocto Project Source Repositories</ulink>"
328 section for information on these repositories.
329 See the
330 "<link linkend='working-with-yocto-project-source-files'>Working With Yocto Project Source Files</link>"
331 section for information on how to set up local Git
332 repositories for related upstream Yocto Project
333 Git repositories.
334 </para></listitem>
335 <listitem><para>
336 Set up the directory for the shared state cache
337 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
338 where it makes sense.
339 For example, set up the sstate cache on a system used
340 by developers in the same organization and share the
341 same source directories on their machines.
342 </para></listitem>
343 <listitem><para>
344 Set up an Autobuilder and have it populate the
345 sstate cache and source directories.
346 </para></listitem>
347 <listitem><para>
348 The Yocto Project community encourages you
349 to send patches to the project to fix bugs or add features.
350 If you do submit patches, follow the project commit
351 guidelines for writing good commit messages.
352 See the "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>"
353 section.
354 </para></listitem>
355 <listitem><para>
356 Send changes to the core sooner than later
357 as others are likely to run into the same issues.
358 For some guidance on mailing lists to use, see the list in the
359 "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>"
360 section.
361 For a description of the available mailing lists, see the
362 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
363 section in the Yocto Project Reference Manual.
364 </para></listitem>
365 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500366 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500367 </orderedlist>
368 </para>
369</section>
370
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500371<section id='submitting-a-defect-against-the-yocto-project'>
372 <title>Submitting a Defect Against the Yocto Project</title>
373
374 <para>
375 Use the Yocto Project implementation of
376 <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink>
377 to submit a defect (bug) against the Yocto Project.
378 For additional information on this implementation of Bugzilla see the
379 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-bugtracker'>Yocto Project Bugzilla</ulink>"
380 section in the Yocto Project Reference Manual.
381 For more detail on any of the following steps, see the Yocto Project
382 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>Bugzilla wiki page</ulink>.
383 </para>
384
385 <para>
386 Use the following general steps to submit a bug"
387
388 <orderedlist>
389 <listitem><para>
390 Open the Yocto Project implementation of
391 <ulink url='&YOCTO_BUGZILLA_URL;'>Bugzilla</ulink>.
392 </para></listitem>
393 <listitem><para>
394 Click "File a Bug" to enter a new bug.
395 </para></listitem>
396 <listitem><para>
397 Choose the appropriate "Classification", "Product", and
398 "Component" for which the bug was found.
399 Bugs for the Yocto Project fall into one of several
400 classifications, which in turn break down into several
401 products and components.
402 For example, for a bug against the
403 <filename>meta-intel</filename> layer, you would choose
404 "Build System, Metadata &amp; Runtime", "BSPs", and
405 "bsps-meta-intel", respectively.
406 </para></listitem>
407 <listitem><para>
408 Choose the "Version" of the Yocto Project for which you found
409 the bug (e.g. &DISTRO;).
410 </para></listitem>
411 <listitem><para>
412 Determine and select the "Severity" of the bug.
413 The severity indicates how the bug impacted your work.
414 </para></listitem>
415 <listitem><para>
416 Choose the "Hardware" that the bug impacts.
417 </para></listitem>
418 <listitem><para>
419 Choose the "Architecture" that the bug impacts.
420 </para></listitem>
421 <listitem><para>
422 Choose a "Documentation change" item for the bug.
423 Fixing a bug might or might not affect the Yocto Project
424 documentation.
425 If you are unsure of the impact to the documentation, select
426 "Don't Know".
427 </para></listitem>
428 <listitem><para>
429 Provide a brief "Summary" of the bug.
430 Try to limit your summary to just a line or two and be sure
431 to capture the essence of the bug.
432 </para></listitem>
433 <listitem><para>
434 Provide a detailed "Description" of the bug.
435 You should provide as much detail as you can about the context,
436 behavior, output, and so forth that surrounds the bug.
437 You can even attach supporting files for output from logs by
438 using the "Add an attachment" button.
439 </para></listitem>
440 <listitem><para>
441 Click the "Submit Bug" button submit the bug.
442 A new Bugzilla number is assigned to the bug and the defect
443 is logged in the bug tracking system.
444 </para></listitem>
445 </orderedlist>
446 Once you file a bug, the bug is processed by the Yocto Project Bug
447 Triage Team and further details concerning the bug are assigned
448 (e.g. priority and owner).
449 You are the "Submitter" of the bug and any further categorization,
450 progress, or comments on the bug result in Bugzilla sending you an
451 automated email concerning the particular change or progress to the
452 bug.
453 </para>
454</section>
455
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500456<section id='how-to-submit-a-change'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500457 <title>Submitting a Change to the Yocto Project</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500458
459 <para>
460 Contributions to the Yocto Project and OpenEmbedded are very welcome.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500461 Because the system is extremely configurable and flexible, we recognize
462 that developers will want to extend, configure or optimize it for
463 their specific uses.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500464 </para>
465
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500466 <para>
467 The Yocto Project uses a mailing list and a patch-based workflow
468 that is similar to the Linux kernel but contains important
469 differences.
470 In general, a mailing list exists through which you can submit
471 patches.
472 You should send patches to the appropriate mailing list so that they
473 can be reviewed and merged by the appropriate maintainer.
474 The specific mailing list you need to use depends on the
475 location of the code you are changing.
476 Each component (e.g. layer) should have a
477 <filename>README</filename> file that indicates where to send
478 the changes and which process to follow.
479 </para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500480
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500481 <para>
482 You can send the patch to the mailing list using whichever approach
483 you feel comfortable with to generate the patch.
484 Once sent, the patch is usually reviewed by the community at large.
485 If somebody has concerns with the patch, they will usually voice
486 their concern over the mailing list.
487 If a patch does not receive any negative reviews, the maintainer of
488 the affected layer typically takes the patch, tests it, and then
489 based on successful testing, merges the patch.
490 </para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500491
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500492 <para id='figuring-out-the-mailing-list-to-use'>
493 The "poky" repository, which is the Yocto Project's reference build
494 environment, is a hybrid repository that contains several
495 individual pieces (e.g. BitBake, Metadata, documentation,
496 and so forth) built using the combo-layer tool.
497 The upstream location used for submitting changes varies by
498 component:
499 <itemizedlist>
500 <listitem><para>
501 <emphasis>Core Metadata:</emphasis>
502 Send your patch to the
503 <ulink url='http://lists.openembedded.org/mailman/listinfo/openembedded-core'>openembedded-core</ulink>
504 mailing list. For example, a change to anything under
505 the <filename>meta</filename> or
506 <filename>scripts</filename> directories should be sent
507 to this mailing list.
508 </para></listitem>
509 <listitem><para>
510 <emphasis>BitBake:</emphasis>
511 For changes to BitBake (i.e. anything under the
512 <filename>bitbake</filename> directory), send your patch
513 to the
514 <ulink url='http://lists.openembedded.org/mailman/listinfo/bitbake-devel'>bitbake-devel</ulink>
515 mailing list.
516 </para></listitem>
517 <listitem><para>
518 <emphasis>"meta-*" trees:</emphasis>
519 These trees contain Metadata.
520 Use the
521 <ulink url='https://lists.yoctoproject.org/listinfo/poky'>poky</ulink>
522 mailing list.
523 </para></listitem>
524 </itemizedlist>
525 </para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500526
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500527 <para>
528 For changes to other layers hosted in the Yocto Project source
529 repositories (i.e. <filename>yoctoproject.org</filename>), tools,
530 and the Yocto Project documentation, use the
531 <ulink url='https://lists.yoctoproject.org/listinfo/yocto'>Yocto Project</ulink>
532 general mailing list.
533 <note>
534 Sometimes a layer's documentation specifies to use a
535 particular mailing list.
536 If so, use that list.
537 </note>
538 For additional recipes that do not fit into the core Metadata, you
539 should determine which layer the recipe should go into and submit
540 the change in the manner recommended by the documentation (e.g.
541 the <filename>README</filename> file) supplied with the layer.
542 If in doubt, please ask on the Yocto general mailing list or on
543 the openembedded-devel mailing list.
544 </para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500545
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500546 <para>
547 You can also push a change upstream and request a maintainer to
548 pull the change into the component's upstream repository.
549 You do this by pushing to a contribution repository that is upstream.
550 See the
551 "<ulink url='&YOCTO_DOCS_REF_URL;#workflows'>Workflows</ulink>"
552 section in the Yocto Project Reference Manual for additional
553 concepts on working in the Yocto Project development environment.
554 </para>
555
556 <para>
557 Two commonly used testing repositories exist for
558 OpenEmbedded-Core:
559 <itemizedlist>
560 <listitem><para>
561 <emphasis>"ross/mut" branch:</emphasis>
562 The "mut" (master-under-test) tree
563 exists in the <filename>poky-contrib</filename> repository
564 in the
565 <ulink url='&YOCTO_GIT_URL;'>Yocto Project source repositories</ulink>.
566 </para></listitem>
567 <listitem><para>
568 <emphasis>"master-next" branch:</emphasis>
569 This branch is part of the main
570 "poky" repository in the Yocto Project source repositories.
571 </para></listitem>
572 </itemizedlist>
573 Maintainers use these branches to test submissions prior to merging
574 patches.
575 Thus, you can get an idea of the status of a patch based on
576 whether the patch has been merged into one of these branches.
577 <note>
578 This system is imperfect and changes can sometimes get lost in the
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500579 flow.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500580 Asking about the status of a patch or change is reasonable if the
581 change has been idle for a while with no feedback.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500582 The Yocto Project does have plans to use
583 <ulink url='https://en.wikipedia.org/wiki/Patchwork_(software)'>Patchwork</ulink>
584 to track the status of patches and also to automatically preview
585 patches.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500586 </note>
587 </para>
588
589 <para>
590 The following sections provide procedures for submitting a change.
591 </para>
592
593 <section id='pushing-a-change-upstream'>
594 <title>Using Scripts to Push a Change Upstream and Request a Pull</title>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500595
596 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500597 Follow this procedure to push a change to an upstream "contrib"
598 Git repository:
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500599 <note>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500600 You can find general Git information on how to push a change
601 upstream in the
602 <ulink url='http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows'>Git Community Book</ulink>.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500603 </note>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500604 <orderedlist>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500605 <listitem><para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500606 <emphasis>Make Your Changes Locally:</emphasis>
607 Make your changes in your local Git repository.
608 You should make small, controlled, isolated changes.
609 Keeping changes small and isolated aids review,
610 makes merging/rebasing easier and keeps the change
611 history clean should anyone need to refer to it in
612 future.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500613 </para></listitem>
614 <listitem><para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500615 <emphasis>Stage Your Changes:</emphasis>
616 Stage your changes by using the <filename>git add</filename>
617 command on each file you changed.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500618 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500619 <listitem><para id='making-sure-you-have-correct-commit-information'>
620 <emphasis>Commit Your Changes:</emphasis>
621 Commit the change by using the
622 <filename>git commit</filename> command.
623 Make sure your commit information follows standards by
624 following these accepted conventions:
625 <itemizedlist>
626 <listitem><para>
627 Be sure to include a "Signed-off-by:" line in the
628 same style as required by the Linux kernel.
629 Adding this line signifies that you, the submitter,
630 have agreed to the Developer's Certificate of
631 Origin 1.1 as follows:
632 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500633 Developer's Certificate of Origin 1.1
634
635 By making a contribution to this project, I certify that:
636
637 (a) The contribution was created in whole or in part by me and I
638 have the right to submit it under the open source license
639 indicated in the file; or
640
641 (b) The contribution is based upon previous work that, to the best
642 of my knowledge, is covered under an appropriate open source
643 license and I have the right under that license to submit that
644 work with modifications, whether created in whole or in part
645 by me, under the same open source license (unless I am
646 permitted to submit under a different license), as indicated
647 in the file; or
648
649 (c) The contribution was provided directly to me by some other
650 person who certified (a), (b) or (c) and I have not modified
651 it.
652
653 (d) I understand and agree that this project and the contribution
654 are public and that a record of the contribution (including all
655 personal information I submit with it, including my sign-off) is
656 maintained indefinitely and may be redistributed consistent with
657 this project or the open source license(s) involved.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500658 </literallayout>
659 </para></listitem>
660 <listitem><para>
661 Provide a single-line summary of the change.
662 and,
663 if more explanation is needed, provide more
664 detail in the body of the commit.
665 This summary is typically viewable in the
666 "shortlist" of changes.
667 Thus, providing something short and descriptive
668 that gives the reader a summary of the change is
669 useful when viewing a list of many commits.
670 You should prefix this short description with the
671 recipe name (if changing a recipe), or else with
672 the short form path to the file being changed.
673 </para></listitem>
674 <listitem><para>
675 For the body of the commit message, provide
676 detailed information that describes what you
677 changed, why you made the change, and the approach
678 you used.
679 It might also be helpful if you mention how you
680 tested the change.
681 Provide as much detail as you can in the body of
682 the commit message.
683 <note>
684 You do not need to provide a more detailed
685 explanation of a change if the change is
686 minor to the point of the single line
687 summary providing all the information.
688 </note>
689 </para></listitem>
690 <listitem><para>
691 If the change addresses a specific bug or issue
692 that is associated with a bug-tracking ID,
693 include a reference to that ID in your detailed
694 description.
695 For example, the Yocto Project uses a specific
696 convention for bug references - any commit that
697 addresses a specific bug should use the following
698 form for the detailed description.
699 Be sure to use the actual bug-tracking ID from
700 Bugzilla for
701 <replaceable>bug-id</replaceable>:
702 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500703 Fixes [YOCTO #<replaceable>bug-id</replaceable>]
704
705 <replaceable>detailed description of change</replaceable>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500706 </literallayout>
707 </para></listitem>
708 </itemizedlist>
709 </para></listitem>
710 <listitem><para>
711 <emphasis>Push Your Commits to a "Contrib" Upstream:</emphasis>
712 If you have arranged for permissions to push to an
713 upstream contrib repository, push the change to that
714 repository:
715 <literallayout class='monospaced'>
716 $ git push <replaceable>upstream_remote_repo</replaceable> <replaceable>local_branch_name</replaceable>
717 </literallayout>
718 For example, suppose you have permissions to push into the
719 upstream <filename>meta-intel-contrib</filename>
720 repository and you are working in a local branch named
721 <replaceable>your_name</replaceable><filename>/README</filename>.
722 The following command pushes your local commits to the
723 <filename>meta-intel-contrib</filename> upstream
724 repository and puts the commit in a branch named
725 <replaceable>your_name</replaceable><filename>/README</filename>:
726 <literallayout class='monospaced'>
727 $ git push meta-intel-contrib <replaceable>your_name</replaceable>/README
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500728 </literallayout>
729 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500730 <listitem><para id='push-determine-who-to-notify'>
731 <emphasis>Determine Who to Notify:</emphasis>
732 Determine the maintainer or the mailing list
733 that you need to notify for the change.</para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500734
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500735 <para>Before submitting any change, you need to be sure
736 who the maintainer is or what mailing list that you need
737 to notify.
738 Use either these methods to find out:
739 <itemizedlist>
740 <listitem><para>
741 <emphasis>Maintenance File:</emphasis>
742 Examine the <filename>maintainers.inc</filename>
743 file, which is located in the
744 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
745 at
746 <filename>meta/conf/distro/include</filename>,
747 to see who is responsible for code.
748 </para></listitem>
749 <listitem><para>
750 <emphasis>Search by File:</emphasis>
751 Using <ulink url='&YOCTO_DOCS_REF_URL;#git'>Git</ulink>,
752 you can enter the following command to bring up a
753 short list of all commits against a specific file:
754 <literallayout class='monospaced'>
755 git shortlog -- <replaceable>filename</replaceable>
756 </literallayout>
757 Just provide the name of the file for which you
758 are interested.
759 The information returned is not ordered by history
760 but does include a list of everyone who has
761 committed grouped by name.
762 From the list, you can see who is responsible for
763 the bulk of the changes against the file.
764 </para></listitem>
765 <listitem><para>
766 <emphasis>Examine the List of Mailing Lists:</emphasis>
767 For a list of the Yocto Project and related mailing
768 lists, see the
769 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>"
770 section in the Yocto Project Reference Manual.
771 </para></listitem>
772 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500773 </para></listitem>
774 <listitem><para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500775 <emphasis>Make a Pull Request:</emphasis>
776 Notify the maintainer or the mailing list that you have
777 pushed a change by making a pull request.</para>
778
779 <para>The Yocto Project provides two scripts that
780 conveniently let you generate and send pull requests to the
781 Yocto Project.
782 These scripts are <filename>create-pull-request</filename>
783 and <filename>send-pull-request</filename>.
784 You can find these scripts in the
785 <filename>scripts</filename> directory within the
786 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
787 (e.g. <filename>~/poky/scripts</filename>).
788 </para>
789
790 <para>Using these scripts correctly formats the requests
791 without introducing any whitespace or HTML formatting.
792 The maintainer that receives your patches either directly
793 or through the mailing list needs to be able to save and
794 apply them directly from your emails.
795 Using these scripts is the preferred method for sending
796 patches.</para>
797
798 <para>First, create the pull request.
799 For example, the following command runs the script,
800 specifies the upstream repository in the contrib directory
801 into which you pushed the change, and provides a subject
802 line in the created patch files:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500803 <literallayout class='monospaced'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500804 $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README"
805 </literallayout>
806 Running this script forms
807 <filename>*.patch</filename> files in a folder named
808 <filename>pull-</filename><replaceable>PID</replaceable>
809 in the current directory.
810 One of the patch files is a cover letter.</para>
811
812 <para>Before running the
813 <filename>send-pull-request</filename> script, you must
814 edit the cover letter patch to insert information about
815 your change.
816 After editing the cover letter, send the pull request.
817 For example, the following command runs the script and
818 specifies the patch directory and email address.
819 In this example, the email address is a mailing list:
820 <literallayout class='monospaced'>
821 $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org
822 </literallayout>
823 You need to follow the prompts as the script is
824 interactive.
825 <note>
826 For help on using these scripts, simply provide the
827 <filename>-h</filename> argument as follows:
828 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500829 $ poky/scripts/create-pull-request -h
830 $ poky/scripts/send-pull-request -h
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500831 </literallayout>
832 </note>
833 </para></listitem>
834 </orderedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500835 </para>
836 </section>
837
838 <section id='submitting-a-patch'>
839 <title>Using Email to Submit a Patch</title>
840
841 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500842 You can submit patches without using the
843 <filename>create-pull-request</filename> and
844 <filename>send-pull-request</filename> scripts described in the
845 previous section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500846 However, keep in mind, the preferred method is to use the scripts.
847 </para>
848
849 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500850 Depending on the components changed, you need to submit the email
851 to a specific mailing list.
852 For some guidance on which mailing list to use, see the
853 <link linkend='figuring-out-the-mailing-list-to-use'>beginning</link>
854 of this section.
855 For a description of all the available mailing lists, see the
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500856 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
857 section in the Yocto Project Reference Manual.
858 </para>
859
860 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500861 Here is the general procedure on how to submit a patch through
862 email without using the scripts:
863 <orderedlist>
864 <listitem><para>
865 <emphasis>Make Your Changes Locally:</emphasis>
866 Make your changes in your local Git repository.
867 You should make small, controlled, isolated changes.
868 Keeping changes small and isolated aids review,
869 makes merging/rebasing easier and keeps the change
870 history clean should anyone need to refer to it in
871 future.
872 </para></listitem>
873 <listitem><para>
874 <emphasis>Stage Your Changes:</emphasis>
875 Stage your changes by using the <filename>git add</filename>
876 command on each file you changed.
877 </para></listitem>
878 <listitem><para>
879 <emphasis>Commit Your Changes:</emphasis>
880 Commit the change by using the
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500881 <filename>git commit --signoff</filename> command.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500882 Using the <filename>--signoff</filename> option identifies
883 you as the person making the change and also satisfies
884 the Developer's Certificate of Origin (DCO) shown earlier.
885 </para>
886
887 <para>When you form a commit, you must follow certain
888 standards established by the Yocto Project development
889 team.
890 See
891 <link linkend='making-sure-you-have-correct-commit-information'>Step 3</link>
892 in the previous section for information on how to
893 provide commit information that meets Yocto Project
894 commit message standards.
895 </para></listitem>
896 <listitem><para>
897 <emphasis>Format the Commit:</emphasis>
898 Format the commit into an email message.
899 To format commits, use the
900 <filename>git format-patch</filename> command.
901 When you provide the command, you must include a revision
902 list or a number of patches as part of the command.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500903 For example, either of these two commands takes your most
904 recent single commit and formats it as an email message in
905 the current directory:
906 <literallayout class='monospaced'>
907 $ git format-patch -1
908 </literallayout>
909 or
910 <literallayout class='monospaced'>
911 $ git format-patch HEAD~
912 </literallayout></para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500913
914 <para>After the command is run, the current directory
915 contains a numbered <filename>.patch</filename> file for
916 the commit.</para>
917
918 <para>If you provide several commits as part of the
919 command, the <filename>git format-patch</filename> command
920 produces a series of numbered files in the current
921 directory – one for each commit.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500922 If you have more than one patch, you should also use the
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500923 <filename>--cover</filename> option with the command,
924 which generates a cover letter as the first "patch" in
925 the series.
926 You can then edit the cover letter to provide a
927 description for the series of patches.
928 For information on the
929 <filename>git format-patch</filename> command,
930 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed
931 using the <filename>man git-format-patch</filename>
932 command.
933 <note>
934 If you are or will be a frequent contributor to the
935 Yocto Project or to OpenEmbedded, you might consider
936 requesting a contrib area and the necessary associated
937 rights.
938 </note>
939 </para></listitem>
940 <listitem><para>
941 <emphasis>Import the Files Into Your Mail Client:</emphasis>
942 Import the files into your mail client by using the
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500943 <filename>git send-email</filename> command.
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500944 <note>
945 In order to use <filename>git send-email</filename>,
946 you must have the proper Git packages installed on
947 your host.
948 For Ubuntu, Debian, and Fedora the package is
949 <filename>git-email</filename>.
950 </note></para>
951
952 <para>The <filename>git send-email</filename> command
953 sends email by using a local or remote Mail Transport Agent
954 (MTA) such as <filename>msmtp</filename>,
955 <filename>sendmail</filename>, or through a direct
956 <filename>smtp</filename> configuration in your Git
957 <filename>~/.gitconfig</filename> file.
958 If you are submitting patches through email only, it is
959 very important that you submit them without any whitespace
960 or HTML formatting that either you or your mailer
961 introduces.
962 The maintainer that receives your patches needs to be able
963 to save and apply them directly from your emails.
964 A good way to verify that what you are sending will be
965 applicable by the maintainer is to do a dry run and send
966 them to yourself and then save and apply them as the
967 maintainer would.</para>
968
969 <para>The <filename>git send-email</filename> command is
970 the preferred method for sending your patches using
971 email since there is no risk of compromising whitespace
972 in the body of the message, which can occur when you use
973 your own mail client.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500974 The command also has several options that let you
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500975 specify recipients and perform further editing of the
976 email message.
977 For information on how to use the
978 <filename>git send-email</filename> command,
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500979 see <filename>GIT-SEND-EMAIL(1)</filename> displayed using
980 the <filename>man git-send-email</filename> command.
981 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500982 </orderedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500983 </para>
984 </section>
985</section>
986</chapter>
987<!--
988vim: expandtab tw=80 ts=4
989-->