werunplugged/unplugged-kernel
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Linux kernel 4.19 for Unplugged OS

Browse Source
This commit is contained in:
ziv committed 2025-10-06 13:52:42 +00:00
commit d30235b04c
80519 files changed
+35800681
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Documentation/process/1.Intro.rst
+266
View File
@@ -0,0 +1,266 @@
Introduction
============
Executive summary
-----------------
The rest of this section covers the scope of the kernel development process
and the kinds of frustrations that developers and their employers can
encounter there. There are a great many reasons why kernel code should be
merged into the official ("mainline") kernel, including automatic
availability to users, community support in many forms, and the ability to
influence the direction of kernel development. Code contributed to the
Linux kernel must be made available under a GPL-compatible license.
:ref:`development_process` introduces the development process, the kernel
release cycle, and the mechanics of the merge window. The various phases in
the patch development, review, and merging cycle are covered. There is some
discussion of tools and mailing lists. Developers wanting to get started
with kernel development are encouraged to track down and fix bugs as an
initial exercise.
:ref:`development_early_stage` covers early-stage project planning, with an
emphasis on involving the development community as soon as possible.
:ref:`development_coding` is about the coding process; several pitfalls which
have been encountered by other developers are discussed. Some requirements for
patches are covered, and there is an introduction to some of the tools
which can help to ensure that kernel patches are correct.
:ref:`development_posting` talks about the process of posting patches for
review. To be taken seriously by the development community, patches must be
properly formatted and described, and they must be sent to the right place.
Following the advice in this section should help to ensure the best
possible reception for your work.
:ref:`development_followthrough` covers what happens after posting patches; the
job is far from done at that point. Working with reviewers is a crucial part
of the development process; this section offers a number of tips on how to
avoid problems at this important stage. Developers are cautioned against
assuming that the job is done when a patch is merged into the mainline.
:ref:`development_advancedtopics` introduces a couple of "advanced" topics:
managing patches with git and reviewing patches posted by others.
:ref:`development_conclusion` concludes the document with pointers to sources
for more information on kernel development.
What this document is about
---------------------------
The Linux kernel, at over 8 million lines of code and well over 1000
contributors to each release, is one of the largest and most active free
software projects in existence. Since its humble beginning in 1991, this
kernel has evolved into a best-of-breed operating system component which
runs on pocket-sized digital music players, desktop PCs, the largest
supercomputers in existence, and all types of systems in between. It is a
robust, efficient, and scalable solution for almost any situation.
With the growth of Linux has come an increase in the number of developers
(and companies) wishing to participate in its development. Hardware
vendors want to ensure that Linux supports their products well, making
those products attractive to Linux users. Embedded systems vendors, who
use Linux as a component in an integrated product, want Linux to be as
capable and well-suited to the task at hand as possible. Distributors and
other software vendors who base their products on Linux have a clear
interest in the capabilities, performance, and reliability of the Linux
kernel. And end users, too, will often wish to change Linux to make it
better suit their needs.
One of the most compelling features of Linux is that it is accessible to
these developers; anybody with the requisite skills can improve Linux and
influence the direction of its development. Proprietary products cannot
offer this kind of openness, which is a characteristic of the free software
process. But, if anything, the kernel is even more open than most other
free software projects. A typical three-month kernel development cycle can
involve over 1000 developers working for more than 100 different companies
(or for no company at all).
Working with the kernel development community is not especially hard. But,
that notwithstanding, many potential contributors have experienced
difficulties when trying to do kernel work. The kernel community has
evolved its own distinct ways of operating which allow it to function
smoothly (and produce a high-quality product) in an environment where
thousands of lines of code are being changed every day. So it is not
surprising that Linux kernel development process differs greatly from
proprietary development methods.
The kernel's development process may come across as strange and
intimidating to new developers, but there are good reasons and solid
experience behind it. A developer who does not understand the kernel
community's ways (or, worse, who tries to flout or circumvent them) will
have a frustrating experience in store. The development community, while
being helpful to those who are trying to learn, has little time for those
who will not listen or who do not care about the development process.
It is hoped that those who read this document will be able to avoid that
frustrating experience. There is a lot of material here, but the effort
involved in reading it will be repaid in short order. The development
community is always in need of developers who will help to make the kernel
better; the following text should help you - or those who work for you -
join our community.
Credits
-------
This document was written by Jonathan Corbet, corbet@lwn.net. It has been
improved by comments from Johannes Berg, James Berry, Alex Chiang, Roland
Dreier, Randy Dunlap, Jake Edge, Jiri Kosina, Matt Mackall, Arthur Marsh,
Amanda McPherson, Andrew Morton, Andrew Price, Tsugikazu Shibata, and
Jochen Voß.
This work was supported by the Linux Foundation; thanks especially to
Amanda McPherson, who saw the value of this effort and made it all happen.
The importance of getting code into the mainline
------------------------------------------------
Some companies and developers occasionally wonder why they should bother
learning how to work with the kernel community and get their code into the
mainline kernel (the "mainline" being the kernel maintained by Linus
Torvalds and used as a base by Linux distributors). In the short term,
contributing code can look like an avoidable expense; it seems easier to
just keep the code separate and support users directly. The truth of the
matter is that keeping code separate ("out of tree") is a false economy.
As a way of illustrating the costs of out-of-tree code, here are a few
relevant aspects of the kernel development process; most of these will be
discussed in greater detail later in this document. Consider:
- Code which has been merged into the mainline kernel is available to all
Linux users. It will automatically be present on all distributions which
enable it. There is no need for driver disks, downloads, or the hassles
of supporting multiple versions of multiple distributions; it all just
works, for the developer and for the user. Incorporation into the
mainline solves a large number of distribution and support problems.
- While kernel developers strive to maintain a stable interface to user
space, the internal kernel API is in constant flux. The lack of a stable
internal interface is a deliberate design decision; it allows fundamental
improvements to be made at any time and results in higher-quality code.
But one result of that policy is that any out-of-tree code requires
constant upkeep if it is to work with new kernels. Maintaining
out-of-tree code requires significant amounts of work just to keep that
code working.
Code which is in the mainline, instead, does not require this work as the
result of a simple rule requiring any developer who makes an API change
to also fix any code that breaks as the result of that change. So code
which has been merged into the mainline has significantly lower
maintenance costs.
- Beyond that, code which is in the kernel will often be improved by other
developers. Surprising results can come from empowering your user
community and customers to improve your product.
- Kernel code is subjected to review, both before and after merging into
the mainline. No matter how strong the original developer's skills are,
this review process invariably finds ways in which the code can be
improved. Often review finds severe bugs and security problems. This is
especially true for code which has been developed in a closed
environment; such code benefits strongly from review by outside
developers. Out-of-tree code is lower-quality code.
- Participation in the development process is your way to influence the
direction of kernel development. Users who complain from the sidelines
are heard, but active developers have a stronger voice - and the ability
to implement changes which make the kernel work better for their needs.
- When code is maintained separately, the possibility that a third party
will contribute a different implementation of a similar feature always
exists. Should that happen, getting your code merged will become much
harder - to the point of impossibility. Then you will be faced with the
unpleasant alternatives of either (1) maintaining a nonstandard feature
out of tree indefinitely, or (2) abandoning your code and migrating your
users over to the in-tree version.
- Contribution of code is the fundamental action which makes the whole
process work. By contributing your code you can add new functionality to
the kernel and provide capabilities and examples which are of use to
other kernel developers. If you have developed code for Linux (or are
thinking about doing so), you clearly have an interest in the continued
success of this platform; contributing code is one of the best ways to
help ensure that success.
All of the reasoning above applies to any out-of-tree kernel code,
including code which is distributed in proprietary, binary-only form.
There are, however, additional factors which should be taken into account
before considering any sort of binary-only kernel code distribution. These
include:
- The legal issues around the distribution of proprietary kernel modules
are cloudy at best; quite a few kernel copyright holders believe that
most binary-only modules are derived products of the kernel and that, as
a result, their distribution is a violation of the GNU General Public
license (about which more will be said below). Your author is not a
lawyer, and nothing in this document can possibly be considered to be
legal advice. The true legal status of closed-source modules can only be
determined by the courts. But the uncertainty which haunts those modules
is there regardless.
- Binary modules greatly increase the difficulty of debugging kernel
problems, to the point that most kernel developers will not even try. So
the distribution of binary-only modules will make it harder for your
users to get support from the community.
- Support is also harder for distributors of binary-only modules, who must
provide a version of the module for every distribution and every kernel
version they wish to support. Dozens of builds of a single module can
be required to provide reasonably comprehensive coverage, and your users
will have to upgrade your module separately every time they upgrade their
kernel.
- Everything that was said above about code review applies doubly to
closed-source code. Since this code is not available at all, it cannot
have been reviewed by the community and will, beyond doubt, have serious
problems.
Makers of embedded systems, in particular, may be tempted to disregard much
of what has been said in this section in the belief that they are shipping
a self-contained product which uses a frozen kernel version and requires no
more development after its release. This argument misses the value of
widespread code review and the value of allowing your users to add
capabilities to your product. But these products, too, have a limited
commercial life, after which a new version must be released. At that
point, vendors whose code is in the mainline and well maintained will be
much better positioned to get the new product ready for market quickly.
Licensing
---------
Code is contributed to the Linux kernel under a number of licenses, but all
code must be compatible with version 2 of the GNU General Public License
(GPLv2), which is the license covering the kernel distribution as a whole.
In practice, that means that all code contributions are covered either by
GPLv2 (with, optionally, language allowing distribution under later
versions of the GPL) or the three-clause BSD license. Any contributions
which are not covered by a compatible license will not be accepted into the
kernel.
Copyright assignments are not required (or requested) for code contributed
to the kernel. All code merged into the mainline kernel retains its
original ownership; as a result, the kernel now has thousands of owners.
One implication of this ownership structure is that any attempt to change
the licensing of the kernel is doomed to almost certain failure. There are
few practical scenarios where the agreement of all copyright holders could
be obtained (or their code removed from the kernel). So, in particular,
there is no prospect of a migration to version 3 of the GPL in the
foreseeable future.
It is imperative that all code contributed to the kernel be legitimately
free software. For that reason, code from anonymous (or pseudonymous)
contributors will not be accepted. All contributors are required to "sign
off" on their code, stating that the code can be distributed with the
kernel under the GPL. Code which has not been licensed as free software by
its owner, or which risks creating copyright-related problems for the
kernel (such as code which derives from reverse-engineering efforts lacking
proper safeguards) cannot be contributed.
Questions about copyright-related issues are common on Linux development
mailing lists. Such questions will normally receive no shortage of
answers, but one should bear in mind that the people answering those
questions are not lawyers and cannot provide legal advice. If you have
legal questions relating to Linux source code, there is no substitute for
talking with a lawyer who understands this field. Relying on answers
obtained on technical mailing lists is a risky affair.
Documentation/process/2.Process.rst
+501
View File
@@ -0,0 +1,501 @@
.. _development_process:
How the development process works
=================================
Linux kernel development in the early 1990's was a pretty loose affair,
with relatively small numbers of users and developers involved. With a
user base in the millions and with some 2,000 developers involved over the
course of one year, the kernel has since had to evolve a number of
processes to keep development happening smoothly. A solid understanding of
how the process works is required in order to be an effective part of it.
The big picture
---------------
The kernel developers use a loosely time-based release process, with a new
major kernel release happening every two or three months. The recent
release history looks like this:
====== =================
4.11 April 30, 2017
4.12 July 2, 2017
4.13 September 3, 2017
4.14 November 12, 2017
4.15 January 28, 2018
4.16 April 1, 2018
====== =================
Every 4.x release is a major kernel release with new features, internal
API changes, and more. A typical 4.x release contain about 13,000
changesets with changes to several hundred thousand lines of code. 4.x is
thus the leading edge of Linux kernel development; the kernel uses a
rolling development model which is continually integrating major changes.
A relatively straightforward discipline is followed with regard to the
merging of patches for each release. At the beginning of each development
cycle, the "merge window" is said to be open. At that time, code which is
deemed to be sufficiently stable (and which is accepted by the development
community) is merged into the mainline kernel. The bulk of changes for a
new development cycle (and all of the major changes) will be merged during
this time, at a rate approaching 1,000 changes ("patches," or "changesets")
per day.
(As an aside, it is worth noting that the changes integrated during the
merge window do not come out of thin air; they have been collected, tested,
and staged ahead of time. How that process works will be described in
detail later on).
The merge window lasts for approximately two weeks. At the end of this
time, Linus Torvalds will declare that the window is closed and release the
first of the "rc" kernels. For the kernel which is destined to be 2.6.40,
for example, the release which happens at the end of the merge window will
be called 2.6.40-rc1. The -rc1 release is the signal that the time to
merge new features has passed, and that the time to stabilize the next
kernel has begun.
Over the next six to ten weeks, only patches which fix problems should be
submitted to the mainline. On occasion a more significant change will be
allowed, but such occasions are rare; developers who try to merge new
features outside of the merge window tend to get an unfriendly reception.
As a general rule, if you miss the merge window for a given feature, the
best thing to do is to wait for the next development cycle. (An occasional
exception is made for drivers for previously-unsupported hardware; if they
touch no in-tree code, they cannot cause regressions and should be safe to
add at any time).
As fixes make their way into the mainline, the patch rate will slow over
time. Linus releases new -rc kernels about once a week; a normal series
will get up to somewhere between -rc6 and -rc9 before the kernel is
considered to be sufficiently stable and the final 2.6.x release is made.
At that point the whole process starts over again.
As an example, here is how the 4.16 development cycle went (all dates in
2018):
============== ===============================
January 28 4.15 stable release
February 11 4.16-rc1, merge window closes
February 18 4.16-rc2
February 25 4.16-rc3
March 4 4.16-rc4
March 11 4.16-rc5
March 18 4.16-rc6
March 25 4.16-rc7
April 1 4.17 stable release
============== ===============================
How do the developers decide when to close the development cycle and create
the stable release? The most significant metric used is the list of
regressions from previous releases. No bugs are welcome, but those which
break systems which worked in the past are considered to be especially
serious. For this reason, patches which cause regressions are looked upon
unfavorably and are quite likely to be reverted during the stabilization
period.
The developers' goal is to fix all known regressions before the stable
release is made. In the real world, this kind of perfection is hard to
achieve; there are just too many variables in a project of this size.
There comes a point where delaying the final release just makes the problem
worse; the pile of changes waiting for the next merge window will grow
larger, creating even more regressions the next time around. So most 4.x
kernels go out with a handful of known regressions though, hopefully, none
of them are serious.
Once a stable release is made, its ongoing maintenance is passed off to the
"stable team," currently consisting of Greg Kroah-Hartman. The stable team
will release occasional updates to the stable release using the 4.x.y
numbering scheme. To be considered for an update release, a patch must (1)
fix a significant bug, and (2) already be merged into the mainline for the
next development kernel. Kernels will typically receive stable updates for
a little more than one development cycle past their initial release. So,
for example, the 4.13 kernel's history looked like:
============== ===============================
September 3 4.13 stable release
September 13 4.13.1
September 20 4.13.2
September 27 4.13.3
October 5 4.13.4
October 12 4.13.5
... ...
November 24 4.13.16
============== ===============================
4.13.16 was the final stable update of the 4.13 release.
Some kernels are designated "long term" kernels; they will receive support
for a longer period. As of this writing, the current long term kernels
and their maintainers are:
====== ====================== ==============================
3.16 Ben Hutchings (very long-term stable kernel)
4.1 Sasha Levin
4.4 Greg Kroah-Hartman (very long-term stable kernel)
4.9 Greg Kroah-Hartman
4.14 Greg Kroah-Hartman
====== ====================== ==============================
The selection of a kernel for long-term support is purely a matter of a
maintainer having the need and the time to maintain that release. There
are no known plans for long-term support for any specific upcoming
release.
The lifecycle of a patch
------------------------
Patches do not go directly from the developer's keyboard into the mainline
kernel. There is, instead, a somewhat involved (if somewhat informal)
process designed to ensure that each patch is reviewed for quality and that
each patch implements a change which is desirable to have in the mainline.
This process can happen quickly for minor fixes, or, in the case of large
and controversial changes, go on for years. Much developer frustration
comes from a lack of understanding of this process or from attempts to
circumvent it.
In the hopes of reducing that frustration, this document will describe how
a patch gets into the kernel. What follows below is an introduction which
describes the process in a somewhat idealized way. A much more detailed
treatment will come in later sections.
The stages that a patch goes through are, generally:
- Design. This is where the real requirements for the patch - and the way
those requirements will be met - are laid out. Design work is often
done without involving the community, but it is better to do this work
in the open if at all possible; it can save a lot of time redesigning
things later.
- Early review. Patches are posted to the relevant mailing list, and
developers on that list reply with any comments they may have. This
process should turn up any major problems with a patch if all goes
well.
- Wider review. When the patch is getting close to ready for mainline
inclusion, it should be accepted by a relevant subsystem maintainer -
though this acceptance is not a guarantee that the patch will make it
all the way to the mainline. The patch will show up in the maintainer's
subsystem tree and into the -next trees (described below). When the
process works, this step leads to more extensive review of the patch and
the discovery of any problems resulting from the integration of this
patch with work being done by others.
- Please note that most maintainers also have day jobs, so merging
your patch may not be their highest priority. If your patch is
getting feedback about changes that are needed, you should either
make those changes or justify why they should not be made. If your
patch has no review complaints but is not being merged by its
appropriate subsystem or driver maintainer, you should be persistent
in updating the patch to the current kernel so that it applies cleanly
and keep sending it for review and merging.
- Merging into the mainline. Eventually, a successful patch will be
merged into the mainline repository managed by Linus Torvalds. More
comments and/or problems may surface at this time; it is important that
the developer be responsive to these and fix any issues which arise.
- Stable release. The number of users potentially affected by the patch
is now large, so, once again, new problems may arise.
- Long-term maintenance. While it is certainly possible for a developer
to forget about code after merging it, that sort of behavior tends to
leave a poor impression in the development community. Merging code
eliminates some of the maintenance burden, in that others will fix
problems caused by API changes. But the original developer should
continue to take responsibility for the code if it is to remain useful
in the longer term.
One of the largest mistakes made by kernel developers (or their employers)
is to try to cut the process down to a single "merging into the mainline"
step. This approach invariably leads to frustration for everybody
involved.
How patches get into the Kernel
-------------------------------
There is exactly one person who can merge patches into the mainline kernel
repository: Linus Torvalds. But, of the over 9,500 patches which went
into the 2.6.38 kernel, only 112 (around 1.3%) were directly chosen by Linus
himself. The kernel project has long since grown to a size where no single
developer could possibly inspect and select every patch unassisted. The
way the kernel developers have addressed this growth is through the use of
a lieutenant system built around a chain of trust.
The kernel code base is logically broken down into a set of subsystems:
networking, specific architecture support, memory management, video
devices, etc. Most subsystems have a designated maintainer, a developer
who has overall responsibility for the code within that subsystem. These
subsystem maintainers are the gatekeepers (in a loose way) for the portion
of the kernel they manage; they are the ones who will (usually) accept a
patch for inclusion into the mainline kernel.
Subsystem maintainers each manage their own version of the kernel source
tree, usually (but certainly not always) using the git source management
tool. Tools like git (and related tools like quilt or mercurial) allow
maintainers to track a list of patches, including authorship information
and other metadata. At any given time, the maintainer can identify which
patches in his or her repository are not found in the mainline.
When the merge window opens, top-level maintainers will ask Linus to "pull"
the patches they have selected for merging from their repositories. If
Linus agrees, the stream of patches will flow up into his repository,
becoming part of the mainline kernel. The amount of attention that Linus
pays to specific patches received in a pull operation varies. It is clear
that, sometimes, he looks quite closely. But, as a general rule, Linus
trusts the subsystem maintainers to not send bad patches upstream.
Subsystem maintainers, in turn, can pull patches from other maintainers.
For example, the networking tree is built from patches which accumulated
first in trees dedicated to network device drivers, wireless networking,
etc. This chain of repositories can be arbitrarily long, though it rarely
exceeds two or three links. Since each maintainer in the chain trusts
those managing lower-level trees, this process is known as the "chain of
trust."
Clearly, in a system like this, getting patches into the kernel depends on
finding the right maintainer. Sending patches directly to Linus is not
normally the right way to go.
Next trees
----------
The chain of subsystem trees guides the flow of patches into the kernel,
but it also raises an interesting question: what if somebody wants to look
at all of the patches which are being prepared for the next merge window?
Developers will be interested in what other changes are pending to see
whether there are any conflicts to worry about; a patch which changes a
core kernel function prototype, for example, will conflict with any other
patches which use the older form of that function. Reviewers and testers
want access to the changes in their integrated form before all of those
changes land in the mainline kernel. One could pull changes from all of
the interesting subsystem trees, but that would be a big and error-prone
job.
The answer comes in the form of -next trees, where subsystem trees are
collected for testing and review. The older of these trees, maintained by
Andrew Morton, is called "-mm" (for memory management, which is how it got
started). The -mm tree integrates patches from a long list of subsystem
trees; it also has some patches aimed at helping with debugging.
Beyond that, -mm contains a significant collection of patches which have
been selected by Andrew directly. These patches may have been posted on a
mailing list, or they may apply to a part of the kernel for which there is
no designated subsystem tree. As a result, -mm operates as a sort of
subsystem tree of last resort; if there is no other obvious path for a
patch into the mainline, it is likely to end up in -mm. Miscellaneous
patches which accumulate in -mm will eventually either be forwarded on to
an appropriate subsystem tree or be sent directly to Linus. In a typical
development cycle, approximately 5-10% of the patches going into the
mainline get there via -mm.
The current -mm patch is available in the "mmotm" (-mm of the moment)
directory at:
http://www.ozlabs.org/~akpm/mmotm/
Use of the MMOTM tree is likely to be a frustrating experience, though;
there is a definite chance that it will not even compile.
The primary tree for next-cycle patch merging is linux-next, maintained by
Stephen Rothwell. The linux-next tree is, by design, a snapshot of what
the mainline is expected to look like after the next merge window closes.
Linux-next trees are announced on the linux-kernel and linux-next mailing
lists when they are assembled; they can be downloaded from:
http://www.kernel.org/pub/linux/kernel/next/
Linux-next has become an integral part of the kernel development process;
all patches merged during a given merge window should really have found
their way into linux-next some time before the merge window opens.
Staging trees
-------------
The kernel source tree contains the drivers/staging/ directory, where
many sub-directories for drivers or filesystems that are on their way to
being added to the kernel tree live. They remain in drivers/staging while
they still need more work; once complete, they can be moved into the
kernel proper. This is a way to keep track of drivers that aren't
up to Linux kernel coding or quality standards, but people may want to use
them and track development.
Greg Kroah-Hartman currently maintains the staging tree. Drivers that
still need work are sent to him, with each driver having its own
subdirectory in drivers/staging/. Along with the driver source files, a
TODO file should be present in the directory as well. The TODO file lists
the pending work that the driver needs for acceptance into the kernel
proper, as well as a list of people that should be Cc'd for any patches to
the driver. Current rules require that drivers contributed to staging
must, at a minimum, compile properly.
Staging can be a relatively easy way to get new drivers into the mainline
where, with luck, they will come to the attention of other developers and
improve quickly. Entry into staging is not the end of the story, though;
code in staging which is not seeing regular progress will eventually be
removed. Distributors also tend to be relatively reluctant to enable
staging drivers. So staging is, at best, a stop on the way toward becoming
a proper mainline driver.
Tools
-----
As can be seen from the above text, the kernel development process depends
heavily on the ability to herd collections of patches in various
directions. The whole thing would not work anywhere near as well as it
does without suitably powerful tools. Tutorials on how to use these tools
are well beyond the scope of this document, but there is space for a few
pointers.
By far the dominant source code management system used by the kernel
community is git. Git is one of a number of distributed version control
systems being developed in the free software community. It is well tuned
for kernel development, in that it performs quite well when dealing with
large repositories and large numbers of patches. It also has a reputation
for being difficult to learn and use, though it has gotten better over
time. Some sort of familiarity with git is almost a requirement for kernel
developers; even if they do not use it for their own work, they'll need git
to keep up with what other developers (and the mainline) are doing.
Git is now packaged by almost all Linux distributions. There is a home
page at:
http://git-scm.com/
That page has pointers to documentation and tutorials.
Among the kernel developers who do not use git, the most popular choice is
almost certainly Mercurial:
http://www.selenic.com/mercurial/
Mercurial shares many features with git, but it provides an interface which
many find easier to use.
The other tool worth knowing about is Quilt:
http://savannah.nongnu.org/projects/quilt/
Quilt is a patch management system, rather than a source code management
system. It does not track history over time; it is, instead, oriented
toward tracking a specific set of changes against an evolving code base.
Some major subsystem maintainers use quilt to manage patches intended to go
upstream. For the management of certain kinds of trees (-mm, for example),
quilt is the best tool for the job.
Mailing lists
-------------
A great deal of Linux kernel development work is done by way of mailing
lists. It is hard to be a fully-functioning member of the community
without joining at least one list somewhere. But Linux mailing lists also
represent a potential hazard to developers, who risk getting buried under a
load of electronic mail, running afoul of the conventions used on the Linux
lists, or both.
Most kernel mailing lists are run on vger.kernel.org; the master list can
be found at:
http://vger.kernel.org/vger-lists.html
There are lists hosted elsewhere, though; a number of them are at
lists.redhat.com.
The core mailing list for kernel development is, of course, linux-kernel.
This list is an intimidating place to be; volume can reach 500 messages per
day, the amount of noise is high, the conversation can be severely
technical, and participants are not always concerned with showing a high
degree of politeness. But there is no other place where the kernel
development community comes together as a whole; developers who avoid this
list will miss important information.
There are a few hints which can help with linux-kernel survival:
- Have the list delivered to a separate folder, rather than your main
mailbox. One must be able to ignore the stream for sustained periods of
time.
- Do not try to follow every conversation - nobody else does. It is
important to filter on both the topic of interest (though note that
long-running conversations can drift away from the original subject
without changing the email subject line) and the people who are
participating.
- Do not feed the trolls. If somebody is trying to stir up an angry
response, ignore them.
- When responding to linux-kernel email (or that on other lists) preserve
the Cc: header for all involved. In the absence of a strong reason (such
as an explicit request), you should never remove recipients. Always make
sure that the person you are responding to is in the Cc: list. This
convention also makes it unnecessary to explicitly ask to be copied on
replies to your postings.
- Search the list archives (and the net as a whole) before asking
questions. Some developers can get impatient with people who clearly
have not done their homework.
- Avoid top-posting (the practice of putting your answer above the quoted
text you are responding to). It makes your response harder to read and
makes a poor impression.
- Ask on the correct mailing list. Linux-kernel may be the general meeting
point, but it is not the best place to find developers from all
subsystems.
The last point - finding the correct mailing list - is a common place for
beginning developers to go wrong. Somebody who asks a networking-related
question on linux-kernel will almost certainly receive a polite suggestion
to ask on the netdev list instead, as that is the list frequented by most
networking developers. Other lists exist for the SCSI, video4linux, IDE,
filesystem, etc. subsystems. The best place to look for mailing lists is
in the MAINTAINERS file packaged with the kernel source.
Getting started with Kernel development
---------------------------------------
Questions about how to get started with the kernel development process are
common - from both individuals and companies. Equally common are missteps
which make the beginning of the relationship harder than it has to be.
Companies often look to hire well-known developers to get a development
group started. This can, in fact, be an effective technique. But it also
tends to be expensive and does not do much to grow the pool of experienced
kernel developers. It is possible to bring in-house developers up to speed
on Linux kernel development, given the investment of a bit of time. Taking
this time can endow an employer with a group of developers who understand
the kernel and the company both, and who can help to train others as well.
Over the medium term, this is often the more profitable approach.
Individual developers are often, understandably, at a loss for a place to
start. Beginning with a large project can be intimidating; one often wants
to test the waters with something smaller first. This is the point where
some developers jump into the creation of patches fixing spelling errors or
minor coding style issues. Unfortunately, such patches create a level of
noise which is distracting for the development community as a whole, so,
increasingly, they are looked down upon. New developers wishing to
introduce themselves to the community will not get the sort of reception
they wish for by these means.
Andrew Morton gives this advice for aspiring kernel developers
::
The #1 project for all kernel beginners should surely be "make sure
that the kernel runs perfectly at all times on all machines which
you can lay your hands on". Usually the way to do this is to work
with others on getting things fixed up (this can require
persistence!) but that's fine - it's a part of kernel development.
(http://lwn.net/Articles/283982/).
In the absence of obvious problems to fix, developers are advised to look
at the current lists of regressions and open bugs in general. There is
never any shortage of issues in need of fixing; by addressing these issues,
developers will gain experience with the process while, at the same time,
building respect with the rest of the development community.
Documentation/process/3.Early-stage.rst
+222
View File
@@ -0,0 +1,222 @@
.. _development_early_stage:
Early-stage planning
====================
When contemplating a Linux kernel development project, it can be tempting
to jump right in and start coding. As with any significant project,
though, much of the groundwork for success is best laid before the first
line of code is written. Some time spent in early planning and
communication can save far more time later on.
Specifying the problem
----------------------
Like any engineering project, a successful kernel enhancement starts with a
clear description of the problem to be solved. In some cases, this step is
easy: when a driver is needed for a specific piece of hardware, for
example. In others, though, it is tempting to confuse the real problem
with the proposed solution, and that can lead to difficulties.
Consider an example: some years ago, developers working with Linux audio
sought a way to run applications without dropouts or other artifacts caused
by excessive latency in the system. The solution they arrived at was a
kernel module intended to hook into the Linux Security Module (LSM)
framework; this module could be configured to give specific applications
access to the realtime scheduler. This module was implemented and sent to
the linux-kernel mailing list, where it immediately ran into problems.
To the audio developers, this security module was sufficient to solve their
immediate problem. To the wider kernel community, though, it was seen as a
misuse of the LSM framework (which is not intended to confer privileges
onto processes which they would not otherwise have) and a risk to system
stability. Their preferred solutions involved realtime scheduling access
via the rlimit mechanism for the short term, and ongoing latency reduction
work in the long term.
The audio community, however, could not see past the particular solution
they had implemented; they were unwilling to accept alternatives. The
resulting disagreement left those developers feeling disillusioned with the
entire kernel development process; one of them went back to an audio list
and posted this:
There are a number of very good Linux kernel developers, but they
tend to get outshouted by a large crowd of arrogant fools. Trying
to communicate user requirements to these people is a waste of
time. They are much too "intelligent" to listen to lesser mortals.
(http://lwn.net/Articles/131776/).
The reality of the situation was different; the kernel developers were far
more concerned about system stability, long-term maintenance, and finding
the right solution to the problem than they were with a specific module.
The moral of the story is to focus on the problem - not a specific solution
- and to discuss it with the development community before investing in the
creation of a body of code.
So, when contemplating a kernel development project, one should obtain
answers to a short set of questions:
- What, exactly, is the problem which needs to be solved?
- Who are the users affected by this problem? Which use cases should the
solution address?
- How does the kernel fall short in addressing that problem now?
Only then does it make sense to start considering possible solutions.
Early discussion
----------------
When planning a kernel development project, it makes great sense to hold
discussions with the community before launching into implementation. Early
communication can save time and trouble in a number of ways:
- It may well be that the problem is addressed by the kernel in ways which
you have not understood. The Linux kernel is large and has a number of
features and capabilities which are not immediately obvious. Not all
kernel capabilities are documented as well as one might like, and it is
easy to miss things. Your author has seen the posting of a complete
driver which duplicated an existing driver that the new author had been
unaware of. Code which reinvents existing wheels is not only wasteful;
it will also not be accepted into the mainline kernel.
- There may be elements of the proposed solution which will not be
acceptable for mainline merging. It is better to find out about
problems like this before writing the code.
- It's entirely possible that other developers have thought about the
problem; they may have ideas for a better solution, and may be willing
to help in the creation of that solution.
Years of experience with the kernel development community have taught a
clear lesson: kernel code which is designed and developed behind closed
doors invariably has problems which are only revealed when the code is
released into the community. Sometimes these problems are severe,
requiring months or years of effort before the code can be brought up to
the kernel community's standards. Some examples include:
- The Devicescape network stack was designed and implemented for
single-processor systems. It could not be merged into the mainline
until it was made suitable for multiprocessor systems. Retrofitting
locking and such into code is a difficult task; as a result, the merging
of this code (now called mac80211) was delayed for over a year.
- The Reiser4 filesystem included a number of capabilities which, in the
core kernel developers' opinion, should have been implemented in the
virtual filesystem layer instead. It also included features which could
not easily be implemented without exposing the system to user-caused
deadlocks. The late revelation of these problems - and refusal to
address some of them - has caused Reiser4 to stay out of the mainline
kernel.
- The AppArmor security module made use of internal virtual filesystem
data structures in ways which were considered to be unsafe and
unreliable. This concern (among others) kept AppArmor out of the
mainline for years.
In each of these cases, a great deal of pain and extra work could have been
avoided with some early discussion with the kernel developers.
Who do you talk to?
-------------------
When developers decide to take their plans public, the next question will
be: where do we start? The answer is to find the right mailing list(s) and
the right maintainer. For mailing lists, the best approach is to look in
the MAINTAINERS file for a relevant place to post. If there is a suitable
subsystem list, posting there is often preferable to posting on
linux-kernel; you are more likely to reach developers with expertise in the
relevant subsystem and the environment may be more supportive.
Finding maintainers can be a bit harder. Again, the MAINTAINERS file is
the place to start. That file tends to not always be up to date, though,
and not all subsystems are represented there. The person listed in the
MAINTAINERS file may, in fact, not be the person who is actually acting in
that role currently. So, when there is doubt about who to contact, a
useful trick is to use git (and "git log" in particular) to see who is
currently active within the subsystem of interest. Look at who is writing
patches, and who, if anybody, is attaching Signed-off-by lines to those
patches. Those are the people who will be best placed to help with a new
development project.
The task of finding the right maintainer is sometimes challenging enough
that the kernel developers have added a script to ease the process:
::
.../scripts/get_maintainer.pl
This script will return the current maintainer(s) for a given file or
directory when given the "-f" option. If passed a patch on the
command line, it will list the maintainers who should probably receive
copies of the patch. There are a number of options regulating how hard
get_maintainer.pl will search for maintainers; please be careful about
using the more aggressive options as you may end up including developers
who have no real interest in the code you are modifying.
If all else fails, talking to Andrew Morton can be an effective way to
track down a maintainer for a specific piece of code.
When to post?
-------------
If possible, posting your plans during the early stages can only be
helpful. Describe the problem being solved and any plans that have been
made on how the implementation will be done. Any information you can
provide can help the development community provide useful input on the
project.
One discouraging thing which can happen at this stage is not a hostile
reaction, but, instead, little or no reaction at all. The sad truth of the
matter is (1) kernel developers tend to be busy, (2) there is no shortage
of people with grand plans and little code (or even prospect of code) to
back them up, and (3) nobody is obligated to review or comment on ideas
posted by others. Beyond that, high-level designs often hide problems
which are only revealed when somebody actually tries to implement those
designs; for that reason, kernel developers would rather see the code.
If a request-for-comments posting yields little in the way of comments, do
not assume that it means there is no interest in the project.
Unfortunately, you also cannot assume that there are no problems with your
idea. The best thing to do in this situation is to proceed, keeping the
community informed as you go.
Getting official buy-in
-----------------------
If your work is being done in a corporate environment - as most Linux
kernel work is - you must, obviously, have permission from suitably
empowered managers before you can post your company's plans or code to a
public mailing list. The posting of code which has not been cleared for
release under a GPL-compatible license can be especially problematic; the
sooner that a company's management and legal staff can agree on the posting
of a kernel development project, the better off everybody involved will be.
Some readers may be thinking at this point that their kernel work is
intended to support a product which does not yet have an officially
acknowledged existence. Revealing their employer's plans on a public
mailing list may not be a viable option. In cases like this, it is worth
considering whether the secrecy is really necessary; there is often no real
need to keep development plans behind closed doors.
That said, there are also cases where a company legitimately cannot
disclose its plans early in the development process. Companies with
experienced kernel developers may choose to proceed in an open-loop manner
on the assumption that they will be able to avoid serious integration
problems later. For companies without that sort of in-house expertise, the
best option is often to hire an outside developer to review the plans under
a non-disclosure agreement. The Linux Foundation operates an NDA program
designed to help with this sort of situation; more information can be found
at:
http://www.linuxfoundation.org/en/NDA_program
This kind of review is often enough to avoid serious problems later on
without requiring public disclosure of the project.
Documentation/process/4.Coding.rst
+422
View File
@@ -0,0 +1,422 @@
.. _development_coding:
Getting the code right
======================
While there is much to be said for a solid and community-oriented design
process, the proof of any kernel development project is in the resulting
code. It is the code which will be examined by other developers and merged
(or not) into the mainline tree. So it is the quality of this code which
will determine the ultimate success of the project.
This section will examine the coding process. We'll start with a look at a
number of ways in which kernel developers can go wrong. Then the focus
will shift toward doing things right and the tools which can help in that
quest.
Pitfalls
---------
Coding style
************
The kernel has long had a standard coding style, described in
:ref:`Documentation/process/coding-style.rst <codingstyle>`. For much of
that time, the policies described in that file were taken as being, at most,
advisory. As a result, there is a substantial amount of code in the kernel
which does not meet the coding style guidelines. The presence of that code
leads to two independent hazards for kernel developers.
The first of these is to believe that the kernel coding standards do not
matter and are not enforced. The truth of the matter is that adding new
code to the kernel is very difficult if that code is not coded according to
the standard; many developers will request that the code be reformatted
before they will even review it. A code base as large as the kernel
requires some uniformity of code to make it possible for developers to
quickly understand any part of it. So there is no longer room for
strangely-formatted code.
Occasionally, the kernel's coding style will run into conflict with an
employer's mandated style. In such cases, the kernel's style will have to
win before the code can be merged. Putting code into the kernel means
giving up a degree of control in a number of ways - including control over
how the code is formatted.
The other trap is to assume that code which is already in the kernel is
urgently in need of coding style fixes. Developers may start to generate
reformatting patches as a way of gaining familiarity with the process, or
as a way of getting their name into the kernel changelogs - or both. But
pure coding style fixes are seen as noise by the development community;
they tend to get a chilly reception. So this type of patch is best
avoided. It is natural to fix the style of a piece of code while working
on it for other reasons, but coding style changes should not be made for
their own sake.
The coding style document also should not be read as an absolute law which
can never be transgressed. If there is a good reason to go against the
style (a line which becomes far less readable if split to fit within the
80-column limit, for example), just do it.
Note that you can also use the ``clang-format`` tool to help you with
these rules, to quickly re-format parts of your code automatically,
and to review full files in order to spot coding style mistakes,
typos and possible improvements. It is also handy for sorting ``#includes``,
for aligning variables/macros, for reflowing text and other similar tasks.
See the file :ref:`Documentation/process/clang-format.rst <clangformat>`
for more details.
Abstraction layers
******************
Computer Science professors teach students to make extensive use of
abstraction layers in the name of flexibility and information hiding.
Certainly the kernel makes extensive use of abstraction; no project
involving several million lines of code could do otherwise and survive.
But experience has shown that excessive or premature abstraction can be
just as harmful as premature optimization. Abstraction should be used to
the level required and no further.
At a simple level, consider a function which has an argument which is
always passed as zero by all callers. One could retain that argument just
in case somebody eventually needs to use the extra flexibility that it
provides. By that time, though, chances are good that the code which
implements this extra argument has been broken in some subtle way which was
never noticed - because it has never been used. Or, when the need for
extra flexibility arises, it does not do so in a way which matches the
programmer's early expectation. Kernel developers will routinely submit
patches to remove unused arguments; they should, in general, not be added
in the first place.
Abstraction layers which hide access to hardware - often to allow the bulk
of a driver to be used with multiple operating systems - are especially
frowned upon. Such layers obscure the code and may impose a performance
penalty; they do not belong in the Linux kernel.
On the other hand, if you find yourself copying significant amounts of code
from another kernel subsystem, it is time to ask whether it would, in fact,
make sense to pull out some of that code into a separate library or to
implement that functionality at a higher level. There is no value in
replicating the same code throughout the kernel.
#ifdef and preprocessor use in general
**************************************
The C preprocessor seems to present a powerful temptation to some C
programmers, who see it as a way to efficiently encode a great deal of
flexibility into a source file. But the preprocessor is not C, and heavy
use of it results in code which is much harder for others to read and
harder for the compiler to check for correctness. Heavy preprocessor use
is almost always a sign of code which needs some cleanup work.
Conditional compilation with #ifdef is, indeed, a powerful feature, and it
is used within the kernel. But there is little desire to see code which is
sprinkled liberally with #ifdef blocks. As a general rule, #ifdef use
should be confined to header files whenever possible.
Conditionally-compiled code can be confined to functions which, if the code
is not to be present, simply become empty. The compiler will then quietly
optimize out the call to the empty function. The result is far cleaner
code which is easier to follow.
C preprocessor macros present a number of hazards, including possible
multiple evaluation of expressions with side effects and no type safety.
If you are tempted to define a macro, consider creating an inline function
instead. The code which results will be the same, but inline functions are
easier to read, do not evaluate their arguments multiple times, and allow
the compiler to perform type checking on the arguments and return value.
Inline functions
****************
Inline functions present a hazard of their own, though. Programmers can
become enamored of the perceived efficiency inherent in avoiding a function
call and fill a source file with inline functions. Those functions,
however, can actually reduce performance. Since their code is replicated
at each call site, they end up bloating the size of the compiled kernel.
That, in turn, creates pressure on the processor's memory caches, which can
slow execution dramatically. Inline functions, as a rule, should be quite
small and relatively rare. The cost of a function call, after all, is not
that high; the creation of large numbers of inline functions is a classic
example of premature optimization.
In general, kernel programmers ignore cache effects at their peril. The
classic time/space tradeoff taught in beginning data structures classes
often does not apply to contemporary hardware. Space *is* time, in that a
larger program will run slower than one which is more compact.
More recent compilers take an increasingly active role in deciding whether
a given function should actually be inlined or not. So the liberal
placement of "inline" keywords may not just be excessive; it could also be
irrelevant.
Locking
*******
In May, 2006, the "Devicescape" networking stack was, with great
fanfare, released under the GPL and made available for inclusion in the
mainline kernel. This donation was welcome news; support for wireless
networking in Linux was considered substandard at best, and the Devicescape
stack offered the promise of fixing that situation. Yet, this code did not
actually make it into the mainline until June, 2007 (2.6.22). What
happened?
This code showed a number of signs of having been developed behind
corporate doors. But one large problem in particular was that it was not
designed to work on multiprocessor systems. Before this networking stack
(now called mac80211) could be merged, a locking scheme needed to be
retrofitted onto it.
Once upon a time, Linux kernel code could be developed without thinking
about the concurrency issues presented by multiprocessor systems. Now,
however, this document is being written on a dual-core laptop. Even on
single-processor systems, work being done to improve responsiveness will
raise the level of concurrency within the kernel. The days when kernel
code could be written without thinking about locking are long past.
Any resource (data structures, hardware registers, etc.) which could be
accessed concurrently by more than one thread must be protected by a lock.
New code should be written with this requirement in mind; retrofitting
locking after the fact is a rather more difficult task. Kernel developers
should take the time to understand the available locking primitives well
enough to pick the right tool for the job. Code which shows a lack of
attention to concurrency will have a difficult path into the mainline.
Regressions
***********
One final hazard worth mentioning is this: it can be tempting to make a
change (which may bring big improvements) which causes something to break
for existing users. This kind of change is called a "regression," and
regressions have become most unwelcome in the mainline kernel. With few
exceptions, changes which cause regressions will be backed out if the
regression cannot be fixed in a timely manner. Far better to avoid the
regression in the first place.
It is often argued that a regression can be justified if it causes things
to work for more people than it creates problems for. Why not make a
change if it brings new functionality to ten systems for each one it
breaks? The best answer to this question was expressed by Linus in July,
2007:
::
So we don't fix bugs by introducing new problems. That way lies
madness, and nobody ever knows if you actually make any real
progress at all. Is it two steps forwards, one step back, or one
step forward and two steps back?
(http://lwn.net/Articles/243460/).
An especially unwelcome type of regression is any sort of change to the
user-space ABI. Once an interface has been exported to user space, it must
be supported indefinitely. This fact makes the creation of user-space
interfaces particularly challenging: since they cannot be changed in
incompatible ways, they must be done right the first time. For this
reason, a great deal of thought, clear documentation, and wide review for
user-space interfaces is always required.
Code checking tools
-------------------
For now, at least, the writing of error-free code remains an ideal that few
of us can reach. What we can hope to do, though, is to catch and fix as
many of those errors as possible before our code goes into the mainline
kernel. To that end, the kernel developers have put together an impressive
array of tools which can catch a wide variety of obscure problems in an
automated way. Any problem caught by the computer is a problem which will
not afflict a user later on, so it stands to reason that the automated
tools should be used whenever possible.
The first step is simply to heed the warnings produced by the compiler.
Contemporary versions of gcc can detect (and warn about) a large number of
potential errors. Quite often, these warnings point to real problems.
Code submitted for review should, as a rule, not produce any compiler
warnings. When silencing warnings, take care to understand the real cause
and try to avoid "fixes" which make the warning go away without addressing
its cause.
Note that not all compiler warnings are enabled by default. Build the
kernel with "make EXTRA_CFLAGS=-W" to get the full set.
The kernel provides several configuration options which turn on debugging
features; most of these are found in the "kernel hacking" submenu. Several
of these options should be turned on for any kernel used for development or
testing purposes. In particular, you should turn on:
- ENABLE_WARN_DEPRECATED, ENABLE_MUST_CHECK, and FRAME_WARN to get an
extra set of warnings for problems like the use of deprecated interfaces
or ignoring an important return value from a function. The output
generated by these warnings can be verbose, but one need not worry about
warnings from other parts of the kernel.
- DEBUG_OBJECTS will add code to track the lifetime of various objects
created by the kernel and warn when things are done out of order. If
you are adding a subsystem which creates (and exports) complex objects
of its own, consider adding support for the object debugging
infrastructure.
- DEBUG_SLAB can find a variety of memory allocation and use errors; it
should be used on most development kernels.
- DEBUG_SPINLOCK, DEBUG_ATOMIC_SLEEP, and DEBUG_MUTEXES will find a
number of common locking errors.
There are quite a few other debugging options, some of which will be
discussed below. Some of them have a significant performance impact and
should not be used all of the time. But some time spent learning the
available options will likely be paid back many times over in short order.
One of the heavier debugging tools is the locking checker, or "lockdep."
This tool will track the acquisition and release of every lock (spinlock or
mutex) in the system, the order in which locks are acquired relative to
each other, the current interrupt environment, and more. It can then
ensure that locks are always acquired in the same order, that the same
interrupt assumptions apply in all situations, and so on. In other words,
lockdep can find a number of scenarios in which the system could, on rare
occasion, deadlock. This kind of problem can be painful (for both
developers and users) in a deployed system; lockdep allows them to be found
in an automated manner ahead of time. Code with any sort of non-trivial
locking should be run with lockdep enabled before being submitted for
inclusion.
As a diligent kernel programmer, you will, beyond doubt, check the return
status of any operation (such as a memory allocation) which can fail. The
fact of the matter, though, is that the resulting failure recovery paths
are, probably, completely untested. Untested code tends to be broken code;
you could be much more confident of your code if all those error-handling
paths had been exercised a few times.
The kernel provides a fault injection framework which can do exactly that,
especially where memory allocations are involved. With fault injection
enabled, a configurable percentage of memory allocations will be made to
fail; these failures can be restricted to a specific range of code.
Running with fault injection enabled allows the programmer to see how the
code responds when things go badly. See
Documentation/fault-injection/fault-injection.txt for more information on
how to use this facility.
Other kinds of errors can be found with the "sparse" static analysis tool.
With sparse, the programmer can be warned about confusion between
user-space and kernel-space addresses, mixture of big-endian and
small-endian quantities, the passing of integer values where a set of bit
flags is expected, and so on. Sparse must be installed separately (it can
be found at https://sparse.wiki.kernel.org/index.php/Main_Page if your
distributor does not package it); it can then be run on the code by adding
"C=1" to your make command.
The "Coccinelle" tool (http://coccinelle.lip6.fr/) is able to find a wide
variety of potential coding problems; it can also propose fixes for those
problems. Quite a few "semantic patches" for the kernel have been packaged
under the scripts/coccinelle directory; running "make coccicheck" will run
through those semantic patches and report on any problems found. See
Documentation/dev-tools/coccinelle.rst for more information.
Other kinds of portability errors are best found by compiling your code for
other architectures. If you do not happen to have an S/390 system or a
Blackfin development board handy, you can still perform the compilation
step. A large set of cross compilers for x86 systems can be found at
http://www.kernel.org/pub/tools/crosstool/
Some time spent installing and using these compilers will help avoid
embarrassment later.
Documentation
-------------
Documentation has often been more the exception than the rule with kernel
development. Even so, adequate documentation will help to ease the merging
of new code into the kernel, make life easier for other developers, and
will be helpful for your users. In many cases, the addition of
documentation has become essentially mandatory.
The first piece of documentation for any patch is its associated
changelog. Log entries should describe the problem being solved, the form
of the solution, the people who worked on the patch, any relevant
effects on performance, and anything else that might be needed to
understand the patch. Be sure that the changelog says *why* the patch is
worth applying; a surprising number of developers fail to provide that
information.
Any code which adds a new user-space interface - including new sysfs or
/proc files - should include documentation of that interface which enables
user-space developers to know what they are working with. See
Documentation/ABI/README for a description of how this documentation should
be formatted and what information needs to be provided.
The file :ref:`Documentation/admin-guide/kernel-parameters.rst
<kernelparameters>` describes all of the kernel's boot-time parameters.
Any patch which adds new parameters should add the appropriate entries to
this file.
Any new configuration options must be accompanied by help text which
clearly explains the options and when the user might want to select them.
Internal API information for many subsystems is documented by way of
specially-formatted comments; these comments can be extracted and formatted
in a number of ways by the "kernel-doc" script. If you are working within
a subsystem which has kerneldoc comments, you should maintain them and add
them, as appropriate, for externally-available functions. Even in areas
which have not been so documented, there is no harm in adding kerneldoc
comments for the future; indeed, this can be a useful activity for
beginning kernel developers. The format of these comments, along with some
information on how to create kerneldoc templates can be found at
:ref:`Documentation/doc-guide/ <doc_guide>`.
Anybody who reads through a significant amount of existing kernel code will
note that, often, comments are most notable by their absence. Once again,
the expectations for new code are higher than they were in the past;
merging uncommented code will be harder. That said, there is little desire
for verbosely-commented code. The code should, itself, be readable, with
comments explaining the more subtle aspects.
Certain things should always be commented. Uses of memory barriers should
be accompanied by a line explaining why the barrier is necessary. The
locking rules for data structures generally need to be explained somewhere.
Major data structures need comprehensive documentation in general.
Non-obvious dependencies between separate bits of code should be pointed
out. Anything which might tempt a code janitor to make an incorrect
"cleanup" needs a comment saying why it is done the way it is. And so on.
Internal API changes
--------------------
The binary interface provided by the kernel to user space cannot be broken
except under the most severe circumstances. The kernel's internal
programming interfaces, instead, are highly fluid and can be changed when
the need arises. If you find yourself having to work around a kernel API,
or simply not using a specific functionality because it does not meet your
needs, that may be a sign that the API needs to change. As a kernel
developer, you are empowered to make such changes.
There are, of course, some catches. API changes can be made, but they need
to be well justified. So any patch making an internal API change should be
accompanied by a description of what the change is and why it is
necessary. This kind of change should also be broken out into a separate
patch, rather than buried within a larger patch.
The other catch is that a developer who changes an internal API is
generally charged with the task of fixing any code within the kernel tree
which is broken by the change. For a widely-used function, this duty can
lead to literally hundreds or thousands of changes - many of which are
likely to conflict with work being done by other developers. Needless to
say, this can be a large job, so it is best to be sure that the
justification is solid. Note that the Coccinelle tool can help with
wide-ranging API changes.
When making an incompatible API change, one should, whenever possible,
ensure that code which has not been updated is caught by the compiler.
This will help you to be sure that you have found all in-tree uses of that
interface. It will also alert developers of out-of-tree code that there is
a change that they need to respond to. Supporting out-of-tree code is not
something that kernel developers need to be worried about, but we also do
not have to make life harder for out-of-tree developers than it needs to
be.
Documentation/process/5.Posting.rst
+326
View File
@@ -0,0 +1,326 @@
.. _development_posting:
Posting patches
===============
Sooner or later, the time comes when your work is ready to be presented to
the community for review and, eventually, inclusion into the mainline
kernel. Unsurprisingly, the kernel development community has evolved a set
of conventions and procedures which are used in the posting of patches;
following them will make life much easier for everybody involved. This
document will attempt to cover these expectations in reasonable detail;
more information can also be found in the files process/submitting-patches.rst,
process/submitting-drivers.rst, and process/submit-checklist.rst in the kernel
documentation directory.
When to post
------------
There is a constant temptation to avoid posting patches before they are
completely "ready." For simple patches, that is not a problem. If the
work being done is complex, though, there is a lot to be gained by getting
feedback from the community before the work is complete. So you should
consider posting in-progress work, or even making a git tree available so
that interested developers can catch up with your work at any time.
When posting code which is not yet considered ready for inclusion, it is a
good idea to say so in the posting itself. Also mention any major work
which remains to be done and any known problems. Fewer people will look at
patches which are known to be half-baked, but those who do will come in
with the idea that they can help you drive the work in the right direction.
Before creating patches
-----------------------
There are a number of things which should be done before you consider
sending patches to the development community. These include:
- Test the code to the extent that you can. Make use of the kernel's
debugging tools, ensure that the kernel will build with all reasonable
combinations of configuration options, use cross-compilers to build for
different architectures, etc.
- Make sure your code is compliant with the kernel coding style
guidelines.
- Does your change have performance implications? If so, you should run
benchmarks showing what the impact (or benefit) of your change is; a
summary of the results should be included with the patch.
- Be sure that you have the right to post the code. If this work was done
for an employer, the employer likely has a right to the work and must be
agreeable with its release under the GPL.
As a general rule, putting in some extra thought before posting code almost
always pays back the effort in short order.
Patch preparation
-----------------
The preparation of patches for posting can be a surprising amount of work,
but, once again, attempting to save time here is not generally advisable
even in the short term.
Patches must be prepared against a specific version of the kernel. As a
general rule, a patch should be based on the current mainline as found in
Linus's git tree. When basing on mainline, start with a well-known release
point - a stable or -rc release - rather than branching off the mainline at
an arbitrary spot.
It may become necessary to make versions against -mm, linux-next, or a
subsystem tree, though, to facilitate wider testing and review. Depending
on the area of your patch and what is going on elsewhere, basing a patch
against these other trees can require a significant amount of work
resolving conflicts and dealing with API changes.
Only the most simple changes should be formatted as a single patch;
everything else should be made as a logical series of changes. Splitting
up patches is a bit of an art; some developers spend a long time figuring
out how to do it in the way that the community expects. There are a few
rules of thumb, however, which can help considerably:
- The patch series you post will almost certainly not be the series of
changes found in your working revision control system. Instead, the
changes you have made need to be considered in their final form, then
split apart in ways which make sense. The developers are interested in
discrete, self-contained changes, not the path you took to get to those
changes.
- Each logically independent change should be formatted as a separate
patch. These changes can be small ("add a field to this structure") or
large (adding a significant new driver, for example), but they should be
conceptually small and amenable to a one-line description. Each patch
should make a specific change which can be reviewed on its own and
verified to do what it says it does.
- As a way of restating the guideline above: do not mix different types of
changes in the same patch. If a single patch fixes a critical security
bug, rearranges a few structures, and reformats the code, there is a
good chance that it will be passed over and the important fix will be
lost.
- Each patch should yield a kernel which builds and runs properly; if your
patch series is interrupted in the middle, the result should still be a
working kernel. Partial application of a patch series is a common
scenario when the "git bisect" tool is used to find regressions; if the
result is a broken kernel, you will make life harder for developers and
users who are engaging in the noble work of tracking down problems.
- Do not overdo it, though. One developer once posted a set of edits
to a single file as 500 separate patches - an act which did not make him
the most popular person on the kernel mailing list. A single patch can
be reasonably large as long as it still contains a single *logical*
change.
- It can be tempting to add a whole new infrastructure with a series of
patches, but to leave that infrastructure unused until the final patch
in the series enables the whole thing. This temptation should be
avoided if possible; if that series adds regressions, bisection will
finger the last patch as the one which caused the problem, even though
the real bug is elsewhere. Whenever possible, a patch which adds new
code should make that code active immediately.
Working to create the perfect patch series can be a frustrating process
which takes quite a bit of time and thought after the "real work" has been
done. When done properly, though, it is time well spent.
Patch formatting and changelogs
-------------------------------
So now you have a perfect series of patches for posting, but the work is
not done quite yet. Each patch needs to be formatted into a message which
quickly and clearly communicates its purpose to the rest of the world. To
that end, each patch will be composed of the following:
- An optional "From" line naming the author of the patch. This line is
only necessary if you are passing on somebody else's patch via email,
but it never hurts to add it when in doubt.
- A one-line description of what the patch does. This message should be
enough for a reader who sees it with no other context to figure out the
scope of the patch; it is the line that will show up in the "short form"
changelogs. This message is usually formatted with the relevant
subsystem name first, followed by the purpose of the patch. For
example:
::
gpio: fix build on CONFIG_GPIO_SYSFS=n
- A blank line followed by a detailed description of the contents of the
patch. This description can be as long as is required; it should say
what the patch does and why it should be applied to the kernel.
- One or more tag lines, with, at a minimum, one Signed-off-by: line from
the author of the patch. Tags will be described in more detail below.
The items above, together, form the changelog for the patch. Writing good
changelogs is a crucial but often-neglected art; it's worth spending
another moment discussing this issue. When writing a changelog, you should
bear in mind that a number of different people will be reading your words.
These include subsystem maintainers and reviewers who need to decide
whether the patch should be included, distributors and other maintainers
trying to decide whether a patch should be backported to other kernels, bug
hunters wondering whether the patch is responsible for a problem they are
chasing, users who want to know how the kernel has changed, and more. A
good changelog conveys the needed information to all of these people in the
most direct and concise way possible.
To that end, the summary line should describe the effects of and motivation
for the change as well as possible given the one-line constraint. The
detailed description can then amplify on those topics and provide any
needed additional information. If the patch fixes a bug, cite the commit
which introduced the bug if possible (and please provide both the commit ID
and the title when citing commits). If a problem is associated with
specific log or compiler output, include that output to help others
searching for a solution to the same problem. If the change is meant to
support other changes coming in later patch, say so. If internal APIs are
changed, detail those changes and how other developers should respond. In
general, the more you can put yourself into the shoes of everybody who will
be reading your changelog, the better that changelog (and the kernel as a
whole) will be.
Needless to say, the changelog should be the text used when committing the
change to a revision control system. It will be followed by:
- The patch itself, in the unified ("-u") patch format. Using the "-p"
option to diff will associate function names with changes, making the
resulting patch easier for others to read.
You should avoid including changes to irrelevant files (those generated by
the build process, for example, or editor backup files) in the patch. The
file "dontdiff" in the Documentation directory can help in this regard;
pass it to diff with the "-X" option.
The tags mentioned above are used to describe how various developers have
been associated with the development of this patch. They are described in
detail in the process/submitting-patches.rst document; what follows here is a
brief summary. Each of these lines has the format:
::
tag: Full Name <email address> optional-other-stuff
The tags in common use are:
- Signed-off-by: this is a developer's certification that he or she has
the right to submit the patch for inclusion into the kernel. It is an
agreement to the Developer's Certificate of Origin, the full text of
which can be found in Documentation/process/submitting-patches.rst. Code
without a proper signoff cannot be merged into the mainline.
- Co-developed-by: states that the patch was also created by another developer
along with the original author. This is useful at times when multiple
people work on a single patch. Note, this person also needs to have a
Signed-off-by: line in the patch as well.
- Acked-by: indicates an agreement by another developer (often a
maintainer of the relevant code) that the patch is appropriate for
inclusion into the kernel.
- Tested-by: states that the named person has tested the patch and found
it to work.
- Reviewed-by: the named developer has reviewed the patch for correctness;
see the reviewer's statement in Documentation/process/submitting-patches.rst
for more detail.
- Reported-by: names a user who reported a problem which is fixed by this
patch; this tag is used to give credit to the (often underappreciated)
people who test our code and let us know when things do not work
correctly.
- Cc: the named person received a copy of the patch and had the
opportunity to comment on it.
Be careful in the addition of tags to your patches: only Cc: is appropriate
for addition without the explicit permission of the person named.
Sending the patch
-----------------
Before you mail your patches, there are a couple of other things you should
take care of:
- Are you sure that your mailer will not corrupt the patches? Patches
which have had gratuitous white-space changes or line wrapping performed
by the mail client will not apply at the other end, and often will not
be examined in any detail. If there is any doubt at all, mail the patch
to yourself and convince yourself that it shows up intact.
Documentation/process/email-clients.rst has some helpful hints on making
specific mail clients work for sending patches.
- Are you sure your patch is free of silly mistakes? You should always
run patches through scripts/checkpatch.pl and address the complaints it
comes up with. Please bear in mind that checkpatch.pl, while being the
embodiment of a fair amount of thought about what kernel patches should
look like, is not smarter than you. If fixing a checkpatch.pl complaint
would make the code worse, don't do it.
Patches should always be sent as plain text. Please do not send them as
attachments; that makes it much harder for reviewers to quote sections of
the patch in their replies. Instead, just put the patch directly into your
message.
When mailing patches, it is important to send copies to anybody who might
be interested in it. Unlike some other projects, the kernel encourages
people to err on the side of sending too many copies; don't assume that the
relevant people will see your posting on the mailing lists. In particular,
copies should go to:
- The maintainer(s) of the affected subsystem(s). As described earlier,
the MAINTAINERS file is the first place to look for these people.
- Other developers who have been working in the same area - especially
those who might be working there now. Using git to see who else has
modified the files you are working on can be helpful.
- If you are responding to a bug report or a feature request, copy the
original poster as well.
- Send a copy to the relevant mailing list, or, if nothing else applies,
the linux-kernel list.
- If you are fixing a bug, think about whether the fix should go into the
next stable update. If so, stable@vger.kernel.org should get a copy of
the patch. Also add a "Cc: stable@vger.kernel.org" to the tags within
the patch itself; that will cause the stable team to get a notification
when your fix goes into the mainline.
When selecting recipients for a patch, it is good to have an idea of who
you think will eventually accept the patch and get it merged. While it
is possible to send patches directly to Linus Torvalds and have him merge
them, things are not normally done that way. Linus is busy, and there are
subsystem maintainers who watch over specific parts of the kernel. Usually
you will be wanting that maintainer to merge your patches. If there is no
obvious maintainer, Andrew Morton is often the patch target of last resort.
Patches need good subject lines. The canonical format for a patch line is
something like:
::
[PATCH nn/mm] subsys: one-line description of the patch
where "nn" is the ordinal number of the patch, "mm" is the total number of
patches in the series, and "subsys" is the name of the affected subsystem.
Clearly, nn/mm can be omitted for a single, standalone patch.
If you have a significant series of patches, it is customary to send an
introductory description as part zero. This convention is not universally
followed though; if you use it, remember that information in the
introduction does not make it into the kernel changelogs. So please ensure
that the patches, themselves, have complete changelog information.
In general, the second and following parts of a multi-part patch should be
sent as a reply to the first part so that they all thread together at the
receiving end. Tools like git and quilt have commands to mail out a set of
patches with the proper threading. If you have a long series, though, and
are using git, please stay away from the --chain-reply-to option to avoid
creating exceptionally deep nesting.
Documentation/process/6.Followthrough.rst
+212
View File
@@ -0,0 +1,212 @@
.. _development_followthrough:
Followthrough
=============
At this point, you have followed the guidelines given so far and, with the
addition of your own engineering skills, have posted a perfect series of
patches. One of the biggest mistakes that even experienced kernel
developers can make is to conclude that their work is now done. In truth,
posting patches indicates a transition into the next stage of the process,
with, possibly, quite a bit of work yet to be done.
It is a rare patch which is so good at its first posting that there is no
room for improvement. The kernel development process recognizes this fact,
and, as a result, is heavily oriented toward the improvement of posted
code. You, as the author of that code, will be expected to work with the
kernel community to ensure that your code is up to the kernel's quality
standards. A failure to participate in this process is quite likely to
prevent the inclusion of your patches into the mainline.
Working with reviewers
----------------------
A patch of any significance will result in a number of comments from other
developers as they review the code. Working with reviewers can be, for
many developers, the most intimidating part of the kernel development
process. Life can be made much easier, though, if you keep a few things in
mind:
- If you have explained your patch well, reviewers will understand its
value and why you went to the trouble of writing it. But that value
will not keep them from asking a fundamental question: what will it be
like to maintain a kernel with this code in it five or ten years later?
Many of the changes you may be asked to make - from coding style tweaks
to substantial rewrites - come from the understanding that Linux will
still be around and under development a decade from now.
- Code review is hard work, and it is a relatively thankless occupation;
people remember who wrote kernel code, but there is little lasting fame
for those who reviewed it. So reviewers can get grumpy, especially when
they see the same mistakes being made over and over again. If you get a
review which seems angry, insulting, or outright offensive, resist the
impulse to respond in kind. Code review is about the code, not about
the people, and code reviewers are not attacking you personally.
- Similarly, code reviewers are not trying to promote their employers'
agendas at the expense of your own. Kernel developers often expect to
be working on the kernel years from now, but they understand that their
employer could change. They truly are, almost without exception,
working toward the creation of the best kernel they can; they are not
trying to create discomfort for their employers' competitors.
What all of this comes down to is that, when reviewers send you comments,
you need to pay attention to the technical observations that they are
making. Do not let their form of expression or your own pride keep that
from happening. When you get review comments on a patch, take the time to
understand what the reviewer is trying to say. If possible, fix the things
that the reviewer is asking you to fix. And respond back to the reviewer:
thank them, and describe how you will answer their questions.
Note that you do not have to agree with every change suggested by
reviewers. If you believe that the reviewer has misunderstood your code,
explain what is really going on. If you have a technical objection to a
suggested change, describe it and justify your solution to the problem. If
your explanations make sense, the reviewer will accept them. Should your
explanation not prove persuasive, though, especially if others start to
agree with the reviewer, take some time to think things over again. It can
be easy to become blinded by your own solution to a problem to the point
that you don't realize that something is fundamentally wrong or, perhaps,
you're not even solving the right problem.
Andrew Morton has suggested that every review comment which does not result
in a code change should result in an additional code comment instead; that
can help future reviewers avoid the questions which came up the first time
around.
One fatal mistake is to ignore review comments in the hope that they will
go away. They will not go away. If you repost code without having
responded to the comments you got the time before, you're likely to find
that your patches go nowhere.
Speaking of reposting code: please bear in mind that reviewers are not
going to remember all the details of the code you posted the last time
around. So it is always a good idea to remind reviewers of previously
raised issues and how you dealt with them; the patch changelog is a good
place for this kind of information. Reviewers should not have to search
through list archives to familiarize themselves with what was said last
time; if you help them get a running start, they will be in a better mood
when they revisit your code.
What if you've tried to do everything right and things still aren't going
anywhere? Most technical disagreements can be resolved through discussion,
but there are times when somebody simply has to make a decision. If you
honestly believe that this decision is going against you wrongly, you can
always try appealing to a higher power. As of this writing, that higher
power tends to be Andrew Morton. Andrew has a great deal of respect in the
kernel development community; he can often unjam a situation which seems to
be hopelessly blocked. Appealing to Andrew should not be done lightly,
though, and not before all other alternatives have been explored. And bear
in mind, of course, that he may not agree with you either.
What happens next
-----------------
If a patch is considered to be a good thing to add to the kernel, and once
most of the review issues have been resolved, the next step is usually
entry into a subsystem maintainer's tree. How that works varies from one
subsystem to the next; each maintainer has his or her own way of doing
things. In particular, there may be more than one tree - one, perhaps,
dedicated to patches planned for the next merge window, and another for
longer-term work.
For patches applying to areas for which there is no obvious subsystem tree
(memory management patches, for example), the default tree often ends up
being -mm. Patches which affect multiple subsystems can also end up going
through the -mm tree.
Inclusion into a subsystem tree can bring a higher level of visibility to a
patch. Now other developers working with that tree will get the patch by
default. Subsystem trees typically feed linux-next as well, making their
contents visible to the development community as a whole. At this point,
there's a good chance that you will get more comments from a new set of
reviewers; these comments need to be answered as in the previous round.
What may also happen at this point, depending on the nature of your patch,
is that conflicts with work being done by others turn up. In the worst
case, heavy patch conflicts can result in some work being put on the back
burner so that the remaining patches can be worked into shape and merged.
Other times, conflict resolution will involve working with the other
developers and, possibly, moving some patches between trees to ensure that
everything applies cleanly. This work can be a pain, but count your
blessings: before the advent of the linux-next tree, these conflicts often
only turned up during the merge window and had to be addressed in a hurry.
Now they can be resolved at leisure, before the merge window opens.
Some day, if all goes well, you'll log on and see that your patch has been
merged into the mainline kernel. Congratulations! Once the celebration is
complete (and you have added yourself to the MAINTAINERS file), though, it
is worth remembering an important little fact: the job still is not done.
Merging into the mainline brings its own challenges.
To begin with, the visibility of your patch has increased yet again. There
may be a new round of comments from developers who had not been aware of
the patch before. It may be tempting to ignore them, since there is no
longer any question of your code being merged. Resist that temptation,
though; you still need to be responsive to developers who have questions or
suggestions.
More importantly, though: inclusion into the mainline puts your code into
the hands of a much larger group of testers. Even if you have contributed
a driver for hardware which is not yet available, you will be surprised by
how many people will build your code into their kernels. And, of course,
where there are testers, there will be bug reports.
The worst sort of bug reports are regressions. If your patch causes a
regression, you'll find an uncomfortable number of eyes upon you;
regressions need to be fixed as soon as possible. If you are unwilling or
unable to fix the regression (and nobody else does it for you), your patch
will almost certainly be removed during the stabilization period. Beyond
negating all of the work you have done to get your patch into the mainline,
having a patch pulled as the result of a failure to fix a regression could
well make it harder for you to get work merged in the future.
After any regressions have been dealt with, there may be other, ordinary
bugs to deal with. The stabilization period is your best opportunity to
fix these bugs and ensure that your code's debut in a mainline kernel
release is as solid as possible. So, please, answer bug reports, and fix
the problems if at all possible. That's what the stabilization period is
for; you can start creating cool new patches once any problems with the old
ones have been taken care of.
And don't forget that there are other milestones which may also create bug
reports: the next mainline stable release, when prominent distributors pick
up a version of the kernel containing your patch, etc. Continuing to
respond to these reports is a matter of basic pride in your work. If that
is insufficient motivation, though, it's also worth considering that the
development community remembers developers who lose interest in their code
after it's merged. The next time you post a patch, they will be evaluating
it with the assumption that you will not be around to maintain it
afterward.
Other things that can happen
-----------------------------
One day, you may open your mail client and see that somebody has mailed you
a patch to your code. That is one of the advantages of having your code
out there in the open, after all. If you agree with the patch, you can
either forward it on to the subsystem maintainer (be sure to include a
proper From: line so that the attribution is correct, and add a signoff of
your own), or send an Acked-by: response back and let the original poster
send it upward.
If you disagree with the patch, send a polite response explaining why. If
possible, tell the author what changes need to be made to make the patch
acceptable to you. There is a certain resistance to merging patches which
are opposed by the author and maintainer of the code, but it only goes so
far. If you are seen as needlessly blocking good work, those patches will
eventually flow around you and get into the mainline anyway. In the Linux
kernel, nobody has absolute veto power over any code. Except maybe Linus.
On very rare occasion, you may see something completely different: another
developer posts a different solution to your problem. At that point,
chances are that one of the two patches will not be merged, and "mine was
here first" is not considered to be a compelling technical argument. If
somebody else's patch displaces yours and gets into the mainline, there is
really only one way to respond: be pleased that your problem got solved and
get on with your work. Having one's work shoved aside in this manner can
be hurtful and discouraging, but the community will remember your reaction
long after they have forgotten whose patch actually got merged.
Documentation/process/7.AdvancedTopics.rst
+178
View File
@@ -0,0 +1,178 @@
.. _development_advancedtopics:
Advanced topics
===============
At this point, hopefully, you have a handle on how the development process
works. There is still more to learn, however! This section will cover a
number of topics which can be helpful for developers wanting to become a
regular part of the Linux kernel development process.
Managing patches with git
-------------------------
The use of distributed version control for the kernel began in early 2002,
when Linus first started playing with the proprietary BitKeeper
application. While BitKeeper was controversial, the approach to software
version management it embodied most certainly was not. Distributed version
control enabled an immediate acceleration of the kernel development
project. In current times, there are several free alternatives to
BitKeeper. For better or for worse, the kernel project has settled on git
as its tool of choice.
Managing patches with git can make life much easier for the developer,
especially as the volume of those patches grows. Git also has its rough
edges and poses certain hazards; it is a young and powerful tool which is
still being civilized by its developers. This document will not attempt to
teach the reader how to use git; that would be sufficient material for a
long document in its own right. Instead, the focus here will be on how git
fits into the kernel development process in particular. Developers who
wish to come up to speed with git will find more information at:
http://git-scm.com/
http://www.kernel.org/pub/software/scm/git/docs/user-manual.html
and on various tutorials found on the web.
The first order of business is to read the above sites and get a solid
understanding of how git works before trying to use it to make patches
available to others. A git-using developer should be able to obtain a copy
of the mainline repository, explore the revision history, commit changes to
the tree, use branches, etc. An understanding of git's tools for the
rewriting of history (such as rebase) is also useful. Git comes with its
own terminology and concepts; a new user of git should know about refs,
remote branches, the index, fast-forward merges, pushes and pulls, detached
heads, etc. It can all be a little intimidating at the outset, but the
concepts are not that hard to grasp with a bit of study.
Using git to generate patches for submission by email can be a good
exercise while coming up to speed.
When you are ready to start putting up git trees for others to look at, you
will, of course, need a server that can be pulled from. Setting up such a
server with git-daemon is relatively straightforward if you have a system
which is accessible to the Internet. Otherwise, free, public hosting sites
(Github, for example) are starting to appear on the net. Established
developers can get an account on kernel.org, but those are not easy to come
by; see http://kernel.org/faq/ for more information.
The normal git workflow involves the use of a lot of branches. Each line
of development can be separated into a separate "topic branch" and
maintained independently. Branches in git are cheap, there is no reason to
not make free use of them. And, in any case, you should not do your
development in any branch which you intend to ask others to pull from.
Publicly-available branches should be created with care; merge in patches
from development branches when they are in complete form and ready to go -
not before.
Git provides some powerful tools which can allow you to rewrite your
development history. An inconvenient patch (one which breaks bisection,
say, or which has some other sort of obvious bug) can be fixed in place or
made to disappear from the history entirely. A patch series can be
rewritten as if it had been written on top of today's mainline, even though
you have been working on it for months. Changes can be transparently
shifted from one branch to another. And so on. Judicious use of git's
ability to revise history can help in the creation of clean patch sets with
fewer problems.
Excessive use of this capability can lead to other problems, though, beyond
a simple obsession for the creation of the perfect project history.
Rewriting history will rewrite the changes contained in that history,
turning a tested (hopefully) kernel tree into an untested one. But, beyond
that, developers cannot easily collaborate if they do not have a shared
view of the project history; if you rewrite history which other developers
have pulled into their repositories, you will make life much more difficult
for those developers. So a simple rule of thumb applies here: history
which has been exported to others should generally be seen as immutable
thereafter.
So, once you push a set of changes to your publicly-available server, those
changes should not be rewritten. Git will attempt to enforce this rule if
you try to push changes which do not result in a fast-forward merge
(i.e. changes which do not share the same history). It is possible to
override this check, and there may be times when it is necessary to rewrite
an exported tree. Moving changesets between trees to avoid conflicts in
linux-next is one example. But such actions should be rare. This is one
of the reasons why development should be done in private branches (which
can be rewritten if necessary) and only moved into public branches when
it's in a reasonably advanced state.
As the mainline (or other tree upon which a set of changes is based)
advances, it is tempting to merge with that tree to stay on the leading
edge. For a private branch, rebasing can be an easy way to keep up with
another tree, but rebasing is not an option once a tree is exported to the
world. Once that happens, a full merge must be done. Merging occasionally
makes good sense, but overly frequent merges can clutter the history
needlessly. Suggested technique in this case is to merge infrequently, and
generally only at specific release points (such as a mainline -rc
release). If you are nervous about specific changes, you can always
perform test merges in a private branch. The git "rerere" tool can be
useful in such situations; it remembers how merge conflicts were resolved
so that you don't have to do the same work twice.
One of the biggest recurring complaints about tools like git is this: the
mass movement of patches from one repository to another makes it easy to
slip in ill-advised changes which go into the mainline below the review
radar. Kernel developers tend to get unhappy when they see that kind of
thing happening; putting up a git tree with unreviewed or off-topic patches
can affect your ability to get trees pulled in the future. Quoting Linus:
::
You can send me patches, but for me to pull a git patch from you, I
need to know that you know what you're doing, and I need to be able
to trust things *without* then having to go and check every
individual change by hand.
(http://lwn.net/Articles/224135/).
To avoid this kind of situation, ensure that all patches within a given
branch stick closely to the associated topic; a "driver fixes" branch
should not be making changes to the core memory management code. And, most
importantly, do not use a git tree to bypass the review process. Post an
occasional summary of the tree to the relevant list, and, when the time is
right, request that the tree be included in linux-next.
If and when others start to send patches for inclusion into your tree,
don't forget to review them. Also ensure that you maintain the correct
authorship information; the git "am" tool does its best in this regard, but
you may have to add a "From:" line to the patch if it has been relayed to
you via a third party.
When requesting a pull, be sure to give all the relevant information: where
your tree is, what branch to pull, and what changes will result from the
pull. The git request-pull command can be helpful in this regard; it will
format the request as other developers expect, and will also check to be
sure that you have remembered to push those changes to the public server.
Reviewing patches
-----------------
Some readers will certainly object to putting this section with "advanced
topics" on the grounds that even beginning kernel developers should be
reviewing patches. It is certainly true that there is no better way to
learn how to program in the kernel environment than by looking at code
posted by others. In addition, reviewers are forever in short supply; by
looking at code you can make a significant contribution to the process as a
whole.
Reviewing code can be an intimidating prospect, especially for a new kernel
developer who may well feel nervous about questioning code - in public -
which has been posted by those with more experience. Even code written by
the most experienced developers can be improved, though. Perhaps the best
piece of advice for reviewers (all reviewers) is this: phrase review
comments as questions rather than criticisms. Asking "how does the lock
get released in this path?" will always work better than stating "the
locking here is wrong."
Different developers will review code from different points of view. Some
are mostly concerned with coding style and whether code lines have trailing
white space. Others will focus primarily on whether the change implemented
by the patch as a whole is a good thing for the kernel or not. Yet others
will check for problematic locking, excessive stack usage, possible
security issues, duplication of code found elsewhere, adequate
documentation, adverse effects on performance, user-space ABI changes, etc.
All types of review, if they lead to better code going into the kernel, are
welcome and worthwhile.
Documentation/process/8.Conclusion.rst
+74
View File
@@ -0,0 +1,74 @@
.. _development_conclusion:
For more information
====================
There are numerous sources of information on Linux kernel development and
related topics. First among those will always be the Documentation
directory found in the kernel source distribution. The top-level process/howto.rst
file is an important starting point; process/submitting-patches.rst and
process/submitting-drivers.rst are also something which all kernel developers should
read. Many internal kernel APIs are documented using the kerneldoc
mechanism; "make htmldocs" or "make pdfdocs" can be used to generate those
documents in HTML or PDF format (though the version of TeX shipped by some
distributions runs into internal limits and fails to process the documents
properly).
Various web sites discuss kernel development at all levels of detail. Your
author would like to humbly suggest http://lwn.net/ as a source;
information on many specific kernel topics can be found via the LWN kernel
index at:
http://lwn.net/Kernel/Index/
Beyond that, a valuable resource for kernel developers is:
http://kernelnewbies.org/
And, of course, one should not forget http://kernel.org/, the definitive
location for kernel release information.
There are a number of books on kernel development:
Linux Device Drivers, 3rd Edition (Jonathan Corbet, Alessandro
Rubini, and Greg Kroah-Hartman). Online at
http://lwn.net/Kernel/LDD3/.
Linux Kernel Development (Robert Love).
Understanding the Linux Kernel (Daniel Bovet and Marco Cesati).
All of these books suffer from a common fault, though: they tend to be
somewhat obsolete by the time they hit the shelves, and they have been on
the shelves for a while now. Still, there is quite a bit of good
information to be found there.
Documentation for git can be found at:
http://www.kernel.org/pub/software/scm/git/docs/
http://www.kernel.org/pub/software/scm/git/docs/user-manual.html
Conclusion
==========
Congratulations to anybody who has made it through this long-winded
document. Hopefully it has provided a helpful understanding of how the
Linux kernel is developed and how you can participate in that process.
In the end, it's the participation that matters. Any open source software
project is no more than the sum of what its contributors put into it. The
Linux kernel has progressed as quickly and as well as it has because it has
been helped by an impressively large group of developers, all of whom are
working to make it better. The kernel is a premier example of what can be
done when thousands of people work together toward a common goal.
The kernel can always benefit from a larger developer base, though. There
is always more work to do. But, just as importantly, most other
participants in the Linux ecosystem can benefit through contributing to the
kernel. Getting code into the mainline is the key to higher code quality,
lower maintenance and distribution costs, a higher level of influence over
the direction of kernel development, and more. It is a situation where
everybody involved wins. Fire up your editor and come join us; you will be
more than welcome.
Documentation/process/adding-syscalls.rst
+574
View File
@@ -0,0 +1,574 @@
Adding a New System Call
========================
This document describes what's involved in adding a new system call to the
Linux kernel, over and above the normal submission advice in
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`.
System Call Alternatives
------------------------
The first thing to consider when adding a new system call is whether one of
the alternatives might be suitable instead. Although system calls are the
most traditional and most obvious interaction points between userspace and the
kernel, there are other possibilities -- choose what fits best for your
interface.
- If the operations involved can be made to look like a filesystem-like
object, it may make more sense to create a new filesystem or device. This
also makes it easier to encapsulate the new functionality in a kernel module
rather than requiring it to be built into the main kernel.
- If the new functionality involves operations where the kernel notifies
userspace that something has happened, then returning a new file
descriptor for the relevant object allows userspace to use
``poll``/``select``/``epoll`` to receive that notification.
- However, operations that don't map to
:manpage:`read(2)`/:manpage:`write(2)`-like operations
have to be implemented as :manpage:`ioctl(2)` requests, which can lead
to a somewhat opaque API.
- If you're just exposing runtime system information, a new node in sysfs
(see ``Documentation/filesystems/sysfs.txt``) or the ``/proc`` filesystem may
be more appropriate. However, access to these mechanisms requires that the
relevant filesystem is mounted, which might not always be the case (e.g.
in a namespaced/sandboxed/chrooted environment). Avoid adding any API to
debugfs, as this is not considered a 'production' interface to userspace.
- If the operation is specific to a particular file or file descriptor, then
an additional :manpage:`fcntl(2)` command option may be more appropriate. However,
:manpage:`fcntl(2)` is a multiplexing system call that hides a lot of complexity, so
this option is best for when the new function is closely analogous to
existing :manpage:`fcntl(2)` functionality, or the new functionality is very simple
(for example, getting/setting a simple flag related to a file descriptor).
- If the operation is specific to a particular task or process, then an
additional :manpage:`prctl(2)` command option may be more appropriate. As
with :manpage:`fcntl(2)`, this system call is a complicated multiplexor so
is best reserved for near-analogs of existing ``prctl()`` commands or
getting/setting a simple flag related to a process.
Designing the API: Planning for Extension
-----------------------------------------
A new system call forms part of the API of the kernel, and has to be supported
indefinitely. As such, it's a very good idea to explicitly discuss the
interface on the kernel mailing list, and it's important to plan for future
extensions of the interface.
(The syscall table is littered with historical examples where this wasn't done,
together with the corresponding follow-up system calls --
``eventfd``/``eventfd2``, ``dup2``/``dup3``, ``inotify_init``/``inotify_init1``,
``pipe``/``pipe2``, ``renameat``/``renameat2`` -- so
learn from the history of the kernel and plan for extensions from the start.)
For simpler system calls that only take a couple of arguments, the preferred
way to allow for future extensibility is to include a flags argument to the
system call. To make sure that userspace programs can safely use flags
between kernel versions, check whether the flags value holds any unknown
flags, and reject the system call (with ``EINVAL``) if it does::
if (flags & ~(THING_FLAG1 | THING_FLAG2 | THING_FLAG3))
return -EINVAL;
(If no flags values are used yet, check that the flags argument is zero.)
For more sophisticated system calls that involve a larger number of arguments,
it's preferred to encapsulate the majority of the arguments into a structure
that is passed in by pointer. Such a structure can cope with future extension
by including a size argument in the structure::
struct xyzzy_params {
u32 size; /* userspace sets p->size = sizeof(struct xyzzy_params) */
u32 param_1;
u64 param_2;
u64 param_3;
};
As long as any subsequently added field, say ``param_4``, is designed so that a
zero value gives the previous behaviour, then this allows both directions of
version mismatch:
- To cope with a later userspace program calling an older kernel, the kernel
code should check that any memory beyond the size of the structure that it
expects is zero (effectively checking that ``param_4 == 0``).
- To cope with an older userspace program calling a newer kernel, the kernel
code can zero-extend a smaller instance of the structure (effectively
setting ``param_4 = 0``).
See :manpage:`perf_event_open(2)` and the ``perf_copy_attr()`` function (in
``kernel/events/core.c``) for an example of this approach.
Designing the API: Other Considerations
---------------------------------------
If your new system call allows userspace to refer to a kernel object, it
should use a file descriptor as the handle for that object -- don't invent a
new type of userspace object handle when the kernel already has mechanisms and
well-defined semantics for using file descriptors.
If your new :manpage:`xyzzy(2)` system call does return a new file descriptor,
then the flags argument should include a value that is equivalent to setting
``O_CLOEXEC`` on the new FD. This makes it possible for userspace to close
the timing window between ``xyzzy()`` and calling
``fcntl(fd, F_SETFD, FD_CLOEXEC)``, where an unexpected ``fork()`` and
``execve()`` in another thread could leak a descriptor to
the exec'ed program. (However, resist the temptation to re-use the actual value
of the ``O_CLOEXEC`` constant, as it is architecture-specific and is part of a
numbering space of ``O_*`` flags that is fairly full.)
If your system call returns a new file descriptor, you should also consider
what it means to use the :manpage:`poll(2)` family of system calls on that file
descriptor. Making a file descriptor ready for reading or writing is the
normal way for the kernel to indicate to userspace that an event has
occurred on the corresponding kernel object.
If your new :manpage:`xyzzy(2)` system call involves a filename argument::
int sys_xyzzy(const char __user *path, ..., unsigned int flags);
you should also consider whether an :manpage:`xyzzyat(2)` version is more appropriate::
int sys_xyzzyat(int dfd, const char __user *path, ..., unsigned int flags);
This allows more flexibility for how userspace specifies the file in question;
in particular it allows userspace to request the functionality for an
already-opened file descriptor using the ``AT_EMPTY_PATH`` flag, effectively
giving an :manpage:`fxyzzy(3)` operation for free::
- xyzzyat(AT_FDCWD, path, ..., 0) is equivalent to xyzzy(path,...)
- xyzzyat(fd, "", ..., AT_EMPTY_PATH) is equivalent to fxyzzy(fd, ...)
(For more details on the rationale of the \*at() calls, see the
:manpage:`openat(2)` man page; for an example of AT_EMPTY_PATH, see the
:manpage:`fstatat(2)` man page.)
If your new :manpage:`xyzzy(2)` system call involves a parameter describing an
offset within a file, make its type ``loff_t`` so that 64-bit offsets can be
supported even on 32-bit architectures.
If your new :manpage:`xyzzy(2)` system call involves privileged functionality,
it needs to be governed by the appropriate Linux capability bit (checked with
a call to ``capable()``), as described in the :manpage:`capabilities(7)` man
page. Choose an existing capability bit that governs related functionality,
but try to avoid combining lots of only vaguely related functions together
under the same bit, as this goes against capabilities' purpose of splitting
the power of root. In particular, avoid adding new uses of the already
overly-general ``CAP_SYS_ADMIN`` capability.
If your new :manpage:`xyzzy(2)` system call manipulates a process other than
the calling process, it should be restricted (using a call to
``ptrace_may_access()``) so that only a calling process with the same
permissions as the target process, or with the necessary capabilities, can
manipulate the target process.
Finally, be aware that some non-x86 architectures have an easier time if
system call parameters that are explicitly 64-bit fall on odd-numbered
arguments (i.e. parameter 1, 3, 5), to allow use of contiguous pairs of 32-bit
registers. (This concern does not apply if the arguments are part of a
structure that's passed in by pointer.)
Proposing the API
-----------------
To make new system calls easy to review, it's best to divide up the patchset
into separate chunks. These should include at least the following items as
distinct commits (each of which is described further below):
- The core implementation of the system call, together with prototypes,
generic numbering, Kconfig changes and fallback stub implementation.
- Wiring up of the new system call for one particular architecture, usually
x86 (including all of x86_64, x86_32 and x32).
- A demonstration of the use of the new system call in userspace via a
selftest in ``tools/testing/selftests/``.
- A draft man-page for the new system call, either as plain text in the
cover letter, or as a patch to the (separate) man-pages repository.
New system call proposals, like any change to the kernel's API, should always
be cc'ed to linux-api@vger.kernel.org.
Generic System Call Implementation
----------------------------------
The main entry point for your new :manpage:`xyzzy(2)` system call will be called
``sys_xyzzy()``, but you add this entry point with the appropriate
``SYSCALL_DEFINEn()`` macro rather than explicitly. The 'n' indicates the
number of arguments to the system call, and the macro takes the system call name
followed by the (type, name) pairs for the parameters as arguments. Using
this macro allows metadata about the new system call to be made available for
other tools.
The new entry point also needs a corresponding function prototype, in
``include/linux/syscalls.h``, marked as asmlinkage to match the way that system
calls are invoked::
asmlinkage long sys_xyzzy(...);
Some architectures (e.g. x86) have their own architecture-specific syscall
tables, but several other architectures share a generic syscall table. Add your
new system call to the generic list by adding an entry to the list in
``include/uapi/asm-generic/unistd.h``::
#define __NR_xyzzy 292
__SYSCALL(__NR_xyzzy, sys_xyzzy)
Also update the __NR_syscalls count to reflect the additional system call, and
note that if multiple new system calls are added in the same merge window,
your new syscall number may get adjusted to resolve conflicts.
The file ``kernel/sys_ni.c`` provides a fallback stub implementation of each
system call, returning ``-ENOSYS``. Add your new system call here too::
COND_SYSCALL(xyzzy);
Your new kernel functionality, and the system call that controls it, should
normally be optional, so add a ``CONFIG`` option (typically to
``init/Kconfig``) for it. As usual for new ``CONFIG`` options:
- Include a description of the new functionality and system call controlled
by the option.
- Make the option depend on EXPERT if it should be hidden from normal users.
- Make any new source files implementing the function dependent on the CONFIG
option in the Makefile (e.g. ``obj-$(CONFIG_XYZZY_SYSCALL) += xyzzy.c``).
- Double check that the kernel still builds with the new CONFIG option turned
off.
To summarize, you need a commit that includes:
- ``CONFIG`` option for the new function, normally in ``init/Kconfig``
- ``SYSCALL_DEFINEn(xyzzy, ...)`` for the entry point
- corresponding prototype in ``include/linux/syscalls.h``
- generic table entry in ``include/uapi/asm-generic/unistd.h``
- fallback stub in ``kernel/sys_ni.c``
x86 System Call Implementation
------------------------------
To wire up your new system call for x86 platforms, you need to update the
master syscall tables. Assuming your new system call isn't special in some
way (see below), this involves a "common" entry (for x86_64 and x32) in
arch/x86/entry/syscalls/syscall_64.tbl::
333 common xyzzy sys_xyzzy
and an "i386" entry in ``arch/x86/entry/syscalls/syscall_32.tbl``::
380 i386 xyzzy sys_xyzzy
Again, these numbers are liable to be changed if there are conflicts in the
relevant merge window.
Compatibility System Calls (Generic)
------------------------------------
For most system calls the same 64-bit implementation can be invoked even when
the userspace program is itself 32-bit; even if the system call's parameters
include an explicit pointer, this is handled transparently.
However, there are a couple of situations where a compatibility layer is
needed to cope with size differences between 32-bit and 64-bit.
The first is if the 64-bit kernel also supports 32-bit userspace programs, and
so needs to parse areas of (``__user``) memory that could hold either 32-bit or
64-bit values. In particular, this is needed whenever a system call argument
is:
- a pointer to a pointer
- a pointer to a struct containing a pointer (e.g. ``struct iovec __user *``)
- a pointer to a varying sized integral type (``time_t``, ``off_t``,
``long``, ...)
- a pointer to a struct containing a varying sized integral type.
The second situation that requires a compatibility layer is if one of the
system call's arguments has a type that is explicitly 64-bit even on a 32-bit
architecture, for example ``loff_t`` or ``__u64``. In this case, a value that
arrives at a 64-bit kernel from a 32-bit application will be split into two
32-bit values, which then need to be re-assembled in the compatibility layer.
(Note that a system call argument that's a pointer to an explicit 64-bit type
does **not** need a compatibility layer; for example, :manpage:`splice(2)`'s arguments of
type ``loff_t __user *`` do not trigger the need for a ``compat_`` system call.)
The compatibility version of the system call is called ``compat_sys_xyzzy()``,
and is added with the ``COMPAT_SYSCALL_DEFINEn()`` macro, analogously to
SYSCALL_DEFINEn. This version of the implementation runs as part of a 64-bit
kernel, but expects to receive 32-bit parameter values and does whatever is
needed to deal with them. (Typically, the ``compat_sys_`` version converts the
values to 64-bit versions and either calls on to the ``sys_`` version, or both of
them call a common inner implementation function.)
The compat entry point also needs a corresponding function prototype, in
``include/linux/compat.h``, marked as asmlinkage to match the way that system
calls are invoked::
asmlinkage long compat_sys_xyzzy(...);
If the system call involves a structure that is laid out differently on 32-bit
and 64-bit systems, say ``struct xyzzy_args``, then the include/linux/compat.h
header file should also include a compat version of the structure (``struct
compat_xyzzy_args``) where each variable-size field has the appropriate
``compat_`` type that corresponds to the type in ``struct xyzzy_args``. The
``compat_sys_xyzzy()`` routine can then use this ``compat_`` structure to
parse the arguments from a 32-bit invocation.
For example, if there are fields::
struct xyzzy_args {
const char __user *ptr;
__kernel_long_t varying_val;
u64 fixed_val;
/* ... */
};
in struct xyzzy_args, then struct compat_xyzzy_args would have::
struct compat_xyzzy_args {
compat_uptr_t ptr;
compat_long_t varying_val;
u64 fixed_val;
/* ... */
};
The generic system call list also needs adjusting to allow for the compat
version; the entry in ``include/uapi/asm-generic/unistd.h`` should use
``__SC_COMP`` rather than ``__SYSCALL``::
#define __NR_xyzzy 292
__SC_COMP(__NR_xyzzy, sys_xyzzy, compat_sys_xyzzy)
To summarize, you need:
- a ``COMPAT_SYSCALL_DEFINEn(xyzzy, ...)`` for the compat entry point
- corresponding prototype in ``include/linux/compat.h``
- (if needed) 32-bit mapping struct in ``include/linux/compat.h``
- instance of ``__SC_COMP`` not ``__SYSCALL`` in
``include/uapi/asm-generic/unistd.h``
Compatibility System Calls (x86)
--------------------------------
To wire up the x86 architecture of a system call with a compatibility version,
the entries in the syscall tables need to be adjusted.
First, the entry in ``arch/x86/entry/syscalls/syscall_32.tbl`` gets an extra
column to indicate that a 32-bit userspace program running on a 64-bit kernel
should hit the compat entry point::
380 i386 xyzzy sys_xyzzy __ia32_compat_sys_xyzzy
Second, you need to figure out what should happen for the x32 ABI version of
the new system call. There's a choice here: the layout of the arguments
should either match the 64-bit version or the 32-bit version.
If there's a pointer-to-a-pointer involved, the decision is easy: x32 is
ILP32, so the layout should match the 32-bit version, and the entry in
``arch/x86/entry/syscalls/syscall_64.tbl`` is split so that x32 programs hit
the compatibility wrapper::
333 64 xyzzy sys_xyzzy
...
555 x32 xyzzy __x32_compat_sys_xyzzy
If no pointers are involved, then it is preferable to re-use the 64-bit system
call for the x32 ABI (and consequently the entry in
arch/x86/entry/syscalls/syscall_64.tbl is unchanged).
In either case, you should check that the types involved in your argument
layout do indeed map exactly from x32 (-mx32) to either the 32-bit (-m32) or
64-bit (-m64) equivalents.
System Calls Returning Elsewhere
--------------------------------
For most system calls, once the system call is complete the user program
continues exactly where it left off -- at the next instruction, with the
stack the same and most of the registers the same as before the system call,
and with the same virtual memory space.
However, a few system calls do things differently. They might return to a
different location (``rt_sigreturn``) or change the memory space
(``fork``/``vfork``/``clone``) or even architecture (``execve``/``execveat``)
of the program.
To allow for this, the kernel implementation of the system call may need to
save and restore additional registers to the kernel stack, allowing complete
control of where and how execution continues after the system call.
This is arch-specific, but typically involves defining assembly entry points
that save/restore additional registers and invoke the real system call entry
point.
For x86_64, this is implemented as a ``stub_xyzzy`` entry point in
``arch/x86/entry/entry_64.S``, and the entry in the syscall table
(``arch/x86/entry/syscalls/syscall_64.tbl``) is adjusted to match::
333 common xyzzy stub_xyzzy
The equivalent for 32-bit programs running on a 64-bit kernel is normally
called ``stub32_xyzzy`` and implemented in ``arch/x86/entry/entry_64_compat.S``,
with the corresponding syscall table adjustment in
``arch/x86/entry/syscalls/syscall_32.tbl``::
380 i386 xyzzy sys_xyzzy stub32_xyzzy
If the system call needs a compatibility layer (as in the previous section)
then the ``stub32_`` version needs to call on to the ``compat_sys_`` version
of the system call rather than the native 64-bit version. Also, if the x32 ABI
implementation is not common with the x86_64 version, then its syscall
table will also need to invoke a stub that calls on to the ``compat_sys_``
version.
For completeness, it's also nice to set up a mapping so that user-mode Linux
still works -- its syscall table will reference stub_xyzzy, but the UML build
doesn't include ``arch/x86/entry/entry_64.S`` implementation (because UML
simulates registers etc). Fixing this is as simple as adding a #define to
``arch/x86/um/sys_call_table_64.c``::
#define stub_xyzzy sys_xyzzy
Other Details
-------------
Most of the kernel treats system calls in a generic way, but there is the
occasional exception that may need updating for your particular system call.
The audit subsystem is one such special case; it includes (arch-specific)
functions that classify some special types of system call -- specifically
file open (``open``/``openat``), program execution (``execve``/``exeveat``) or
socket multiplexor (``socketcall``) operations. If your new system call is
analogous to one of these, then the audit system should be updated.
More generally, if there is an existing system call that is analogous to your
new system call, it's worth doing a kernel-wide grep for the existing system
call to check there are no other special cases.
Testing
-------
A new system call should obviously be tested; it is also useful to provide
reviewers with a demonstration of how user space programs will use the system
call. A good way to combine these aims is to include a simple self-test
program in a new directory under ``tools/testing/selftests/``.
For a new system call, there will obviously be no libc wrapper function and so
the test will need to invoke it using ``syscall()``; also, if the system call
involves a new userspace-visible structure, the corresponding header will need
to be installed to compile the test.
Make sure the selftest runs successfully on all supported architectures. For
example, check that it works when compiled as an x86_64 (-m64), x86_32 (-m32)
and x32 (-mx32) ABI program.
For more extensive and thorough testing of new functionality, you should also
consider adding tests to the Linux Test Project, or to the xfstests project
for filesystem-related changes.
- https://linux-test-project.github.io/
- git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
Man Page
--------
All new system calls should come with a complete man page, ideally using groff
markup, but plain text will do. If groff is used, it's helpful to include a
pre-rendered ASCII version of the man page in the cover email for the
patchset, for the convenience of reviewers.
The man page should be cc'ed to linux-man@vger.kernel.org
For more details, see https://www.kernel.org/doc/man-pages/patches.html
Do not call System Calls in the Kernel
--------------------------------------
System calls are, as stated above, interaction points between userspace and
the kernel. Therefore, system call functions such as ``sys_xyzzy()`` or
``compat_sys_xyzzy()`` should only be called from userspace via the syscall
table, but not from elsewhere in the kernel. If the syscall functionality is
useful to be used within the kernel, needs to be shared between an old and a
new syscall, or needs to be shared between a syscall and its compatibility
variant, it should be implemented by means of a "helper" function (such as
``kern_xyzzy()``). This kernel function may then be called within the
syscall stub (``sys_xyzzy()``), the compatibility syscall stub
(``compat_sys_xyzzy()``), and/or other kernel code.
At least on 64-bit x86, it will be a hard requirement from v4.17 onwards to not
call system call functions in the kernel. It uses a different calling
convention for system calls where ``struct pt_regs`` is decoded on-the-fly in a
syscall wrapper which then hands processing over to the actual syscall function.
This means that only those parameters which are actually needed for a specific
syscall are passed on during syscall entry, instead of filling in six CPU
registers with random user space content all the time (which may cause serious
trouble down the call chain).
Moreover, rules on how data may be accessed may differ between kernel data and
user data. This is another reason why calling ``sys_xyzzy()`` is generally a
bad idea.
Exceptions to this rule are only allowed in architecture-specific overrides,
architecture-specific compatibility wrappers, or other code in arch/.
References and Sources
----------------------
- LWN article from Michael Kerrisk on use of flags argument in system calls:
https://lwn.net/Articles/585415/
- LWN article from Michael Kerrisk on how to handle unknown flags in a system
call: https://lwn.net/Articles/588444/
- LWN article from Jake Edge describing constraints on 64-bit system call
arguments: https://lwn.net/Articles/311630/
- Pair of LWN articles from David Drysdale that describe the system call
implementation paths in detail for v3.14:
- https://lwn.net/Articles/604287/
- https://lwn.net/Articles/604515/
- Architecture-specific requirements for system calls are discussed in the
:manpage:`syscall(2)` man-page:
http://man7.org/linux/man-pages/man2/syscall.2.html#NOTES
- Collated emails from Linus Torvalds discussing the problems with ``ioctl()``:
http://yarchive.net/comp/linux/ioctl.html
- "How to not invent kernel interfaces", Arnd Bergmann,
http://www.ukuug.org/events/linux2007/2007/papers/Bergmann.pdf
- LWN article from Michael Kerrisk on avoiding new uses of CAP_SYS_ADMIN:
https://lwn.net/Articles/486306/
- Recommendation from Andrew Morton that all related information for a new
system call should come in the same email thread:
https://lkml.org/lkml/2014/7/24/641
- Recommendation from Michael Kerrisk that a new system call should come with
a man page: https://lkml.org/lkml/2014/6/13/309
- Suggestion from Thomas Gleixner that x86 wire-up should be in a separate
commit: https://lkml.org/lkml/2014/11/19/254
- Suggestion from Greg Kroah-Hartman that it's good for new system calls to
come with a man-page & selftest: https://lkml.org/lkml/2014/3/19/710
- Discussion from Michael Kerrisk of new system call vs. :manpage:`prctl(2)` extension:
https://lkml.org/lkml/2014/6/3/411
- Suggestion from Ingo Molnar that system calls that involve multiple
arguments should encapsulate those arguments in a struct, which includes a
size field for future extensibility: https://lkml.org/lkml/2015/7/30/117
- Numbering oddities arising from (re-)use of O_* numbering space flags:
- commit 75069f2b5bfb ("vfs: renumber FMODE_NONOTIFY and add to uniqueness
check")
- commit 12ed2e36c98a ("fanotify: FMODE_NONOTIFY and __O_SYNC in sparc
conflict")
- commit bb458c644a59 ("Safer ABI for O_TMPFILE")
- Discussion from Matthew Wilcox about restrictions on 64-bit arguments:
https://lkml.org/lkml/2008/12/12/187
- Recommendation from Greg Kroah-Hartman that unknown flags should be
policed: https://lkml.org/lkml/2014/7/17/577
- Recommendation from Linus Torvalds that x32 system calls should prefer
compatibility with 64-bit versions rather than 32-bit versions:
https://lkml.org/lkml/2011/8/31/244
Documentation/process/applying-patches.rst
+421
View File
@@ -0,0 +1,421 @@
.. _applying_patches:
Applying Patches To The Linux Kernel
++++++++++++++++++++++++++++++++++++
Original by:
Jesper Juhl, August 2005
.. note::
This document is obsolete. In most cases, rather than using ``patch``
manually, you'll almost certainly want to look at using Git instead.
A frequently asked question on the Linux Kernel Mailing List is how to apply
a patch to the kernel or, more specifically, what base kernel a patch for
one of the many trees/branches should be applied to. Hopefully this document
will explain this to you.
In addition to explaining how to apply and revert patches, a brief
description of the different kernel trees (and examples of how to apply
their specific patches) is also provided.
What is a patch?
================
A patch is a small text document containing a delta of changes between two
different versions of a source tree. Patches are created with the ``diff``
program.
To correctly apply a patch you need to know what base it was generated from
and what new version the patch will change the source tree into. These
should both be present in the patch file metadata or be possible to deduce
from the filename.
How do I apply or revert a patch?
=================================
You apply a patch with the ``patch`` program. The patch program reads a diff
(or patch) file and makes the changes to the source tree described in it.
Patches for the Linux kernel are generated relative to the parent directory
holding the kernel source dir.
This means that paths to files inside the patch file contain the name of the
kernel source directories it was generated against (or some other directory
names like "a/" and "b/").
Since this is unlikely to match the name of the kernel source dir on your
local machine (but is often useful info to see what version an otherwise
unlabeled patch was generated against) you should change into your kernel
source directory and then strip the first element of the path from filenames
in the patch file when applying it (the ``-p1`` argument to ``patch`` does
this).
To revert a previously applied patch, use the -R argument to patch.
So, if you applied a patch like this::
patch -p1 < ../patch-x.y.z
You can revert (undo) it like this::
patch -R -p1 < ../patch-x.y.z
How do I feed a patch/diff file to ``patch``?
=============================================
This (as usual with Linux and other UNIX like operating systems) can be
done in several different ways.
In all the examples below I feed the file (in uncompressed form) to patch
via stdin using the following syntax::
patch -p1 < path/to/patch-x.y.z
If you just want to be able to follow the examples below and don't want to
know of more than one way to use patch, then you can stop reading this
section here.
Patch can also get the name of the file to use via the -i argument, like
this::
patch -p1 -i path/to/patch-x.y.z
If your patch file is compressed with gzip or xz and you don't want to
uncompress it before applying it, then you can feed it to patch like this
instead::
xzcat path/to/patch-x.y.z.xz | patch -p1
bzcat path/to/patch-x.y.z.gz | patch -p1
If you wish to uncompress the patch file by hand first before applying it
(what I assume you've done in the examples below), then you simply run
gunzip or xz on the file -- like this::
gunzip patch-x.y.z.gz
xz -d patch-x.y.z.xz
Which will leave you with a plain text patch-x.y.z file that you can feed to
patch via stdin or the ``-i`` argument, as you prefer.
A few other nice arguments for patch are ``-s`` which causes patch to be silent
except for errors which is nice to prevent errors from scrolling out of the
screen too fast, and ``--dry-run`` which causes patch to just print a listing of
what would happen, but doesn't actually make any changes. Finally ``--verbose``
tells patch to print more information about the work being done.
Common errors when patching
===========================
When patch applies a patch file it attempts to verify the sanity of the
file in different ways.
Checking that the file looks like a valid patch file and checking the code
around the bits being modified matches the context provided in the patch are
just two of the basic sanity checks patch does.
If patch encounters something that doesn't look quite right it has two
options. It can either refuse to apply the changes and abort or it can try
to find a way to make the patch apply with a few minor changes.
One example of something that's not 'quite right' that patch will attempt to
fix up is if all the context matches, the lines being changed match, but the
line numbers are different. This can happen, for example, if the patch makes
a change in the middle of the file but for some reasons a few lines have
been added or removed near the beginning of the file. In that case
everything looks good it has just moved up or down a bit, and patch will
usually adjust the line numbers and apply the patch.
Whenever patch applies a patch that it had to modify a bit to make it fit
it'll tell you about it by saying the patch applied with **fuzz**.
You should be wary of such changes since even though patch probably got it
right it doesn't /always/ get it right, and the result will sometimes be
wrong.
When patch encounters a change that it can't fix up with fuzz it rejects it
outright and leaves a file with a ``.rej`` extension (a reject file). You can
read this file to see exactly what change couldn't be applied, so you can
go fix it up by hand if you wish.
If you don't have any third-party patches applied to your kernel source, but
only patches from kernel.org and you apply the patches in the correct order,
and have made no modifications yourself to the source files, then you should
never see a fuzz or reject message from patch. If you do see such messages
anyway, then there's a high risk that either your local source tree or the
patch file is corrupted in some way. In that case you should probably try
re-downloading the patch and if things are still not OK then you'd be advised
to start with a fresh tree downloaded in full from kernel.org.
Let's look a bit more at some of the messages patch can produce.
If patch stops and presents a ``File to patch:`` prompt, then patch could not
find a file to be patched. Most likely you forgot to specify -p1 or you are
in the wrong directory. Less often, you'll find patches that need to be
applied with ``-p0`` instead of ``-p1`` (reading the patch file should reveal if
this is the case -- if so, then this is an error by the person who created
the patch but is not fatal).
If you get ``Hunk #2 succeeded at 1887 with fuzz 2 (offset 7 lines).`` or a
message similar to that, then it means that patch had to adjust the location
of the change (in this example it needed to move 7 lines from where it
expected to make the change to make it fit).
The resulting file may or may not be OK, depending on the reason the file
was different than expected.
This often happens if you try to apply a patch that was generated against a
different kernel version than the one you are trying to patch.
If you get a message like ``Hunk #3 FAILED at 2387.``, then it means that the
patch could not be applied correctly and the patch program was unable to
fuzz its way through. This will generate a ``.rej`` file with the change that
caused the patch to fail and also a ``.orig`` file showing you the original
content that couldn't be changed.
If you get ``Reversed (or previously applied) patch detected! Assume -R? [n]``
then patch detected that the change contained in the patch seems to have
already been made.
If you actually did apply this patch previously and you just re-applied it
in error, then just say [n]o and abort this patch. If you applied this patch
previously and actually intended to revert it, but forgot to specify -R,
then you can say [**y**]es here to make patch revert it for you.
This can also happen if the creator of the patch reversed the source and
destination directories when creating the patch, and in that case reverting
the patch will in fact apply it.
A message similar to ``patch: **** unexpected end of file in patch`` or
``patch unexpectedly ends in middle of line`` means that patch could make no
sense of the file you fed to it. Either your download is broken, you tried to
feed patch a compressed patch file without uncompressing it first, or the patch
file that you are using has been mangled by a mail client or mail transfer
agent along the way somewhere, e.g., by splitting a long line into two lines.
Often these warnings can easily be fixed by joining (concatenating) the
two lines that had been split.
As I already mentioned above, these errors should never happen if you apply
a patch from kernel.org to the correct version of an unmodified source tree.
So if you get these errors with kernel.org patches then you should probably
assume that either your patch file or your tree is broken and I'd advise you
to start over with a fresh download of a full kernel tree and the patch you
wish to apply.
Are there any alternatives to ``patch``?
========================================
Yes there are alternatives.
You can use the ``interdiff`` program (http://cyberelk.net/tim/patchutils/) to
generate a patch representing the differences between two patches and then
apply the result.
This will let you move from something like 4.7.2 to 4.7.3 in a single
step. The -z flag to interdiff will even let you feed it patches in gzip or
bzip2 compressed form directly without the use of zcat or bzcat or manual
decompression.
Here's how you'd go from 4.7.2 to 4.7.3 in a single step::
interdiff -z ../patch-4.7.2.gz ../patch-4.7.3.gz | patch -p1
Although interdiff may save you a step or two you are generally advised to
do the additional steps since interdiff can get things wrong in some cases.
Another alternative is ``ketchup``, which is a python script for automatic
downloading and applying of patches (http://www.selenic.com/ketchup/).
Other nice tools are diffstat, which shows a summary of changes made by a
patch; lsdiff, which displays a short listing of affected files in a patch
file, along with (optionally) the line numbers of the start of each patch;
and grepdiff, which displays a list of the files modified by a patch where
the patch contains a given regular expression.
Where can I download the patches?
=================================
The patches are available at http://kernel.org/
Most recent patches are linked from the front page, but they also have
specific homes.
The 4.x.y (-stable) and 4.x patches live at
https://www.kernel.org/pub/linux/kernel/v4.x/
The -rc patches live at
https://www.kernel.org/pub/linux/kernel/v4.x/testing/
The 4.x kernels
===============
These are the base stable releases released by Linus. The highest numbered
release is the most recent.
If regressions or other serious flaws are found, then a -stable fix patch
will be released (see below) on top of this base. Once a new 4.x base
kernel is released, a patch is made available that is a delta between the
previous 4.x kernel and the new one.
To apply a patch moving from 4.6 to 4.7, you'd do the following (note
that such patches do **NOT** apply on top of 4.x.y kernels but on top of the
base 4.x kernel -- if you need to move from 4.x.y to 4.x+1 you need to
first revert the 4.x.y patch).
Here are some examples::
# moving from 4.6 to 4.7
$ cd ~/linux-4.6 # change to kernel source dir
$ patch -p1 < ../patch-4.7 # apply the 4.7 patch
$ cd ..
$ mv linux-4.6 linux-4.7 # rename source dir
# moving from 4.6.1 to 4.7
$ cd ~/linux-4.6.1 # change to kernel source dir
$ patch -p1 -R < ../patch-4.6.1 # revert the 4.6.1 patch
# source dir is now 4.6
$ patch -p1 < ../patch-4.7 # apply new 4.7 patch
$ cd ..
$ mv linux-4.6.1 linux-4.7 # rename source dir
The 4.x.y kernels
=================
Kernels with 3-digit versions are -stable kernels. They contain small(ish)
critical fixes for security problems or significant regressions discovered
in a given 4.x kernel.
This is the recommended branch for users who want the most recent stable
kernel and are not interested in helping test development/experimental
versions.
If no 4.x.y kernel is available, then the highest numbered 4.x kernel is
the current stable kernel.
.. note::
The -stable team usually do make incremental patches available as well
as patches against the latest mainline release, but I only cover the
non-incremental ones below. The incremental ones can be found at
https://www.kernel.org/pub/linux/kernel/v4.x/incr/
These patches are not incremental, meaning that for example the 4.7.3
patch does not apply on top of the 4.7.2 kernel source, but rather on top
of the base 4.7 kernel source.
So, in order to apply the 4.7.3 patch to your existing 4.7.2 kernel
source you have to first back out the 4.7.2 patch (so you are left with a
base 4.7 kernel source) and then apply the new 4.7.3 patch.
Here's a small example::
$ cd ~/linux-4.7.2 # change to the kernel source dir
$ patch -p1 -R < ../patch-4.7.2 # revert the 4.7.2 patch
$ patch -p1 < ../patch-4.7.3 # apply the new 4.7.3 patch
$ cd ..
$ mv linux-4.7.2 linux-4.7.3 # rename the kernel source dir
The -rc kernels
===============
These are release-candidate kernels. These are development kernels released
by Linus whenever he deems the current git (the kernel's source management
tool) tree to be in a reasonably sane state adequate for testing.
These kernels are not stable and you should expect occasional breakage if
you intend to run them. This is however the most stable of the main
development branches and is also what will eventually turn into the next
stable kernel, so it is important that it be tested by as many people as
possible.
This is a good branch to run for people who want to help out testing
development kernels but do not want to run some of the really experimental
stuff (such people should see the sections about -next and -mm kernels below).
The -rc patches are not incremental, they apply to a base 4.x kernel, just
like the 4.x.y patches described above. The kernel version before the -rcN
suffix denotes the version of the kernel that this -rc kernel will eventually
turn into.
So, 4.8-rc5 means that this is the fifth release candidate for the 4.8
kernel and the patch should be applied on top of the 4.7 kernel source.
Here are 3 examples of how to apply these patches::
# first an example of moving from 4.7 to 4.8-rc3
$ cd ~/linux-4.7 # change to the 4.7 source dir
$ patch -p1 < ../patch-4.8-rc3 # apply the 4.8-rc3 patch
$ cd ..
$ mv linux-4.7 linux-4.8-rc3 # rename the source dir
# now let's move from 4.8-rc3 to 4.8-rc5
$ cd ~/linux-4.8-rc3 # change to the 4.8-rc3 dir
$ patch -p1 -R < ../patch-4.8-rc3 # revert the 4.8-rc3 patch
$ patch -p1 < ../patch-4.8-rc5 # apply the new 4.8-rc5 patch
$ cd ..
$ mv linux-4.8-rc3 linux-4.8-rc5 # rename the source dir
# finally let's try and move from 4.7.3 to 4.8-rc5
$ cd ~/linux-4.7.3 # change to the kernel source dir
$ patch -p1 -R < ../patch-4.7.3 # revert the 4.7.3 patch
$ patch -p1 < ../patch-4.8-rc5 # apply new 4.8-rc5 patch
$ cd ..
$ mv linux-4.7.3 linux-4.8-rc5 # rename the kernel source dir
The -mm patches and the linux-next tree
=======================================
The -mm patches are experimental patches released by Andrew Morton.
In the past, -mm tree were used to also test subsystem patches, but this
function is now done via the
`linux-next <https://www.kernel.org/doc/man-pages/linux-next.html>`
tree. The Subsystem maintainers push their patches first to linux-next,
and, during the merge window, sends them directly to Linus.
The -mm patches serve as a sort of proving ground for new features and other
experimental patches that aren't merged via a subsystem tree.
Once such patches has proved its worth in -mm for a while Andrew pushes
it on to Linus for inclusion in mainline.
The linux-next tree is daily updated, and includes the -mm patches.
Both are in constant flux and contains many experimental features, a
lot of debugging patches not appropriate for mainline etc., and is the most
experimental of the branches described in this document.
These patches are not appropriate for use on systems that are supposed to be
stable and they are more risky to run than any of the other branches (make
sure you have up-to-date backups -- that goes for any experimental kernel but
even more so for -mm patches or using a Kernel from the linux-next tree).
Testing of -mm patches and linux-next is greatly appreciated since the whole
point of those are to weed out regressions, crashes, data corruption bugs,
build breakage (and any other bug in general) before changes are merged into
the more stable mainline Linus tree.
But testers of -mm and linux-next should be aware that breakages are
more common than in any other tree.
This concludes this list of explanations of the various kernel trees.
I hope you are now clear on how to apply the various patches and help testing
the kernel.
Thank you's to Randy Dunlap, Rolf Eike Beer, Linus Torvalds, Bodo Eggert,
Johannes Stezenbach, Grant Coady, Pavel Machek and others that I may have
forgotten for their reviews and contributions to this document.
Documentation/process/changes.rst
+487
View File
@@ -0,0 +1,487 @@
.. _changes:
Minimal requirements to compile the Kernel
++++++++++++++++++++++++++++++++++++++++++
Intro
=====
This document is designed to provide a list of the minimum levels of
software necessary to run the 4.x kernels.
This document is originally based on my "Changes" file for 2.0.x kernels
and therefore owes credit to the same people as that file (Jared Mauch,
Axel Boldt, Alessandro Sigala, and countless other users all over the
'net).
Current Minimal Requirements
****************************
Upgrade to at **least** these software revisions before thinking you've
encountered a bug! If you're unsure what version you're currently
running, the suggested command should tell you.
Again, keep in mind that this list assumes you are already functionally
running a Linux kernel. Also, not all tools are necessary on all
systems; obviously, if you don't have any ISDN hardware, for example,
you probably needn't concern yourself with isdn4k-utils.
====================== =============== ========================================
Program Minimal version Command to check the version
====================== =============== ========================================
GNU C 4.6 gcc --version
GNU make 3.81 make --version
binutils 2.20 ld -v
flex 2.5.35 flex --version
bison 2.0 bison --version
util-linux 2.10o fdformat --version
kmod 13 depmod -V
e2fsprogs 1.41.4 e2fsck -V
jfsutils 1.1.3 fsck.jfs -V
reiserfsprogs 3.6.3 reiserfsck -V
xfsprogs 2.6.0 xfs_db -V
squashfs-tools 4.0 mksquashfs -version
btrfs-progs 0.18 btrfsck
pcmciautils 004 pccardctl -V
quota-tools 3.09 quota -V
PPP 2.4.0 pppd --version
isdn4k-utils 3.1pre1 isdnctrl 2>&1|grep version
nfs-utils 1.0.5 showmount --version
procps 3.2.0 ps --version
oprofile 0.9 oprofiled --version
udev 081 udevd --version
grub 0.93 grub --version || grub-install --version
mcelog 0.6 mcelog --version
iptables 1.4.2 iptables -V
openssl & libcrypto 1.0.0 openssl version
bc 1.06.95 bc --version
Sphinx\ [#f1]_ 1.3 sphinx-build --version
====================== =============== ========================================
.. [#f1] Sphinx is needed only to build the Kernel documentation
Kernel compilation
******************
GCC
---
The gcc version requirements may vary depending on the type of CPU in your
computer.
Make
----
You will need GNU make 3.81 or later to build the kernel.
Binutils
--------
The build system has, as of 4.13, switched to using thin archives (`ar T`)
rather than incremental linking (`ld -r`) for built-in.a intermediate steps.
This requires binutils 2.20 or newer.
pkg-config
----------
The build system, as of 4.18, requires pkg-config to check for installed
kconfig tools and to determine flags settings for use in
'make {g,x}config'. Previously pkg-config was being used but not
verified or documented.
Flex
----
Since Linux 4.16, the build system generates lexical analyzers
during build. This requires flex 2.5.35 or later.
Bison
-----
Since Linux 4.16, the build system generates parsers
during build. This requires bison 2.0 or later.
Perl
----
You will need perl 5 and the following modules: ``Getopt::Long``,
``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
BC
--
You will need bc to build kernels 3.10 and higher
OpenSSL
-------
Module signing and external certificate handling use the OpenSSL program and
crypto library to do key creation and signature generation.
You will need openssl to build kernels 3.7 and higher if module signing is
enabled. You will also need openssl development packages to build kernels 4.3
and higher.
System utilities
****************
Architectural changes
---------------------
DevFS has been obsoleted in favour of udev
(http://www.kernel.org/pub/linux/utils/kernel/hotplug/)
32-bit UID support is now in place. Have fun!
Linux documentation for functions is transitioning to inline
documentation via specially-formatted comments near their
definitions in the source. These comments can be combined with ReST
files the Documentation/ directory to make enriched documentation, which can
then be converted to PostScript, HTML, LaTex, ePUB and PDF files.
In order to convert from ReST format to a format of your choice, you'll need
Sphinx.
Util-linux
----------
New versions of util-linux provide ``fdisk`` support for larger disks,
support new options to mount, recognize more supported partition
types, have a fdformat which works with 2.4 kernels, and similar goodies.
You'll probably want to upgrade.
Ksymoops
--------
If the unthinkable happens and your kernel oopses, you may need the
ksymoops tool to decode it, but in most cases you don't.
It is generally preferred to build the kernel with ``CONFIG_KALLSYMS`` so
that it produces readable dumps that can be used as-is (this also
produces better output than ksymoops). If for some reason your kernel
is not build with ``CONFIG_KALLSYMS`` and you have no way to rebuild and
reproduce the Oops with that option, then you can still decode that Oops
with ksymoops.
Mkinitrd
--------
These changes to the ``/lib/modules`` file tree layout also require that
mkinitrd be upgraded.
E2fsprogs
---------
The latest version of ``e2fsprogs`` fixes several bugs in fsck and
debugfs. Obviously, it's a good idea to upgrade.
JFSutils
--------
The ``jfsutils`` package contains the utilities for the file system.
The following utilities are available:
- ``fsck.jfs`` - initiate replay of the transaction log, and check
and repair a JFS formatted partition.
- ``mkfs.jfs`` - create a JFS formatted partition.
- other file system utilities are also available in this package.
Reiserfsprogs
-------------
The reiserfsprogs package should be used for reiserfs-3.6.x
(Linux kernels 2.4.x). It is a combined package and contains working
versions of ``mkreiserfs``, ``resize_reiserfs``, ``debugreiserfs`` and
``reiserfsck``. These utils work on both i386 and alpha platforms.
Xfsprogs
--------
The latest version of ``xfsprogs`` contains ``mkfs.xfs``, ``xfs_db``, and the
``xfs_repair`` utilities, among others, for the XFS filesystem. It is
architecture independent and any version from 2.0.0 onward should
work correctly with this version of the XFS kernel code (2.6.0 or
later is recommended, due to some significant improvements).
PCMCIAutils
-----------
PCMCIAutils replaces ``pcmcia-cs``. It properly sets up
PCMCIA sockets at system startup and loads the appropriate modules
for 16-bit PCMCIA devices if the kernel is modularized and the hotplug
subsystem is used.
Quota-tools
-----------
Support for 32 bit uid's and gid's is required if you want to use
the newer version 2 quota format. Quota-tools version 3.07 and
newer has this support. Use the recommended version or newer
from the table above.
Intel IA32 microcode
--------------------
A driver has been added to allow updating of Intel IA32 microcode,
accessible as a normal (misc) character device. If you are not using
udev you may need to::
mkdir /dev/cpu
mknod /dev/cpu/microcode c 10 184
chmod 0644 /dev/cpu/microcode
as root before you can use this. You'll probably also want to
get the user-space microcode_ctl utility to use with this.
udev
----
``udev`` is a userspace application for populating ``/dev`` dynamically with
only entries for devices actually present. ``udev`` replaces the basic
functionality of devfs, while allowing persistent device naming for
devices.
FUSE
----
Needs libfuse 2.4.0 or later. Absolute minimum is 2.3.0 but mount
options ``direct_io`` and ``kernel_cache`` won't work.
Networking
**********
General changes
---------------
If you have advanced network configuration needs, you should probably
consider using the network tools from ip-route2.
Packet Filter / NAT
-------------------
The packet filtering and NAT code uses the same tools like the previous 2.4.x
kernel series (iptables). It still includes backwards-compatibility modules
for 2.2.x-style ipchains and 2.0.x-style ipfwadm.
PPP
---
The PPP driver has been restructured to support multilink and to
enable it to operate over diverse media layers. If you use PPP,
upgrade pppd to at least 2.4.0.
If you are not using udev, you must have the device file /dev/ppp
which can be made by::
mknod /dev/ppp c 108 0
as root.
Isdn4k-utils
------------
Due to changes in the length of the phone number field, isdn4k-utils
needs to be recompiled or (preferably) upgraded.
NFS-utils
---------
In ancient (2.4 and earlier) kernels, the nfs server needed to know
about any client that expected to be able to access files via NFS. This
information would be given to the kernel by ``mountd`` when the client
mounted the filesystem, or by ``exportfs`` at system startup. exportfs
would take information about active clients from ``/var/lib/nfs/rmtab``.
This approach is quite fragile as it depends on rmtab being correct
which is not always easy, particularly when trying to implement
fail-over. Even when the system is working well, ``rmtab`` suffers from
getting lots of old entries that never get removed.
With modern kernels we have the option of having the kernel tell mountd
when it gets a request from an unknown host, and mountd can give
appropriate export information to the kernel. This removes the
dependency on ``rmtab`` and means that the kernel only needs to know about
currently active clients.
To enable this new functionality, you need to::
mount -t nfsd nfsd /proc/fs/nfsd
before running exportfs or mountd. It is recommended that all NFS
services be protected from the internet-at-large by a firewall where
that is possible.
mcelog
------
On x86 kernels the mcelog utility is needed to process and log machine check
events when ``CONFIG_X86_MCE`` is enabled. Machine check events are errors
reported by the CPU. Processing them is strongly encouraged.
Kernel documentation
********************
Sphinx
------
Please see :ref:`sphinx_install` in ``Documentation/doc-guide/sphinx.rst``
for details about Sphinx requirements.
Getting updated software
========================
Kernel compilation
******************
gcc
---
- <ftp://ftp.gnu.org/gnu/gcc/>
Make
----
- <ftp://ftp.gnu.org/gnu/make/>
Binutils
--------
- <https://www.kernel.org/pub/linux/devel/binutils/>
Flex
----
- <https://github.com/westes/flex/releases>
Bison
-----
- <ftp://ftp.gnu.org/gnu/bison/>
OpenSSL
-------
- <https://www.openssl.org/>
System utilities
****************
Util-linux
----------
- <https://www.kernel.org/pub/linux/utils/util-linux/>
Kmod
----
- <https://www.kernel.org/pub/linux/utils/kernel/kmod/>
- <https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git>
Ksymoops
--------
- <https://www.kernel.org/pub/linux/utils/kernel/ksymoops/v2.4/>
Mkinitrd
--------
- <https://code.launchpad.net/initrd-tools/main>
E2fsprogs
---------
- <http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.29.tar.gz>
JFSutils
--------
- <http://jfs.sourceforge.net/>
Reiserfsprogs
-------------
- <http://www.kernel.org/pub/linux/utils/fs/reiserfs/>
Xfsprogs
--------
- <ftp://oss.sgi.com/projects/xfs/>
Pcmciautils
-----------
- <https://www.kernel.org/pub/linux/utils/kernel/pcmcia/>
Quota-tools
-----------
- <http://sourceforge.net/projects/linuxquota/>
Intel P6 microcode
------------------
- <https://downloadcenter.intel.com/>
udev
----
- <http://www.freedesktop.org/software/systemd/man/udev.html>
FUSE
----
- <https://github.com/libfuse/libfuse/releases>
mcelog
------
- <http://www.mcelog.org/>
Networking
**********
PPP
---
- <ftp://ftp.samba.org/pub/ppp/>
Isdn4k-utils
------------
- <ftp://ftp.isdn4linux.de/pub/isdn4linux/utils/>
NFS-utils
---------
- <http://sourceforge.net/project/showfiles.php?group_id=14>
Iptables
--------
- <http://www.iptables.org/downloads.html>
Ip-route2
---------
- <https://www.kernel.org/pub/linux/utils/net/iproute2/>
OProfile
--------
- <http://oprofile.sf.net/download/>
NFS-Utils
---------
- <http://nfs.sourceforge.net/>
Kernel documentation
********************
Sphinx
------
- <http://www.sphinx-doc.org/>
Documentation/process/clang-format.rst
+184
View File
@@ -0,0 +1,184 @@
.. _clangformat:
clang-format
============
``clang-format`` is a tool to format C/C++/... code according to
a set of rules and heuristics. Like most tools, it is not perfect
nor covers every single case, but it is good enough to be helpful.
``clang-format`` can be used for several purposes:
- Quickly reformat a block of code to the kernel style. Specially useful
when moving code around and aligning/sorting. See clangformatreformat_.
- Spot style mistakes, typos and possible improvements in files
you maintain, patches you review, diffs, etc. See clangformatreview_.
- Help you follow the coding style rules, specially useful for those
new to kernel development or working at the same time in several
projects with different coding styles.
Its configuration file is ``.clang-format`` in the root of the kernel tree.
The rules contained there try to approximate the most common kernel
coding style. They also try to follow :ref:`Documentation/process/coding-style.rst <codingstyle>`
as much as possible. Since not all the kernel follows the same style,
it is possible that you may want to tweak the defaults for a particular
subsystem or folder. To do so, you can override the defaults by writing
another ``.clang-format`` file in a subfolder.
The tool itself has already been included in the repositories of popular
Linux distributions for a long time. Search for ``clang-format`` in
your repositories. Otherwise, you can either download pre-built
LLVM/clang binaries or build the source code from:
http://releases.llvm.org/download.html
See more information about the tool at:
https://clang.llvm.org/docs/ClangFormat.html
https://clang.llvm.org/docs/ClangFormatStyleOptions.html
.. _clangformatreview:
Review files and patches for coding style
-----------------------------------------
By running the tool in its inline mode, you can review full subsystems,
folders or individual files for code style mistakes, typos or improvements.
To do so, you can run something like::
# Make sure your working directory is clean!
clang-format -i kernel/*.[ch]
And then take a look at the git diff.
Counting the lines of such a diff is also useful for improving/tweaking
the style options in the configuration file; as well as testing new
``clang-format`` features/versions.
``clang-format`` also supports reading unified diffs, so you can review
patches and git diffs easily. See the documentation at:
https://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting
To avoid ``clang-format`` formatting some portion of a file, you can do::
int formatted_code;
// clang-format off
void unformatted_code ;
// clang-format on
void formatted_code_again;
While it might be tempting to use this to keep a file always in sync with
``clang-format``, specially if you are writing new files or if you are
a maintainer, please note that people might be running different
``clang-format`` versions or not have it available at all. Therefore,
you should probably refrain yourself from using this in kernel sources;
at least until we see if ``clang-format`` becomes commonplace.
.. _clangformatreformat:
Reformatting blocks of code
---------------------------
By using an integration with your text editor, you can reformat arbitrary
blocks (selections) of code with a single keystroke. This is specially
useful when moving code around, for complex code that is deeply intended,
for multi-line macros (and aligning their backslashes), etc.
Remember that you can always tweak the changes afterwards in those cases
where the tool did not do an optimal job. But as a first approximation,
it can be very useful.
There are integrations for many popular text editors. For some of them,
like vim, emacs, BBEdit and Visual Studio you can find support built-in.
For instructions, read the appropiate section at:
https://clang.llvm.org/docs/ClangFormat.html
For Atom, Eclipse, Sublime Text, Visual Studio Code, XCode and other
editors and IDEs you should be able to find ready-to-use plugins.
For this use case, consider using a secondary ``.clang-format``
so that you can tweak a few options. See clangformatextra_.
.. _clangformatmissing:
Missing support
---------------
``clang-format`` is missing support for some things that are common
in kernel code. They are easy to remember, so if you use the tool
regularly, you will quickly learn to avoid/ignore those.
In particular, some very common ones you will notice are:
- Aligned blocks of one-line ``#defines``, e.g.::
#define TRACING_MAP_BITS_DEFAULT 11
#define TRACING_MAP_BITS_MAX 17
#define TRACING_MAP_BITS_MIN 7
vs.::
#define TRACING_MAP_BITS_DEFAULT 11
#define TRACING_MAP_BITS_MAX 17
#define TRACING_MAP_BITS_MIN 7
- Aligned designated initializers, e.g.::
static const struct file_operations uprobe_events_ops = {
.owner = THIS_MODULE,
.open = probes_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release,
.write = probes_write,
};
vs.::
static const struct file_operations uprobe_events_ops = {
.owner = THIS_MODULE,
.open = probes_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release,
.write = probes_write,
};
.. _clangformatextra:
Extra features/options
----------------------
Some features/style options are not enabled by default in the configuration
file in order to minimize the differences between the output and the current
code. In other words, to make the difference as small as possible,
which makes reviewing full-file style, as well diffs and patches as easy
as possible.
In other cases (e.g. particular subsystems/folders/files), the kernel style
might be different and enabling some of these options may approximate
better the style there.
For instance:
- Aligning assignments (``AlignConsecutiveAssignments``).
- Aligning declarations (``AlignConsecutiveDeclarations``).
- Reflowing text in comments (``ReflowComments``).
- Sorting ``#includes`` (``SortIncludes``).
They are typically useful for block re-formatting, rather than full-file.
You might want to create another ``.clang-format`` file and use that one
from your editor/IDE instead.
Documentation/process/code-of-conduct-interpretation.rst
+156
View File
@@ -0,0 +1,156 @@
.. _code_of_conduct_interpretation:
Linux Kernel Contributor Covenant Code of Conduct Interpretation
================================================================
The :ref:`code_of_conduct` is a general document meant to
provide a set of rules for almost any open source community. Every
open-source community is unique and the Linux kernel is no exception.
Because of this, this document describes how we in the Linux kernel
community will interpret it. We also do not expect this interpretation
to be static over time, and will adjust it as needed.
The Linux kernel development effort is a very personal process compared
to "traditional" ways of developing software. Your contributions and
ideas behind them will be carefully reviewed, often resulting in
critique and criticism. The review will almost always require
improvements before the material can be included in the
kernel. Know that this happens because everyone involved wants to see
the best possible solution for the overall success of Linux. This
development process has been proven to create the most robust operating
system kernel ever, and we do not want to do anything to cause the
quality of submission and eventual result to ever decrease.
Maintainers
-----------
The Code of Conduct uses the term "maintainers" numerous times. In the
kernel community, a "maintainer" is anyone who is responsible for a
subsystem, driver, or file, and is listed in the MAINTAINERS file in the
kernel source tree.
Responsibilities
----------------
The Code of Conduct mentions rights and responsibilities for
maintainers, and this needs some further clarifications.
First and foremost, it is a reasonable expectation to have maintainers
lead by example.
That being said, our community is vast and broad, and there is no new
requirement for maintainers to unilaterally handle how other people
behave in the parts of the community where they are active. That
responsibility is upon all of us, and ultimately the Code of Conduct
documents final escalation paths in case of unresolved concerns
regarding conduct issues.
Maintainers should be willing to help when problems occur, and work with
others in the community when needed. Do not be afraid to reach out to
the Technical Advisory Board (TAB) or other maintainers if you're
uncertain how to handle situations that come up. It will not be
considered a violation report unless you want it to be. If you are
uncertain about approaching the TAB or any other maintainers, please
reach out to our conflict mediator, Mishi Choudhary <mishi@linux.com>.
In the end, "be kind to each other" is really what the end goal is for
everybody. We know everyone is human and we all fail at times, but the
primary goal for all of us should be to work toward amicable resolutions
of problems. Enforcement of the code of conduct will only be a last
resort option.
Our goal of creating a robust and technically advanced operating system
and the technical complexity involved naturally require expertise and
decision-making.
The required expertise varies depending on the area of contribution. It
is determined mainly by context and technical complexity and only
secondary by the expectations of contributors and maintainers.
Both the expertise expectations and decision-making are subject to
discussion, but at the very end there is a basic necessity to be able to
make decisions in order to make progress. This prerogative is in the
hands of maintainers and project's leadership and is expected to be used
in good faith.
As a consequence, setting expertise expectations, making decisions and
rejecting unsuitable contributions are not viewed as a violation of the
Code of Conduct.
While maintainers are in general welcoming to newcomers, their capacity
of helping contributors overcome the entry hurdles is limited, so they
have to set priorities. This, also, is not to be seen as a violation of
the Code of Conduct. The kernel community is aware of that and provides
entry level programs in various forms like kernelnewbies.org.
Scope
-----
The Linux kernel community primarily interacts on a set of public email
lists distributed around a number of different servers controlled by a
number of different companies or individuals. All of these lists are
defined in the MAINTAINERS file in the kernel source tree. Any emails
sent to those mailing lists are considered covered by the Code of
Conduct.
Developers who use the kernel.org bugzilla, and other subsystem bugzilla
or bug tracking tools should follow the guidelines of the Code of
Conduct. The Linux kernel community does not have an "official" project
email address, or "official" social media address. Any activity
performed using a kernel.org email account must follow the Code of
Conduct as published for kernel.org, just as any individual using a
corporate email account must follow the specific rules of that
corporation.
The Code of Conduct does not prohibit continuing to include names, email
addresses, and associated comments in mailing list messages, kernel
change log messages, or code comments.
Interaction in other forums is covered by whatever rules apply to said
forums and is in general not covered by the Code of Conduct. Exceptions
may be considered for extreme circumstances.
Contributions submitted for the kernel should use appropriate language.
Content that already exists predating the Code of Conduct will not be
addressed now as a violation. Inappropriate language can be seen as a
bug, though; such bugs will be fixed more quickly if any interested
parties submit patches to that effect. Expressions that are currently
part of the user/kernel API, or reflect terminology used in published
standards or specifications, are not considered bugs.
Enforcement
-----------
The address listed in the Code of Conduct goes to the Code of Conduct
Committee. The exact members receiving these emails at any given time
are listed at https://kernel.org/code-of-conduct.html. Members can not
access reports made before they joined or after they have left the
committee.
The initial Code of Conduct Committee consists of volunteer members of
the TAB, as well as a professional mediator acting as a neutral third
party. The first task of the committee is to establish documented
processes, which will be made public.
Any member of the committee, including the mediator, can be contacted
directly if a reporter does not wish to include the full committee in a
complaint or concern.
The Code of Conduct Committee reviews the cases according to the
processes (see above) and consults with the TAB as needed and
appropriate, for instance to request and receive information about the
kernel community.
Any decisions by the committee will be brought to the TAB, for
implementation of enforcement with the relevant maintainers if needed.
A decision by the Code of Conduct Committee can be overturned by the TAB
by a two-thirds vote.
At quarterly intervals, the Code of Conduct Committee and TAB will
provide a report summarizing the anonymised reports that the Code of
Conduct committee has received and their status, as well details of any
overridden decisions including complete and identifiable voting details.
We expect to establish a different process for Code of Conduct Committee
staffing beyond the bootstrap period. This document will be updated
with that information when this occurs.
Documentation/process/code-of-conduct.rst
+86
View File
@@ -0,0 +1,86 @@
.. _code_of_conduct:
Contributor Covenant Code of Conduct
++++++++++++++++++++++++++++++++++++
Our Pledge
==========
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and
expression, level of experience, education, socio-economic status, nationality,
personal appearance, race, religion, or sexual identity and orientation.
Our Standards
=============
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others’ private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
Our Responsibilities
====================
Maintainers are responsible for clarifying the standards of acceptable behavior
and are expected to take appropriate and fair corrective action in response to
any instances of unacceptable behavior.
Maintainers have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, or to ban temporarily or permanently any
contributor for other behaviors that they deem inappropriate, threatening,
offensive, or harmful.
Scope
=====
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
Enforcement
===========
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the Code of Conduct Committee at
<conduct@kernel.org>. All complaints will be reviewed and investigated
and will result in a response that is deemed necessary and appropriate
to the circumstances. The Code of Conduct Committee is obligated to
maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted
separately.
Attribution
===========
This Code of Conduct is adapted from the Contributor Covenant, version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
Interpretation
==============
See the :ref:`code_of_conduct_interpretation` document for how the Linux
kernel community will be interpreting this document.
Documentation/process/coding-style.rst
+1084
View File
File diff suppressed because it is too large. Load diff
Documentation/process/conf.py
+10
View File
@@ -0,0 +1,10 @@
# -*- coding: utf-8; mode: python -*-
project = 'Linux Kernel Development Documentation'
tags.add("subproject")
latex_documents = [
('index', 'process.tex', 'Linux Kernel Development Documentation',
'The kernel development community', 'manual'),
]
Documentation/process/development-process.rst
+28
View File
@@ -0,0 +1,28 @@
.. _development_process_main:
A guide to the Kernel Development Process
=========================================
Contents:
.. toctree::
:numbered:
:maxdepth: 2
1.Intro
2.Process
3.Early-stage
4.Coding
5.Posting
6.Followthrough
7.AdvancedTopics
8.Conclusion
The purpose of this document is to help developers (and their managers)
work with the development community with a minimum of frustration. It is
an attempt to document how this community works in a way which is
accessible to those who are not intimately familiar with Linux kernel
development (or, indeed, free software development in general). While
there is some technical material here, this is very much a process-oriented
discussion which does not require a deep knowledge of kernel programming to
understand.
Documentation/process/email-clients.rst
+324
View File
@@ -0,0 +1,324 @@
.. _email_clients:
Email clients info for Linux
============================
Git
---
These days most developers use ``git send-email`` instead of regular
email clients. The man page for this is quite good. On the receiving
end, maintainers use ``git am`` to apply the patches.
If you are new to ``git`` then send your first patch to yourself. Save it
as raw text including all the headers. Run ``git am raw_email.txt`` and
then review the changelog with ``git log``. When that works then send
the patch to the appropriate mailing list(s).
General Preferences
-------------------
Patches for the Linux kernel are submitted via email, preferably as
inline text in the body of the email. Some maintainers accept
attachments, but then the attachments should have content-type
``text/plain``. However, attachments are generally frowned upon because
it makes quoting portions of the patch more difficult in the patch
review process.
Email clients that are used for Linux kernel patches should send the
patch text untouched. For example, they should not modify or delete tabs
or spaces, even at the beginning or end of lines.
Don't send patches with ``format=flowed``. This can cause unexpected
and unwanted line breaks.
Don't let your email client do automatic word wrapping for you.
This can also corrupt your patch.
Email clients should not modify the character set encoding of the text.
Emailed patches should be in ASCII or UTF-8 encoding only.
If you configure your email client to send emails with UTF-8 encoding,
you avoid some possible charset problems.
Email clients should generate and maintain References: or In-Reply-To:
headers so that mail threading is not broken.
Copy-and-paste (or cut-and-paste) usually does not work for patches
because tabs are converted to spaces. Using xclipboard, xclip, and/or
xcutsel may work, but it's best to test this for yourself or just avoid
copy-and-paste.
Don't use PGP/GPG signatures in mail that contains patches.
This breaks many scripts that read and apply the patches.
(This should be fixable.)
It's a good idea to send a patch to yourself, save the received message,
and successfully apply it with 'patch' before sending patches to Linux
mailing lists.
Some email client (MUA) hints
-----------------------------
Here are some specific MUA configuration hints for editing and sending
patches for the Linux kernel. These are not meant to be complete
software package configuration summaries.
Legend:
- TUI = text-based user interface
- GUI = graphical user interface
Alpine (TUI)
************
Config options:
In the :menuselection:`Sending Preferences` section:
- :menuselection:`Do Not Send Flowed Text` must be ``enabled``
- :menuselection:`Strip Whitespace Before Sending` must be ``disabled``
When composing the message, the cursor should be placed where the patch
should appear, and then pressing :kbd:`CTRL-R` let you specify the patch file
to insert into the message.
Claws Mail (GUI)
****************
Works. Some people use this successfully for patches.
To insert a patch use :menuselection:`Message-->Insert` File (:kbd:`CTRL-I`)
or an external editor.
If the inserted patch has to be edited in the Claws composition window
"Auto wrapping" in
:menuselection:`Configuration-->Preferences-->Compose-->Wrapping` should be
disabled.
Evolution (GUI)
***************
Some people use this successfully for patches.
When composing mail select: Preformat
from :menuselection:`Format-->Paragraph Style-->Preformatted` (:kbd:`CTRL-7`)
or the toolbar
Then use:
:menuselection:`Insert-->Text File...` (:kbd:`ALT-N x`)
to insert the patch.
You can also ``diff -Nru old.c new.c | xclip``, select
:menuselection:`Preformat`, then paste with the middle button.
Kmail (GUI)
***********
Some people use Kmail successfully for patches.
The default setting of not composing in HTML is appropriate; do not
enable it.
When composing an email, under options, uncheck "word wrap". The only
disadvantage is any text you type in the email will not be word-wrapped
so you will have to manually word wrap text before the patch. The easiest
way around this is to compose your email with word wrap enabled, then save
it as a draft. Once you pull it up again from your drafts it is now hard
word-wrapped and you can uncheck "word wrap" without losing the existing
wrapping.
At the bottom of your email, put the commonly-used patch delimiter before
inserting your patch: three hyphens (``---``).
Then from the :menuselection:`Message` menu item, select insert file and
choose your patch.
As an added bonus you can customise the message creation toolbar menu
and put the :menuselection:`insert file` icon there.
Make the composer window wide enough so that no lines wrap. As of
KMail 1.13.5 (KDE 4.5.4), KMail will apply word wrapping when sending
the email if the lines wrap in the composer window. Having word wrapping
disabled in the Options menu isn't enough. Thus, if your patch has very
long lines, you must make the composer window very wide before sending
the email. See: https://bugs.kde.org/show_bug.cgi?id=174034
You can safely GPG sign attachments, but inlined text is preferred for
patches so do not GPG sign them. Signing patches that have been inserted
as inlined text will make them tricky to extract from their 7-bit encoding.
If you absolutely must send patches as attachments instead of inlining
them as text, right click on the attachment and select properties, and
highlight :menuselection:`Suggest automatic display` to make the attachment
inlined to make it more viewable.
When saving patches that are sent as inlined text, select the email that
contains the patch from the message list pane, right click and select
:menuselection:`save as`. You can use the whole email unmodified as a patch
if it was properly composed. There is no option currently to save the email
when you are actually viewing it in its own window -- there has been a request
filed at kmail's bugzilla and hopefully this will be addressed. Emails are
saved as read-write for user only so you will have to chmod them to make them
group and world readable if you copy them elsewhere.
Lotus Notes (GUI)
*****************
Run away from it.
IBM Verse (Web GUI)
*******************
See Lotus Notes.
Mutt (TUI)
**********
Plenty of Linux developers use ``mutt``, so it must work pretty well.
Mutt doesn't come with an editor, so whatever editor you use should be
used in a way that there are no automatic linebreaks. Most editors have
an :menuselection:`insert file` option that inserts the contents of a file
unaltered.
To use ``vim`` with mutt::
set editor="vi"
If using xclip, type the command::
:set paste
before middle button or shift-insert or use::
:r filename
if you want to include the patch inline.
(a)ttach works fine without ``set paste``.
You can also generate patches with ``git format-patch`` and then use Mutt
to send them::
$ mutt -H 0001-some-bug-fix.patch
Config options:
It should work with default settings.
However, it's a good idea to set the ``send_charset`` to::
set send_charset="us-ascii:utf-8"
Mutt is highly customizable. Here is a minimum configuration to start
using Mutt to send patches through Gmail::
# .muttrc
# ================ IMAP ====================
set imap_user = 'yourusername@gmail.com'
set imap_pass = 'yourpassword'
set spoolfile = imaps://imap.gmail.com/INBOX
set folder = imaps://imap.gmail.com/
set record="imaps://imap.gmail.com/[Gmail]/Sent Mail"
set postponed="imaps://imap.gmail.com/[Gmail]/Drafts"
set mbox="imaps://imap.gmail.com/[Gmail]/All Mail"
# ================ SMTP ====================
set smtp_url = "smtp://username@smtp.gmail.com:587/"
set smtp_pass = $imap_pass
set ssl_force_tls = yes # Require encrypted connection
# ================ Composition ====================
set editor = `echo \$EDITOR`
set edit_headers = yes # See the headers when editing
set charset = UTF-8 # value of $LANG; also fallback for send_charset
# Sender, email address, and sign-off line must match
unset use_domain # because joe@localhost is just embarrassing
set realname = "YOUR NAME"
set from = "username@gmail.com"
set use_from = yes
The Mutt docs have lots more information:
http://dev.mutt.org/trac/wiki/UseCases/Gmail
http://dev.mutt.org/doc/manual.html
Pine (TUI)
**********
Pine has had some whitespace truncation issues in the past, but these
should all be fixed now.
Use alpine (pine's successor) if you can.
Config options:
- ``quell-flowed-text`` is needed for recent versions
- the ``no-strip-whitespace-before-send`` option is needed
Sylpheed (GUI)
**************
- Works well for inlining text (or using attachments).
- Allows use of an external editor.
- Is slow on large folders.
- Won't do TLS SMTP auth over a non-SSL connection.
- Has a helpful ruler bar in the compose window.
- Adding addresses to address book doesn't understand the display name
properly.
Thunderbird (GUI)
*****************
Thunderbird is an Outlook clone that likes to mangle text, but there are ways
to coerce it into behaving.
- Allow use of an external editor:
The easiest thing to do with Thunderbird and patches is to use an
"external editor" extension and then just use your favorite ``$EDITOR``
for reading/merging patches into the body text. To do this, download
and install the extension, then add a button for it using
:menuselection:`View-->Toolbars-->Customize...` and finally just click on it
when in the :menuselection:`Compose` dialog.
Please note that "external editor" requires that your editor must not
fork, or in other words, the editor must not return before closing.
You may have to pass additional flags or change the settings of your
editor. Most notably if you are using gvim then you must pass the -f
option to gvim by putting ``/usr/bin/gvim -f`` (if the binary is in
``/usr/bin``) to the text editor field in :menuselection:`external editor`
settings. If you are using some other editor then please read its manual
to find out how to do this.
To beat some sense out of the internal editor, do this:
- Edit your Thunderbird config settings so that it won't use ``format=flowed``.
Go to :menuselection:`edit-->preferences-->advanced-->config editor` to bring up
the thunderbird's registry editor.
- Set ``mailnews.send_plaintext_flowed`` to ``false``
- Set ``mailnews.wraplength`` from ``72`` to ``0``
- :menuselection:`View-->Message Body As-->Plain Text`
- :menuselection:`View-->Character Encoding-->Unicode (UTF-8)`
TkRat (GUI)
***********
Works. Use "Insert file..." or external editor.
Gmail (Web GUI)
***************
Does not work for sending patches.
Gmail web client converts tabs to spaces automatically.
At the same time it wraps lines every 78 chars with CRLF style line breaks
although tab2space problem can be solved with external editor.
Another problem is that Gmail will base64-encode any message that has a
non-ASCII character. That includes things like European names.
Documentation/process/howto.rst
+635
View File
@@ -0,0 +1,635 @@
HOWTO do Linux kernel development
=================================
This is the be-all, end-all document on this topic. It contains
instructions on how to become a Linux kernel developer and how to learn
to work with the Linux kernel development community. It tries to not
contain anything related to the technical aspects of kernel programming,
but will help point you in the right direction for that.
If anything in this document becomes out of date, please send in patches
to the maintainer of this file, who is listed at the bottom of the
document.
Introduction
------------
So, you want to learn how to become a Linux kernel developer? Or you
have been told by your manager, "Go write a Linux driver for this
device." This document's goal is to teach you everything you need to
know to achieve this by describing the process you need to go through,
and hints on how to work with the community. It will also try to
explain some of the reasons why the community works like it does.
The kernel is written mostly in C, with some architecture-dependent
parts written in assembly. A good understanding of C is required for
kernel development. Assembly (any architecture) is not required unless
you plan to do low-level development for that architecture. Though they
are not a good substitute for a solid C education and/or years of
experience, the following books are good for, if anything, reference:
- "The C Programming Language" by Kernighan and Ritchie [Prentice Hall]
- "Practical C Programming" by Steve Oualline [O'Reilly]
- "C: A Reference Manual" by Harbison and Steele [Prentice Hall]
The kernel is written using GNU C and the GNU toolchain. While it
adheres to the ISO C89 standard, it uses a number of extensions that are
not featured in the standard. The kernel is a freestanding C
environment, with no reliance on the standard C library, so some
portions of the C standard are not supported. Arbitrary long long
divisions and floating point are not allowed. It can sometimes be
difficult to understand the assumptions the kernel has on the toolchain
and the extensions that it uses, and unfortunately there is no
definitive reference for them. Please check the gcc info pages (`info
gcc`) for some information on them.
Please remember that you are trying to learn how to work with the
existing development community. It is a diverse group of people, with
high standards for coding, style and procedure. These standards have
been created over time based on what they have found to work best for
such a large and geographically dispersed team. Try to learn as much as
possible about these standards ahead of time, as they are well
documented; do not expect people to adapt to you or your company's way
of doing things.
Legal Issues
------------
The Linux kernel source code is released under the GPL. Please see the
file, COPYING, in the main directory of the source tree, for details on
the license. If you have further questions about the license, please
contact a lawyer, and do not ask on the Linux kernel mailing list. The
people on the mailing lists are not lawyers, and you should not rely on
their statements on legal matters.
For common questions and answers about the GPL, please see:
https://www.gnu.org/licenses/gpl-faq.html
Documentation
-------------
The Linux kernel source tree has a large range of documents that are
invaluable for learning how to interact with the kernel community. When
new features are added to the kernel, it is recommended that new
documentation files are also added which explain how to use the feature.
When a kernel change causes the interface that the kernel exposes to
userspace to change, it is recommended that you send the information or
a patch to the manual pages explaining the change to the manual pages
maintainer at mtk.manpages@gmail.com, and CC the list
linux-api@vger.kernel.org.
Here is a list of files that are in the kernel source tree that are
required reading:
:ref:`Documentation/admin-guide/README.rst <readme>`
This file gives a short background on the Linux kernel and describes
what is necessary to do to configure and build the kernel. People
who are new to the kernel should start here.
:ref:`Documentation/process/changes.rst <changes>`
This file gives a list of the minimum levels of various software
packages that are necessary to build and run the kernel
successfully.
:ref:`Documentation/process/coding-style.rst <codingstyle>`
This describes the Linux kernel coding style, and some of the
rationale behind it. All new code is expected to follow the
guidelines in this document. Most maintainers will only accept
patches if these rules are followed, and many people will only
review code if it is in the proper style.
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>` and :ref:`Documentation/process/submitting-drivers.rst <submittingdrivers>`
These files describe in explicit detail how to successfully create
and send a patch, including (but not limited to):
- Email contents
- Email format
- Who to send it to
Following these rules will not guarantee success (as all patches are
subject to scrutiny for content and style), but not following them
will almost always prevent it.
Other excellent descriptions of how to create patches properly are:
"The Perfect Patch"
https://www.ozlabs.org/~akpm/stuff/tpp.txt
"Linux kernel patch submission format"
http://linux.yyz.us/patch-format.html
:ref:`Documentation/process/stable-api-nonsense.rst <stable_api_nonsense>`
This file describes the rationale behind the conscious decision to
not have a stable API within the kernel, including things like:
- Subsystem shim-layers (for compatibility?)
- Driver portability between Operating Systems.
- Mitigating rapid change within the kernel source tree (or
preventing rapid change)
This document is crucial for understanding the Linux development
philosophy and is very important for people moving to Linux from
development on other Operating Systems.
:ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`
If you feel you have found a security problem in the Linux kernel,
please follow the steps in this document to help notify the kernel
developers, and help solve the issue.
:ref:`Documentation/process/management-style.rst <managementstyle>`
This document describes how Linux kernel maintainers operate and the
shared ethos behind their methodologies. This is important reading
for anyone new to kernel development (or anyone simply curious about
it), as it resolves a lot of common misconceptions and confusion
about the unique behavior of kernel maintainers.
:ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>`
This file describes the rules on how the stable kernel releases
happen, and what to do if you want to get a change into one of these
releases.
:ref:`Documentation/process/kernel-docs.rst <kernel_docs>`
A list of external documentation that pertains to kernel
development. Please consult this list if you do not find what you
are looking for within the in-kernel documentation.
:ref:`Documentation/process/applying-patches.rst <applying_patches>`
A good introduction describing exactly what a patch is and how to
apply it to the different development branches of the kernel.
The kernel also has a large number of documents that can be
automatically generated from the source code itself or from
ReStructuredText markups (ReST), like this one. This includes a
full description of the in-kernel API, and rules on how to handle
locking properly.
All such documents can be generated as PDF or HTML by running::
make pdfdocs
make htmldocs
respectively from the main kernel source directory.
The documents that uses ReST markup will be generated at Documentation/output.
They can also be generated on LaTeX and ePub formats with::
make latexdocs
make epubdocs
Becoming A Kernel Developer
---------------------------
If you do not know anything about Linux kernel development, you should
look at the Linux KernelNewbies project:
https://kernelnewbies.org
It consists of a helpful mailing list where you can ask almost any type
of basic kernel development question (make sure to search the archives
first, before asking something that has already been answered in the
past.) It also has an IRC channel that you can use to ask questions in
real-time, and a lot of helpful documentation that is useful for
learning about Linux kernel development.
The website has basic information about code organization, subsystems,
and current projects (both in-tree and out-of-tree). It also describes
some basic logistical information, like how to compile a kernel and
apply a patch.
If you do not know where you want to start, but you want to look for
some task to start doing to join into the kernel development community,
go to the Linux Kernel Janitor's project:
https://kernelnewbies.org/KernelJanitors
It is a great place to start. It describes a list of relatively simple
problems that need to be cleaned up and fixed within the Linux kernel
source tree. Working with the developers in charge of this project, you
will learn the basics of getting your patch into the Linux kernel tree,
and possibly be pointed in the direction of what to go work on next, if
you do not already have an idea.
Before making any actual modifications to the Linux kernel code, it is
imperative to understand how the code in question works. For this
purpose, nothing is better than reading through it directly (most tricky
bits are commented well), perhaps even with the help of specialized
tools. One such tool that is particularly recommended is the Linux
Cross-Reference project, which is able to present source code in a
self-referential, indexed webpage format. An excellent up-to-date
repository of the kernel code may be found at:
http://lxr.free-electrons.com/
The development process
-----------------------
Linux kernel development process currently consists of a few different
main kernel "branches" and lots of different subsystem-specific kernel
branches. These different branches are:
- main 4.x kernel tree
- 4.x.y -stable kernel tree
- 4.x -git kernel patches
- subsystem specific kernel trees and patches
- the 4.x -next kernel tree for integration tests
4.x kernel tree
~~~~~~~~~~~~~~~
4.x kernels are maintained by Linus Torvalds, and can be found on
https://kernel.org in the pub/linux/kernel/v4.x/ directory. Its development
process is as follows:
- As soon as a new kernel is released a two weeks window is open,
during this period of time maintainers can submit big diffs to
Linus, usually the patches that have already been included in the
-next kernel for a few weeks. The preferred way to submit big changes
is using git (the kernel's source management tool, more information
can be found at https://git-scm.com/) but plain patches are also just
fine.
- After two weeks a -rc1 kernel is released and the focus is on making the
new kernel as rock solid as possible. Most of the patches at this point
should fix a regression. Bugs that have always existed are not
regressions, so only push these kinds of fixes if they are important.
Please note that a whole new driver (or filesystem) might be accepted
after -rc1 because there is no risk of causing regressions with such a
change as long as the change is self-contained and does not affect areas
outside of the code that is being added. git can be used to send
patches to Linus after -rc1 is released, but the patches need to also be
sent to a public mailing list for review.
- A new -rc is released whenever Linus deems the current git tree to
be in a reasonably sane state adequate for testing. The goal is to
release a new -rc kernel every week.
- Process continues until the kernel is considered "ready", the
process should last around 6 weeks.
It is worth mentioning what Andrew Morton wrote on the linux-kernel
mailing list about kernel releases:
*"Nobody knows when a kernel will be released, because it's
released according to perceived bug status, not according to a
preconceived timeline."*
4.x.y -stable kernel tree
~~~~~~~~~~~~~~~~~~~~~~~~~
Kernels with 3-part versions are -stable kernels. They contain
relatively small and critical fixes for security problems or significant
regressions discovered in a given 4.x kernel.
This is the recommended branch for users who want the most recent stable
kernel and are not interested in helping test development/experimental
versions.
If no 4.x.y kernel is available, then the highest numbered 4.x
kernel is the current stable kernel.
4.x.y are maintained by the "stable" team <stable@vger.kernel.org>, and
are released as needs dictate. The normal release period is approximately
two weeks, but it can be longer if there are no pressing problems. A
security-related problem, instead, can cause a release to happen almost
instantly.
The file Documentation/process/stable-kernel-rules.rst in the kernel tree
documents what kinds of changes are acceptable for the -stable tree, and
how the release process works.
4.x -git patches
~~~~~~~~~~~~~~~~
These are daily snapshots of Linus' kernel tree which are managed in a
git repository (hence the name.) These patches are usually released
daily and represent the current state of Linus' tree. They are more
experimental than -rc kernels since they are generated automatically
without even a cursory glance to see if they are sane.
Subsystem Specific kernel trees and patches
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The maintainers of the various kernel subsystems --- and also many
kernel subsystem developers --- expose their current state of
development in source repositories. That way, others can see what is
happening in the different areas of the kernel. In areas where
development is rapid, a developer may be asked to base his submissions
onto such a subsystem kernel tree so that conflicts between the
submission and other already ongoing work are avoided.
Most of these repositories are git trees, but there are also other SCMs
in use, or patch queues being published as quilt series. Addresses of
these subsystem repositories are listed in the MAINTAINERS file. Many
of them can be browsed at https://git.kernel.org/.
Before a proposed patch is committed to such a subsystem tree, it is
subject to review which primarily happens on mailing lists (see the
respective section below). For several kernel subsystems, this review
process is tracked with the tool patchwork. Patchwork offers a web
interface which shows patch postings, any comments on a patch or
revisions to it, and maintainers can mark patches as under review,
accepted, or rejected. Most of these patchwork sites are listed at
https://patchwork.kernel.org/.
4.x -next kernel tree for integration tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before updates from subsystem trees are merged into the mainline 4.x
tree, they need to be integration-tested. For this purpose, a special
testing repository exists into which virtually all subsystem trees are
pulled on an almost daily basis:
https://git.kernel.org/?p=linux/kernel/git/next/linux-next.git
This way, the -next kernel gives a summary outlook onto what will be
expected to go into the mainline kernel at the next merge period.
Adventurous testers are very welcome to runtime-test the -next kernel.
Bug Reporting
-------------
https://bugzilla.kernel.org is where the Linux kernel developers track kernel
bugs. Users are encouraged to report all bugs that they find in this
tool. For details on how to use the kernel bugzilla, please see:
https://bugzilla.kernel.org/page.cgi?id=faq.html
The file admin-guide/reporting-bugs.rst in the main kernel source directory has a good
template for how to report a possible kernel bug, and details what kind
of information is needed by the kernel developers to help track down the
problem.
Managing bug reports
--------------------
One of the best ways to put into practice your hacking skills is by fixing
bugs reported by other people. Not only you will help to make the kernel
more stable, you'll learn to fix real world problems and you will improve
your skills, and other developers will be aware of your presence. Fixing
bugs is one of the best ways to get merits among other developers, because
not many people like wasting time fixing other people's bugs.
To work in the already reported bug reports, go to https://bugzilla.kernel.org.
Mailing lists
-------------
As some of the above documents describe, the majority of the core kernel
developers participate on the Linux Kernel Mailing list. Details on how
to subscribe and unsubscribe from the list can be found at:
http://vger.kernel.org/vger-lists.html#linux-kernel
There are archives of the mailing list on the web in many different
places. Use a search engine to find these archives. For example:
http://dir.gmane.org/gmane.linux.kernel
It is highly recommended that you search the archives about the topic
you want to bring up, before you post it to the list. A lot of things
already discussed in detail are only recorded at the mailing list
archives.
Most of the individual kernel subsystems also have their own separate
mailing list where they do their development efforts. See the
MAINTAINERS file for a list of what these lists are for the different
groups.
Many of the lists are hosted on kernel.org. Information on them can be
found at:
http://vger.kernel.org/vger-lists.html
Please remember to follow good behavioral habits when using the lists.
Though a bit cheesy, the following URL has some simple guidelines for
interacting with the list (or any list):
http://www.albion.com/netiquette/
If multiple people respond to your mail, the CC: list of recipients may
get pretty large. Don't remove anybody from the CC: list without a good
reason, or don't reply only to the list address. Get used to receiving the
mail twice, one from the sender and the one from the list, and don't try
to tune that by adding fancy mail-headers, people will not like it.
Remember to keep the context and the attribution of your replies intact,
keep the "John Kernelhacker wrote ...:" lines at the top of your reply, and
add your statements between the individual quoted sections instead of
writing at the top of the mail.
If you add patches to your mail, make sure they are plain readable text
as stated in Documentation/process/submitting-patches.rst.
Kernel developers don't want to deal with
attachments or compressed patches; they may want to comment on
individual lines of your patch, which works only that way. Make sure you
use a mail program that does not mangle spaces and tab characters. A
good first test is to send the mail to yourself and try to apply your
own patch by yourself. If that doesn't work, get your mail program fixed
or change it until it works.
Above all, please remember to show respect to other subscribers.
Working with the community
--------------------------
The goal of the kernel community is to provide the best possible kernel
there is. When you submit a patch for acceptance, it will be reviewed
on its technical merits and those alone. So, what should you be
expecting?
- criticism
- comments
- requests for change
- requests for justification
- silence
Remember, this is part of getting your patch into the kernel. You have
to be able to take criticism and comments about your patches, evaluate
them at a technical level and either rework your patches or provide
clear and concise reasoning as to why those changes should not be made.
If there are no responses to your posting, wait a few days and try
again, sometimes things get lost in the huge volume.
What should you not do?
- expect your patch to be accepted without question
- become defensive
- ignore comments
- resubmit the patch without making any of the requested changes
In a community that is looking for the best technical solution possible,
there will always be differing opinions on how beneficial a patch is.
You have to be cooperative, and willing to adapt your idea to fit within
the kernel. Or at least be willing to prove your idea is worth it.
Remember, being wrong is acceptable as long as you are willing to work
toward a solution that is right.
It is normal that the answers to your first patch might simply be a list
of a dozen things you should correct. This does **not** imply that your
patch will not be accepted, and it is **not** meant against you
personally. Simply correct all issues raised against your patch and
resend it.
Differences between the kernel community and corporate structures
-----------------------------------------------------------------
The kernel community works differently than most traditional corporate
development environments. Here are a list of things that you can try to
do to avoid problems:
Good things to say regarding your proposed changes:
- "This solves multiple problems."
- "This deletes 2000 lines of code."
- "Here is a patch that explains what I am trying to describe."
- "I tested it on 5 different architectures..."
- "Here is a series of small patches that..."
- "This increases performance on typical machines..."
Bad things you should avoid saying:
- "We did it this way in AIX/ptx/Solaris, so therefore it must be
good..."
- "I've being doing this for 20 years, so..."
- "This is required for my company to make money"
- "This is for our Enterprise product line."
- "Here is my 1000 page design document that describes my idea"
- "I've been working on this for 6 months..."
- "Here's a 5000 line patch that..."
- "I rewrote all of the current mess, and here it is..."
- "I have a deadline, and this patch needs to be applied now."
Another way the kernel community is different than most traditional
software engineering work environments is the faceless nature of
interaction. One benefit of using email and irc as the primary forms of
communication is the lack of discrimination based on gender or race.
The Linux kernel work environment is accepting of women and minorities
because all you are is an email address. The international aspect also
helps to level the playing field because you can't guess gender based on
a person's name. A man may be named Andrea and a woman may be named Pat.
Most women who have worked in the Linux kernel and have expressed an
opinion have had positive experiences.
The language barrier can cause problems for some people who are not
comfortable with English. A good grasp of the language can be needed in
order to get ideas across properly on mailing lists, so it is
recommended that you check your emails to make sure they make sense in
English before sending them.
Break up your changes
---------------------
The Linux kernel community does not gladly accept large chunks of code
dropped on it all at once. The changes need to be properly introduced,
discussed, and broken up into tiny, individual portions. This is almost
the exact opposite of what companies are used to doing. Your proposal
should also be introduced very early in the development process, so that
you can receive feedback on what you are doing. It also lets the
community feel that you are working with them, and not simply using them
as a dumping ground for your feature. However, don't send 50 emails at
one time to a mailing list, your patch series should be smaller than
that almost all of the time.
The reasons for breaking things up are the following:
1) Small patches increase the likelihood that your patches will be
applied, since they don't take much time or effort to verify for
correctness. A 5 line patch can be applied by a maintainer with
barely a second glance. However, a 500 line patch may take hours to
review for correctness (the time it takes is exponentially
proportional to the size of the patch, or something).
Small patches also make it very easy to debug when something goes
wrong. It's much easier to back out patches one by one than it is
to dissect a very large patch after it's been applied (and broken
something).
2) It's important not only to send small patches, but also to rewrite
and simplify (or simply re-order) patches before submitting them.
Here is an analogy from kernel developer Al Viro:
*"Think of a teacher grading homework from a math student. The
teacher does not want to see the student's trials and errors
before they came up with the solution. They want to see the
cleanest, most elegant answer. A good student knows this, and
would never submit her intermediate work before the final
solution.*
*The same is true of kernel development. The maintainers and
reviewers do not want to see the thought process behind the
solution to the problem one is solving. They want to see a
simple and elegant solution."*
It may be challenging to keep the balance between presenting an elegant
solution and working together with the community and discussing your
unfinished work. Therefore it is good to get early in the process to
get feedback to improve your work, but also keep your changes in small
chunks that they may get already accepted, even when your whole task is
not ready for inclusion now.
Also realize that it is not acceptable to send patches for inclusion
that are unfinished and will be "fixed up later."
Justify your change
-------------------
Along with breaking up your patches, it is very important for you to let
the Linux community know why they should add this change. New features
must be justified as being needed and useful.
Document your change
--------------------
When sending in your patches, pay special attention to what you say in
the text in your email. This information will become the ChangeLog
information for the patch, and will be preserved for everyone to see for
all time. It should describe the patch completely, containing:
- why the change is necessary
- the overall design approach in the patch
- implementation details
- testing results
For more details on what this should all look like, please see the
ChangeLog section of the document:
"The Perfect Patch"
http://www.ozlabs.org/~akpm/stuff/tpp.txt
All of these things are sometimes very hard to do. It can take years to
perfect these practices (if at all). It's a continuous process of
improvement that requires a lot of patience and determination. But
don't give up, it's possible. Many have done it before, and each had to
start exactly where you are now.
----------
Thanks to Paolo Ciarrocchi who allowed the "Development Process"
(https://lwn.net/Articles/94386/) section
to be based on text he had written, and to Randy Dunlap and Gerrit
Huizenga for some of the list of things you should and should not say.
Also thanks to Pat Mochel, Hanna Linder, Randy Dunlap, Kay Sievers,
Vojtech Pavlik, Jan Kara, Josh Boyer, Kees Cook, Andrew Morton, Andi
Kleen, Vadim Lobanov, Jesper Juhl, Adrian Bunk, Keri Harris, Frans Pop,
David A. Wheeler, Junio Hamano, Michael Kerrisk, and Alex Shepard for
their review, comments, and contributions. Without their help, this
document would not have been possible.
Maintainer: Greg Kroah-Hartman <greg@kroah.com>
Documentation/process/index.rst
+63
View File
@@ -0,0 +1,63 @@
.. raw:: latex
\renewcommand\thesection*
\renewcommand\thesubsection*
.. _process_index:
Working with the kernel development community
=============================================
So you want to be a Linux kernel developer? Welcome! While there is a lot
to be learned about the kernel in a technical sense, it is also important
to learn about how our community works. Reading these documents will make
it much easier for you to get your changes merged with a minimum of
trouble.
Below are the essential guides that every developer should read.
.. toctree::
:maxdepth: 1
howto
code-of-conduct
code-of-conduct-interpretation
development-process
submitting-patches
coding-style
maintainer-pgp-guide
email-clients
kernel-enforcement-statement
kernel-driver-statement
Other guides to the community that are of interest to most developers are:
.. toctree::
:maxdepth: 1
changes
submitting-drivers
stable-api-nonsense
management-style
stable-kernel-rules
submit-checklist
kernel-docs
These are some overall technical guides that have been put here for now for
lack of a better place.
.. toctree::
:maxdepth: 1
applying-patches
adding-syscalls
magic-number
volatile-considered-harmful
clang-format
.. only:: subproject and html
Indices
=======
* :ref:`genindex`
Documentation/process/kernel-docs.rst
+620
View File
@@ -0,0 +1,620 @@
.. _kernel_docs:
Index of Documentation for People Interested in Writing and/or Understanding the Linux Kernel
=============================================================================================
Juan-Mariano de Goyeneche <jmseyas@dit.upm.es>
The need for a document like this one became apparent in the
linux-kernel mailing list as the same questions, asking for pointers
to information, appeared again and again.
Fortunately, as more and more people get to GNU/Linux, more and more
get interested in the Kernel. But reading the sources is not always
enough. It is easy to understand the code, but miss the concepts, the
philosophy and design decisions behind this code.
Unfortunately, not many documents are available for beginners to
start. And, even if they exist, there was no "well-known" place which
kept track of them. These lines try to cover this lack. All documents
available on line known by the author are listed, while some reference
books are also mentioned.
PLEASE, if you know any paper not listed here or write a new document,
send me an e-mail, and I'll include a reference to it here. Any
corrections, ideas or comments are also welcomed.
The papers that follow are listed in no particular order. All are
cataloged with the following fields: the document's "Title", the
"Author"/s, the "URL" where they can be found, some "Keywords" helpful
when searching for specific topics, and a brief "Description" of the
Document.
Enjoy!
.. note::
The documents on each section of this document are ordered by its
published date, from the newest to the oldest.
Docs at the Linux Kernel tree
-----------------------------
The Sphinx books should be built with ``make {htmldocs | pdfdocs | epubdocs}``.
* Name: **linux/Documentation**
:Author: Many.
:Location: Documentation/
:Keywords: text files, Sphinx.
:Description: Documentation that comes with the kernel sources,
inside the Documentation directory. Some pages from this document
(including this document itself) have been moved there, and might
be more up to date than the web version.
On-line docs
------------
* Title: **Linux Kernel Mailing List Glossary**
:Author: various
:URL: https://kernelnewbies.org/KernelGlossary
:Date: rolling version
:Keywords: glossary, terms, linux-kernel.
:Description: From the introduction: "This glossary is intended as
a brief description of some of the acronyms and terms you may hear
during discussion of the Linux kernel".
* Title: **Tracing the Way of Data in a TCP Connection through the Linux Kernel**
:Author: Richard Sailer
:URL: https://archive.org/details/linux_kernel_data_flow_short_paper
:Date: 2016
:Keywords: Linux Kernel Networking, TCP, tracing, ftrace
:Description: A seminar paper explaining ftrace and how to use it for
understanding linux kernel internals,
illustrated at tracing the way of a TCP packet through the kernel.
:Abstract: *This short paper outlines the usage of ftrace a tracing framework
as a tool to understand a running Linux system.
Having obtained a trace-log a kernel hacker can read and understand
source code more determined and with context.
In a detailed example this approach is demonstrated in tracing
and the way of data in a TCP Connection through the kernel.
Finally this trace-log is used as base for more a exact conceptual
exploration and description of the Linux TCP/IP implementation.*
* Title: **On submitting kernel Patches**
:Author: Andi Kleen
:URL: http://halobates.de/on-submitting-kernel-patches.pdf
:Date: 2008
:Keywords: patches, review process, types of submissions, basic rules, case studies
:Description: This paper gives several experience values on what types of patches
there are and how likley they get merged.
:Abstract:
[...]. This paper examines some common problems for
submitting larger changes and some strategies to avoid problems.
* Title: **Overview of the Virtual File System**
:Author: Richard Gooch.
:URL: http://www.mjmwired.net/kernel/Documentation/filesystems/vfs.txt
:Date: 2007
:Keywords: VFS, File System, mounting filesystems, opening files,
dentries, dcache.
:Description: Brief introduction to the Linux Virtual File System.
What is it, how it works, operations taken when opening a file or
mounting a file system and description of important data
structures explaining the purpose of each of their entries.
* Title: **Linux Device Drivers, Third Edition**
:Author: Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman
:URL: http://lwn.net/Kernel/LDD3/
:Date: 2005
:Description: A 600-page book covering the (2.6.10) driver
programming API and kernel hacking in general. Available under the
Creative Commons Attribution-ShareAlike 2.0 license.
:note: You can also :ref:`purchase a copy from O'Reilly or elsewhere <ldd3_published>`.
* Title: **Writing an ALSA Driver**
:Author: Takashi Iwai <tiwai@suse.de>
:URL: http://www.alsa-project.org/~iwai/writing-an-alsa-driver/index.html
:Date: 2005
:Keywords: ALSA, sound, soundcard, driver, lowlevel, hardware.
:Description: Advanced Linux Sound Architecture for developers,
both at kernel and user-level sides. ALSA is the Linux kernel
sound architecture in the 2.6 kernel version.
* Title: **Linux PCMCIA Programmer's Guide**
:Author: David Hinds.
:URL: http://pcmcia-cs.sourceforge.net/ftp/doc/PCMCIA-PROG.html
:Date: 2003
:Keywords: PCMCIA.
:Description: "This document describes how to write kernel device
drivers for the Linux PCMCIA Card Services interface. It also
describes how to write user-mode utilities for communicating with
Card Services.
* Title: **Linux Kernel Module Programming Guide**
:Author: Ori Pomerantz.
:URL: http://tldp.org/LDP/lkmpg/2.6/html/index.html
:Date: 2001
:Keywords: modules, GPL book, /proc, ioctls, system calls,
interrupt handlers .
:Description: Very nice 92 pages GPL book on the topic of modules
programming. Lots of examples.
* Title: **Global spinlock list and usage**
:Author: Rick Lindsley.
:URL: http://lse.sourceforge.net/lockhier/global-spin-lock
:Date: 2001
:Keywords: spinlock.
:Description: This is an attempt to document both the existence and
usage of the spinlocks in the Linux 2.4.5 kernel. Comprehensive
list of spinlocks showing when they are used, which functions
access them, how each lock is acquired, under what conditions it
is held, whether interrupts can occur or not while it is held...
* Title: **A Linux vm README**
:Author: Kanoj Sarcar.
:URL: http://kos.enix.org/pub/linux-vmm.html
:Date: 2001
:Keywords: virtual memory, mm, pgd, vma, page, page flags, page
cache, swap cache, kswapd.
:Description: Telegraphic, short descriptions and definitions
relating the Linux virtual memory implementation.
* Title: **Video4linux Drivers, Part 1: Video-Capture Device**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/406
:Date: 2000
:Keywords: video4linux, driver, video capture, capture devices,
camera driver.
:Description: The title says it all.
* Title: **Video4linux Drivers, Part 2: Video-capture Devices**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/429
:Date: 2000
:Keywords: video4linux, driver, video capture, capture devices,
camera driver, control, query capabilities, capability, facility.
:Description: The title says it all.
* Title: **Linux IP Networking. A Guide to the Implementation and Modification of the Linux Protocol Stack.**
:Author: Glenn Herrin.
:URL: http://www.cs.unh.edu/cnrg/gherrin
:Date: 2000
:Keywords: network, networking, protocol, IP, UDP, TCP, connection,
socket, receiving, transmitting, forwarding, routing, packets,
modules, /proc, sk_buff, FIB, tags.
:Description: Excellent paper devoted to the Linux IP Networking,
explaining anything from the kernel's to the user space
configuration tools' code. Very good to get a general overview of
the kernel networking implementation and understand all steps
packets follow from the time they are received at the network
device till they are delivered to applications. The studied kernel
code is from 2.2.14 version. Provides code for a working packet
dropper example.
* Title: **How To Make Sure Your Driver Will Work On The Power Macintosh**
:Author: Paul Mackerras.
:URL: http://www.linux-mag.com/id/261
:Date: 1999
:Keywords: Mac, Power Macintosh, porting, drivers, compatibility.
:Description: The title says it all.
* Title: **An Introduction to SCSI Drivers**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/284
:Date: 1999
:Keywords: SCSI, device, driver.
:Description: The title says it all.
* Title: **Advanced SCSI Drivers And Other Tales**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/307
:Date: 1999
:Keywords: SCSI, device, driver, advanced.
:Description: The title says it all.
* Title: **Writing Linux Mouse Drivers**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/330
:Date: 1999
:Keywords: mouse, driver, gpm.
:Description: The title says it all.
* Title: **More on Mouse Drivers**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/356
:Date: 1999
:Keywords: mouse, driver, gpm, races, asynchronous I/O.
:Description: The title still says it all.
* Title: **Writing Video4linux Radio Driver**
:Author: Alan Cox.
:URL: http://www.linux-mag.com/id/381
:Date: 1999
:Keywords: video4linux, driver, radio, radio devices.
:Description: The title says it all.
* Title: **I/O Event Handling Under Linux**
:Author: Richard Gooch.
:URL: http://web.mit.edu/~yandros/doc/io-events.html
:Date: 1999
:Keywords: IO, I/O, select(2), poll(2), FDs, aio_read(2), readiness
event queues.
:Description: From the Introduction: "I/O Event handling is about
how your Operating System allows you to manage a large number of
open files (file descriptors in UNIX/POSIX, or FDs) in your
application. You want the OS to notify you when FDs become active
(have data ready to be read or are ready for writing). Ideally you
want a mechanism that is scalable. This means a large number of
inactive FDs cost very little in memory and CPU time to manage".
* Title: **(nearly) Complete Linux Loadable Kernel Modules. The definitive guide for hackers, virus coders and system administrators.**
:Author: pragmatic/THC.
:URL: http://packetstormsecurity.org/docs/hack/LKM_HACKING.html
:Date: 1999
:Keywords: syscalls, intercept, hide, abuse, symbol table.
:Description: Interesting paper on how to abuse the Linux kernel in
order to intercept and modify syscalls, make
files/directories/processes invisible, become root, hijack ttys,
write kernel modules based virus... and solutions for admins to
avoid all those abuses.
:Notes: For 2.0.x kernels. Gives guidances to port it to 2.2.x
kernels.
* Name: **Linux Virtual File System**
:Author: Peter J. Braam.
:URL: http://www.coda.cs.cmu.edu/doc/talks/linuxvfs/
:Date: 1998
:Keywords: slides, VFS, inode, superblock, dentry, dcache.
:Description: Set of slides, presumably from a presentation on the
Linux VFS layer. Covers version 2.1.x, with dentries and the
dcache.
* Title: **The Venus kernel interface**
:Author: Peter J. Braam.
:URL: http://www.coda.cs.cmu.edu/doc/html/kernel-venus-protocol.html
:Date: 1998
:Keywords: coda, filesystem, venus, cache manager.
:Description: "This document describes the communication between
Venus and kernel level file system code needed for the operation
of the Coda filesystem. This version document is meant to describe
the current interface (version 1.0) as well as improvements we
envisage".
* Title: **Design and Implementation of the Second Extended Filesystem**
:Author: Rémy Card, Theodore Ts'o, Stephen Tweedie.
:URL: http://web.mit.edu/tytso/www/linux/ext2intro.html
:Date: 1998
:Keywords: ext2, linux fs history, inode, directory, link, devices,
VFS, physical structure, performance, benchmarks, ext2fs library,
ext2fs tools, e2fsck.
:Description: Paper written by three of the top ext2 hackers.
Covers Linux filesystems history, ext2 motivation, ext2 features,
design, physical structure on disk, performance, benchmarks,
e2fsck's passes description... A must read!
:Notes: This paper was first published in the Proceedings of the
First Dutch International Symposium on Linux, ISBN 90-367-0385-9.
* Title: **The Linux RAID-1, 4, 5 Code**
:Author: Ingo Molnar, Gadi Oxman and Miguel de Icaza.
:URL: http://www.linuxjournal.com/article.php?sid=2391
:Date: 1997
:Keywords: RAID, MD driver.
:Description: Linux Journal Kernel Korner article. Here is its
:Abstract: *A description of the implementation of the RAID-1,
RAID-4 and RAID-5 personalities of the MD device driver in the
Linux kernel, providing users with high performance and reliable,
secondary-storage capability using software*.
* Title: **Linux Kernel Hackers' Guide**
:Author: Michael K. Johnson.
:URL: http://www.tldp.org/LDP/khg/HyperNews/get/khg.html
:Date: 1997
:Keywords: device drivers, files, VFS, kernel interface, character vs
block devices, hardware interrupts, scsi, DMA, access to user memory,
memory allocation, timers.
:Description: A guide designed to help you get up to speed on the
concepts that are not intuitevly obvious, and to document the internal
structures of Linux.
* Title: **Dynamic Kernels: Modularized Device Drivers**
:Author: Alessandro Rubini.
:URL: http://www.linuxjournal.com/article.php?sid=1219
:Date: 1996
:Keywords: device driver, module, loading/unloading modules,
allocating resources.
:Description: Linux Journal Kernel Korner article. Here is its
:Abstract: *This is the first of a series of four articles
co-authored by Alessandro Rubini and Georg Zezchwitz which present
a practical approach to writing Linux device drivers as kernel
loadable modules. This installment presents an introduction to the
topic, preparing the reader to understand next month's
installment*.
* Title: **Dynamic Kernels: Discovery**
:Author: Alessandro Rubini.
:URL: http://www.linuxjournal.com/article.php?sid=1220
:Date: 1996
:Keywords: character driver, init_module, clean_up module,
autodetection, mayor number, minor number, file operations,
open(), close().
:Description: Linux Journal Kernel Korner article. Here is its
:Abstract: *This article, the second of four, introduces part of
the actual code to create custom module implementing a character
device driver. It describes the code for module initialization and
cleanup, as well as the open() and close() system calls*.
* Title: **The Devil's in the Details**
:Author: Georg v. Zezschwitz and Alessandro Rubini.
:URL: http://www.linuxjournal.com/article.php?sid=1221
:Date: 1996
:Keywords: read(), write(), select(), ioctl(), blocking/non
blocking mode, interrupt handler.
:Description: Linux Journal Kernel Korner article. Here is its
:Abstract: *This article, the third of four on writing character
device drivers, introduces concepts of reading, writing, and using
ioctl-calls*.
* Title: **Dissecting Interrupts and Browsing DMA**
:Author: Alessandro Rubini and Georg v. Zezschwitz.
:URL: http://www.linuxjournal.com/article.php?sid=1222
:Date: 1996
:Keywords: interrupts, irqs, DMA, bottom halves, task queues.
:Description: Linux Journal Kernel Korner article. Here is its
:Abstract: *This is the fourth in a series of articles about
writing character device drivers as loadable kernel modules. This
month, we further investigate the field of interrupt handling.
Though it is conceptually simple, practical limitations and
constraints make this an ''interesting'' part of device driver
writing, and several different facilities have been provided for
different situations. We also investigate the complex topic of
DMA*.
* Title: **Device Drivers Concluded**
:Author: Georg v. Zezschwitz.
:URL: http://www.linuxjournal.com/article.php?sid=1287
:Date: 1996
:Keywords: address spaces, pages, pagination, page management,
demand loading, swapping, memory protection, memory mapping, mmap,
virtual memory areas (VMAs), vremap, PCI.
:Description: Finally, the above turned out into a five articles
series. This latest one's introduction reads: "This is the last of
five articles about character device drivers. In this final
section, Georg deals with memory mapping devices, beginning with
an overall description of the Linux memory management concepts".
* Title: **Network Buffers And Memory Management**
:Author: Alan Cox.
:URL: http://www.linuxjournal.com/article.php?sid=1312
:Date: 1996
:Keywords: sk_buffs, network devices, protocol/link layer
variables, network devices flags, transmit, receive,
configuration, multicast.
:Description: Linux Journal Kernel Korner.
:Abstract: *Writing a network device driver for Linux is fundamentally
simple---most of the complexity (other than talking to the
hardware) involves managing network packets in memory*.
* Title: **Analysis of the Ext2fs structure**
:Author: Louis-Dominique Dubeau.
:URL: http://teaching.csse.uwa.edu.au/units/CITS2002/fs-ext2/
:Date: 1994
:Keywords: ext2, filesystem, ext2fs.
:Description: Description of ext2's blocks, directories, inodes,
bitmaps, invariants...
Published books
---------------
* Title: **Linux Treiber entwickeln**
:Author: Jürgen Quade, Eva-Katharina Kunst
:Publisher: dpunkt.verlag
:Date: Oct 2015 (4th edition)
:Pages: 688
:ISBN: 978-3-86490-288-8
:Note: German. The third edition from 2011 is
much cheaper and still quite up-to-date.
* Title: **Linux Kernel Networking: Implementation and Theory**
:Author: Rami Rosen
:Publisher: Apress
:Date: December 22, 2013
:Pages: 648
:ISBN: 978-1430261964
* Title: **Embedded Linux Primer: A practical Real-World Approach, 2nd Edition**
:Author: Christopher Hallinan
:Publisher: Pearson
:Date: November, 2010
:Pages: 656
:ISBN: 978-0137017836
* Title: **Linux Kernel Development, 3rd Edition**
:Author: Robert Love
:Publisher: Addison-Wesley
:Date: July, 2010
:Pages: 440
:ISBN: 978-0672329463
* Title: **Essential Linux Device Drivers**
:Author: Sreekrishnan Venkateswaran
:Published: Prentice Hall
:Date: April, 2008
:Pages: 744
:ISBN: 978-0132396554
.. _ldd3_published:
* Title: **Linux Device Drivers, 3rd Edition**
:Authors: Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman
:Publisher: O'Reilly & Associates
:Date: 2005
:Pages: 636
:ISBN: 0-596-00590-3
:Notes: Further information in
http://www.oreilly.com/catalog/linuxdrive3/
PDF format, URL: http://lwn.net/Kernel/LDD3/
* Title: **Linux Kernel Internals**
:Author: Michael Beck
:Publisher: Addison-Wesley
:Date: 1997
:ISBN: 0-201-33143-8 (second edition)
* Title: **Programmation Linux 2.0 API systeme et fonctionnement du noyau**
:Author: Remy Card, Eric Dumas, Franck Mevel
:Publisher: Eyrolles
:Date: 1997
:Pages: 520
:ISBN: 2-212-08932-5
:Notes: French
* Title: **The Design and Implementation of the 4.4 BSD UNIX Operating System**
:Author: Marshall Kirk McKusick, Keith Bostic, Michael J. Karels,
John S. Quarterman
:Publisher: Addison-Wesley
:Date: 1996
:ISBN: 0-201-54979-4
* Title: **Unix internals -- the new frontiers**
:Author: Uresh Vahalia
:Publisher: Prentice Hall
:Date: 1996
:Pages: 600
:ISBN: 0-13-101908-2
* Title: **Programming for the real world - POSIX.4**
:Author: Bill O. Gallmeister
:Publisher: O'Reilly & Associates, Inc
:Date: 1995
:Pages: 552
:ISBN: I-56592-074-0
:Notes: Though not being directly about Linux, Linux aims to be
POSIX. Good reference.
* Title: **UNIX Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers**
:Author: Curt Schimmel
:Publisher: Addison Wesley
:Date: June, 1994
:Pages: 432
:ISBN: 0-201-63338-8
* Title: **The Design and Implementation of the 4.3 BSD UNIX Operating System**
:Author: Samuel J. Leffler, Marshall Kirk McKusick, Michael J
Karels, John S. Quarterman
:Publisher: Addison-Wesley
:Date: 1989 (reprinted with corrections on October, 1990)
:ISBN: 0-201-06196-1
* Title: **The Design of the UNIX Operating System**
:Author: Maurice J. Bach
:Publisher: Prentice Hall
:Date: 1986
:Pages: 471
:ISBN: 0-13-201757-1
Miscellaneous
-------------
* Name: **Cross-Referencing Linux**
:URL: http://lxr.free-electrons.com/
:Keywords: Browsing source code.
:Description: Another web-based Linux kernel source code browser.
Lots of cross references to variables and functions. You can see
where they are defined and where they are used.
* Name: **Linux Weekly News**
:URL: http://lwn.net
:Keywords: latest kernel news.
:Description: The title says it all. There's a fixed kernel section
summarizing developers' work, bug fixes, new features and versions
produced during the week. Published every Thursday.
* Name: **The home page of Linux-MM**
:Author: The Linux-MM team.
:URL: http://linux-mm.org/
:Keywords: memory management, Linux-MM, mm patches, TODO, docs,
mailing list.
:Description: Site devoted to Linux Memory Management development.
Memory related patches, HOWTOs, links, mm developers... Don't miss
it if you are interested in memory management development!
* Name: **Kernel Newbies IRC Channel and Website**
:URL: http://www.kernelnewbies.org
:Keywords: IRC, newbies, channel, asking doubts.
:Description: #kernelnewbies on irc.oftc.net.
#kernelnewbies is an IRC network dedicated to the 'newbie'
kernel hacker. The audience mostly consists of people who are
learning about the kernel, working on kernel projects or
professional kernel hackers that want to help less seasoned kernel
people.
#kernelnewbies is on the OFTC IRC Network.
Try irc.oftc.net as your server and then /join #kernelnewbies.
The kernelnewbies website also hosts articles, documents, FAQs...
* Name: **linux-kernel mailing list archives and search engines**
:URL: http://vger.kernel.org/vger-lists.html
:URL: http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html
:URL: http://groups.google.com/group/mlist.linux.kernel
:Keywords: linux-kernel, archives, search.
:Description: Some of the linux-kernel mailing list archivers. If
you have a better/another one, please let me know.
-------
Document last updated on Tue 2016-Sep-20
This document is based on:
http://www.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html
Documentation/process/kernel-driver-statement.rst
+200
View File
@@ -0,0 +1,200 @@
Kernel Driver Statement
-----------------------
Position Statement on Linux Kernel Modules
==========================================
We, the undersigned Linux kernel developers, consider any closed-source
Linux kernel module or driver to be harmful and undesirable. We have
repeatedly found them to be detrimental to Linux users, businesses, and
the greater Linux ecosystem. Such modules negate the openness,
stability, flexibility, and maintainability of the Linux development
model and shut their users off from the expertise of the Linux
community. Vendors that provide closed-source kernel modules force their
customers to give up key Linux advantages or choose new vendors.
Therefore, in order to take full advantage of the cost savings and
shared support benefits open source has to offer, we urge vendors to
adopt a policy of supporting their customers on Linux with open-source
kernel code.
We speak only for ourselves, and not for any company we might work for
today, have in the past, or will in the future.
- Dave Airlie
- Nick Andrew
- Jens Axboe
- Ralf Baechle
- Felipe Balbi
- Ohad Ben-Cohen
- Muli Ben-Yehuda
- Jiri Benc
- Arnd Bergmann
- Thomas Bogendoerfer
- Vitaly Bordug
- James Bottomley
- Josh Boyer
- Neil Brown
- Mark Brown
- David Brownell
- Michael Buesch
- Franck Bui-Huu
- Adrian Bunk
- François Cami
- Ralph Campbell
- Luiz Fernando N. Capitulino
- Mauro Carvalho Chehab
- Denis Cheng
- Jonathan Corbet
- Glauber Costa
- Alan Cox
- Magnus Damm
- Ahmed S. Darwish
- Robert P. J. Day
- Hans de Goede
- Arnaldo Carvalho de Melo
- Helge Deller
- Jean Delvare
- Mathieu Desnoyers
- Sven-Thorsten Dietrich
- Alexey Dobriyan
- Daniel Drake
- Alex Dubov
- Randy Dunlap
- Michael Ellerman
- Pekka Enberg
- Jan Engelhardt
- Mark Fasheh
- J. Bruce Fields
- Larry Finger
- Jeremy Fitzhardinge
- Mike Frysinger
- Kumar Gala
- Robin Getz
- Liam Girdwood
- Jan-Benedict Glaw
- Thomas Gleixner
- Brice Goglin
- Cyrill Gorcunov
- Andy Gospodarek
- Thomas Graf
- Krzysztof Halasa
- Harvey Harrison
- Stephen Hemminger
- Michael Hennerich
- Tejun Heo
- Benjamin Herrenschmidt
- Kristian Høgsberg
- Henrique de Moraes Holschuh
- Marcel Holtmann
- Mike Isely
- Takashi Iwai
- Olof Johansson
- Dave Jones
- Jesper Juhl
- Matthias Kaehlcke
- Kenji Kaneshige
- Jan Kara
- Jeremy Kerr
- Russell King
- Olaf Kirch
- Roel Kluin
- Hans-Jürgen Koch
- Auke Kok
- Peter Korsgaard
- Jiri Kosina
- Aaro Koskinen
- Mariusz Kozlowski
- Greg Kroah-Hartman
- Michael Krufky
- Aneesh Kumar
- Clemens Ladisch
- Christoph Lameter
- Gunnar Larisch
- Anders Larsen
- Grant Likely
- John W. Linville
- Yinghai Lu
- Tony Luck
- Pavel Machek
- Matt Mackall
- Paul Mackerras
- Roland McGrath
- Patrick McHardy
- Kyle McMartin
- Paul Menage
- Thierry Merle
- Eric Miao
- Akinobu Mita
- Ingo Molnar
- James Morris
- Andrew Morton
- Paul Mundt
- Oleg Nesterov
- Luca Olivetti
- S.Çağlar Onur
- Pierre Ossman
- Keith Owens
- Venkatesh Pallipadi
- Nick Piggin
- Nicolas Pitre
- Evgeniy Polyakov
- Richard Purdie
- Mike Rapoport
- Sam Ravnborg
- Gerrit Renker
- Stefan Richter
- David Rientjes
- Luis R. Rodriguez
- Stefan Roese
- Francois Romieu
- Rami Rosen
- Stephen Rothwell
- Maciej W. Rozycki
- Mark Salyzyn
- Yoshinori Sato
- Deepak Saxena
- Holger Schurig
- Amit Shah
- Yoshihiro Shimoda
- Sergei Shtylyov
- Kay Sievers
- Sebastian Siewior
- Rik Snel
- Jes Sorensen
- Alexey Starikovskiy
- Alan Stern
- Timur Tabi
- Hirokazu Takata
- Eliezer Tamir
- Eugene Teo
- Doug Thompson
- FUJITA Tomonori
- Dmitry Torokhov
- Marcelo Tosatti
- Steven Toth
- Theodore Tso
- Matthias Urlichs
- Geert Uytterhoeven
- Arjan van de Ven
- Ivo van Doorn
- Rik van Riel
- Wim Van Sebroeck
- Hans Verkuil
- Horst H. von Brand
- Dmitri Vorobiev
- Anton Vorontsov
- Daniel Walker
- Johannes Weiner
- Harald Welte
- Matthew Wilcox
- Dan J. Williams
- Darrick J. Wong
- David Woodhouse
- Chris Wright
- Bryan Wu
- Rafael J. Wysocki
- Herbert Xu
- Vlad Yasevich
- Peter Zijlstra
- Bartlomiej Zolnierkiewicz
Documentation/process/kernel-enforcement-statement.rst
+161
View File
@@ -0,0 +1,161 @@
Linux Kernel Enforcement Statement
----------------------------------
As developers of the Linux kernel, we have a keen interest in how our software
is used and how the license for our software is enforced. Compliance with the
reciprocal sharing obligations of GPL-2.0 is critical to the long-term
sustainability of our software and community.
Although there is a right to enforce the separate copyright interests in the
contributions made to our community, we share an interest in ensuring that
individual enforcement actions are conducted in a manner that benefits our
community and do not have an unintended negative impact on the health and
growth of our software ecosystem. In order to deter unhelpful enforcement
actions, we agree that it is in the best interests of our development
community to undertake the following commitment to users of the Linux kernel
on behalf of ourselves and any successors to our copyright interests:
Notwithstanding the termination provisions of the GPL-2.0, we agree that
it is in the best interests of our development community to adopt the
following provisions of GPL-3.0 as additional permissions under our
license with respect to any non-defensive assertion of rights under the
license.
However, if you cease all violation of this License, then your license
from a particular copyright holder is reinstated (a) provisionally,
unless and until the copyright holder explicitly and finally
terminates your license, and (b) permanently, if the copyright holder
fails to notify you of the violation by some reasonable means prior to
60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Our intent in providing these assurances is to encourage more use of the
software. We want companies and individuals to use, modify and distribute
this software. We want to work with users in an open and transparent way to
eliminate any uncertainty about our expectations regarding compliance or
enforcement that might limit adoption of our software. We view legal action
as a last resort, to be initiated only when other community efforts have
failed to resolve the problem.
Finally, once a non-compliance issue is resolved, we hope the user will feel
welcome to join us in our efforts on this project. Working together, we will
be stronger.
Except where noted below, we speak only for ourselves, and not for any company
we might work for today, have in the past, or will in the future.
- Laura Abbott
- Bjorn Andersson (Linaro)
- Andrea Arcangeli
- Neil Armstrong
- Jens Axboe
- Pablo Neira Ayuso
- Khalid Aziz
- Ralf Baechle
- Felipe Balbi
- Arnd Bergmann
- Ard Biesheuvel
- Tim Bird
- Paolo Bonzini
- Christian Borntraeger
- Mark Brown (Linaro)
- Paul Burton
- Javier Martinez Canillas
- Rob Clark
- Kees Cook (Google)
- Jonathan Corbet
- Dennis Dalessandro
- Vivien Didelot (Savoir-faire Linux)
- Hans de Goede
- Mel Gorman (SUSE)
- Sven Eckelmann
- Alex Elder (Linaro)
- Fabio Estevam
- Larry Finger
- Bhumika Goyal
- Andy Gross
- Juergen Gross
- Shawn Guo
- Ulf Hansson
- Stephen Hemminger (Microsoft)
- Tejun Heo
- Rob Herring
- Masami Hiramatsu
- Michal Hocko
- Simon Horman
- Johan Hovold (Hovold Consulting AB)
- Christophe JAILLET
- Olof Johansson
- Lee Jones (Linaro)
- Heiner Kallweit
- Srinivas Kandagatla
- Jan Kara
- Shuah Khan (Samsung)
- David Kershner
- Jaegeuk Kim
- Namhyung Kim
- Colin Ian King
- Jeff Kirsher
- Greg Kroah-Hartman (Linux Foundation)
- Christian König
- Vinod Koul
- Krzysztof Kozlowski
- Viresh Kumar
- Aneesh Kumar K.V
- Julia Lawall
- Doug Ledford
- Chuck Lever (Oracle)
- Daniel Lezcano
- Shaohua Li
- Xin Long
- Tony Luck
- Catalin Marinas (Arm Ltd)
- Mike Marshall
- Chris Mason
- Paul E. McKenney
- Arnaldo Carvalho de Melo
- David S. Miller
- Ingo Molnar
- Kuninori Morimoto
- Trond Myklebust
- Martin K. Petersen (Oracle)
- Borislav Petkov
- Jiri Pirko
- Josh Poimboeuf
- Sebastian Reichel (Collabora)
- Guenter Roeck
- Joerg Roedel
- Leon Romanovsky
- Steven Rostedt (VMware)
- Frank Rowand
- Ivan Safonov
- Anna Schumaker
- Jes Sorensen
- K.Y. Srinivasan
- David Sterba (SUSE)
- Heiko Stuebner
- Jiri Kosina (SUSE)
- Willy Tarreau
- Dmitry Torokhov
- Linus Torvalds
- Thierry Reding
- Rik van Riel
- Luis R. Rodriguez
- Geert Uytterhoeven (Glider bvba)
- Eduardo Valentin (Amazon.com)
- Daniel Vetter
- Linus Walleij
- Richard Weinberger
- Dan Williams
- Rafael J. Wysocki
- Arvind Yadav
- Masahiro Yamada
- Wei Yongjun
- Lv Zheng
- Marc Zyngier (Arm Ltd)
Documentation/process/license-rules.rst
+372
View File
@@ -0,0 +1,372 @@
.. SPDX-License-Identifier: GPL-2.0
Linux kernel licensing rules
============================
The Linux Kernel is provided under the terms of the GNU General Public
License version 2 only (GPL-2.0), as provided in LICENSES/preferred/GPL-2.0,
with an explicit syscall exception described in
LICENSES/exceptions/Linux-syscall-note, as described in the COPYING file.
This documentation file provides a description of how each source file
should be annotated to make its license clear and unambiguous.
It doesn't replace the Kernel's license.
The license described in the COPYING file applies to the kernel source
as a whole, though individual source files can have a different license
which is required to be compatible with the GPL-2.0::
GPL-1.0+ : GNU General Public License v1.0 or later
GPL-2.0+ : GNU General Public License v2.0 or later
LGPL-2.0 : GNU Library General Public License v2 only
LGPL-2.0+ : GNU Library General Public License v2 or later
LGPL-2.1 : GNU Lesser General Public License v2.1 only
LGPL-2.1+ : GNU Lesser General Public License v2.1 or later
Aside from that, individual files can be provided under a dual license,
e.g. one of the compatible GPL variants and alternatively under a
permissive license like BSD, MIT etc.
The User-space API (UAPI) header files, which describe the interface of
user-space programs to the kernel are a special case. According to the
note in the kernel COPYING file, the syscall interface is a clear boundary,
which does not extend the GPL requirements to any software which uses it to
communicate with the kernel. Because the UAPI headers must be includable
into any source files which create an executable running on the Linux
kernel, the exception must be documented by a special license expression.
The common way of expressing the license of a source file is to add the
matching boilerplate text into the top comment of the file. Due to
formatting, typos etc. these "boilerplates" are hard to validate for
tools which are used in the context of license compliance.
An alternative to boilerplate text is the use of Software Package Data
Exchange (SPDX) license identifiers in each source file. SPDX license
identifiers are machine parsable and precise shorthands for the license
under which the content of the file is contributed. SPDX license
identifiers are managed by the SPDX Workgroup at the Linux Foundation and
have been agreed on by partners throughout the industry, tool vendors, and
legal teams. For further information see https://spdx.org/
The Linux kernel requires the precise SPDX identifier in all source files.
The valid identifiers used in the kernel are explained in the section
`License identifiers`_ and have been retrieved from the official SPDX
license list at https://spdx.org/licenses/ along with the license texts.
License identifier syntax
-------------------------
1. Placement:
The SPDX license identifier in kernel files shall be added at the first
possible line in a file which can contain a comment. For the majority
or files this is the first line, except for scripts which require the
'#!PATH_TO_INTERPRETER' in the first line. For those scripts the SPDX
identifier goes into the second line.
|
2. Style:
The SPDX license identifier is added in form of a comment. The comment
style depends on the file type::
C source: // SPDX-License-Identifier: <SPDX License Expression>
C header: /* SPDX-License-Identifier: <SPDX License Expression> */
ASM: /* SPDX-License-Identifier: <SPDX License Expression> */
scripts: # SPDX-License-Identifier: <SPDX License Expression>
.rst: .. SPDX-License-Identifier: <SPDX License Expression>
.dts{i}: // SPDX-License-Identifier: <SPDX License Expression>
If a specific tool cannot handle the standard comment style, then the
appropriate comment mechanism which the tool accepts shall be used. This
is the reason for having the "/\* \*/" style comment in C header
files. There was build breakage observed with generated .lds files where
'ld' failed to parse the C++ comment. This has been fixed by now, but
there are still older assembler tools which cannot handle C++ style
comments.
|
3. Syntax:
A <SPDX License Expression> is either an SPDX short form license
identifier found on the SPDX License List, or the combination of two
SPDX short form license identifiers separated by "WITH" when a license
exception applies. When multiple licenses apply, an expression consists
of keywords "AND", "OR" separating sub-expressions and surrounded by
"(", ")" .
License identifiers for licenses like [L]GPL with the 'or later' option
are constructed by using a "+" for indicating the 'or later' option.::
// SPDX-License-Identifier: GPL-2.0+
// SPDX-License-Identifier: LGPL-2.1+
WITH should be used when there is a modifier to a license needed.
For example, the linux kernel UAPI files use the expression::
// SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
// SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note
Other examples using WITH exceptions found in the kernel are::
// SPDX-License-Identifier: GPL-2.0 WITH mif-exception
// SPDX-License-Identifier: GPL-2.0+ WITH GCC-exception-2.0
Exceptions can only be used with particular License identifiers. The
valid License identifiers are listed in the tags of the exception text
file. For details see the point `Exceptions`_ in the chapter `License
identifiers`_.
OR should be used if the file is dual licensed and only one license is
to be selected. For example, some dtsi files are available under dual
licenses::
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
Examples from the kernel for license expressions in dual licensed files::
// SPDX-License-Identifier: GPL-2.0 OR MIT
// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
// SPDX-License-Identifier: GPL-2.0 OR Apache-2.0
// SPDX-License-Identifier: GPL-2.0 OR MPL-1.1
// SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) OR MIT
// SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause OR OpenSSL
AND should be used if the file has multiple licenses whose terms all
apply to use the file. For example, if code is inherited from another
project and permission has been given to put it in the kernel, but the
original license terms need to remain in effect::
// SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) AND MIT
Another other example where both sets of license terms need to be
adhered to is::
// SPDX-License-Identifier: GPL-1.0+ AND LGPL-2.1+
License identifiers
-------------------
The licenses currently used, as well as the licenses for code added to the
kernel, can be broken down into:
1. _`Preferred licenses`:
Whenever possible these licenses should be used as they are known to be
fully compatible and widely used. These licenses are available from the
directory::
LICENSES/preferred/
in the kernel source tree.
The files in this directory contain the full license text and
`Metatags`_. The file names are identical to the SPDX license
identifier which shall be used for the license in source files.
Examples::
LICENSES/preferred/GPL-2.0
Contains the GPL version 2 license text and the required metatags::
LICENSES/preferred/MIT
Contains the MIT license text and the required metatags
_`Metatags`:
The following meta tags must be available in a license file:
- Valid-License-Identifier:
One or more lines which declare which License Identifiers are valid
inside the project to reference this particular license text. Usually
this is a single valid identifier, but e.g. for licenses with the 'or
later' options two identifiers are valid.
- SPDX-URL:
The URL of the SPDX page which contains additional information related
to the license.
- Usage-Guidance:
Freeform text for usage advice. The text must include correct examples
for the SPDX license identifiers as they should be put into source
files according to the `License identifier syntax`_ guidelines.
- License-Text:
All text after this tag is treated as the original license text
File format examples::
Valid-License-Identifier: GPL-2.0
Valid-License-Identifier: GPL-2.0+
SPDX-URL: https://spdx.org/licenses/GPL-2.0.html
Usage-Guide:
To use this license in source code, put one of the following SPDX
tag/value pairs into a comment according to the placement
guidelines in the licensing rules documentation.
For 'GNU General Public License (GPL) version 2 only' use:
SPDX-License-Identifier: GPL-2.0
For 'GNU General Public License (GPL) version 2 or any later version' use:
SPDX-License-Identifier: GPL-2.0+
License-Text:
Full license text
::
SPDX-License-Identifier: MIT
SPDX-URL: https://spdx.org/licenses/MIT.html
Usage-Guide:
To use this license in source code, put the following SPDX
tag/value pair into a comment according to the placement
guidelines in the licensing rules documentation.
SPDX-License-Identifier: MIT
License-Text:
Full license text
|
2. Not recommended licenses:
These licenses should only be used for existing code or for importing
code from a different project. These licenses are available from the
directory::
LICENSES/other/
in the kernel source tree.
The files in this directory contain the full license text and
`Metatags`_. The file names are identical to the SPDX license
identifier which shall be used for the license in source files.
Examples::
LICENSES/other/ISC
Contains the Internet Systems Consortium license text and the required
metatags::
LICENSES/other/ZLib
Contains the ZLIB license text and the required metatags.
Metatags:
The metatag requirements for 'other' licenses are identical to the
requirements of the `Preferred licenses`_.
File format example::
Valid-License-Identifier: ISC
SPDX-URL: https://spdx.org/licenses/ISC.html
Usage-Guide:
Usage of this license in the kernel for new code is discouraged
and it should solely be used for importing code from an already
existing project.
To use this license in source code, put the following SPDX
tag/value pair into a comment according to the placement
guidelines in the licensing rules documentation.
SPDX-License-Identifier: ISC
License-Text:
Full license text
|
3. _`Exceptions`:
Some licenses can be amended with exceptions which grant certain rights
which the original license does not. These exceptions are available
from the directory::
LICENSES/exceptions/
in the kernel source tree. The files in this directory contain the full
exception text and the required `Exception Metatags`_.
Examples::
LICENSES/exceptions/Linux-syscall-note
Contains the Linux syscall exception as documented in the COPYING
file of the Linux kernel, which is used for UAPI header files.
e.g. /\* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note \*/::
LICENSES/exceptions/GCC-exception-2.0
Contains the GCC 'linking exception' which allows to link any binary
independent of its license against the compiled version of a file marked
with this exception. This is required for creating runnable executables
from source code which is not compatible with the GPL.
_`Exception Metatags`:
The following meta tags must be available in an exception file:
- SPDX-Exception-Identifier:
One exception identifier which can be used with SPDX license
identifiers.
- SPDX-URL:
The URL of the SPDX page which contains additional information related
to the exception.
- SPDX-Licenses:
A comma separated list of SPDX license identifiers for which the
exception can be used.
- Usage-Guidance:
Freeform text for usage advice. The text must be followed by correct
examples for the SPDX license identifiers as they should be put into
source files according to the `License identifier syntax`_ guidelines.
- Exception-Text:
All text after this tag is treated as the original exception text
File format examples::
SPDX-Exception-Identifier: Linux-syscall-note
SPDX-URL: https://spdx.org/licenses/Linux-syscall-note.html
SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-1.0+, LGPL-2.0, LGPL-2.0+, LGPL-2.1, LGPL-2.1+
Usage-Guidance:
This exception is used together with one of the above SPDX-Licenses
to mark user-space API (uapi) header files so they can be included
into non GPL compliant user-space application code.
To use this exception add it with the keyword WITH to one of the
identifiers in the SPDX-Licenses tag:
SPDX-License-Identifier: <SPDX-License> WITH Linux-syscall-note
Exception-Text:
Full exception text
::
SPDX-Exception-Identifier: GCC-exception-2.0
SPDX-URL: https://spdx.org/licenses/GCC-exception-2.0.html
SPDX-Licenses: GPL-2.0, GPL-2.0+
Usage-Guidance:
The "GCC Runtime Library exception 2.0" is used together with one
of the above SPDX-Licenses for code imported from the GCC runtime
library.
To use this exception add it with the keyword WITH to one of the
identifiers in the SPDX-Licenses tag:
SPDX-License-Identifier: <SPDX-License> WITH GCC-exception-2.0
Exception-Text:
Full exception text
All SPDX license identifiers and exceptions must have a corresponding file
in the LICENSE subdirectories. This is required to allow tool
verification (e.g. checkpatch.pl) and to have the licenses ready to read
and extract right from the source, which is recommended by various FOSS
organizations, e.g. the `FSFE REUSE initiative <https://reuse.software/>`_.
Documentation/process/magic-number.rst
+161
View File
@@ -0,0 +1,161 @@
Linux magic numbers
===================
This file is a registry of magic numbers which are in use. When you
add a magic number to a structure, you should also add it to this
file, since it is best if the magic numbers used by various structures
are unique.
It is a **very** good idea to protect kernel data structures with magic
numbers. This allows you to check at run time whether (a) a structure
has been clobbered, or (b) you've passed the wrong structure to a
routine. This last is especially useful --- particularly when you are
passing pointers to structures via a void * pointer. The tty code,
for example, does this frequently to pass driver-specific and line
discipline-specific structures back and forth.
The way to use magic numbers is to declare them at the beginning of
the structure, like so::
struct tty_ldisc {
int magic;
...
};
Please follow this discipline when you are adding future enhancements
to the kernel! It has saved me countless hours of debugging,
especially in the screwy cases where an array has been overrun and
structures following the array have been overwritten. Using this
discipline, these cases get detected quickly and safely.
Changelog::
Theodore Ts'o
31 Mar 94
The magic table is current to Linux 2.1.55.
Michael Chastain
<mailto:mec@shout.net>
22 Sep 1997
Now it should be up to date with Linux 2.1.112. Because
we are in feature freeze time it is very unlikely that
something will change before 2.2.x. The entries are
sorted by number field.
Krzysztof G. Baranowski
<mailto: kgb@knm.org.pl>
29 Jul 1998
Updated the magic table to Linux 2.5.45. Right over the feature freeze,
but it is possible that some new magic numbers will sneak into the
kernel before 2.6.x yet.
Petr Baudis
<pasky@ucw.cz>
03 Nov 2002
Updated the magic table to Linux 2.5.74.
Fabian Frederick
<ffrederick@users.sourceforge.net>
09 Jul 2003
===================== ================ ======================== ==========================================
Magic Name Number Structure File
===================== ================ ======================== ==========================================
PG_MAGIC 'P' pg_{read,write}_hdr ``include/linux/pg.h``
CMAGIC 0x0111 user ``include/linux/a.out.h``
MKISS_DRIVER_MAGIC 0x04bf mkiss_channel ``drivers/net/mkiss.h``
HDLC_MAGIC 0x239e n_hdlc ``drivers/char/n_hdlc.c``
APM_BIOS_MAGIC 0x4101 apm_user ``arch/x86/kernel/apm_32.c``
CYCLADES_MAGIC 0x4359 cyclades_port ``include/linux/cyclades.h``
DB_MAGIC 0x4442 fc_info ``drivers/net/iph5526_novram.c``
DL_MAGIC 0x444d fc_info ``drivers/net/iph5526_novram.c``
FASYNC_MAGIC 0x4601 fasync_struct ``include/linux/fs.h``
FF_MAGIC 0x4646 fc_info ``drivers/net/iph5526_novram.c``
ISICOM_MAGIC 0x4d54 isi_port ``include/linux/isicom.h``
PTY_MAGIC 0x5001 ``drivers/char/pty.c``
PPP_MAGIC 0x5002 ppp ``include/linux/if_pppvar.h``
SERIAL_MAGIC 0x5301 async_struct ``include/linux/serial.h``
SSTATE_MAGIC 0x5302 serial_state ``include/linux/serial.h``
SLIP_MAGIC 0x5302 slip ``drivers/net/slip.h``
STRIP_MAGIC 0x5303 strip ``drivers/net/strip.c``
X25_ASY_MAGIC 0x5303 x25_asy ``drivers/net/x25_asy.h``
SIXPACK_MAGIC 0x5304 sixpack ``drivers/net/hamradio/6pack.h``
AX25_MAGIC 0x5316 ax_disp ``drivers/net/mkiss.h``
TTY_MAGIC 0x5401 tty_struct ``include/linux/tty.h``
MGSL_MAGIC 0x5401 mgsl_info ``drivers/char/synclink.c``
TTY_DRIVER_MAGIC 0x5402 tty_driver ``include/linux/tty_driver.h``
MGSLPC_MAGIC 0x5402 mgslpc_info ``drivers/char/pcmcia/synclink_cs.c``
TTY_LDISC_MAGIC 0x5403 tty_ldisc ``include/linux/tty_ldisc.h``
USB_SERIAL_MAGIC 0x6702 usb_serial ``drivers/usb/serial/usb-serial.h``
FULL_DUPLEX_MAGIC 0x6969 ``drivers/net/ethernet/dec/tulip/de2104x.c``
USB_BLUETOOTH_MAGIC 0x6d02 usb_bluetooth ``drivers/usb/class/bluetty.c``
RFCOMM_TTY_MAGIC 0x6d02 ``net/bluetooth/rfcomm/tty.c``
USB_SERIAL_PORT_MAGIC 0x7301 usb_serial_port ``drivers/usb/serial/usb-serial.h``
CG_MAGIC 0x00090255 ufs_cylinder_group ``include/linux/ufs_fs.h``
RPORT_MAGIC 0x00525001 r_port ``drivers/char/rocket_int.h``
LSEMAGIC 0x05091998 lse ``drivers/fc4/fc.c``
GDTIOCTL_MAGIC 0x06030f07 gdth_iowr_str ``drivers/scsi/gdth_ioctl.h``
RIEBL_MAGIC 0x09051990 ``drivers/net/atarilance.c``
NBD_REQUEST_MAGIC 0x12560953 nbd_request ``include/linux/nbd.h``
RED_MAGIC2 0x170fc2a5 (any) ``mm/slab.c``
BAYCOM_MAGIC 0x19730510 baycom_state ``drivers/net/baycom_epp.c``
ISDN_X25IFACE_MAGIC 0x1e75a2b9 isdn_x25iface_proto_data ``drivers/isdn/isdn_x25iface.h``
ECP_MAGIC 0x21504345 cdkecpsig ``include/linux/cdk.h``
LSOMAGIC 0x27091997 lso ``drivers/fc4/fc.c``
LSMAGIC 0x2a3b4d2a ls ``drivers/fc4/fc.c``
WANPIPE_MAGIC 0x414C4453 sdla_{dump,exec} ``include/linux/wanpipe.h``
CS_CARD_MAGIC 0x43525553 cs_card ``sound/oss/cs46xx.c``
LABELCL_MAGIC 0x4857434c labelcl_info_s ``include/asm/ia64/sn/labelcl.h``
ISDN_ASYNC_MAGIC 0x49344C01 modem_info ``include/linux/isdn.h``
CTC_ASYNC_MAGIC 0x49344C01 ctc_tty_info ``drivers/s390/net/ctctty.c``
ISDN_NET_MAGIC 0x49344C02 isdn_net_local_s ``drivers/isdn/i4l/isdn_net_lib.h``
SAVEKMSG_MAGIC2 0x4B4D5347 savekmsg ``arch/*/amiga/config.c``
CS_STATE_MAGIC 0x4c4f4749 cs_state ``sound/oss/cs46xx.c``
SLAB_C_MAGIC 0x4f17a36d kmem_cache ``mm/slab.c``
COW_MAGIC 0x4f4f4f4d cow_header_v1 ``arch/um/drivers/ubd_user.c``
I810_CARD_MAGIC 0x5072696E i810_card ``sound/oss/i810_audio.c``
TRIDENT_CARD_MAGIC 0x5072696E trident_card ``sound/oss/trident.c``
ROUTER_MAGIC 0x524d4157 wan_device [in ``wanrouter.h`` pre 3.9]
SAVEKMSG_MAGIC1 0x53415645 savekmsg ``arch/*/amiga/config.c``
GDA_MAGIC 0x58464552 gda ``arch/mips/include/asm/sn/gda.h``
RED_MAGIC1 0x5a2cf071 (any) ``mm/slab.c``
EEPROM_MAGIC_VALUE 0x5ab478d2 lanai_dev ``drivers/atm/lanai.c``
HDLCDRV_MAGIC 0x5ac6e778 hdlcdrv_state ``include/linux/hdlcdrv.h``
PCXX_MAGIC 0x5c6df104 channel ``drivers/char/pcxx.h``
KV_MAGIC 0x5f4b565f kernel_vars_s ``arch/mips/include/asm/sn/klkernvars.h``
I810_STATE_MAGIC 0x63657373 i810_state ``sound/oss/i810_audio.c``
TRIDENT_STATE_MAGIC 0x63657373 trient_state ``sound/oss/trident.c``
M3_CARD_MAGIC 0x646e6f50 m3_card ``sound/oss/maestro3.c``
FW_HEADER_MAGIC 0x65726F66 fw_header ``drivers/atm/fore200e.h``
SLOT_MAGIC 0x67267321 slot ``drivers/hotplug/cpqphp.h``
SLOT_MAGIC 0x67267322 slot ``drivers/hotplug/acpiphp.h``
LO_MAGIC 0x68797548 nbd_device ``include/linux/nbd.h``
OPROFILE_MAGIC 0x6f70726f super_block ``drivers/oprofile/oprofilefs.h``
M3_STATE_MAGIC 0x734d724d m3_state ``sound/oss/maestro3.c``
VMALLOC_MAGIC 0x87654320 snd_alloc_track ``sound/core/memory.c``
KMALLOC_MAGIC 0x87654321 snd_alloc_track ``sound/core/memory.c``
PWC_MAGIC 0x89DC10AB pwc_device ``drivers/usb/media/pwc.h``
NBD_REPLY_MAGIC 0x96744668 nbd_reply ``include/linux/nbd.h``
ENI155_MAGIC 0xa54b872d midway_eprom ``drivers/atm/eni.h``
CODA_MAGIC 0xC0DAC0DA coda_file_info ``fs/coda/coda_fs_i.h``
DPMEM_MAGIC 0xc0ffee11 gdt_pci_sram ``drivers/scsi/gdth.h``
YAM_MAGIC 0xF10A7654 yam_port ``drivers/net/hamradio/yam.c``
CCB_MAGIC 0xf2691ad2 ccb ``drivers/scsi/ncr53c8xx.c``
QUEUE_MAGIC_FREE 0xf7e1c9a3 queue_entry ``drivers/scsi/arm/queue.c``
QUEUE_MAGIC_USED 0xf7e1cc33 queue_entry ``drivers/scsi/arm/queue.c``
HTB_CMAGIC 0xFEFAFEF1 htb_class ``net/sched/sch_htb.c``
NMI_MAGIC 0x48414d4d455201 nmi_s ``arch/mips/include/asm/sn/nmi.h``
===================== ================ ======================== ==========================================
Note that there are also defined special per-driver magic numbers in sound
memory management. See ``include/sound/sndmagic.h`` for complete list of them. Many
OSS sound drivers have their magic numbers constructed from the soundcard PCI
ID - these are not listed here as well.
HFS is another larger user of magic numbers - you can find them in
``fs/hfs/hfs.h``.
Documentation/process/maintainer-pgp-guide.rst
+964
View File
@@ -0,0 +1,964 @@
.. _pgpguide:
===========================
Kernel Maintainer PGP guide
===========================
:Author: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
This document is aimed at Linux kernel developers, and especially at
subsystem maintainers. It contains a subset of information discussed in
the more general "`Protecting Code Integrity`_" guide published by the
Linux Foundation. Please read that document for more in-depth discussion
on some of the topics mentioned in this guide.
.. _`Protecting Code Integrity`: https://github.com/lfit/itpol/blob/master/protecting-code-integrity.md
The role of PGP in Linux Kernel development
===========================================
PGP helps ensure the integrity of the code that is produced by the Linux
kernel development community and, to a lesser degree, establish trusted
communication channels between developers via PGP-signed email exchange.
The Linux kernel source code is available in two main formats:
- Distributed source repositories (git)
- Periodic release snapshots (tarballs)
Both git repositories and tarballs carry PGP signatures of the kernel
developers who create official kernel releases. These signatures offer a
cryptographic guarantee that downloadable versions made available via
kernel.org or any other mirrors are identical to what these developers
have on their workstations. To this end:
- git repositories provide PGP signatures on all tags
- tarballs provide detached PGP signatures with all downloads
.. _devs_not_infra:
Trusting the developers, not infrastructure
-------------------------------------------
Ever since the 2011 compromise of core kernel.org systems, the main
operating principle of the Kernel Archives project has been to assume
that any part of the infrastructure can be compromised at any time. For
this reason, the administrators have taken deliberate steps to emphasize
that trust must always be placed with developers and never with the code
hosting infrastructure, regardless of how good the security practices
for the latter may be.
The above guiding principle is the reason why this guide is needed. We
want to make sure that by placing trust into developers we do not simply
shift the blame for potential future security incidents to someone else.
The goal is to provide a set of guidelines developers can use to create
a secure working environment and safeguard the PGP keys used to
establish the integrity of the Linux kernel itself.
.. _pgp_tools:
PGP tools
=========
Use GnuPG v2
------------
Your distro should already have GnuPG installed by default, you just
need to verify that you are using version 2.x and not the legacy 1.4
release -- many distributions still package both, with the default
``gpg`` command invoking GnuPG v.1. To check, run::
$ gpg --version | head -n1
If you see ``gpg (GnuPG) 1.4.x``, then you are using GnuPG v.1. Try the
``gpg2`` command (if you don't have it, you may need to install the
gnupg2 package)::
$ gpg2 --version | head -n1
If you see ``gpg (GnuPG) 2.x.x``, then you are good to go. This guide
will assume you have the version 2.2 of GnuPG (or later). If you are
using version 2.0 of GnuPG, then some of the commands in this guide will
not work, and you should consider installing the latest 2.2 version of
GnuPG. Versions of gnupg-2.1.11 and later should be compatible for the
purposes of this guide as well.
If you have both ``gpg`` and ``gpg2`` commands, you should make sure you
are always using GnuPG v2, not the legacy version. You can enforce this
by setting the appropriate alias::
$ alias gpg=gpg2
You can put that in your ``.bashrc`` to make sure it's always the case.
Configure gpg-agent options
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The GnuPG agent is a helper tool that will start automatically whenever
you use the ``gpg`` command and run in the background with the purpose
of caching the private key passphrase. There are two options you should
know in order to tweak when the passphrase should be expired from cache:
- ``default-cache-ttl`` (seconds): If you use the same key again before
the time-to-live expires, the countdown will reset for another period.
The default is 600 (10 minutes).
- ``max-cache-ttl`` (seconds): Regardless of how recently you've used
the key since initial passphrase entry, if the maximum time-to-live
countdown expires, you'll have to enter the passphrase again. The
default is 30 minutes.
If you find either of these defaults too short (or too long), you can
edit your ``~/.gnupg/gpg-agent.conf`` file to set your own values::
# set to 30 minutes for regular ttl, and 2 hours for max ttl
default-cache-ttl 1800
max-cache-ttl 7200
.. note::
It is no longer necessary to start gpg-agent manually at the
beginning of your shell session. You may want to check your rc files
to remove anything you had in place for older versions of GnuPG, as
it may not be doing the right thing any more.
Set up a refresh cronjob
~~~~~~~~~~~~~~~~~~~~~~~~
You will need to regularly refresh your keyring in order to get the
latest changes on other people's public keys, which is best done with a
daily cronjob::
@daily /usr/bin/gpg2 --refresh >/dev/null 2>&1
Check the full path to your ``gpg`` or ``gpg2`` command and use the
``gpg2`` command if regular ``gpg`` for you is the legacy GnuPG v.1.
.. _master_key:
Protect your master PGP key
===========================
This guide assumes that you already have a PGP key that you use for Linux
kernel development purposes. If you do not yet have one, please see the
"`Protecting Code Integrity`_" document mentioned earlier for guidance
on how to create a new one.
You should also make a new key if your current one is weaker than 2048 bits
(RSA).
Master key vs. Subkeys
----------------------
Subkeys are fully independent PGP keypairs that are tied to the "master"
key using certifying key signatures (certificates). It is important to
understand the following:
1. There are no technical differences between the "master key" and "subkeys."
2. At creation time, we assign functional limitations to each key by
giving it specific capabilities.
3. A PGP key can have 4 capabilities:
- **[S]** key can be used for signing
- **[E]** key can be used for encryption
- **[A]** key can be used for authentication
- **[C]** key can be used for certifying other keys
4. A single key may have multiple capabilities.
5. A subkey is fully independent from the master key. A message
encrypted to a subkey cannot be decrypted with the master key. If you
lose your private subkey, it cannot be recreated from the master key
in any way.
The key carrying the **[C]** (certify) capability is considered the
"master" key because it is the only key that can be used to indicate
relationship with other keys. Only the **[C]** key can be used to:
- add or revoke other keys (subkeys) with S/E/A capabilities
- add, change or revoke identities (uids) associated with the key
- add or change the expiration date on itself or any subkey
- sign other people's keys for web of trust purposes
By default, GnuPG creates the following when generating new keys:
- A master key carrying both Certify and Sign capabilities (**[SC]**)
- A separate subkey with the Encryption capability (**[E]**)
If you used the default parameters when generating your key, then that
is what you will have. You can verify by running ``gpg --list-secret-keys``,
for example::
sec rsa2048 2018-01-23 [SC] [expires: 2020-01-23]
000000000000000000000000AAAABBBBCCCCDDDD
uid [ultimate] Alice Dev <adev@kernel.org>
ssb rsa2048 2018-01-23 [E] [expires: 2020-01-23]
Any key carrying the **[C]** capability is your master key, regardless
of any other capabilities it may have assigned to it.
The long line under the ``sec`` entry is your key fingerprint --
whenever you see ``[fpr]`` in the examples below, that 40-character
string is what it refers to.
Ensure your passphrase is strong
--------------------------------
GnuPG uses passphrases to encrypt your private keys before storing them on
disk. This way, even if your ``.gnupg`` directory is leaked or stolen in
its entirety, the attackers cannot use your private keys without first
obtaining the passphrase to decrypt them.
It is absolutely essential that your private keys are protected by a
strong passphrase. To set it or change it, use::
$ gpg --change-passphrase [fpr]
Create a separate Signing subkey
--------------------------------
Our goal is to protect your master key by moving it to offline media, so
if you only have a combined **[SC]** key, then you should create a separate
signing subkey::
$ gpg --quick-addkey [fpr] ed25519 sign
Remember to tell the keyservers about this change, so others can pull down
your new subkey::
$ gpg --send-key [fpr]
.. note:: ECC support in GnuPG
GnuPG 2.1 and later has full support for Elliptic Curve
Cryptography, with ability to combine ECC subkeys with traditional
RSA master keys. The main upside of ECC cryptography is that it is
much faster computationally and creates much smaller signatures when
compared byte for byte with 2048+ bit RSA keys. Unless you plan on
using a smartcard device that does not support ECC operations, we
recommend that you create an ECC signing subkey for your kernel
work.
If for some reason you prefer to stay with RSA subkeys, just replace
"ed25519" with "rsa2048" in the above command.
Back up your master key for disaster recovery
---------------------------------------------
The more signatures you have on your PGP key from other developers, the
more reasons you have to create a backup version that lives on something
other than digital media, for disaster recovery reasons.
The best way to create a printable hardcopy of your private key is by
using the ``paperkey`` software written for this very purpose. See ``man
paperkey`` for more details on the output format and its benefits over
other solutions. Paperkey should already be packaged for most
distributions.
Run the following command to create a hardcopy backup of your private
key::
$ gpg --export-secret-key [fpr] | paperkey -o /tmp/key-backup.txt
Print out that file (or pipe the output straight to lpr), then take a
pen and write your passphrase on the margin of the paper. **This is
strongly recommended** because the key printout is still encrypted with
that passphrase, and if you ever change it you will not remember what it
used to be when you had created the backup -- *guaranteed*.
Put the resulting printout and the hand-written passphrase into an envelope
and store in a secure and well-protected place, preferably away from your
home, such as your bank vault.
.. note::
Your printer is probably no longer a simple dumb device connected to
your parallel port, but since the output is still encrypted with
your passphrase, printing out even to "cloud-integrated" modern
printers should remain a relatively safe operation. One option is to
change the passphrase on your master key immediately after you are
done with paperkey.
Back up your whole GnuPG directory
----------------------------------
.. warning::
**!!!Do not skip this step!!!**
It is important to have a readily available backup of your PGP keys
should you need to recover them. This is different from the
disaster-level preparedness we did with ``paperkey``. You will also rely
on these external copies whenever you need to use your Certify key --
such as when making changes to your own key or signing other people's
keys after conferences and summits.
Start by getting a small USB "thumb" drive (preferably two!) that you
will use for backup purposes. You will need to encrypt them using LUKS
-- refer to your distro's documentation on how to accomplish this.
For the encryption passphrase, you can use the same one as on your
master key.
Once the encryption process is over, re-insert the USB drive and make
sure it gets properly mounted. Copy your entire ``.gnupg`` directory
over to the encrypted storage::
$ cp -a ~/.gnupg /media/disk/foo/gnupg-backup
You should now test to make sure everything still works::
$ gpg --homedir=/media/disk/foo/gnupg-backup --list-key [fpr]
If you don't get any errors, then you should be good to go. Unmount the
USB drive, distinctly label it so you don't blow it away next time you
need to use a random USB drive, and put in a safe place -- but not too
far away, because you'll need to use it every now and again for things
like editing identities, adding or revoking subkeys, or signing other
people's keys.
Remove the master key from your homedir
----------------------------------------
The files in our home directory are not as well protected as we like to
think. They can be leaked or stolen via many different means:
- by accident when making quick homedir copies to set up a new workstation
- by systems administrator negligence or malice
- via poorly secured backups
- via malware in desktop apps (browsers, pdf viewers, etc)
- via coercion when crossing international borders
Protecting your key with a good passphrase greatly helps reduce the risk
of any of the above, but passphrases can be discovered via keyloggers,
shoulder-surfing, or any number of other means. For this reason, the
recommended setup is to remove your master key from your home directory
and store it on offline storage.
.. warning::
Please see the previous section and make sure you have backed up
your GnuPG directory in its entirety. What we are about to do will
render your key useless if you do not have a usable backup!
First, identify the keygrip of your master key::
$ gpg --with-keygrip --list-key [fpr]
The output will be something like this::
pub rsa2048 2018-01-24 [SC] [expires: 2020-01-24]
000000000000000000000000AAAABBBBCCCCDDDD
Keygrip = 1111000000000000000000000000000000000000
uid [ultimate] Alice Dev <adev@kernel.org>
sub rsa2048 2018-01-24 [E] [expires: 2020-01-24]
Keygrip = 2222000000000000000000000000000000000000
sub ed25519 2018-01-24 [S]
Keygrip = 3333000000000000000000000000000000000000
Find the keygrip entry that is beneath the ``pub`` line (right under the
master key fingerprint). This will correspond directly to a file in your
``~/.gnupg`` directory::
$ cd ~/.gnupg/private-keys-v1.d
$ ls
1111000000000000000000000000000000000000.key
2222000000000000000000000000000000000000.key
3333000000000000000000000000000000000000.key
All you have to do is simply remove the .key file that corresponds to
the master keygrip::
$ cd ~/.gnupg/private-keys-v1.d
$ rm 1111000000000000000000000000000000000000.key
Now, if you issue the ``--list-secret-keys`` command, it will show that
the master key is missing (the ``#`` indicates it is not available)::
$ gpg --list-secret-keys
sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24]
000000000000000000000000AAAABBBBCCCCDDDD
uid [ultimate] Alice Dev <adev@kernel.org>
ssb rsa2048 2018-01-24 [E] [expires: 2020-01-24]
ssb ed25519 2018-01-24 [S]
You should also remove any ``secring.gpg`` files in the ``~/.gnupg``
directory, which are left over from earlier versions of GnuPG.
If you don't have the "private-keys-v1.d" directory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you do not have a ``~/.gnupg/private-keys-v1.d`` directory, then your
secret keys are still stored in the legacy ``secring.gpg`` file used by
GnuPG v1. Making any changes to your key, such as changing the
passphrase or adding a subkey, should automatically convert the old
``secring.gpg`` format to use ``private-keys-v1.d`` instead.
Once you get that done, make sure to delete the obsolete ``secring.gpg``
file, which still contains your private keys.
.. _smartcards:
Move the subkeys to a dedicated crypto device
=============================================
Even though the master key is now safe from being leaked or stolen, the
subkeys are still in your home directory. Anyone who manages to get
their hands on those will be able to decrypt your communication or fake
your signatures (if they know the passphrase). Furthermore, each time a
GnuPG operation is performed, the keys are loaded into system memory and
can be stolen from there by sufficiently advanced malware (think
Meltdown and Spectre).
The best way to completely protect your keys is to move them to a
specialized hardware device that is capable of smartcard operations.
The benefits of smartcards
--------------------------
A smartcard contains a cryptographic chip that is capable of storing
private keys and performing crypto operations directly on the card
itself. Because the key contents never leave the smartcard, the
operating system of the computer into which you plug in the hardware
device is not able to retrieve the private keys themselves. This is very
different from the encrypted USB storage device we used earlier for
backup purposes -- while that USB device is plugged in and mounted, the
operating system is able to access the private key contents.
Using external encrypted USB media is not a substitute to having a
smartcard-capable device.
Available smartcard devices
---------------------------
Unless all your laptops and workstations have smartcard readers, the
easiest is to get a specialized USB device that implements smartcard
functionality. There are several options available:
- `Nitrokey Start`_: Open hardware and Free Software, based on FSI
Japan's `Gnuk`_. Offers support for ECC keys, but fewest security
features (such as resistance to tampering or some side-channel
attacks).
- `Nitrokey Pro`_: Similar to the Nitrokey Start, but more
tamper-resistant and offers more security features, but no ECC
support.
- `Yubikey 4`_: proprietary hardware and software, but cheaper than
Nitrokey Pro and comes available in the USB-C form that is more useful
with newer laptops. Offers additional security features such as FIDO
U2F, but no ECC.
`LWN has a good review`_ of some of the above models, as well as several
others. If you want to use ECC keys, your best bet among commercially
available devices is the Nitrokey Start.
.. note::
If you are listed in MAINTAINERS or have an account at kernel.org,
you `qualify for a free Nitrokey Start`_ courtesy of The Linux
Foundation.
.. _`Nitrokey Start`: https://shop.nitrokey.com/shop/product/nitrokey-start-6
.. _`Nitrokey Pro`: https://shop.nitrokey.com/shop/product/nitrokey-pro-3
.. _`Yubikey 4`: https://www.yubico.com/product/yubikey-4-series/
.. _Gnuk: http://www.fsij.org/doc-gnuk/
.. _`LWN has a good review`: https://lwn.net/Articles/736231/
.. _`qualify for a free Nitrokey Start`: https://www.kernel.org/nitrokey-digital-tokens-for-kernel-developers.html
Configure your smartcard device
-------------------------------
Your smartcard device should Just Work (TM) the moment you plug it into
any modern Linux workstation. You can verify it by running::
$ gpg --card-status
If you see full smartcard details, then you are good to go.
Unfortunately, troubleshooting all possible reasons why things may not
be working for you is way beyond the scope of this guide. If you are
having trouble getting the card to work with GnuPG, please seek help via
usual support channels.
To configure your smartcard, you will need to use the GnuPG menu system, as
there are no convenient command-line switches::
$ gpg --card-edit
[...omitted...]
gpg/card> admin
Admin commands are allowed
gpg/card> passwd
You should set the user PIN (1), Admin PIN (3), and the Reset Code (4).
Please make sure to record and store these in a safe place -- especially
the Admin PIN and the Reset Code (which allows you to completely wipe
the smartcard). You so rarely need to use the Admin PIN, that you will
inevitably forget what it is if you do not record it.
Getting back to the main card menu, you can also set other values (such
as name, sex, login data, etc), but it's not necessary and will
additionally leak information about your smartcard should you lose it.
.. note::
Despite having the name "PIN", neither the user PIN nor the admin
PIN on the card need to be numbers.
.. warning::
Some devices may require that you move the subkeys onto the device
before you can change the passphrase. Please check the documentation
provided by the device manufacturer.
Move the subkeys to your smartcard
----------------------------------
Exit the card menu (using "q") and save all changes. Next, let's move
your subkeys onto the smartcard. You will need both your PGP key
passphrase and the admin PIN of the card for most operations::
$ gpg --edit-key [fpr]
Secret subkeys are available.
pub rsa2048/AAAABBBBCCCCDDDD
created: 2018-01-23 expires: 2020-01-23 usage: SC
trust: ultimate validity: ultimate
ssb rsa2048/1111222233334444
created: 2018-01-23 expires: never usage: E
ssb ed25519/5555666677778888
created: 2017-12-07 expires: never usage: S
[ultimate] (1). Alice Dev <adev@kernel.org>
gpg>
Using ``--edit-key`` puts us into the menu mode again, and you will
notice that the key listing is a little different. From here on, all
commands are done from inside this menu mode, as indicated by ``gpg>``.
First, let's select the key we'll be putting onto the card -- you do
this by typing ``key 1`` (it's the first one in the listing, the **[E]**
subkey)::
gpg> key 1
In the output, you should now see ``ssb*`` on the **[E]** key. The ``*``
indicates which key is currently "selected." It works as a *toggle*,
meaning that if you type ``key 1`` again, the ``*`` will disappear and
the key will not be selected any more.
Now, let's move that key onto the smartcard::
gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2
Since it's our **[E]** key, it makes sense to put it into the Encryption
slot. When you submit your selection, you will be prompted first for
your PGP key passphrase, and then for the admin PIN. If the command
returns without an error, your key has been moved.
**Important**: Now type ``key 1`` again to unselect the first key, and
``key 2`` to select the **[S]** key::
gpg> key 1
gpg> key 2
gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1
You can use the **[S]** key both for Signature and Authentication, but
we want to make sure it's in the Signature slot, so choose (1). Once
again, if your command returns without an error, then the operation was
successful::
gpg> q
Save changes? (y/N) y
Saving the changes will delete the keys you moved to the card from your
home directory (but it's okay, because we have them in our backups
should we need to do this again for a replacement smartcard).
Verifying that the keys were moved
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you perform ``--list-secret-keys`` now, you will see a subtle
difference in the output::
$ gpg --list-secret-keys
sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24]
000000000000000000000000AAAABBBBCCCCDDDD
uid [ultimate] Alice Dev <adev@kernel.org>
ssb> rsa2048 2018-01-24 [E] [expires: 2020-01-24]
ssb> ed25519 2018-01-24 [S]
The ``>`` in the ``ssb>`` output indicates that the subkey is only
available on the smartcard. If you go back into your secret keys
directory and look at the contents there, you will notice that the
``.key`` files there have been replaced with stubs::
$ cd ~/.gnupg/private-keys-v1.d
$ strings *.key | grep 'private-key'
The output should contain ``shadowed-private-key`` to indicate that
these files are only stubs and the actual content is on the smartcard.
Verifying that the smartcard is functioning
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To verify that the smartcard is working as intended, you can create a
signature::
$ echo "Hello world" | gpg --clearsign > /tmp/test.asc
$ gpg --verify /tmp/test.asc
This should ask for your smartcard PIN on your first command, and then
show "Good signature" after you run ``gpg --verify``.
Congratulations, you have successfully made it extremely difficult to
steal your digital developer identity!
Other common GnuPG operations
-----------------------------
Here is a quick reference for some common operations you'll need to do
with your PGP key.
Mounting your master key offline storage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You will need your master key for any of the operations below, so you
will first need to mount your backup offline storage and tell GnuPG to
use it::
$ export GNUPGHOME=/media/disk/foo/gnupg-backup
$ gpg --list-secret-keys
You want to make sure that you see ``sec`` and not ``sec#`` in the
output (the ``#`` means the key is not available and you're still using
your regular home directory location).
Extending key expiration date
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The master key has the default expiration date of 2 years from the date
of creation. This is done both for security reasons and to make obsolete
keys eventually disappear from keyservers.
To extend the expiration on your key by a year from current date, just
run::
$ gpg --quick-set-expire [fpr] 1y
You can also use a specific date if that is easier to remember (e.g.
your birthday, January 1st, or Canada Day)::
$ gpg --quick-set-expire [fpr] 2020-07-01
Remember to send the updated key back to keyservers::
$ gpg --send-key [fpr]
Updating your work directory after any changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After you make any changes to your key using the offline storage, you will
want to import these changes back into your regular working directory::
$ gpg --export | gpg --homedir ~/.gnupg --import
$ unset GNUPGHOME
Using gpg-agent over ssh
~~~~~~~~~~~~~~~~~~~~~~~~
You can forward your gpg-agent over ssh if you need to sign tags or
commits on a remote system. Please refer to the instructions provided
on the GnuPG wiki:
- `Agent Forwarding over SSH`_
It works more smoothly if you can modify the sshd server settings on the
remote end.
.. _`Agent Forwarding over SSH`: https://wiki.gnupg.org/AgentForwarding
Using PGP with Git
==================
One of the core features of Git is its decentralized nature -- once a
repository is cloned to your system, you have full history of the
project, including all of its tags, commits and branches. However, with
hundreds of cloned repositories floating around, how does anyone verify
that their copy of linux.git has not been tampered with by a malicious
third party?
Or what happens if a backdoor is discovered in the code and the "Author"
line in the commit says it was done by you, while you're pretty sure you
had `nothing to do with it`_?
To address both of these issues, Git introduced PGP integration. Signed
tags prove the repository integrity by assuring that its contents are
exactly the same as on the workstation of the developer who created the
tag, while signed commits make it nearly impossible for someone to
impersonate you without having access to your PGP keys.
.. _`nothing to do with it`: https://github.com/jayphelps/git-blame-someone-else
Configure git to use your PGP key
---------------------------------
If you only have one secret key in your keyring, then you don't really
need to do anything extra, as it becomes your default key. However, if
you happen to have multiple secret keys, you can tell git which key
should be used (``[fpr]`` is the fingerprint of your key)::
$ git config --global user.signingKey [fpr]
**IMPORTANT**: If you have a distinct ``gpg2`` command, then you should
tell git to always use it instead of the legacy ``gpg`` from version 1::
$ git config --global gpg.program gpg2
$ git config --global gpgv.program gpgv2
How to work with signed tags
----------------------------
To create a signed tag, simply pass the ``-s`` switch to the tag
command::
$ git tag -s [tagname]
Our recommendation is to always sign git tags, as this allows other
developers to ensure that the git repository they are pulling from has
not been maliciously altered.
How to verify signed tags
~~~~~~~~~~~~~~~~~~~~~~~~~
To verify a signed tag, simply use the ``verify-tag`` command::
$ git verify-tag [tagname]
If you are pulling a tag from another fork of the project repository,
git should automatically verify the signature at the tip you're pulling
and show you the results during the merge operation::
$ git pull [url] tags/sometag
The merge message will contain something like this::
Merge tag 'sometag' of [url]
[Tag message]
# gpg: Signature made [...]
# gpg: Good signature from [...]
If you are verifying someone else's git tag, then you will need to
import their PGP key. Please refer to the
":ref:`verify_identities`" section below.
.. note::
If you get "``gpg: Can't check signature: unknown pubkey
algorithm``" error, you need to tell git to use gpgv2 for
verification, so it properly processes signatures made by ECC keys.
See instructions at the start of this section.
Configure git to always sign annotated tags
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Chances are, if you're creating an annotated tag, you'll want to sign
it. To force git to always sign annotated tags, you can set a global
configuration option::
$ git config --global tag.forceSignAnnotated true
How to work with signed commits
-------------------------------
It is easy to create signed commits, but it is much more difficult to
use them in Linux kernel development, since it relies on patches sent to
the mailing list, and this workflow does not preserve PGP commit
signatures. Furthermore, when rebasing your repository to match
upstream, even your own PGP commit signatures will end up discarded. For
this reason, most kernel developers don't bother signing their commits
and will ignore signed commits in any external repositories that they
rely upon in their work.
However, if you have your working git tree publicly available at some
git hosting service (kernel.org, infradead.org, ozlabs.org, or others),
then the recommendation is that you sign all your git commits even if
upstream developers do not directly benefit from this practice.
We recommend this for the following reasons:
1. Should there ever be a need to perform code forensics or track code
provenance, even externally maintained trees carrying PGP commit
signatures will be valuable for such purposes.
2. If you ever need to re-clone your local repository (for example,
after a disk failure), this lets you easily verify the repository
integrity before resuming your work.
3. If someone needs to cherry-pick your commits, this allows them to
quickly verify their integrity before applying them.
Creating signed commits
~~~~~~~~~~~~~~~~~~~~~~~
To create a signed commit, you just need to pass the ``-S`` flag to the
``git commit`` command (it's capital ``-S`` due to collision with
another flag)::
$ git commit -S
Configure git to always sign commits
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can tell git to always sign commits::
git config --global commit.gpgSign true
.. note::
Make sure you configure ``gpg-agent`` before you turn this on.
.. _verify_identities:
How to verify kernel developer identities
=========================================
Signing tags and commits is easy, but how does one go about verifying
that the key used to sign something belongs to the actual kernel
developer and not to a malicious imposter?
Configure auto-key-retrieval using WKD and DANE
-----------------------------------------------
If you are not already someone with an extensive collection of other
developers' public keys, then you can jumpstart your keyring by relying
on key auto-discovery and auto-retrieval. GnuPG can piggyback on other
delegated trust technologies, namely DNSSEC and TLS, to get you going if
the prospect of starting your own Web of Trust from scratch is too
daunting.
Add the following to your ``~/.gnupg/gpg.conf``::
auto-key-locate wkd,dane,local
auto-key-retrieve
DNS-Based Authentication of Named Entities ("DANE") is a method for
publishing public keys in DNS and securing them using DNSSEC signed
zones. Web Key Directory ("WKD") is the alternative method that uses
https lookups for the same purpose. When using either DANE or WKD for
looking up public keys, GnuPG will validate DNSSEC or TLS certificates,
respectively, before adding auto-retrieved public keys to your local
keyring.
Kernel.org publishes the WKD for all developers who have kernel.org
accounts. Once you have the above changes in your ``gpg.conf``, you can
auto-retrieve the keys for Linus Torvalds and Greg Kroah-Hartman (if you
don't already have them)::
$ gpg --locate-keys torvalds@kernel.org gregkh@kernel.org
If you have a kernel.org account, then you should `add the kernel.org
UID to your key`_ to make WKD more useful to other kernel developers.
.. _`add the kernel.org UID to your key`: https://korg.wiki.kernel.org/userdoc/mail#adding_a_kernelorg_uid_to_your_pgp_key
Web of Trust (WOT) vs. Trust on First Use (TOFU)
------------------------------------------------
PGP incorporates a trust delegation mechanism known as the "Web of
Trust." At its core, this is an attempt to replace the need for
centralized Certification Authorities of the HTTPS/TLS world. Instead of
various software makers dictating who should be your trusted certifying
entity, PGP leaves this responsibility to each user.
Unfortunately, very few people understand how the Web of Trust works.
While it remains an important aspect of the OpenPGP specification,
recent versions of GnuPG (2.2 and above) have implemented an alternative
mechanism called "Trust on First Use" (TOFU). You can think of TOFU as
"the SSH-like approach to trust." With SSH, the first time you connect
to a remote system, its key fingerprint is recorded and remembered. If
the key changes in the future, the SSH client will alert you and refuse
to connect, forcing you to make a decision on whether you choose to
trust the changed key or not. Similarly, the first time you import
someone's PGP key, it is assumed to be valid. If at any point in the
future GnuPG comes across another key with the same identity, both the
previously imported key and the new key will be marked as invalid and
you will need to manually figure out which one to keep.
We recommend that you use the combined TOFU+PGP trust model (which is
the new default in GnuPG v2). To set it, add (or modify) the
``trust-model`` setting in ``~/.gnupg/gpg.conf``::
trust-model tofu+pgp
How to use keyservers (more) safely
-----------------------------------
If you get a "No public key" error when trying to validate someone's
tag, then you should attempt to lookup that key using a keyserver. It is
important to keep in mind that there is absolutely no guarantee that the
key you retrieve from PGP keyservers belongs to the actual person --
that much is by design. You are supposed to use the Web of Trust to
establish key validity.
How to properly maintain the Web of Trust is beyond the scope of this
document, simply because doing it properly requires both effort and
dedication that tends to be beyond the caring threshold of most human
beings. Here are some shortcuts that will help you reduce the risk of
importing a malicious key.
First, let's say you've tried to run ``git verify-tag`` but it returned
an error saying the key is not found::
$ git verify-tag sunxi-fixes-for-4.15-2
gpg: Signature made Sun 07 Jan 2018 10:51:55 PM EST
gpg: using RSA key DA73759BF8619E484E5A3B47389A54219C0F2430
gpg: issuer "wens@...org"
gpg: Can't check signature: No public key
Let's query the keyserver for more info about that key fingerprint (the
fingerprint probably belongs to a subkey, so we can't use it directly
without finding out the ID of the master key it is associated with)::
$ gpg --search DA73759BF8619E484E5A3B47389A54219C0F2430
gpg: data source: hkp://keys.gnupg.net
(1) Chen-Yu Tsai <wens@...org>
4096 bit RSA key C94035C21B4F2AEB, created: 2017-03-14, expires: 2019-03-15
Keys 1-1 of 1 for "DA73759BF8619E484E5A3B47389A54219C0F2430". Enter number(s), N)ext, or Q)uit > q
Locate the ID of the master key in the output, in our example
``C94035C21B4F2AEB``. Now display the key of Linus Torvalds that you
have on your keyring::
$ gpg --list-key torvalds@kernel.org
pub rsa2048 2011-09-20 [SC]
ABAF11C65A2970B130ABE3C479BE3E4300411886
uid [ unknown] Linus Torvalds <torvalds@kernel.org>
sub rsa2048 2011-09-20 [E]
Next, open the `PGP pathfinder`_. In the "From" field, paste the key
fingerprint of Linus Torvalds from the output above. In the "To" field,
paste they key-id you found via ``gpg --search`` of the unknown key, and
check the results:
- `Finding paths to Linus`_
If you get a few decent trust paths, then it's a pretty good indication
that it is a valid key. You can add it to your keyring from the
keyserver now::
$ gpg --recv-key C94035C21B4F2AEB
This process is not perfect, and you are obviously trusting the
administrators of the PGP Pathfinder service to not be malicious (in
fact, this goes against :ref:`devs_not_infra`). However, if you
do not carefully maintain your own web of trust, then it is a marked
improvement over blindly trusting keyservers.
.. _`PGP pathfinder`: https://pgp.cs.uu.nl/
.. _`Finding paths to Linus`: https://pgp.cs.uu.nl/paths/79BE3E4300411886/to/C94035C21B4F2AEB.html
Documentation/process/management-style.rst
+289
View File
@@ -0,0 +1,289 @@
.. _managementstyle:
Linux kernel management style
=============================
This is a short document describing the preferred (or made up, depending
on who you ask) management style for the linux kernel. It's meant to
mirror the process/coding-style.rst document to some degree, and mainly written to
avoid answering [#f1]_ the same (or similar) questions over and over again.
Management style is very personal and much harder to quantify than
simple coding style rules, so this document may or may not have anything
to do with reality. It started as a lark, but that doesn't mean that it
might not actually be true. You'll have to decide for yourself.
Btw, when talking about "kernel manager", it's all about the technical
lead persons, not the people who do traditional management inside
companies. If you sign purchase orders or you have any clue about the
budget of your group, you're almost certainly not a kernel manager.
These suggestions may or may not apply to you.
First off, I'd suggest buying "Seven Habits of Highly Effective
People", and NOT read it. Burn it, it's a great symbolic gesture.
.. [#f1] This document does so not so much by answering the question, but by
making it painfully obvious to the questioner that we don't have a clue
to what the answer is.
Anyway, here goes:
.. _decisions:
1) Decisions
------------
Everybody thinks managers make decisions, and that decision-making is
important. The bigger and more painful the decision, the bigger the
manager must be to make it. That's very deep and obvious, but it's not
actually true.
The name of the game is to **avoid** having to make a decision. In
particular, if somebody tells you "choose (a) or (b), we really need you
to decide on this", you're in trouble as a manager. The people you
manage had better know the details better than you, so if they come to
you for a technical decision, you're screwed. You're clearly not
competent to make that decision for them.
(Corollary:if the people you manage don't know the details better than
you, you're also screwed, although for a totally different reason.
Namely that you are in the wrong job, and that **they** should be managing
your brilliance instead).
So the name of the game is to **avoid** decisions, at least the big and
painful ones. Making small and non-consequential decisions is fine, and
makes you look like you know what you're doing, so what a kernel manager
needs to do is to turn the big and painful ones into small things where
nobody really cares.
It helps to realize that the key difference between a big decision and a
small one is whether you can fix your decision afterwards. Any decision
can be made small by just always making sure that if you were wrong (and
you **will** be wrong), you can always undo the damage later by
backtracking. Suddenly, you get to be doubly managerial for making
**two** inconsequential decisions - the wrong one **and** the right one.
And people will even see that as true leadership (*cough* bullshit
*cough*).
Thus the key to avoiding big decisions becomes to just avoiding to do
things that can't be undone. Don't get ushered into a corner from which
you cannot escape. A cornered rat may be dangerous - a cornered manager
is just pitiful.
It turns out that since nobody would be stupid enough to ever really let
a kernel manager have huge fiscal responsibility **anyway**, it's usually
fairly easy to backtrack. Since you're not going to be able to waste
huge amounts of money that you might not be able to repay, the only
thing you can backtrack on is a technical decision, and there
back-tracking is very easy: just tell everybody that you were an
incompetent nincompoop, say you're sorry, and undo all the worthless
work you had people work on for the last year. Suddenly the decision
you made a year ago wasn't a big decision after all, since it could be
easily undone.
It turns out that some people have trouble with this approach, for two
reasons:
- admitting you were an idiot is harder than it looks. We all like to
maintain appearances, and coming out in public to say that you were
wrong is sometimes very hard indeed.
- having somebody tell you that what you worked on for the last year
wasn't worthwhile after all can be hard on the poor lowly engineers
too, and while the actual **work** was easy enough to undo by just
deleting it, you may have irrevocably lost the trust of that
engineer. And remember: "irrevocable" was what we tried to avoid in
the first place, and your decision ended up being a big one after
all.
Happily, both of these reasons can be mitigated effectively by just
admitting up-front that you don't have a friggin' clue, and telling
people ahead of the fact that your decision is purely preliminary, and
might be the wrong thing. You should always reserve the right to change
your mind, and make people very **aware** of that. And it's much easier
to admit that you are stupid when you haven't **yet** done the really
stupid thing.
Then, when it really does turn out to be stupid, people just roll their
eyes and say "Oops, not again".
This preemptive admission of incompetence might also make the people who
actually do the work also think twice about whether it's worth doing or
not. After all, if **they** aren't certain whether it's a good idea, you
sure as hell shouldn't encourage them by promising them that what they
work on will be included. Make them at least think twice before they
embark on a big endeavor.
Remember: they'd better know more about the details than you do, and
they usually already think they have the answer to everything. The best
thing you can do as a manager is not to instill confidence, but rather a
healthy dose of critical thinking on what they do.
Btw, another way to avoid a decision is to plaintively just whine "can't
we just do both?" and look pitiful. Trust me, it works. If it's not
clear which approach is better, they'll eventually figure it out. The
answer may end up being that both teams get so frustrated by the
situation that they just give up.
That may sound like a failure, but it's usually a sign that there was
something wrong with both projects, and the reason the people involved
couldn't decide was that they were both wrong. You end up coming up
smelling like roses, and you avoided yet another decision that you could
have screwed up on.
2) People
---------
Most people are idiots, and being a manager means you'll have to deal
with it, and perhaps more importantly, that **they** have to deal with
**you**.
It turns out that while it's easy to undo technical mistakes, it's not
as easy to undo personality disorders. You just have to live with
theirs - and yours.
However, in order to prepare yourself as a kernel manager, it's best to
remember not to burn any bridges, bomb any innocent villagers, or
alienate too many kernel developers. It turns out that alienating people
is fairly easy, and un-alienating them is hard. Thus "alienating"
immediately falls under the heading of "not reversible", and becomes a
no-no according to :ref:`decisions`.
There's just a few simple rules here:
(1) don't call people d*ckheads (at least not in public)
(2) learn how to apologize when you forgot rule (1)
The problem with #1 is that it's very easy to do, since you can say
"you're a d*ckhead" in millions of different ways [#f2]_, sometimes without
even realizing it, and almost always with a white-hot conviction that
you are right.
And the more convinced you are that you are right (and let's face it,
you can call just about **anybody** a d*ckhead, and you often **will** be
right), the harder it ends up being to apologize afterwards.
To solve this problem, you really only have two options:
- get really good at apologies
- spread the "love" out so evenly that nobody really ends up feeling
like they get unfairly targeted. Make it inventive enough, and they
might even be amused.
The option of being unfailingly polite really doesn't exist. Nobody will
trust somebody who is so clearly hiding their true character.
.. [#f2] Paul Simon sang "Fifty Ways to Leave Your Lover", because quite
frankly, "A Million Ways to Tell a Developer They're a D*ckhead" doesn't
scan nearly as well. But I'm sure he thought about it.
3) People II - the Good Kind
----------------------------
While it turns out that most people are idiots, the corollary to that is
sadly that you are one too, and that while we can all bask in the secure
knowledge that we're better than the average person (let's face it,
nobody ever believes that they're average or below-average), we should
also admit that we're not the sharpest knife around, and there will be
other people that are less of an idiot than you are.
Some people react badly to smart people. Others take advantage of them.
Make sure that you, as a kernel maintainer, are in the second group.
Suck up to them, because they are the people who will make your job
easier. In particular, they'll be able to make your decisions for you,
which is what the game is all about.
So when you find somebody smarter than you are, just coast along. Your
management responsibilities largely become ones of saying "Sounds like a
good idea - go wild", or "That sounds good, but what about xxx?". The
second version in particular is a great way to either learn something
new about "xxx" or seem **extra** managerial by pointing out something the
smarter person hadn't thought about. In either case, you win.
One thing to look out for is to realize that greatness in one area does
not necessarily translate to other areas. So you might prod people in
specific directions, but let's face it, they might be good at what they
do, and suck at everything else. The good news is that people tend to
naturally gravitate back to what they are good at, so it's not like you
are doing something irreversible when you **do** prod them in some
direction, just don't push too hard.
4) Placing blame
----------------
Things will go wrong, and people want somebody to blame. Tag, you're it.
It's not actually that hard to accept the blame, especially if people
kind of realize that it wasn't **all** your fault. Which brings us to the
best way of taking the blame: do it for someone else. You'll feel good
for taking the fall, they'll feel good about not getting blamed, and the
person who lost their whole 36GB porn-collection because of your
incompetence will grudgingly admit that you at least didn't try to weasel
out of it.
Then make the developer who really screwed up (if you can find them) know
**in_private** that they screwed up. Not just so they can avoid it in the
future, but so that they know they owe you one. And, perhaps even more
importantly, they're also likely the person who can fix it. Because, let's
face it, it sure ain't you.
Taking the blame is also why you get to be manager in the first place.
It's part of what makes people trust you, and allow you the potential
glory, because you're the one who gets to say "I screwed up". And if
you've followed the previous rules, you'll be pretty good at saying that
by now.
5) Things to avoid
------------------
There's one thing people hate even more than being called "d*ckhead",
and that is being called a "d*ckhead" in a sanctimonious voice. The
first you can apologize for, the second one you won't really get the
chance. They likely will no longer be listening even if you otherwise
do a good job.
We all think we're better than anybody else, which means that when
somebody else puts on airs, it **really** rubs us the wrong way. You may
be morally and intellectually superior to everybody around you, but
don't try to make it too obvious unless you really **intend** to irritate
somebody [#f3]_.
Similarly, don't be too polite or subtle about things. Politeness easily
ends up going overboard and hiding the problem, and as they say, "On the
internet, nobody can hear you being subtle". Use a big blunt object to
hammer the point in, because you can't really depend on people getting
your point otherwise.
Some humor can help pad both the bluntness and the moralizing. Going
overboard to the point of being ridiculous can drive a point home
without making it painful to the recipient, who just thinks you're being
silly. It can thus help get through the personal mental block we all
have about criticism.
.. [#f3] Hint: internet newsgroups that are not directly related to your work
are great ways to take out your frustrations at other people. Write
insulting posts with a sneer just to get into a good flame every once in
a while, and you'll feel cleansed. Just don't crap too close to home.
6) Why me?
----------
Since your main responsibility seems to be to take the blame for other
peoples mistakes, and make it painfully obvious to everybody else that
you're incompetent, the obvious question becomes one of why do it in the
first place?
First off, while you may or may not get screaming teenage girls (or
boys, let's not be judgmental or sexist here) knocking on your dressing
room door, you **will** get an immense feeling of personal accomplishment
for being "in charge". Never mind the fact that you're really leading
by trying to keep up with everybody else and running after them as fast
as you can. Everybody will still think you're the person in charge.
It's a great job if you can hack it.
Documentation/process/stable-api-nonsense.rst
+205
View File
@@ -0,0 +1,205 @@
.. _stable_api_nonsense:
The Linux Kernel Driver Interface
==================================
(all of your questions answered and then some)
Greg Kroah-Hartman <greg@kroah.com>
This is being written to try to explain why Linux **does not have a binary
kernel interface, nor does it have a stable kernel interface**.
.. note::
Please realize that this article describes the **in kernel** interfaces, not
the kernel to userspace interfaces.
The kernel to userspace interface is the one that application programs use,
the syscall interface. That interface is **very** stable over time, and
will not break. I have old programs that were built on a pre 0.9something
kernel that still work just fine on the latest 2.6 kernel release.
That interface is the one that users and application programmers can count
on being stable.
Executive Summary
-----------------
You think you want a stable kernel interface, but you really do not, and
you don't even know it. What you want is a stable running driver, and
you get that only if your driver is in the main kernel tree. You also
get lots of other good benefits if your driver is in the main kernel
tree, all of which has made Linux into such a strong, stable, and mature
operating system which is the reason you are using it in the first
place.
Intro
-----
It's only the odd person who wants to write a kernel driver that needs
to worry about the in-kernel interfaces changing. For the majority of
the world, they neither see this interface, nor do they care about it at
all.
First off, I'm not going to address **any** legal issues about closed
source, hidden source, binary blobs, source wrappers, or any other term
that describes kernel drivers that do not have their source code
released under the GPL. Please consult a lawyer if you have any legal
questions, I'm a programmer and hence, I'm just going to be describing
the technical issues here (not to make light of the legal issues, they
are real, and you do need to be aware of them at all times.)
So, there are two main topics here, binary kernel interfaces and stable
kernel source interfaces. They both depend on each other, but we will
discuss the binary stuff first to get it out of the way.
Binary Kernel Interface
-----------------------
Assuming that we had a stable kernel source interface for the kernel, a
binary interface would naturally happen too, right? Wrong. Please
consider the following facts about the Linux kernel:
- Depending on the version of the C compiler you use, different kernel
data structures will contain different alignment of structures, and
possibly include different functions in different ways (putting
functions inline or not.) The individual function organization
isn't that important, but the different data structure padding is
very important.
- Depending on what kernel build options you select, a wide range of
different things can be assumed by the kernel:
- different structures can contain different fields
- Some functions may not be implemented at all, (i.e. some locks
compile away to nothing for non-SMP builds.)
- Memory within the kernel can be aligned in different ways,
depending on the build options.
- Linux runs on a wide range of different processor architectures.
There is no way that binary drivers from one architecture will run
on another architecture properly.
Now a number of these issues can be addressed by simply compiling your
module for the exact specific kernel configuration, using the same exact
C compiler that the kernel was built with. This is sufficient if you
want to provide a module for a specific release version of a specific
Linux distribution. But multiply that single build by the number of
different Linux distributions and the number of different supported
releases of the Linux distribution and you quickly have a nightmare of
different build options on different releases. Also realize that each
Linux distribution release contains a number of different kernels, all
tuned to different hardware types (different processor types and
different options), so for even a single release you will need to create
multiple versions of your module.
Trust me, you will go insane over time if you try to support this kind
of release, I learned this the hard way a long time ago...
Stable Kernel Source Interfaces
-------------------------------
This is a much more "volatile" topic if you talk to people who try to
keep a Linux kernel driver that is not in the main kernel tree up to
date over time.
Linux kernel development is continuous and at a rapid pace, never
stopping to slow down. As such, the kernel developers find bugs in
current interfaces, or figure out a better way to do things. If they do
that, they then fix the current interfaces to work better. When they do
so, function names may change, structures may grow or shrink, and
function parameters may be reworked. If this happens, all of the
instances of where this interface is used within the kernel are fixed up
at the same time, ensuring that everything continues to work properly.
As a specific examples of this, the in-kernel USB interfaces have
undergone at least three different reworks over the lifetime of this
subsystem. These reworks were done to address a number of different
issues:
- A change from a synchronous model of data streams to an asynchronous
one. This reduced the complexity of a number of drivers and
increased the throughput of all USB drivers such that we are now
running almost all USB devices at their maximum speed possible.
- A change was made in the way data packets were allocated from the
USB core by USB drivers so that all drivers now needed to provide
more information to the USB core to fix a number of documented
deadlocks.
This is in stark contrast to a number of closed source operating systems
which have had to maintain their older USB interfaces over time. This
provides the ability for new developers to accidentally use the old
interfaces and do things in improper ways, causing the stability of the
operating system to suffer.
In both of these instances, all developers agreed that these were
important changes that needed to be made, and they were made, with
relatively little pain. If Linux had to ensure that it will preserve a
stable source interface, a new interface would have been created, and
the older, broken one would have had to be maintained over time, leading
to extra work for the USB developers. Since all Linux USB developers do
their work on their own time, asking programmers to do extra work for no
gain, for free, is not a possibility.
Security issues are also very important for Linux. When a
security issue is found, it is fixed in a very short amount of time. A
number of times this has caused internal kernel interfaces to be
reworked to prevent the security problem from occurring. When this
happens, all drivers that use the interfaces were also fixed at the
same time, ensuring that the security problem was fixed and could not
come back at some future time accidentally. If the internal interfaces
were not allowed to change, fixing this kind of security problem and
insuring that it could not happen again would not be possible.
Kernel interfaces are cleaned up over time. If there is no one using a
current interface, it is deleted. This ensures that the kernel remains
as small as possible, and that all potential interfaces are tested as
well as they can be (unused interfaces are pretty much impossible to
test for validity.)
What to do
----------
So, if you have a Linux kernel driver that is not in the main kernel
tree, what are you, a developer, supposed to do? Releasing a binary
driver for every different kernel version for every distribution is a
nightmare, and trying to keep up with an ever changing kernel interface
is also a rough job.
Simple, get your kernel driver into the main kernel tree (remember we
are talking about GPL released drivers here, if your code doesn't fall
under this category, good luck, you are on your own here, you leech
<insert link to leech comment from Andrew and Linus here>.) If your
driver is in the tree, and a kernel interface changes, it will be fixed
up by the person who did the kernel change in the first place. This
ensures that your driver is always buildable, and works over time, with
very little effort on your part.
The very good side effects of having your driver in the main kernel tree
are:
- The quality of the driver will rise as the maintenance costs (to the
original developer) will decrease.
- Other developers will add features to your driver.
- Other people will find and fix bugs in your driver.
- Other people will find tuning opportunities in your driver.
- Other people will update the driver for you when external interface
changes require it.
- The driver automatically gets shipped in all Linux distributions
without having to ask the distros to add it.
As Linux supports a larger number of different devices "out of the box"
than any other operating system, and it supports these devices on more
different processor architectures than any other operating system, this
proven type of development model must be doing something right :)
------
Thanks to Randy Dunlap, Andrew Morton, David Brownell, Hanna Linder,
Robert Love, and Nishanth Aravamudan for their review and comments on
early drafts of this paper.
Documentation/process/stable-kernel-rules.rst
+184
View File
@@ -0,0 +1,184 @@
.. _stable_kernel_rules:
Everything you ever wanted to know about Linux -stable releases
===============================================================
Rules on what kind of patches are accepted, and which ones are not, into the
"-stable" tree:
- It must be obviously correct and tested.
- It cannot be bigger than 100 lines, with context.
- It must fix only one thing.
- It must fix a real bug that bothers people (not a, "This could be a
problem..." type thing).
- It must fix a problem that causes a build error (but not for things
marked CONFIG_BROKEN), an oops, a hang, data corruption, a real
security issue, or some "oh, that's not good" issue. In short, something
critical.
- Serious issues as reported by a user of a distribution kernel may also
be considered if they fix a notable performance or interactivity issue.
As these fixes are not as obvious and have a higher risk of a subtle
regression they should only be submitted by a distribution kernel
maintainer and include an addendum linking to a bugzilla entry if it
exists and additional information on the user-visible impact.
- New device IDs and quirks are also accepted.
- No "theoretical race condition" issues, unless an explanation of how the
race can be exploited is also provided.
- It cannot contain any "trivial" fixes in it (spelling changes,
whitespace cleanups, etc).
- It must follow the
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
rules.
- It or an equivalent fix must already exist in Linus' tree (upstream).
Procedure for submitting patches to the -stable tree
----------------------------------------------------
- If the patch covers files in net/ or drivers/net please follow netdev stable
submission guidelines as described in
:ref:`Documentation/networking/netdev-FAQ.rst <netdev-FAQ>`
after first checking the stable networking queue at
https://patchwork.ozlabs.org/bundle/davem/stable/?series=&submitter=&state=*&q=&archive=
to ensure the requested patch is not already queued up.
- Security patches should not be handled (solely) by the -stable review
process but should follow the procedures in
:ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`.
For all other submissions, choose one of the following procedures
-----------------------------------------------------------------
.. _option_1:
Option 1
********
To have the patch automatically included in the stable tree, add the tag
.. code-block:: none
Cc: stable@vger.kernel.org
in the sign-off area. Once the patch is merged it will be applied to
the stable tree without anything else needing to be done by the author
or subsystem maintainer.
.. _option_2:
Option 2
********
After the patch has been merged to Linus' tree, send an email to
stable@vger.kernel.org containing the subject of the patch, the commit ID,
why you think it should be applied, and what kernel version you wish it to
be applied to.
.. _option_3:
Option 3
********
Send the patch, after verifying that it follows the above rules, to
stable@vger.kernel.org. You must note the upstream commit ID in the
changelog of your submission, as well as the kernel version you wish
it to be applied to.
:ref:`option_1` is **strongly** preferred, is the easiest and most common.
:ref:`option_2` and :ref:`option_3` are more useful if the patch isn't deemed
worthy at the time it is applied to a public git tree (for instance, because
it deserves more regression testing first). :ref:`option_3` is especially
useful if the patch needs some special handling to apply to an older kernel
(e.g., if API's have changed in the meantime).
Note that for :ref:`option_3`, if the patch deviates from the original
upstream patch (for example because it had to be backported) this must be very
clearly documented and justified in the patch description.
The upstream commit ID must be specified with a separate line above the commit
text, like this:
.. code-block:: none
commit <sha1> upstream.
Additionally, some patches submitted via Option 1 may have additional patch
prerequisites which can be cherry-picked. This can be specified in the following
format in the sign-off area:
.. code-block:: none
Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idle
Cc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidle
Cc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logic
Cc: <stable@vger.kernel.org> # 3.3.x
Signed-off-by: Ingo Molnar <mingo@elte.hu>
The tag sequence has the meaning of:
.. code-block:: none
git cherry-pick a1f84a3
git cherry-pick 1b9508f
git cherry-pick fd21073
git cherry-pick <this commit>
Also, some patches may have kernel version prerequisites. This can be
specified in the following format in the sign-off area:
.. code-block:: none
Cc: <stable@vger.kernel.org> # 3.3.x
The tag has the meaning of:
.. code-block:: none
git cherry-pick <this commit>
For each "-stable" tree starting with the specified version.
Following the submission:
- The sender will receive an ACK when the patch has been accepted into the
queue, or a NAK if the patch is rejected. This response might take a few
days, according to the developer's schedules.
- If accepted, the patch will be added to the -stable queue, for review by
other developers and by the relevant subsystem maintainer.
Review cycle
------------
- When the -stable maintainers decide for a review cycle, the patches will be
sent to the review committee, and the maintainer of the affected area of
the patch (unless the submitter is the maintainer of the area) and CC: to
the linux-kernel mailing list.
- The review committee has 48 hours in which to ACK or NAK the patch.
- If the patch is rejected by a member of the committee, or linux-kernel
members object to the patch, bringing up issues that the maintainers and
members did not realize, the patch will be dropped from the queue.
- At the end of the review cycle, the ACKed patches will be added to the
latest -stable release, and a new -stable release will happen.
- Security patches will be accepted into the -stable tree directly from the
security kernel team, and not go through the normal review cycle.
Contact the kernel security team for more details on this procedure.
Trees
-----
- The queues of patches, for both completed versions and in progress
versions can be found at:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git
- The finalized and tagged releases of all stable kernels can be found
in separate branches per version at:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Review committee
----------------
- This is made up of a number of kernel developers who have volunteered for
this task, and a few that haven't.
Documentation/process/submit-checklist.rst
+122
View File
@@ -0,0 +1,122 @@
.. _submitchecklist:
Linux Kernel patch submission checklist
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here are some basic things that developers should do if they want to see their
kernel patch submissions accepted more quickly.
These are all above and beyond the documentation that is provided in
:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`
and elsewhere regarding submitting Linux kernel patches.
1) If you use a facility then #include the file that defines/declares
that facility. Don't depend on other header files pulling in ones
that you use.
2) Builds cleanly:
a) with applicable or modified ``CONFIG`` options ``=y``, ``=m``, and
``=n``. No ``gcc`` warnings/errors, no linker warnings/errors.
b) Passes ``allnoconfig``, ``allmodconfig``
c) Builds successfully when using ``O=builddir``
3) Builds on multiple CPU architectures by using local cross-compile tools
or some other build farm.
4) ppc64 is a good architecture for cross-compilation checking because it
tends to use ``unsigned long`` for 64-bit quantities.
5) Check your patch for general style as detailed in
:ref:`Documentation/process/coding-style.rst <codingstyle>`.
Check for trivial violations with the patch style checker prior to
submission (``scripts/checkpatch.pl``).
You should be able to justify all violations that remain in
your patch.
6) Any new or modified ``CONFIG`` options do not muck up the config menu and
default to off unless they meet the exception criteria documented in
``Documentation/kbuild/kconfig-language.txt`` Menu attributes: default value.
7) All new ``Kconfig`` options have help text.
8) Has been carefully reviewed with respect to relevant ``Kconfig``
combinations. This is very hard to get right with testing -- brainpower
pays off here.
9) Check cleanly with sparse.
10) Use ``make checkstack`` and ``make namespacecheck`` and fix any problems
that they find.
.. note::
``checkstack`` does not point out problems explicitly,
but any one function that uses more than 512 bytes on the stack is a
candidate for change.
11) Include :ref:`kernel-doc <kernel_doc>` to document global kernel APIs.
(Not required for static functions, but OK there also.) Use
``make htmldocs`` or ``make pdfdocs`` to check the
:ref:`kernel-doc <kernel_doc>` and fix any issues.
12) Has been tested with ``CONFIG_PREEMPT``, ``CONFIG_DEBUG_PREEMPT``,
``CONFIG_DEBUG_SLAB``, ``CONFIG_DEBUG_PAGEALLOC``, ``CONFIG_DEBUG_MUTEXES``,
``CONFIG_DEBUG_SPINLOCK``, ``CONFIG_DEBUG_ATOMIC_SLEEP``,
``CONFIG_PROVE_RCU`` and ``CONFIG_DEBUG_OBJECTS_RCU_HEAD`` all
simultaneously enabled.
13) Has been build- and runtime tested with and without ``CONFIG_SMP`` and
``CONFIG_PREEMPT.``
14) If the patch affects IO/Disk, etc: has been tested with and without
``CONFIG_LBDAF.``
15) All codepaths have been exercised with all lockdep features enabled.
16) All new ``/proc`` entries are documented under ``Documentation/``
17) All new kernel boot parameters are documented in
``Documentation/admin-guide/kernel-parameters.rst``.
18) All new module parameters are documented with ``MODULE_PARM_DESC()``
19) All new userspace interfaces are documented in ``Documentation/ABI/``.
See ``Documentation/ABI/README`` for more information.
Patches that change userspace interfaces should be CCed to
linux-api@vger.kernel.org.
20) Check that it all passes ``make headers_check``.
21) Has been checked with injection of at least slab and page-allocation
failures. See ``Documentation/fault-injection/``.
If the new code is substantial, addition of subsystem-specific fault
injection might be appropriate.
22) Newly-added code has been compiled with ``gcc -W`` (use
``make EXTRA_CFLAGS=-W``). This will generate lots of noise, but is good
for finding bugs like "warning: comparison between signed and unsigned".
23) Tested after it has been merged into the -mm patchset to make sure
that it still works with all of the other queued patches and various
changes in the VM, VFS, and other subsystems.
24) All memory barriers {e.g., ``barrier()``, ``rmb()``, ``wmb()``} need a
comment in the source code that explains the logic of what they are doing
and why.
25) If any ioctl's are added by the patch, then also update
``Documentation/ioctl/ioctl-number.txt``.
26) If your modified source code depends on or uses any of the kernel
APIs or features that are related to the following ``Kconfig`` symbols,
then test multiple builds with the related ``Kconfig`` symbols disabled
and/or ``=m`` (if that option is available) [not all of these at the
same time, just various/random combinations of them]:
``CONFIG_SMP``, ``CONFIG_SYSFS``, ``CONFIG_PROC_FS``, ``CONFIG_INPUT``, ``CONFIG_PCI``, ``CONFIG_BLOCK``, ``CONFIG_PM``, ``CONFIG_MAGIC_SYSRQ``,
``CONFIG_NET``, ``CONFIG_INET=n`` (but latter with ``CONFIG_NET=y``).
Documentation/process/submitting-drivers.rst
+191
View File
@@ -0,0 +1,191 @@
.. _submittingdrivers:
Submitting Drivers For The Linux Kernel
=======================================
This document is intended to explain how to submit device drivers to the
various kernel trees. Note that if you are interested in video card drivers
you should probably talk to XFree86 (http://www.xfree86.org/) and/or X.Org
(http://x.org/) instead.
.. note::
This document is old and has seen little maintenance in recent years; it
should probably be updated or, perhaps better, just deleted. Most of
what is here can be found in the other development documents anyway.
Oh, and we don't really recommend submitting changes to XFree86 :)
Also read the Documentation/process/submitting-patches.rst document.
Allocating Device Numbers
-------------------------
Major and minor numbers for block and character devices are allocated
by the Linux assigned name and number authority (currently this is
Torben Mathiasen). The site is http://www.lanana.org/. This
also deals with allocating numbers for devices that are not going to
be submitted to the mainstream kernel.
See Documentation/admin-guide/devices.rst for more information on this.
If you don't use assigned numbers then when your device is submitted it will
be given an assigned number even if that is different from values you may
have shipped to customers before.
Who To Submit Drivers To
------------------------
Linux 2.0:
No new drivers are accepted for this kernel tree.
Linux 2.2:
No new drivers are accepted for this kernel tree.
Linux 2.4:
If the code area has a general maintainer then please submit it to
the maintainer listed in MAINTAINERS in the kernel file. If the
maintainer does not respond or you cannot find the appropriate
maintainer then please contact Willy Tarreau <w@1wt.eu>.
Linux 2.6 and upper:
The same rules apply as 2.4 except that you should follow linux-kernel
to track changes in API's. The final contact point for Linux 2.6+
submissions is Andrew Morton.
What Criteria Determine Acceptance
----------------------------------
Licensing:
The code must be released to us under the
GNU General Public License. We don't insist on any kind
of exclusive GPL licensing, and if you wish the driver
to be useful to other communities such as BSD you may well
wish to release under multiple licenses.
See accepted licenses at include/linux/module.h
Copyright:
The copyright owner must agree to use of GPL.
It's best if the submitter and copyright owner
are the same person/entity. If not, the name of
the person/entity authorizing use of GPL should be
listed in case it's necessary to verify the will of
the copyright owner.
Interfaces:
If your driver uses existing interfaces and behaves like
other drivers in the same class it will be much more likely
to be accepted than if it invents gratuitous new ones.
If you need to implement a common API over Linux and NT
drivers do it in userspace.
Code:
Please use the Linux style of code formatting as documented
in :ref:`Documentation/process/coding-style.rst <codingStyle>`.
If you have sections of code
that need to be in other formats, for example because they
are shared with a windows driver kit and you want to
maintain them just once separate them out nicely and note
this fact.
Portability:
Pointers are not always 32bits, not all computers are little
endian, people do not all have floating point and you
shouldn't use inline x86 assembler in your driver without
careful thought. Pure x86 drivers generally are not popular.
If you only have x86 hardware it is hard to test portability
but it is easy to make sure the code can easily be made
portable.
Clarity:
It helps if anyone can see how to fix the driver. It helps
you because you get patches not bug reports. If you submit a
driver that intentionally obfuscates how the hardware works
it will go in the bitbucket.
PM support:
Since Linux is used on many portable and desktop systems, your
driver is likely to be used on such a system and therefore it
should support basic power management by implementing, if
necessary, the .suspend and .resume methods used during the
system-wide suspend and resume transitions. You should verify
that your driver correctly handles the suspend and resume, but
if you are unable to ensure that, please at least define the
.suspend method returning the -ENOSYS ("Function not
implemented") error. You should also try to make sure that your
driver uses as little power as possible when it's not doing
anything. For the driver testing instructions see
Documentation/power/drivers-testing.txt and for a relatively
complete overview of the power management issues related to
drivers see Documentation/driver-api/pm/devices.rst.
Control:
In general if there is active maintenance of a driver by
the author then patches will be redirected to them unless
they are totally obvious and without need of checking.
If you want to be the contact and update point for the
driver it is a good idea to state this in the comments,
and include an entry in MAINTAINERS for your driver.
What Criteria Do Not Determine Acceptance
-----------------------------------------
Vendor:
Being the hardware vendor and maintaining the driver is
often a good thing. If there is a stable working driver from
other people already in the tree don't expect 'we are the
vendor' to get your driver chosen. Ideally work with the
existing driver author to build a single perfect driver.
Author:
It doesn't matter if a large Linux company wrote the driver,
or you did. Nobody has any special access to the kernel
tree. Anyone who tells you otherwise isn't telling the
whole story.
Resources
---------
Linux kernel master tree:
ftp.\ *country_code*\ .kernel.org:/pub/linux/kernel/...
where *country_code* == your country code, such as
**us**, **uk**, **fr**, etc.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git
Linux kernel mailing list:
linux-kernel@vger.kernel.org
[mail majordomo@vger.kernel.org to subscribe]
Linux Device Drivers, Third Edition (covers 2.6.10):
http://lwn.net/Kernel/LDD3/ (free version)
LWN.net:
Weekly summary of kernel development activity - http://lwn.net/
2.6 API changes:
http://lwn.net/Articles/2.6-kernel-api/
Porting drivers from prior kernels to 2.6:
http://lwn.net/Articles/driver-porting/
KernelNewbies:
Documentation and assistance for new kernel programmers
http://kernelnewbies.org/
Linux USB project:
http://www.linux-usb.org/
How to NOT write kernel driver by Arjan van de Ven:
http://www.fenrus.org/how-to-not-write-a-device-driver-paper.pdf
Kernel Janitor:
http://kernelnewbies.org/KernelJanitors
GIT, Fast Version Control System:
http://git-scm.com/
Documentation/process/submitting-patches.rst
+842
View File
@@ -0,0 +1,842 @@
.. _submittingpatches:
Submitting patches: the essential guide to getting your code into the kernel
============================================================================
For a person or company who wishes to submit a change to the Linux
kernel, the process can sometimes be daunting if you're not familiar
with "the system." This text is a collection of suggestions which
can greatly increase the chances of your change being accepted.
This document contains a large number of suggestions in a relatively terse
format. For detailed information on how the kernel development process
works, see :ref:`Documentation/process <development_process_main>`.
Also, read :ref:`Documentation/process/submit-checklist.rst <submitchecklist>`
for a list of items to check before
submitting code. If you are submitting a driver, also read
:ref:`Documentation/process/submitting-drivers.rst <submittingdrivers>`;
for device tree binding patches, read
Documentation/devicetree/bindings/submitting-patches.txt.
Many of these steps describe the default behavior of the ``git`` version
control system; if you use ``git`` to prepare your patches, you'll find much
of the mechanical work done for you, though you'll still need to prepare
and document a sensible set of patches. In general, use of ``git`` will make
your life as a kernel developer easier.
0) Obtain a current source tree
-------------------------------
If you do not have a repository with the current kernel source handy, use
``git`` to obtain one. You'll want to start with the mainline repository,
which can be grabbed with::
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Note, however, that you may not want to develop against the mainline tree
directly. Most subsystem maintainers run their own trees and want to see
patches prepared against those trees. See the **T:** entry for the subsystem
in the MAINTAINERS file to find that tree, or simply ask the maintainer if
the tree is not listed there.
It is still possible to download kernel releases via tarballs (as described
in the next section), but that is the hard way to do kernel development.
1) ``diff -up``
---------------
If you must generate your patches by hand, use ``diff -up`` or ``diff -uprN``
to create patches. Git generates patches in this form by default; if
you're using ``git``, you can skip this section entirely.
All changes to the Linux kernel occur in the form of patches, as
generated by :manpage:`diff(1)`. When creating your patch, make sure to
create it in "unified diff" format, as supplied by the ``-u`` argument
to :manpage:`diff(1)`.
Also, please use the ``-p`` argument which shows which C function each
change is in - that makes the resultant ``diff`` a lot easier to read.
Patches should be based in the root kernel source directory,
not in any lower subdirectory.
To create a patch for a single file, it is often sufficient to do::
SRCTREE= linux
MYFILE= drivers/net/mydriver.c
cd $SRCTREE
cp $MYFILE $MYFILE.orig
vi $MYFILE # make your change
cd ..
diff -up $SRCTREE/$MYFILE{.orig,} > /tmp/patch
To create a patch for multiple files, you should unpack a "vanilla",
or unmodified kernel source tree, and generate a ``diff`` against your
own source tree. For example::
MYSRC= /devel/linux
tar xvfz linux-3.19.tar.gz
mv linux-3.19 linux-3.19-vanilla
diff -uprN -X linux-3.19-vanilla/Documentation/dontdiff \
linux-3.19-vanilla $MYSRC > /tmp/patch
``dontdiff`` is a list of files which are generated by the kernel during
the build process, and should be ignored in any :manpage:`diff(1)`-generated
patch.
Make sure your patch does not include any extra files which do not
belong in a patch submission. Make sure to review your patch -after-
generating it with :manpage:`diff(1)`, to ensure accuracy.
If your changes produce a lot of deltas, you need to split them into
individual patches which modify things in logical stages; see
:ref:`split_changes`. This will facilitate review by other kernel developers,
very important if you want your patch accepted.
If you're using ``git``, ``git rebase -i`` can help you with this process. If
you're not using ``git``, ``quilt`` <http://savannah.nongnu.org/projects/quilt>
is another popular alternative.
.. _describe_changes:
2) Describe your changes
------------------------
Describe your problem. Whether your patch is a one-line bug fix or
5000 lines of a new feature, there must be an underlying problem that
motivated you to do this work. Convince the reviewer that there is a
problem worth fixing and that it makes sense for them to read past the
first paragraph.
Describe user-visible impact. Straight up crashes and lockups are
pretty convincing, but not all bugs are that blatant. Even if the
problem was spotted during code review, describe the impact you think
it can have on users. Keep in mind that the majority of Linux
installations run kernels from secondary stable trees or
vendor/product-specific trees that cherry-pick only specific patches
from upstream, so include anything that could help route your change
downstream: provoking circumstances, excerpts from dmesg, crash
descriptions, performance regressions, latency spikes, lockups, etc.
Quantify optimizations and trade-offs. If you claim improvements in
performance, memory consumption, stack footprint, or binary size,
include numbers that back them up. But also describe non-obvious
costs. Optimizations usually aren't free but trade-offs between CPU,
memory, and readability; or, when it comes to heuristics, between
different workloads. Describe the expected downsides of your
optimization so that the reviewer can weigh costs against benefits.
Once the problem is established, describe what you are actually doing
about it in technical detail. It's important to describe the change
in plain English for the reviewer to verify that the code is behaving
as you intend it to.
The maintainer will thank you if you write your patch description in a
form which can be easily pulled into Linux's source code management
system, ``git``, as a "commit log". See :ref:`explicit_in_reply_to`.
Solve only one problem per patch. If your description starts to get
long, that's a sign that you probably need to split up your patch.
See :ref:`split_changes`.
When you submit or resubmit a patch or patch series, include the
complete patch description and justification for it. Don't just
say that this is version N of the patch (series). Don't expect the
subsystem maintainer to refer back to earlier patch versions or referenced
URLs to find the patch description and put that into the patch.
I.e., the patch (series) and its description should be self-contained.
This benefits both the maintainers and reviewers. Some reviewers
probably didn't even receive earlier versions of the patch.
Describe your changes in imperative mood, e.g. "make xyzzy do frotz"
instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy
to do frotz", as if you are giving orders to the codebase to change
its behaviour.
If the patch fixes a logged bug entry, refer to that bug entry by
number and URL. If the patch follows from a mailing list discussion,
give a URL to the mailing list archive; use the https://lkml.kernel.org/
redirector with a ``Message-Id``, to ensure that the links cannot become
stale.
However, try to make your explanation understandable without external
resources. In addition to giving a URL to a mailing list archive or
bug, summarize the relevant points of the discussion that led to the
patch as submitted.
If you want to refer to a specific commit, don't just refer to the
SHA-1 ID of the commit. Please also include the oneline summary of
the commit, to make it easier for reviewers to know what it is about.
Example::
Commit e21d2170f36602ae2708 ("video: remove unnecessary
platform_set_drvdata()") removed the unnecessary
platform_set_drvdata(), but left the variable "dev" unused,
delete it.
You should also be sure to use at least the first twelve characters of the
SHA-1 ID. The kernel repository holds a *lot* of objects, making
collisions with shorter IDs a real possibility. Bear in mind that, even if
there is no collision with your six-character ID now, that condition may
change five years from now.
If your patch fixes a bug in a specific commit, e.g. you found an issue using
``git bisect``, please use the 'Fixes:' tag with the first 12 characters of
the SHA-1 ID, and the one line summary. For example::
Fixes: e21d2170f366 ("video: remove unnecessary platform_set_drvdata()")
The following ``git config`` settings can be used to add a pretty format for
outputting the above style in the ``git log`` or ``git show`` commands::
[core]
abbrev = 12
[pretty]
fixes = Fixes: %h (\"%s\")
.. _split_changes:
3) Separate your changes
------------------------
Separate each **logical change** into a separate patch.
For example, if your changes include both bug fixes and performance
enhancements for a single driver, separate those changes into two
or more patches. If your changes include an API update, and a new
driver which uses that new API, separate those into two patches.
On the other hand, if you make a single change to numerous files,
group those changes into a single patch. Thus a single logical change
is contained within a single patch.
The point to remember is that each patch should make an easily understood
change that can be verified by reviewers. Each patch should be justifiable
on its own merits.
If one patch depends on another patch in order for a change to be
complete, that is OK. Simply note **"this patch depends on patch X"**
in your patch description.
When dividing your change into a series of patches, take special care to
ensure that the kernel builds and runs properly after each patch in the
series. Developers using ``git bisect`` to track down a problem can end up
splitting your patch series at any point; they will not thank you if you
introduce bugs in the middle.
If you cannot condense your patch set into a smaller set of patches,
then only post say 15 or so at a time and wait for review and integration.
4) Style-check your changes
---------------------------
Check your patch for basic style violations, details of which can be
found in
:ref:`Documentation/process/coding-style.rst <codingstyle>`.
Failure to do so simply wastes
the reviewers time and will get your patch rejected, probably
without even being read.
One significant exception is when moving code from one file to
another -- in this case you should not modify the moved code at all in
the same patch which moves it. This clearly delineates the act of
moving the code and your changes. This greatly aids review of the
actual differences and allows tools to better track the history of
the code itself.
Check your patches with the patch style checker prior to submission
(scripts/checkpatch.pl). Note, though, that the style checker should be
viewed as a guide, not as a replacement for human judgment. If your code
looks better with a violation then its probably best left alone.
The checker reports at three levels:
- ERROR: things that are very likely to be wrong
- WARNING: things requiring careful review
- CHECK: things requiring thought
You should be able to justify all violations that remain in your
patch.
5) Select the recipients for your patch
---------------------------------------
You should always copy the appropriate subsystem maintainer(s) on any patch
to code that they maintain; look through the MAINTAINERS file and the
source code revision history to see who those maintainers are. The
script scripts/get_maintainer.pl can be very useful at this step. If you
cannot find a maintainer for the subsystem you are working on, Andrew
Morton (akpm@linux-foundation.org) serves as a maintainer of last resort.
You should also normally choose at least one mailing list to receive a copy
of your patch set. linux-kernel@vger.kernel.org functions as a list of
last resort, but the volume on that list has caused a number of developers
to tune it out. Look in the MAINTAINERS file for a subsystem-specific
list; your patch will probably get more attention there. Please do not
spam unrelated lists, though.
Many kernel-related lists are hosted on vger.kernel.org; you can find a
list of them at http://vger.kernel.org/vger-lists.html. There are
kernel-related lists hosted elsewhere as well, though.
Do not send more than 15 patches at once to the vger mailing lists!!!
Linus Torvalds is the final arbiter of all changes accepted into the
Linux kernel. His e-mail address is <torvalds@linux-foundation.org>.
He gets a lot of e-mail, and, at this point, very few patches go through
Linus directly, so typically you should do your best to -avoid-
sending him e-mail.
If you have a patch that fixes an exploitable security bug, send that patch
to security@kernel.org. For severe bugs, a short embargo may be considered
to allow distributors to get the patch out to users; in such cases,
obviously, the patch should not be sent to any public lists.
Patches that fix a severe bug in a released kernel should be directed
toward the stable maintainers by putting a line like this::
Cc: stable@vger.kernel.org
into the sign-off area of your patch (note, NOT an email recipient). You
should also read
:ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>`
in addition to this file.
Note, however, that some subsystem maintainers want to come to their own
conclusions on which patches should go to the stable trees. The networking
maintainer, in particular, would rather not see individual developers
adding lines like the above to their patches.
If changes affect userland-kernel interfaces, please send the MAN-PAGES
maintainer (as listed in the MAINTAINERS file) a man-pages patch, or at
least a notification of the change, so that some information makes its way
into the manual pages. User-space API changes should also be copied to
linux-api@vger.kernel.org.
For small patches you may want to CC the Trivial Patch Monkey
trivial@kernel.org which collects "trivial" patches. Have a look
into the MAINTAINERS file for its current manager.
Trivial patches must qualify for one of the following rules:
- Spelling fixes in documentation
- Spelling fixes for errors which could break :manpage:`grep(1)`
- Warning fixes (cluttering with useless warnings is bad)
- Compilation fixes (only if they are actually correct)
- Runtime fixes (only if they actually fix things)
- Removing use of deprecated functions/macros
- Contact detail and documentation fixes
- Non-portable code replaced by portable code (even in arch-specific,
since people copy, as long as it's trivial)
- Any fix by the author/maintainer of the file (ie. patch monkey
in re-transmission mode)
6) No MIME, no links, no compression, no attachments. Just plain text
----------------------------------------------------------------------
Linus and other kernel developers need to be able to read and comment
on the changes you are submitting. It is important for a kernel
developer to be able to "quote" your changes, using standard e-mail
tools, so that they may comment on specific portions of your code.
For this reason, all patches should be submitted by e-mail "inline".
.. warning::
Be wary of your editor's word-wrap corrupting your patch,
if you choose to cut-n-paste your patch.
Do not attach the patch as a MIME attachment, compressed or not.
Many popular e-mail applications will not always transmit a MIME
attachment as plain text, making it impossible to comment on your
code. A MIME attachment also takes Linus a bit more time to process,
decreasing the likelihood of your MIME-attached change being accepted.
Exception: If your mailer is mangling patches then someone may ask
you to re-send them using MIME.
See :ref:`Documentation/process/email-clients.rst <email_clients>`
for hints about configuring your e-mail client so that it sends your patches
untouched.
7) E-mail size
--------------
Large changes are not appropriate for mailing lists, and some
maintainers. If your patch, uncompressed, exceeds 300 kB in size,
it is preferred that you store your patch on an Internet-accessible
server, and provide instead a URL (link) pointing to your patch. But note
that if your patch exceeds 300 kB, it almost certainly needs to be broken up
anyway.
8) Respond to review comments
-----------------------------
Your patch will almost certainly get comments from reviewers on ways in
which the patch can be improved. You must respond to those comments;
ignoring reviewers is a good way to get ignored in return. Review comments
or questions that do not lead to a code change should almost certainly
bring about a comment or changelog entry so that the next reviewer better
understands what is going on.
Be sure to tell the reviewers what changes you are making and to thank them
for their time. Code review is a tiring and time-consuming process, and
reviewers sometimes get grumpy. Even in that case, though, respond
politely and address the problems they have pointed out.
9) Don't get discouraged - or impatient
---------------------------------------
After you have submitted your change, be patient and wait. Reviewers are
busy people and may not get to your patch right away.
Once upon a time, patches used to disappear into the void without comment,
but the development process works more smoothly than that now. You should
receive comments within a week or so; if that does not happen, make sure
that you have sent your patches to the right place. Wait for a minimum of
one week before resubmitting or pinging reviewers - possibly longer during
busy times like merge windows.
10) Include PATCH in the subject
--------------------------------
Due to high e-mail traffic to Linus, and to linux-kernel, it is common
convention to prefix your subject line with [PATCH]. This lets Linus
and other kernel developers more easily distinguish patches from other
e-mail discussions.
11) Sign your work - the Developer's Certificate of Origin
----------------------------------------------------------
To improve tracking of who did what, especially with patches that can
percolate to their final resting place in the kernel through several
layers of maintainers, we've introduced a "sign-off" procedure on
patches that are being emailed around.
The sign-off is a simple line at the end of the explanation for the
patch, which certifies that you wrote it or otherwise have the right to
pass it on as an open-source patch. The rules are pretty simple: if you
can certify the below:
Developer's Certificate of Origin 1.1
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
then you just add a line saying::
Signed-off-by: Random J Developer <random@developer.example.org>
using your real name (sorry, no pseudonyms or anonymous contributions.)
Some people also put extra tags at the end. They'll just be ignored for
now, but you can do this to mark internal company procedures or just
point out some special detail about the sign-off.
If you are a subsystem or branch maintainer, sometimes you need to slightly
modify patches you receive in order to merge them, because the code is not
exactly the same in your tree and the submitters'. If you stick strictly to
rule (c), you should ask the submitter to rediff, but this is a totally
counter-productive waste of time and energy. Rule (b) allows you to adjust
the code, but then it is very impolite to change one submitter's code and
make him endorse your bugs. To solve this problem, it is recommended that
you add a line between the last Signed-off-by header and yours, indicating
the nature of your changes. While there is nothing mandatory about this, it
seems like prepending the description with your mail and/or name, all
enclosed in square brackets, is noticeable enough to make it obvious that
you are responsible for last-minute changes. Example::
Signed-off-by: Random J Developer <random@developer.example.org>
[lucky@maintainer.example.org: struct foo moved from foo.c to foo.h]
Signed-off-by: Lucky K Maintainer <lucky@maintainer.example.org>
This practice is particularly helpful if you maintain a stable branch and
want at the same time to credit the author, track changes, merge the fix,
and protect the submitter from complaints. Note that under no circumstances
can you change the author's identity (the From header), as it is the one
which appears in the changelog.
Special note to back-porters: It seems to be a common and useful practice
to insert an indication of the origin of a patch at the top of the commit
message (just after the subject line) to facilitate tracking. For instance,
here's what we see in a 3.x-stable release::
Date: Tue Oct 7 07:26:38 2014 -0400
libata: Un-break ATA blacklist
commit 1c40279960bcd7d52dbdf1d466b20d24b99176c8 upstream.
And here's what might appear in an older kernel once a patch is backported::
Date: Tue May 13 22:12:27 2008 +0200
wireless, airo: waitbusy() won't delay
[backport of 2.6 commit b7acbdfbd1f277c1eb23f344f899cfa4cd0bf36a]
Whatever the format, this information provides a valuable help to people
tracking your trees, and to people trying to troubleshoot bugs in your
tree.
12) When to use Acked-by:, Cc:, and Co-Developed-by:
-------------------------------------------------------
The Signed-off-by: tag indicates that the signer was involved in the
development of the patch, or that he/she was in the patch's delivery path.
If a person was not directly involved in the preparation or handling of a
patch but wishes to signify and record their approval of it then they can
ask to have an Acked-by: line added to the patch's changelog.
Acked-by: is often used by the maintainer of the affected code when that
maintainer neither contributed to nor forwarded the patch.
Acked-by: is not as formal as Signed-off-by:. It is a record that the acker
has at least reviewed the patch and has indicated acceptance. Hence patch
mergers will sometimes manually convert an acker's "yep, looks good to me"
into an Acked-by: (but note that it is usually better to ask for an
explicit ack).
Acked-by: does not necessarily indicate acknowledgement of the entire patch.
For example, if a patch affects multiple subsystems and has an Acked-by: from
one subsystem maintainer then this usually indicates acknowledgement of just
the part which affects that maintainer's code. Judgement should be used here.
When in doubt people should refer to the original discussion in the mailing
list archives.
If a person has had the opportunity to comment on a patch, but has not
provided such comments, you may optionally add a ``Cc:`` tag to the patch.
This is the only tag which might be added without an explicit action by the
person it names - but it should indicate that this person was copied on the
patch. This tag documents that potentially interested parties
have been included in the discussion.
A Co-Developed-by: states that the patch was also created by another developer
along with the original author. This is useful at times when multiple people
work on a single patch. Note, this person also needs to have a Signed-off-by:
line in the patch as well.
13) Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes:
--------------------------------------------------------------------------
The Reported-by tag gives credit to people who find bugs and report them and it
hopefully inspires them to help us again in the future. Please note that if
the bug was reported in private, then ask for permission first before using the
Reported-by tag.
A Tested-by: tag indicates that the patch has been successfully tested (in
some environment) by the person named. This tag informs maintainers that
some testing has been performed, provides a means to locate testers for
future patches, and ensures credit for the testers.
Reviewed-by:, instead, indicates that the patch has been reviewed and found
acceptable according to the Reviewer's Statement:
Reviewer's statement of oversight
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By offering my Reviewed-by: tag, I state that:
(a) I have carried out a technical review of this patch to
evaluate its appropriateness and readiness for inclusion into
the mainline kernel.
(b) Any problems, concerns, or questions relating to the patch
have been communicated back to the submitter. I am satisfied
with the submitter's response to my comments.
(c) While there may be things that could be improved with this
submission, I believe that it is, at this time, (1) a
worthwhile modification to the kernel, and (2) free of known
issues which would argue against its inclusion.
(d) While I have reviewed the patch and believe it to be sound, I
do not (unless explicitly stated elsewhere) make any
warranties or guarantees that it will achieve its stated
purpose or function properly in any given situation.
A Reviewed-by tag is a statement of opinion that the patch is an
appropriate modification of the kernel without any remaining serious
technical issues. Any interested reviewer (who has done the work) can
offer a Reviewed-by tag for a patch. This tag serves to give credit to
reviewers and to inform maintainers of the degree of review which has been
done on the patch. Reviewed-by: tags, when supplied by reviewers known to
understand the subject area and to perform thorough reviews, will normally
increase the likelihood of your patch getting into the kernel.
A Suggested-by: tag indicates that the patch idea is suggested by the person
named and ensures credit to the person for the idea. Please note that this
tag should not be added without the reporter's permission, especially if the
idea was not posted in a public forum. That said, if we diligently credit our
idea reporters, they will, hopefully, be inspired to help us again in the
future.
A Fixes: tag indicates that the patch fixes an issue in a previous commit. It
is used to make it easy to determine where a bug originated, which can help
review a bug fix. This tag also assists the stable kernel team in determining
which stable kernel versions should receive your fix. This is the preferred
method for indicating a bug fixed by the patch. See :ref:`describe_changes`
for more details.
.. _the_canonical_patch_format:
14) The canonical patch format
------------------------------
This section describes how the patch itself should be formatted. Note
that, if you have your patches stored in a ``git`` repository, proper patch
formatting can be had with ``git format-patch``. The tools cannot create
the necessary text, though, so read the instructions below anyway.
The canonical patch subject line is::
Subject: [PATCH 001/123] subsystem: summary phrase
The canonical patch message body contains the following:
- A ``from`` line specifying the patch author, followed by an empty
line (only needed if the person sending the patch is not the author).
- The body of the explanation, line wrapped at 75 columns, which will
be copied to the permanent changelog to describe this patch.
- An empty line.
- The ``Signed-off-by:`` lines, described above, which will
also go in the changelog.
- A marker line containing simply ``---``.
- Any additional comments not suitable for the changelog.
- The actual patch (``diff`` output).
The Subject line format makes it very easy to sort the emails
alphabetically by subject line - pretty much any email reader will
support that - since because the sequence number is zero-padded,
the numerical and alphabetic sort is the same.
The ``subsystem`` in the email's Subject should identify which
area or subsystem of the kernel is being patched.
The ``summary phrase`` in the email's Subject should concisely
describe the patch which that email contains. The ``summary
phrase`` should not be a filename. Do not use the same ``summary
phrase`` for every patch in a whole patch series (where a ``patch
series`` is an ordered sequence of multiple, related patches).
Bear in mind that the ``summary phrase`` of your email becomes a
globally-unique identifier for that patch. It propagates all the way
into the ``git`` changelog. The ``summary phrase`` may later be used in
developer discussions which refer to the patch. People will want to
google for the ``summary phrase`` to read discussion regarding that
patch. It will also be the only thing that people may quickly see
when, two or three months later, they are going through perhaps
thousands of patches using tools such as ``gitk`` or ``git log
--oneline``.
For these reasons, the ``summary`` must be no more than 70-75
characters, and it must describe both what the patch changes, as well
as why the patch might be necessary. It is challenging to be both
succinct and descriptive, but that is what a well-written summary
should do.
The ``summary phrase`` may be prefixed by tags enclosed in square
brackets: "Subject: [PATCH <tag>...] <summary phrase>". The tags are
not considered part of the summary phrase, but describe how the patch
should be treated. Common tags might include a version descriptor if
the multiple versions of the patch have been sent out in response to
comments (i.e., "v1, v2, v3"), or "RFC" to indicate a request for
comments. If there are four patches in a patch series the individual
patches may be numbered like this: 1/4, 2/4, 3/4, 4/4. This assures
that developers understand the order in which the patches should be
applied and that they have reviewed or applied all of the patches in
the patch series.
A couple of example Subjects::
Subject: [PATCH 2/5] ext2: improve scalability of bitmap searching
Subject: [PATCH v2 01/27] x86: fix eflags tracking
The ``from`` line must be the very first line in the message body,
and has the form:
From: Original Author <author@example.com>
The ``from`` line specifies who will be credited as the author of the
patch in the permanent changelog. If the ``from`` line is missing,
then the ``From:`` line from the email header will be used to determine
the patch author in the changelog.
The explanation body will be committed to the permanent source
changelog, so should make sense to a competent reader who has long
since forgotten the immediate details of the discussion that might
have led to this patch. Including symptoms of the failure which the
patch addresses (kernel log messages, oops messages, etc.) is
especially useful for people who might be searching the commit logs
looking for the applicable patch. If a patch fixes a compile failure,
it may not be necessary to include _all_ of the compile failures; just
enough that it is likely that someone searching for the patch can find
it. As in the ``summary phrase``, it is important to be both succinct as
well as descriptive.
The ``---`` marker line serves the essential purpose of marking for patch
handling tools where the changelog message ends.
One good use for the additional comments after the ``---`` marker is for
a ``diffstat``, to show what files have changed, and the number of
inserted and deleted lines per file. A ``diffstat`` is especially useful
on bigger patches. Other comments relevant only to the moment or the
maintainer, not suitable for the permanent changelog, should also go
here. A good example of such comments might be ``patch changelogs``
which describe what has changed between the v1 and v2 version of the
patch.
If you are going to include a ``diffstat`` after the ``---`` marker, please
use ``diffstat`` options ``-p 1 -w 70`` so that filenames are listed from
the top of the kernel source tree and don't use too much horizontal
space (easily fit in 80 columns, maybe with some indentation). (``git``
generates appropriate diffstats by default.)
See more details on the proper patch format in the following
references.
.. _explicit_in_reply_to:
15) Explicit In-Reply-To headers
--------------------------------
It can be helpful to manually add In-Reply-To: headers to a patch
(e.g., when using ``git send-email``) to associate the patch with
previous relevant discussion, e.g. to link a bug fix to the email with
the bug report. However, for a multi-patch series, it is generally
best to avoid using In-Reply-To: to link to older versions of the
series. This way multiple versions of the patch don't become an
unmanageable forest of references in email clients. If a link is
helpful, you can use the https://lkml.kernel.org/ redirector (e.g., in
the cover email text) to link to an earlier version of the patch series.
16) Sending ``git pull`` requests
---------------------------------
If you have a series of patches, it may be most convenient to have the
maintainer pull them directly into the subsystem repository with a
``git pull`` operation. Note, however, that pulling patches from a developer
requires a higher degree of trust than taking patches from a mailing list.
As a result, many subsystem maintainers are reluctant to take pull
requests, especially from new, unknown developers. If in doubt you can use
the pull request as the cover letter for a normal posting of the patch
series, giving the maintainer the option of using either.
A pull request should have [GIT PULL] in the subject line. The
request itself should include the repository name and the branch of
interest on a single line; it should look something like::
Please pull from
git://jdelvare.pck.nerim.net/jdelvare-2.6 i2c-for-linus
to get these changes:
A pull request should also include an overall message saying what will be
included in the request, a ``git shortlog`` listing of the patches
themselves, and a ``diffstat`` showing the overall effect of the patch series.
The easiest way to get all this information together is, of course, to let
``git`` do it for you with the ``git request-pull`` command.
Some maintainers (including Linus) want to see pull requests from signed
commits; that increases their confidence that the request actually came
from you. Linus, in particular, will not pull from public hosting sites
like GitHub in the absence of a signed tag.
The first step toward creating such tags is to make a GNUPG key and get it
signed by one or more core kernel developers. This step can be hard for
new developers, but there is no way around it. Attending conferences can
be a good way to find developers who can sign your key.
Once you have prepared a patch series in ``git`` that you wish to have somebody
pull, create a signed tag with ``git tag -s``. This will create a new tag
identifying the last commit in the series and containing a signature
created with your private key. You will also have the opportunity to add a
changelog-style message to the tag; this is an ideal place to describe the
effects of the pull request as a whole.
If the tree the maintainer will be pulling from is not the repository you
are working from, don't forget to push the signed tag explicitly to the
public tree.
When generating your pull request, use the signed tag as the target. A
command like this will do the trick::
git request-pull master git://my.public.tree/linux.git my-signed-tag
References
----------
Andrew Morton, "The perfect patch" (tpp).
<http://www.ozlabs.org/~akpm/stuff/tpp.txt>
Jeff Garzik, "Linux kernel patch submission format".
<http://linux.yyz.us/patch-format.html>
Greg Kroah-Hartman, "How to piss off a kernel subsystem maintainer".
<http://www.kroah.com/log/linux/maintainer.html>
<http://www.kroah.com/log/linux/maintainer-02.html>
<http://www.kroah.com/log/linux/maintainer-03.html>
<http://www.kroah.com/log/linux/maintainer-04.html>
<http://www.kroah.com/log/linux/maintainer-05.html>
<http://www.kroah.com/log/linux/maintainer-06.html>
NO!!!! No more huge patch bombs to linux-kernel@vger.kernel.org people!
<https://lkml.org/lkml/2005/7/11/336>
Kernel Documentation/process/coding-style.rst:
:ref:`Documentation/process/coding-style.rst <codingstyle>`
Linus Torvalds's mail on the canonical patch format:
<http://lkml.org/lkml/2005/4/7/183>
Andi Kleen, "On submitting kernel patches"
Some strategies to get difficult or controversial changes in.
http://halobates.de/on-submitting-patches.pdf
Documentation/process/volatile-considered-harmful.rst
+125
View File
@@ -0,0 +1,125 @@
.. _volatile_considered_harmful:
Why the "volatile" type class should not be used
------------------------------------------------
C programmers have often taken volatile to mean that the variable could be
changed outside of the current thread of execution; as a result, they are
sometimes tempted to use it in kernel code when shared data structures are
being used. In other words, they have been known to treat volatile types
as a sort of easy atomic variable, which they are not. The use of volatile in
kernel code is almost never correct; this document describes why.
The key point to understand with regard to volatile is that its purpose is
to suppress optimization, which is almost never what one really wants to
do. In the kernel, one must protect shared data structures against
unwanted concurrent access, which is very much a different task. The
process of protecting against unwanted concurrency will also avoid almost
all optimization-related problems in a more efficient way.
Like volatile, the kernel primitives which make concurrent access to data
safe (spinlocks, mutexes, memory barriers, etc.) are designed to prevent
unwanted optimization. If they are being used properly, there will be no
need to use volatile as well. If volatile is still necessary, there is
almost certainly a bug in the code somewhere. In properly-written kernel
code, volatile can only serve to slow things down.
Consider a typical block of kernel code::
spin_lock(&the_lock);
do_something_on(&shared_data);
do_something_else_with(&shared_data);
spin_unlock(&the_lock);
If all the code follows the locking rules, the value of shared_data cannot
change unexpectedly while the_lock is held. Any other code which might
want to play with that data will be waiting on the lock. The spinlock
primitives act as memory barriers - they are explicitly written to do so -
meaning that data accesses will not be optimized across them. So the
compiler might think it knows what will be in shared_data, but the
spin_lock() call, since it acts as a memory barrier, will force it to
forget anything it knows. There will be no optimization problems with
accesses to that data.
If shared_data were declared volatile, the locking would still be
necessary. But the compiler would also be prevented from optimizing access
to shared_data _within_ the critical section, when we know that nobody else
can be working with it. While the lock is held, shared_data is not
volatile. When dealing with shared data, proper locking makes volatile
unnecessary - and potentially harmful.
The volatile storage class was originally meant for memory-mapped I/O
registers. Within the kernel, register accesses, too, should be protected
by locks, but one also does not want the compiler "optimizing" register
accesses within a critical section. But, within the kernel, I/O memory
accesses are always done through accessor functions; accessing I/O memory
directly through pointers is frowned upon and does not work on all
architectures. Those accessors are written to prevent unwanted
optimization, so, once again, volatile is unnecessary.
Another situation where one might be tempted to use volatile is
when the processor is busy-waiting on the value of a variable. The right
way to perform a busy wait is::
while (my_variable != what_i_want)
cpu_relax();
The cpu_relax() call can lower CPU power consumption or yield to a
hyperthreaded twin processor; it also happens to serve as a compiler
barrier, so, once again, volatile is unnecessary. Of course, busy-
waiting is generally an anti-social act to begin with.
There are still a few rare situations where volatile makes sense in the
kernel:
- The above-mentioned accessor functions might use volatile on
architectures where direct I/O memory access does work. Essentially,
each accessor call becomes a little critical section on its own and
ensures that the access happens as expected by the programmer.
- Inline assembly code which changes memory, but which has no other
visible side effects, risks being deleted by GCC. Adding the volatile
keyword to asm statements will prevent this removal.
- The jiffies variable is special in that it can have a different value
every time it is referenced, but it can be read without any special
locking. So jiffies can be volatile, but the addition of other
variables of this type is strongly frowned upon. Jiffies is considered
to be a "stupid legacy" issue (Linus's words) in this regard; fixing it
would be more trouble than it is worth.
- Pointers to data structures in coherent memory which might be modified
by I/O devices can, sometimes, legitimately be volatile. A ring buffer
used by a network adapter, where that adapter changes pointers to
indicate which descriptors have been processed, is an example of this
type of situation.
For most code, none of the above justifications for volatile apply. As a
result, the use of volatile is likely to be seen as a bug and will bring
additional scrutiny to the code. Developers who are tempted to use
volatile should take a step back and think about what they are truly trying
to accomplish.
Patches to remove volatile variables are generally welcome - as long as
they come with a justification which shows that the concurrency issues have
been properly thought through.
References
==========
[1] http://lwn.net/Articles/233481/
[2] http://lwn.net/Articles/233482/
Credits
=======
Original impetus and research by Randy Dunlap
Written by Jonathan Corbet
Improvements via comments from Satyam Sharma, Johannes Stezenbach, Jesper
Juhl, Heikki Orsila, H. Peter Anvin, Philipp Hahn, and Stefan
Richter.