Conventions — Plone Documentation v4.3

Introduction ¶

We've modeled the following rules and recommendations based on the following documents:

PEP8

PEP257

Rope project

Google Style Guide

Pylons Coding Style

Tim Pope on Git commit messages

All Python code in this package should be PEP8 valid. This includes adhering to the 80-char line length. If you absolutely need to break this rule, append # noPEP8 to the offending line to skip it in syntax checks.

Note

Configuring your editor to display a line at 79th column helps a lot here and saves time.

Note

The line length rule also applies to non-python source files, such as .zcml files, but is a bit more relaxed there. It explicitly does not aply to documentation .rst files. For rst files, use semantic linebreaks. See the Plone rst styleguide for the reasoning behind it.

Breaking lines ¶

Based on code we love to look at (Pyramid, Requests, etc.), we allow the following two styles for breaking long lines into blocks:

Break into next line with one additional indent block.

                              foo = do_something(
    very_long_argument='foo', another_very_long_argument='bar')

# For functions the ): needs to be placed on the following line
def some_func(
    very_long_argument='foo', another_very_long_argument='bar'
):

                            

If this still doesn't fit the 80-char limit, break into multiple lines.

                              foo = dict(
    very_long_argument='foo',
    another_very_long_argument='bar',
)

a_long_list = [
    "a_fairly_long_string",
    "quite_a_long_string_indeed",
    "an_exceptionally_long_string_of_characters",
]

                            

Arguments on first line, directly after the opening parenthesis are forbidden when breaking lines.

The last argument line needs to have a trailing comma (to be nice to the next developer coming in to add something as an argument and minimize VCS diffs in these cases).

The closing parenthesis or bracket needs to have the same indentation level as the first line.

Each line can only contain a single argument.

The same style applies to dicts, lists, return calls, etc.

This package follows all rules above, check out the source to see them in action.

autopep8 ¶

Making old code pep8 compliant can be a lot of work. There is a tool that can automatically do some of this work for you: autopep8. This fixes various issues, for example fixing indentation to be a multiple of four. Just install it with pip and call it like this:

                          pip install autopep8
autopep8 -i filename.py
autopep8 -i -r directory

                        

It is best to first run autopep8 in the default non aggressive mode, which means it only does whitespace changes. To run this recursively on the current directory, changing files in place:

autopep8 -i -r .

Quickly check the changes and then commit them.

WARNING: be very careful when running this in a skins directory, if you run it there at all. It will make changes to the top of the file like this, which completely breaks the skin script:

                          -##parameters=policy_in=''
+# parameters=policy_in=''

With those safe changes out of the way, you can move on to a second, more aggresive round:

autopep8 -i --aggressive -r .

Check these changes more thoroughly. At the very least check if Plone can still start in the foreground and that there are no failures or errors in the tests.

Not all changes are always safe. You can ignore some checks:

autopep8 -i --ignore W690,E711,E721 --aggressive -r .

This skips the following changes:

W690: Fix various deprecated code (via lib2to3). (Can be bad for Python 2.4.)
E721: Use isinstance() instead of comparing types directly. (There are uses of this in for example GenericSetup and plone.api that must not be fixed.)
E711: Fix comparison with None. (This can break SQLAlchemy code.)

You can check what would be changed by one specific code:

autopep8 --diff --select E309 -r .

Indentation ¶

For Python files, we stick with the PEP 8 recommondation: Use 4 spaces per indentation level.

For ZCML and XML (GenericSetup) files, we recommend the Zope Toolkit's coding style on ZCML

                        Indentation of 2 characters to show nesting, 4 characters to list attributes
on separate lines. This distinction makes it easier to see the difference
between attributes and nested elements.

                      

EditorConfig ¶

EditorConfig provides a way to share the same configuration for all major source code editors.

You only need to install the plugin for your editor of choice, and add the following configuration on ~/.editorconfig.

                          [*]
indent_style = space
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8

[{*.py,*.cfg}]
indent_size = 4

[{*.html,*.dtml,*.pt,*.zpt,*.xml,*.zcml,*.js}]
indent_size = 2

[Makefile]
indent_style = tab

                        

Quoting ¶

For strings and such prefer using single quotes over double quotes. The reason is that sometimes you do need to write a bit of HTML in your python code, and HTML feels more natural with double quotes so you wrap HTML string into single quotes. And if you are using single quotes for this reason, then be consistent and use them everywhere.

There are two exceptions to this rule:

docstrings should always use double quotes (as per PEP-257).
if you want to use single quotes in your string, double quotes might make more sense so you don't have to escape those single quotes.

                        # GOOD
print 'short'
print 'A longer string, but still using single quotes.'

# BAD
print "short"
print "A long string."

# EXCEPTIONS
print "I want to use a 'single quote' in my string."
"""This is a docstring."""

                      

Docstrings style ¶

Read and follow http://www.python.org/dev/peps/pep-0257/. There is one exception though: We reject BDFL's recommendation about inserting a blank line between the last paragraph in a multi-line docstring and its closing quotes as it's Emacs specific and two Emacs users here on the Beer & Wine Sprint both support our way.

The content of the docstring must be written in the active first-person form, e.g. "Calculate X from Y" or "Determine the exact foo of bar".

                        def foo():
    """Single line docstring."""

def bar():
    """Multi-line docstring.

    With the additional lines indented with the beginning quote and a
    newline preceding the ending quote.
    """

                      

If you wanna be extra nice, you are encouraged to document your method's parameters and their return values in a reST field list syntax.

                        :param foo: blah blah
:type foo: string
:param bar: blah blah
:type bar: int
:returns: something

                      

Check out the plone.api source for more usage examples. Also, see the following for examples on how to write good Sphinxy docstrings: http://stackoverflow.com/questions/4547849/good-examples-of-python-docstrings-for-sphinx.

Unit tests style ¶

Read http://www.voidspace.org.uk/python/articles/unittest2.shtml to learn what is new in unittest2 and use it.

This is not true for in-line documentation tests. Those still use old unittest test-cases, so you cannot use assertIn and similar.

String formatting ¶

As per http://docs.python.org/2/library/stdtypes.html#str.format, we should prefer the new style string formatting (.format()) over the old one (% ()).

Also use numbering, like so:

                        # GOOD
print "{0} is not {1}".format(1, 2)

and not like this:

                        # BAD
print "{} is not {}".format(1, 2)
print "%s is not %s" % (1, 2)

                      

because Python 2.6 supports only explicitly numbered placeholders.

About imports ¶

Don't use * to import everything from a module, because if you do, pyflakes cannot detect undefined names (W404).
Don't use commas to import multiple things on a single line. Some developers use IDEs (like Eclipse) or tools (such as mr.igor) that expect one import per line. Let's be nice to them.

Don't use relative paths, again to be nice to people using certain IDEs and tools. Also Google Python Style Guide recommends against it.

                            # GOOD
from plone.app.testing import something
from zope.component import getMultiAdapter
from zope.component import getSiteManager

                          

instead of

                            # BAD
from plone.app.testing import *
from zope.component import getMultiAdapter, getSiteManager

                          

Don't catch ImportError to detect whether a package is available or not, as it might hide circular import errors. Instead, use pkg_resources.get_distribution and catch DistributionNotFound. More background at http://do3.cc/blog/2010/08/20/do-not-catch-import-errors,-use-pkg_resources/.

                            # GOOD
import pkg_resources

try:
    pkg_resources.get_distribution('plone.dexterity')
except pkg_resources.DistributionNotFound:
    HAS_DEXTERITY = False
else:
    HAS_DEXTERITY = True

                          

instead of

                            # BAD
try:
    import plone.dexterity
    HAVE_DEXTERITY = True
except ImportError:
    HAVE_DEXTERITY = False

                          

Grouping and sorting ¶

Since Plone has such a huge code base, we don't want to lose developer time figuring out into which group some import goes (standard lib?, external package?, etc.). So we just sort everything alphabetically and insert one blank line between from foo import bar and import baz blocks. Conditional imports come last. Again, we do not distinguish between what is standard lib, external package or internal package in order to save time and avoid the hassle of explaining which is which.

                          # GOOD
from __future__ import division
from Acquisition import aq_inner
from plone.api import portal
from plone.api.exc import MissingParameterError
from Products.CMFCore.interfaces import ISiteRoot
from Products.CMFCore.WorkflowCore import WorkflowException

import pkg_resources
import random

try:
    pkg_resources.get_distribution('plone.dexterity')
except pkg_resources.DistributionNotFound:
    HAS_DEXTERITY = False
else:
    HAS_DEXTERITY = True

                        

Declaring dependencies ¶

All direct dependencies should be declared in install_requires or extras_require sections in setup.py. Dependencies, which are not needed for a production environment (like "develop" or "test" dependencies) or are optional (like "Archetypes" or "Dexterity" flavors of the same package) should go in extras_require. Remember to document how to enable specific features (and think of using zcml:condition statements, if you have such optional features).

Generally all direct dependencies (packages directly imported or used in ZCML) should be declared, even if they would already be pulled in by other dependencies. This explicitness reduces possible runtime errors and gives a good overview on the complexity of a package.

For example, if you depend on Products.CMFPlone and use getToolByName from Products.CMFCore, you should also declare the CMFCore dependency explicitly, even though it's pulled in by Plone itself. If you use namespace packages from the Zope distribution like Products.Five you should explicitly declare Zope as dependency.

Inside each group of dependencies, lines should be sorted alphabetically.

Versioning scheme ¶

For software versions, use a sequence-based versioning scheme, which is compatible with setuptools:

MAJOR.MINOR[.MICRO][STATUS]

The way, setuptools interprets versions is intuitive:

1.0 < 1.1dev < 1.1a1 < 1.1a2 < 1.1b < 1.1rc1 < 1.1 < 1.1.1

You can test it with setuptools:

                        >>> from pkg_resources import parse_version
>>> parse_version('1.0') < parse_version('1.1.dev')
... < parse_version('1.1.a1') < parse_version('1.1.a2')
... < parse_version('1.1.b') < parse_version('1.1.rc1')
... < parse_version('1.1') < parse_version('1.1.1')

                      

Setuptools recommends to seperate parts with a dot. The website about semantic versioning is also worth a read.

Restructured Text versus Plain Text ¶

Use the Restructured Text (.rst file extension) format instead of plain text files (.txt file extension) for all documentation, including doctest files. This way you get nice syntax highlighting and formating in recent text editors, on GitHub and with Sphinx.

Tracking changes ¶

Feature-level changes to code are tracked inside CHANGES.rst. The title of the CHANGES.rst file should be Changelog. Example:

                        Changelog
=========

1.0.0-dev (Unreleased)
----------------------

- Added feature Z.
  [github_userid1]

- Removed Y.
  [github_userid2]


1.0.0-alpha.1 (2012-12-12)
--------------------------

- Fixed Bug X.
  [github_userid1]

                      

Add an entry every time you add/remove a feature, fix a bug, etc. on top of the current development changes block.

Sphinx Documentation ¶

Un-documented code is broken code.

For every feature you add to the codebase you should also add documentation for it to docs/.

After adding/modifying documentation, run make to re-generate your docs.

Publicly available documentation on http://api.plone.org is automatically generated from these source files, periodically. So when you push changes to master on GitHub you should soon be able to see them published on api.plone.org.

Read the reStructuredText Primer to brush up on your reST skills.

Example:

                        def add(a, b):
    """Calculate the sum of the two parameters.

    Also see the :func:`mod.path.my_func`, :meth:`mod.path.MyClass.method`
    and :attr:`mod.path.MY_CONSTANT` for more details.

    :param a: The first operand.
    :type a: :class:`mod.path.A`

    :param b: The second operand.
    :type b: :class:`mod.path.B`

    :rtype: int
    :return: The sum of the operands.
    :raise: `KeyError`, if the operands are not the correct type.
    """

                      

Attributes are documented using the #: marker above the attribute. The documentation may span multiple lines.

                        #: Description of the constant value
MY_CONSTANT = 0xc0ffee

class Foobar(object):

    #: Description of the class variable which spans over
    #: multiple lines
    FOO = 1

                      

Travis Continuous Integration ¶

On every push to GitHub, Travis runs all tests and syntax validation checks and reports build outcome to the #sprint IRC channel and the person who committed the last change.

Travis is configured with the .travis.yml file located in the root of this package.

Git workflow & branching model ¶

Our repository on GitHub has the following layout:

feature branches: all development for new features must be done in dedicated branches, normally one branch per feature,
master branch: when features get completed they are merged into the master branch; bugfixes are commited directly on the master branch,
tags: whenever we create a new release we tag the repository so we can later re-trace our steps, re-release versions, etc.

Release process for Plone packages ¶

To keep the Plone software stack maintainable, the Python egg release process must be automated to high degree. This happens by enforcing Python packaging best practices and then making automated releases using the zest.releaser tool.

Anyone with necessary PyPi permissions must be able to make a new release by running the fullrelease command

... which includes ...

All releases must be hosted on PyPi
All versions must be tagged at version control
Each package must have README.rst with links to the version control repository and issue tracker
CHANGES.txt (docs/HISTORY.txt in some packages) must be always up-to-date and must contain list of functional changes which may affect package users.
CHANGES.txt must contain release dates
README.rst and CHANGES.txt must be visible on PyPi
Released eggs must contain generated gettext .mo files, but these files must not be committed to the repository (files can be created with zest.pocompile addon)
.gitignore and MANIFEST.in must reflect the files going to egg (must include page template, po files)

More information

High quality automated package releases for Python with zest.releaser.

                          Capitalized, short (50 chars or less) summary

More detailed explanatory text, if necessary.  Wrap it to about 72
characters or so.  In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body.  The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.

Write your commit message in the imperative: "Fix bug" and not "Fixed bug"
or "Fixes bug."  This convention matches up with commit messages generated
by commands like git merge and git revert.

Further paragraphs come after blank lines.

- Bullet points are okay, too
- Typically a hyphen or asterisk is used for the bullet, preceded by a
  single space, with blank lines in between, but conventions vary here
- Use a hanging indent

                        

Github flavored markdown is also useful in commit messages.

Squashing commits ¶

In order to keep a clear and concise git history, it is good practice to squash commits before merging. Use git rebase --interactive to squash all commits that you think are unnecessary.

Conventions¶

Introduction ¶

Line length ¶

Breaking lines ¶

autopep8 ¶

Indentation ¶

EditorConfig ¶

Quoting ¶

Docstrings style ¶

Unit tests style ¶

String formatting ¶

About imports ¶

Grouping and sorting ¶

Declaring dependencies ¶

Versioning scheme ¶

Restructured Text versus Plain Text ¶

Tracking changes ¶

Sphinx Documentation ¶

Travis Continuous Integration ¶

Git workflow & branching model ¶

Release process for Plone packages ¶

Setting up Git ¶

Enhanced git prompt ¶

Git dotfiles ¶

Git Commit Message Style ¶

Squashing commits ¶