Skip to content

Latest commit

 

History

History
915 lines (592 loc) · 25.1 KB

CONTRIBUTING.rst

File metadata and controls

915 lines (592 loc) · 25.1 KB
Raw

Contributing

Welcome!

This document is fairly extensive and you aren't really expected to study this in detail for small contributions;

The most important rule is that contributing must be easy and that the community is friendly and not nitpicking on details, such as coding style.

If you're reporting a bug you should read the Reporting bugs section below to ensure that your bug report contains enough information to successfully diagnose the issue, and if you're contributing code you should try to mimic the conventions you see surrounding the code you're working on, but in the end all patches will be cleaned up by the person merging the changes so don't worry too much.

Code of Conduct

Everyone interacting in the project's code bases, issue trackers, chat rooms, and mailing lists is expected to follow the Faust Code of Conduct.

As contributors and maintainers of these projects, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.

We are committed to making participation in these projects a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality.

Examples of unacceptable behavior by participants include:

  • The use of sexualized language or imagery
  • Personal attacks
  • Trolling or insulting/derogatory comments
  • Public or private harassment
  • Publishing other's private information, such as physical or electronic addresses, without explicit permission
  • Other unethical or unprofessional conduct.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. By adopting this Code of Conduct, project maintainers commit themselves to fairly and consistently applying these principles to every aspect of managing this project. Project maintainers who do not follow or enforce the Code of Conduct may be permanently removed from the project team.

This code of conduct applies both within project spaces and in public spaces when an individual is representing the project or its community.

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.

This Code of Conduct is adapted from the Contributor Covenant, version 1.2.0 available at http://contributor-covenant.org/version/1/2/0/.

Reporting Bugs

Security

You must never report security related issues, vulnerabilities or bugs including sensitive information to the bug tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to security@celeryproject.org.

If you'd like to submit the information encrypted our PGP key is:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.15 (Darwin)

mQENBFJpWDkBCADFIc9/Fpgse4owLNvsTC7GYfnJL19XO0hnL99sPx+DPbfr+cSE
9wiU+Wp2TfUX7pCLEGrODiEP6ZCZbgtiPgId+JYvMxpP6GXbjiIlHRw1EQNH8RlX
cVxy3rQfVv8PGGiJuyBBjxzvETHW25htVAZ5TI1+CkxmuyyEYqgZN2fNd0wEU19D
+c10G1gSECbCQTCbacLSzdpngAt1Gkrc96r7wGHBBSvDaGDD2pFSkVuTLMbIRrVp
lnKOPMsUijiip2EMr2DvfuXiUIUvaqInTPNWkDynLoh69ib5xC19CSVLONjkKBsr
Pe+qAY29liBatatpXsydY7GIUzyBT3MzgMJlABEBAAG0MUNlbGVyeSBTZWN1cml0
eSBUZWFtIDxzZWN1cml0eUBjZWxlcnlwcm9qZWN0Lm9yZz6JATgEEwECACIFAlJp
WDkCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOArFOUDCicIw1IH/26f
CViDC7/P13jr+srRdjAsWvQztia9HmTlY8cUnbmkR9w6b6j3F2ayw8VhkyFWgYEJ
wtPBv8mHKADiVSFARS+0yGsfCkia5wDSQuIv6XqRlIrXUyqJbmF4NUFTyCZYoh+C
ZiQpN9xGhFPr5QDlMx2izWg1rvWlG1jY2Es1v/xED3AeCOB1eUGvRe/uJHKjGv7J
rj0pFcptZX+WDF22AN235WYwgJM6TrNfSu8sv8vNAQOVnsKcgsqhuwomSGsOfMQj
LFzIn95MKBBU1G5wOs7JtwiV9jefGqJGBO2FAvOVbvPdK/saSnB+7K36dQcIHqms
5hU4Xj0RIJiod5idlRC5AQ0EUmlYOQEIAJs8OwHMkrdcvy9kk2HBVbdqhgAREMKy
gmphDp7prRL9FqSY/dKpCbG0u82zyJypdb7QiaQ5pfPzPpQcd2dIcohkkh7G3E+e
hS2L9AXHpwR26/PzMBXyr2iNnNc4vTksHvGVDxzFnRpka6vbI/hrrZmYNYh9EAiv
uhE54b3/XhXwFgHjZXb9i8hgJ3nsO0pRwvUAM1bRGMbvf8e9F+kqgV0yWYNnh6QL
4Vpl1+epqp2RKPHyNQftbQyrAHXT9kQF9pPlx013MKYaFTADscuAp4T3dy7xmiwS
crqMbZLzfrxfFOsNxTUGE5vmJCcm+mybAtRo4aV6ACohAO9NevMx8pUAEQEAAYkB
HwQYAQIACQUCUmlYOQIbDAAKCRDgKxTlAwonCNFbB/9esir/f7TufE+isNqErzR/
aZKZo2WzZR9c75kbqo6J6DYuUHe6xI0OZ2qZ60iABDEZAiNXGulysFLCiPdatQ8x
8zt3DF9BMkEck54ZvAjpNSern6zfZb1jPYWZq3TKxlTs/GuCgBAuV4i5vDTZ7xK/
aF+OFY5zN7ciZHkqLgMiTZ+RhqRcK6FhVBP/Y7d9NlBOcDBTxxE1ZO1ute6n7guJ
ciw4hfoRk8qNN19szZuq3UU64zpkM2sBsIFM9tGF2FADRxiOaOWZHmIyVZriPFqW
RUwjSjs7jBVNq0Vy4fCu/5+e+XLOUBOoqtM5W7ELt0t1w9tXebtPEetV86in8fU2
=0chn
-----END PGP PUBLIC KEY BLOCK-----

Other bugs

Bugs can always be described to the slack-channel, but the best way to report an issue and to ensure a timely response is to use the issue tracker.

  1. Create a GitHub account.

You need to create a GitHub account to be able to create new issues and participate in the discussion.

  1. Determine if your bug is really a bug.

You shouldn't file a bug if you're requesting support. For that you can use the slack-channel.

  1. Make sure your bug hasn't already been reported.

Search through the appropriate Issue tracker. If a bug like yours was found, check if you have new information that could be reported to help the developers fix the bug.

  1. Check if you're using the latest version.

A bug could be fixed by some other improvements and fixes - it might not have an existing report in the bug tracker. Make sure you're using the latest release of Faust.

  1. Collect information about the bug.

To have the best chance of having a bug fixed, we need to be able to easily reproduce the conditions that caused it. Most of the time this information will be from a Python traceback message, though some bugs might be in design, spelling or other errors on the website/docs/code.

  1. If the error is from a Python traceback, include it in the bug report.

  2. We also need to know what platform you're running (Windows, macOS, Linux, etc.), the version of your Python interpreter, and the version of Faust, and related packages that you were running when the bug occurred.

  3. If you're reporting a race condition or a deadlock, tracebacks can be hard to get or might not be that useful. Try to inspect the process to get more diagnostic data. Some ideas:

    • Collect tracing data using strace`_(Linux), ``dtruss` (macOS), and ktrace (BSD), ltrace, and lsof.

  4. Include the output from the faust report command:

    $ faust -A proj report

    This will also include your configuration settings and it try to remove values for keys known to be sensitive, but make sure you also verify the information before submitting so that it doesn't contain confidential information like API tokens and authentication credentials.

  1. Submit the bug.

By default GitHub will email you to let you know when new comments have been made on your bug. In the event you've turned this feature off, you should check back on occasion to ensure you don't miss any questions a developer trying to fix the bug might ask.

Issue Trackers

Bugs for a package in the Faust ecosystem should be reported to the relevant issue tracker.

If you're unsure of the origin of the bug you can ask the slack-channel, or just use the Faust issue tracker.

Contributors guide to the code base

There's a separate section for internal details, including details about the code base and a style guide.

Read developers-guide for more!

Versions

Version numbers consists of a major version, minor version and a release number. Faust uses the versioning semantics described by SemVer: http://semver.org.

Stable releases are published at PyPI while development releases are only available in the GitHub git repository as tags. All version tags starts with “v”, so version 0.8.0 is the tag v0.8.0.

Branches

Current active version branches:

You can see the state of any branch by looking at the Changelog:

https://github.com/robinhood/faust/blob/master/Changelog.rst

If the branch is in active development the topmost version info should contain meta-data like:

2.4.0
======
:release-date: TBA
:status: DEVELOPMENT
:branch: dev (git calls this master)

The status field can be one of:

  • PLANNING

    The branch is currently experimental and in the planning stage.

  • DEVELOPMENT

    The branch is in active development, but the test suite should be passing and the product should be working and possible for users to test.

  • FROZEN

    The branch is frozen, and no more features will be accepted. When a branch is frozen the focus is on testing the version as much as possible before it is released.

dev branch

The dev branch (called "master" by git), is where development of the next version happens.

Maintenance branches

Maintenance branches are named after the version -- for example, the maintenance branch for the 2.2.x series is named 2.2.

Previously these were named releaseXX-maint.

The versions we currently maintain is:

  • 1.0

    This is the current series.

Archived branches

Archived branches are kept for preserving history only, and theoretically someone could provide patches for these if they depend on a series that's no longer officially supported.

An archived version is named X.Y-archived.

Our currently archived branches are:

We don't currently have any archived branches.

Feature branches

Major new features are worked on in dedicated branches. There's no strict naming requirement for these branches.

Feature branches are removed once they've been merged into a release branch.

Tags

  • Tags are used exclusively for tagging releases. A release tag is named with the format vX.Y.Z -- for example v2.3.1.
  • Experimental releases contain an additional identifier vX.Y.Z-id -- for example v3.0.0-rc1.
  • Experimental tags may be removed after the official release.

Working on Features & Patches

Note

Contributing to Faust should be as simple as possible, so none of these steps should be considered mandatory.

You can even send in patches by email if that's your preferred work method. We won't like you any less, any contribution you make is always appreciated!

However following these steps may make maintainers life easier, and may mean that your changes will be accepted sooner.

Forking and setting up the repository

Create your fork

First you need to fork the Faust repository, a good introduction to this is in the GitHub Guide: Fork a Repo.

After you have cloned the repository you should checkout your copy to a directory on your machine:

$ git clone git@github.com:username/faust.git

When the repository is cloned enter the directory to set up easy access to upstream changes:

$ cd faust
$ git remote add upstream https://github.com/robinhood/faust.git
$ git fetch upstream

If you need to pull in new changes from upstream you should always use the --rebase option to git pull:

$ git pull --rebase upstream master

With this option you don't clutter the history with merging commit notes. See Rebasing merge commits in git. If you want to learn more about rebasing see the Rebase section in the GitHub guides.

Start Developing

To start developing Faust you should install the requirements and setup the development environment so that Python uses the Faust development directory.

To do so run:

$ make develop

If you want to install requirements manually you should at least install the git pre-commit hooks (the make develop command above automatically runs this as well):

$ make hooks

If you also want to install C extensions, including the RocksDB bindings then you can use make cdevelop instead of make develop:

$ make cdevelop

Note

If you need to work on a different branch than the one git calls master, you can fetch and checkout a remote branch like this:

$ git checkout --track -b 2.0-devel origin/2.0-devel

Running the test suite

To run the Faust test suite you need to install a few dependencies. A complete list of the dependencies needed are located in requirements/test.txt.

Both the stable and the development version have testing related dependencies, so install these:

$ pip install -U -r requirements/test.txt
$ pip install -U -r requirements/default.txt

After installing the dependencies required, you can now execute the test suite by calling py.test <pytest:

$ py.test

This will run the unit tests, functional tests and doc example tests, but not integration tests or stress tests.

Some useful options to py.test are:

  • -x

    Stop running the tests at the first test that fails.

  • -s

    Don't capture output

  • -v

    Run with verbose output.

If you want to run the tests for a single test file only you can do so like this:

$ py.test t/unit/test_app.py

Creating pull requests

When your feature/bugfix is complete you may want to submit a pull requests so that it can be reviewed by the maintainers.

Creating pull requests is easy, and also let you track the progress of your contribution. Read the Pull Requests section in the GitHub Guide to learn how this is done.

You can also attach pull requests to existing issues by following the steps outlined here: http://bit.ly/koJoso

Running the tests on all supported Python versions

There's a tox configuration file in the top directory of the distribution.

To run the tests for all supported Python versions simply execute:

$ tox

Use the tox -e option if you only want to test specific Python versions:

$ tox -e 2.7

Building the documentation

To build the documentation you need to install the dependencies listed in requirements/docs.txt:

$ pip install -U -r requirements/docs.txt

After these dependencies are installed you should be able to build the docs by running:

$ cd docs
$ rm -rf _build
$ make html

Make sure there are no errors or warnings in the build output. After building succeeds the documentation is available at _build/html.

Verifying your contribution

To use these tools you need to install a few dependencies. These dependencies can be found in requirements/dist.txt.

Installing the dependencies:

$ pip install -U -r requirements/dist.txt

pyflakes & PEP-8

To ensure that your changes conform to 8 and to run pyflakes execute:

$ make flakecheck

To not return a negative exit code when this command fails use the flakes target instead:

$ make flakes

API reference

To make sure that all modules have a corresponding section in the API reference please execute:

$ make apicheck
$ make indexcheck

If files are missing you can add them by copying an existing reference file.

If the module is internal it should be part of the internal reference located in docs/internals/reference/. If the module is public it should be located in docs/reference/.

For example if reference is missing for the module faust.worker.awesome and this module is considered part of the public API, use the following steps:

Use an existing file as a template:

$ cd docs/reference/
$ cp faust.schedules.rst faust.worker.awesome.rst

Edit the file using your favorite editor:

$ vim faust.worker.awesome.rst

    # change every occurrence of ``faust.schedules`` to
    # ``faust.worker.awesome``

Edit the index using your favorite editor:

$ vim index.rst

    # Add ``faust.worker.awesome`` to the index.

Commit your changes:

# Add the file to git
$ git add faust.worker.awesome.rst
$ git add index.rst
$ git commit faust.worker.awesome.rst index.rst \
    -m "Adds reference for faust.worker.awesome"

Configuration Reference

To make sure that all settings have a corresponding section in the configuration reference, please execute:

$ make configcheck

If settings are missing from there an error is produced, and you can proceed by documenting the settings in docs/userguide/settings.rst.

Coding Style

You should probably be able to pick up the coding style from surrounding code, but it is a good idea to be aware of the following conventions.

  • We use static types and the mypy type checker to verify them.

    Python code must import these static types when using them, so to keep static types lightweight we define interfaces for classes in faust/types/.

    For example for the fauts.App class, there is a corresponding faust.types.app.AppT; for faust.Channel there is a faust.types.channels.ChannelT and similarly for most other classes in the library.

    We suffer some duplication because of this, but it keeps static typing imports fast and reduces the need for recursive imports.

    In some cases recursive imports still happen, in that case you can "trick" the type checker into importing it, while regular Python does not:

    if typing.TYPE_CHECKING:
    from faust.app import App
else:
    class App: ...  # noqa

  • All Python code must follow the 8 guidelines.

pep8 is a utility you can use to verify that your code is following the conventions.

  • Docstrings must follow the 257 conventions, and use the following style.

    Do this:

    def method(self, arg: str) -> None:
    """Short description.

    More details.

    """

    or:

    def method(self, arg: str) -> None:
    """Short description."""

    but not this:

    def method(self, arg: str) -> None:
    """
    Short description.
    """

  • Lines shouldn't exceed 78 columns.

    You can enforce this in vim by setting the textwidth option:

    set textwidth=78

    If adhering to this limit makes the code less readable, you have one more character to go on. This means 78 is a soft limit, and 79 is the hard limit :)

  • Import order

    • Python standard library
    • Third-party packages.
    • Other modules from the current package.

    or in case of code using Django:

    • Python standard library (import xxx)
    • Third-party packages.
    • Django packages.
    • Other modules from the current package.

    Within these sections the imports should be sorted by module name.

    Example:

    import threading
import time
from collections import deque
from Queue import Queue, Empty

from .platforms import Pidfile
from .five import zip_longest, items, range
from .utils.time import maybe_timedelta

  • Wild-card imports must not be used (from xxx import *).

Contributing features requiring additional libraries

Some features like a new result backend may require additional libraries that the user must install.

We use setuptools extra_requires for this, and all new optional features that require third-party libraries must be added.

  1. Add a new requirements file in requirements/extras

    For the RocksDB store this is requirements/extras/rocksdb.txt, and the file looks like this:

    python-rocksdb

    These are pip requirement files so you can have version specifiers and multiple packages are separated by newline. A more complex example could be:

    # python-rocksdb 2.0 breaks Foo
python-rocksdb>=1.0,<2.0
thrift

  2. Modify setup.py

    After the requirements file is added you need to add it as an option to setup.py in the EXTENSIONS section:

    EXTENSIONS = {
    'debug',
    'fast',
    'rocksdb',
    'uvloop',
}

  3. Document the new feature in docs/includes/installation.txt

    You must add your feature to the list in the bundles section of docs/includes/installation.txt.

    After you've made changes to this file you need to render the distro README file:

    $ pip install -U requirements/dist.txt
$ make readme

Contacts

This is a list of people that can be contacted for questions regarding the official git repositories, PyPI packages Read the Docs pages.

If the issue isn't an emergency then it's better to report an issue.

Committers

Ask Solem

github:https://github.com/ask
twitter:http://twitter.com/#!/asksol

Vineet Goel

github:https://github.com/vineet-rh
twitter:https://twitter.com/#!/vineetik

Arpan Shah

github:https://github.com/arpanshah29

Packages

Faust

git:https://github.com/robinhood/faust
CI:http://travis-ci.org/#!/robinhood/faust
Windows-CI:https://ci.appveyor.com/project/ask/faust
PyPI:faust
docs:https://faust.readthedocs.io

Mode

git:https://github.com/ask/mode
CI:http://travis-ci.org/#!/ask/mode
Windows-CI:https://ci.appveyor.com/project/ask/mode
PyPI:Mode
docs:http://mode.readthedocs.io/

Release Procedure

Updating the version number

The version number must be updated two places:

  • faust/__init__.py
  • docs/include/introduction.txt

After you have changed these files you must render the README files. There's a script to convert sphinx syntax to generic reStructured Text syntax, and the make target readme does this for you:

$ make readme

Now commit the changes:

$ git commit -a -m "Bumps version to X.Y.Z"

and make a new version tag:

$ git tag vX.Y.Z
$ git push --tags

Releasing

Commands to make a new public stable release:

$ make distcheck  # checks pep8, autodoc index, runs tests and more
$ make dist  # NOTE: Runs git clean -xdf and removes files not in the repo.
$ python setup.py sdist upload --sign --identity='Celery Security Team'
$ python setup.py bdist_wheel upload --sign --identity='Celery Security Team'

If this is a new release series then you also need to do the following:

  • Go to the Read The Docs management interface at:

    http://readthedocs.org/projects/faust/?fromdocs=faust

  • Enter "Edit project"

    Change default branch to the branch of this series, for example, use the 1.0 branch for the 1.0 series.

  • Also add the previous version under the "versions" tab.