known incompatibilities:
new features:
experimentally introduce a new pytest.yield_fixture decorator which accepts exactly the same parameters as pytest.fixture but mandates a yield statement instead of a return statement from fixture functions. This allows direct integration with “with-style” context managers in fixture functions and generally avoids registering of finalization callbacks in favour of treating the “after-yield” as teardown code. Thanks Andreas Pelme, Vladimir Keleshev, Floris Bruynooghe, Ronny Pfannschmidt and many others for discussions.
allow boolean expression directly with skipif/xfail if a “reason” is also specified. Rework skipping documentation to recommend “condition as booleans” because it prevents surprises when importing markers between modules. Specifying conditions as strings will remain fully supported.
reporting: color the last line red or green depending if failures/errors occured or everything passed. thanks Christian Theunert.
make “import pdb ; pdb.set_trace()” work natively wrt capturing (no “-s” needed anymore), making pytest.set_trace() a mere shortcut.
fix issue181: –pdb now also works on collect errors (and on internal errors) . This was implemented by a slight internal refactoring and the introduction of a new hook pytest_exception_interact hook (see next item).
fix issue341: introduce new experimental hook for IDEs/terminals to intercept debugging: pytest_exception_interact(node, call, report).
new monkeypatch.setattr() variant to provide a shorter invocation for patching out classes/functions from modules:
monkeypatch.setattr(“requests.get”, myfunc)
will replace the “get” function of the “requests” module with myfunc.
fix issue322: tearDownClass is not run if setUpClass failed. Thanks Mathieu Agopian for the initial fix. Also make all of pytest/nose finalizer mimick the same generic behaviour: if a setupX exists and fails, don’t run teardownX. This internally introduces a new method “node.addfinalizer()” helper which can only be called during the setup phase of a node.
simplify pytest.mark.parametrize() signature: allow to pass a CSV-separated string to specify argnames. For example: pytest.mark.parametrize("input,expected", [(1,2), (2,3)]) works as well as the previous: pytest.mark.parametrize(("input", "expected"), ...).
add support for setUpModule/tearDownModule detection, thanks Brian Okken.
integrate tab-completion on options through use of “argcomplete”. Thanks Anthon van der Neut for the PR.
change option names to be hyphen-separated long options but keep the old spelling backward compatible. py.test -h will only show the hyphenated version, for example “–collect-only” but “–collectonly” will remain valid as well (for backward-compat reasons). Many thanks to Anthon van der Neut for the implementation and to Hynek Schlawack for pushing us.
fix issue 308 - allow to mark/xfail/skip individual parameter sets when parametrizing. Thanks Brianna Laugher.
call new experimental pytest_load_initial_conftests hook to allow 3rd party plugins to do something before a conftest is loaded.
Bug fixes:
fix issue202 - better automatic names for parametrized test functions
fix issue139 - introduce @pytest.fixture which allows direct scoping and parametrization of funcarg factories.
fix issue198 - conftest fixtures were not found on windows32 in some circumstances with nested directory structures due to path manipulation issues
fix issue193 skip test functions with were parametrized with empty parameter sets
fix python3.3 compat, mostly reporting bits that previously depended on dict ordering
introduce re-ordering of tests by resource and parametrization setup which takes precedence to the usual file-ordering
fix issue185 monkeypatching time.time does not cause pytest to fail
fix issue172 duplicate call of pytest.fixture decoratored setup_module functions
fix junitxml=path construction so that if tests change the current working directory and the path is a relative path it is constructed correctly from the original current working dir.
fix “python setup.py test” example to cause a proper “errno” return
fix issue165 - fix broken doc links and mention stackoverflow for FAQ
catch unicode-issues when writing failure representations to terminal to prevent the whole session from crashing
fix xfail/skip confusion: a skip-mark or an imperative pytest.skip will now take precedence before xfail-markers because we can’t determine xfail/xpass status in case of a skip. see also: http://stackoverflow.com/questions/11105828/in-py-test-when-i-explicitly-skip-a-test-that-is-marked-as-xfail-how-can-i-get
always report installed 3rd party plugins in the header of a test run
fix issue160: a failing setup of an xfail-marked tests should be reported as xfail (not xpass)
fix issue128: show captured output when capsys/capfd are used
fix issue179: propperly show the dependency chain of factories
pluginmanager.register(...) now raises ValueError if the plugin has been already registered or the name is taken
fix issue159: improve http://pytest.org/latest/faq.html especially with respect to the “magic” history, also mention pytest-django, trial and unittest integration.
make request.keywords and node.keywords writable. All descendant collection nodes will see keyword values. Keywords are dictionaries containing markers and other info.
fix issue 178: xml binary escapes are now wrapped in py.xml.raw
fix issue 176: correctly catch the builtin AssertionError even when we replaced AssertionError with a subclass on the python level
factory discovery no longer fails with magic global callables that provide no sane __code__ object (mock.call for example)
fix issue 182: testdir.inprocess_run now considers passed plugins
before calling into a test
fix issue 191: add unittest TestCase runTest method support
fix issue 156: monkeypatch correctly handles class level descriptors
reporting refinements:
tackle issue32 - speed up test runs of very quick test functions by reducing the relative overhead
fix issue30 - extended xfail/skipif handling and improved reporting. If you have a syntax error in your skip/xfail expressions you now get nice error reports.
Also you can now access module globals from xfail/skipif expressions so that this for example works now:
import pytest
import mymodule
@pytest.mark.skipif("mymodule.__version__[0] == "1")
def test_function():
pass
This will not run the test function if the module’s version string does not start with a “1”. Note that specifying a string instead of a boolean expressions allows py.test to report meaningful information when summarizing a test run as to what conditions lead to skipping (or xfail-ing) tests.
fix issue28 - setup_method and pytest_generate_tests work together The setup_method fixture method now gets called also for test function invocations generated from the pytest_generate_tests hook.
fix issue27 - collectonly and keyword-selection (-k) now work together Also, if you do “py.test –collectonly -q” you now get a flat list of test ids that you can use to paste to the py.test commandline in order to execute a particular test.
fix issue25 avoid reported problems with –pdb and python3.2/encodings output
fix issue23 - tmpdir argument now works on Python3.2 and WindowsXP Starting with Python3.2 os.symlink may be supported. By requiring a newer py lib version the py.path.local() implementation acknowledges this.
fixed typos in the docs (thanks Victor Garcia, Brianna Laugher) and particular thanks to Laura Creighton who also revieved parts of the documentation.
fix slighly wrong output of verbose progress reporting for classes (thanks Amaury)
more precise (avoiding of) deprecation warnings for node.Class|Function accesses
avoid std unittest assertion helper code in tracebacks (thanks Ronny)
fix issue103: introduce py.test.raises as context manager, examples:
with py.test.raises(ZeroDivisionError):
x = 0
1 / x
with py.test.raises(RuntimeError) as excinfo:
call_something()
# you may do extra checks on excinfo.value|type|traceback here
(thanks Ronny Pfannschmidt)
Funcarg factories can now dynamically apply a marker to a test invocation. This is for example useful if a factory provides parameters to a test which are expected-to-fail:
def pytest_funcarg__arg(request):
request.applymarker(py.test.mark.xfail(reason="flaky config"))
...
def test_function(arg):
...
improved error reporting on collection and import errors. This makes use of a more general mechanism, namely that for custom test item/collect nodes node.repr_failure(excinfo) is now uniformly called so that you can override it to return a string error representation of your choice which is going to be reported as a (red) string.
introduce ‘–junitprefix=STR’ option to prepend a prefix to all reports in the junitxml file.
issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail.
issue102: introduce new –maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the ‘-x’ or ‘–exitfirst’ option which is now equivalent to ‘–maxfail=1’. Both ‘-x’ and ‘–maxfail’ will now also print a line near the end indicating the Interruption.
issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list.
improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed)
You can use any combination of ‘fsxX’ with the ‘-r’ extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99.
make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don’t have global state.
deprecate –report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of ‘xfsX’ characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test.
allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature.
add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:
def pytest_ignore_collect(path):
return path.check(link=1)
to prevent even a collection try of any tests in symlinked dirs.
new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module.
extend and refine xfail mechanism: @py.test.mark.xfail(run=False) do not run the decorated test @py.test.mark.xfail(reason="...") prints the reason string in xfail summaries specifiying --runxfail on command line virtually ignores xfail markers
expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string
expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking
(issue85) fix junitxml plugin to handle tests with non-ascii output
fix/refine python3 compatibility (thanks Benjamin Peterson)
fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around.
fixes for handling of unicode exception values and unprintable objects
(issue87) fix unboundlocal error in assertionold code
(issue86) improve documentation for looponfailing
refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method
ship distribute_setup.py version 0.6.10
added links to the new capturelog and coverage plugins
refined usage and options for “py.cleanup”:
py.cleanup # remove "*.pyc" and "*$py.class" (jython) files
py.cleanup -e .swp -e .cache # also remove files with these extensions
py.cleanup -s # remove "build" and "dist" directory next to setup.py files
py.cleanup -d # also remove empty directories
py.cleanup -a # synonym for "-s -d -e 'pip-log.txt'"
py.cleanup -n # dry run, only show what would be removed
add a new option “py.test –funcargs” which shows available funcargs and their help strings (docstrings on their respective factory function) for a given test path
display a short and concise traceback if a funcarg lookup fails
early-load “conftest.py” files in non-dot first-level sub directories. allows to conveniently keep and access test-related options in a test subdir and still add command line options.
fix issue67: new super-short traceback-printing option: “–tb=line” will print a single line for each failing (python) test indicating its filename, lineno and the failure value
fix issue78: always call python-level teardown functions even if the according setup failed. This includes refinements for calling setup_module/class functions which will now only be called once instead of the previous behaviour where they’d be called multiple times if they raise an exception (including a Skipped exception). Any exception will be re-corded and associated with all tests in the according module/class scope.
fix issue63: assume <40 columns to be a bogus terminal width, default to 80
fix pdb debugging to be in the correct frame on raises-related errors
update apipkg.py to fix an issue where recursive imports might unnecessarily break importing
fix plugin links
XXX lots of things missing here XXX
This is a fairly complete list of changes between 0.9 and 0.9.1, which can serve as a reference for developers.