blob: 077b7da112d949cb3fe54ffce47abe31596ba99f [file] [log] [blame]
Release Notes
This page briefly summarizes the KUnit changes in each Linux release.
For more specific documentation on KUnit features, see the in-kernel
.. _v5.5:
- KUnit introduced, including all the basic functionality.
- ``kunitconfig`` was renamed to ``.kunitconfig`` later in 5.5 (`commit
.. _v5.6:
- KUnit tests can be built as modules, `docs
.. _v5.7:
- switched to using ``.kunit/`` as the default build dir (and
``.kunitconfig`` => ``.kunit/.kunitconfig``).
- ``CONFIG_KUNIT_DEBUGFS`` to expose per-suite test results in
``/sys/kernel/debug/kunit/<suite>/results``, `docs
.. _v5.8:
- ```` now supports separate ``config/build/exec/parse`` commands in
addition to ``run``.
- You can use these if you don't want to let `` run`` do everything
for you, e.g. if you want to run manually but parse the results using
- ``CONFIG_KUNIT_ALL_TESTS`` introduced
- This is just a convention. New tests should use it like so:
.. code-block:: none
tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
depends on KUNIT
This builds unit tests for foo.
See `the upstream documentation
for the authoritative source on how to write Kconfig entries.
.. _v5.9:
- Named resources: tests can store and retrieve data via string keys.
- This is a power feature intended to be a cleaner alternative to using
global variables in tests.
- See `test_kasan.c
<>`__ for an
example that uses ``"kasan_data"``.
.. _v5.10:
- KASAN violations cause the current KUnit test case to be marked FAILED.
- Note: KASAN doesn't currently work on UML.
.. _v5.11:
- Parameterized testing: see the `docs
.. _v5.12:
- You can pass in a path to a kunitconfig file instead of using
``.kunit/.kunitconfig``, e.g.
.. code-block:: shell
$ ./tools/testing/kunit/ run --kunitconfig=fs/ext4/.kunitconfig
- You can tell ```` to run a subset of built suites by passing in a
glob argument, e.g.
.. code-block:: shell
# Only run "list" suites
$ ./tools/testing/kunit/ run '*list*'
.. _v5.13:
- Can call ``kunit_fail_current_test()`` from anywhere to fail the current KUnit test (e.g. if an invariant is broken, a UBSAN violation is detected, etc.)
- You can pass a directory ``--kunitconfig`` and it'll implicitly append
``.kunitconfig``, e.g. `` run --kunitconfig=lib/kunit``.
.. _v5.14:
- ```` can run tests on non-UML architectures (using QEMU), `docs
- It's now possible to generate code coverage reports when using UML, `docs
- Tests can call ``kunit_skip()`` to bail out and mark a test as "SKIPPED"
instead of "PASSED" or "FAILED"
.. _v5.15:
- UBSAN violations cause the current KUnit test case to be marked FAILED.
- Can pass kernel commandline parameters from ```` via
- KUnit prints a summary per suite of how many test cases passed/failed to
dmesg, e.g.
.. code-block:: shell
# property-entry: pass:7 fail:0 skip:0 total:7
.. _v5.16:
- You can filter on test names now in addition to suite names, e.g.
.. code-block:: shell
# Only run tests that contain "foo"
$ ./tools/testing/kunit/ run '*.*foo*'
# Only run tests in "mysuite" that contain "foo"
$ ./tools/testing/kunit/ run 'mysuite.*foo*'
- You can ask ```` to run each suite/test by itself. This can be useful
to track down issues due to non-hermetic tests
.. code-block:: shell
$ ./tools/testing/kunit/ run --run_isolated=suite
$ ./tools/testing/kunit/ run --run_isolated=test
- ````'s parsing logic was largely rewritten, so the output looks
slightly different
- Parsed output is also now mostly printed out in real time.
- ```` with ``--raw_output`` will print output in real time.
- In-kernel documentation has been significantly revamped.
.. _v5.17:
- Print out parsed test results in real time.
- Allow tweaking Kconfig options from command-line, e.g.
.. code-block:: shell
$ ./tools/testing/kunit/ run --kconfig_add=MY_CONFIG=y
- Reconfigure kunit kernel when the used ``kunitconfig`` has changed
- Before, if you removed an entry from ``.kunitconfig``, ```` would
not reconfigure, so your edit would have no effect.
- This now also works if you remove a ``--kconfig_add`` flag.
- Print out parameters as sub-test cases (both in raw and parsed output), e.g.:
.. code-block::
[12:37:14] =============== ext4_inode_test (1 subtest) ================
[12:37:14] ======= inode_test_xtimestamp_decoding (16 subtests) =======
[12:37:14] [PASSED] 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
[12:37:14] [PASSED] 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
[12:37:14] ========= [PASSED] inode_test_xtimestamp_decoding ==========
[12:37:14] ================= [PASSED] ext4_inode_test =================
[12:37:14] ============================================================
.. _v5.18:
- Drastically decrease stack usage of ``KUNIT_EXPECT_`` macros.
- Before, we largely relied on the compiler to optimize away some internal
structs, but that didn't always work.
- Now on UML (x86_64), ``KUNIT_EXPECT_EQ()`` uses 32 bytes instead of 88.
.. _v5.19:
- Add support for suite-level init/exit functions.
- ``suite_init``/``suite_exit`` are like ``init`` and ``exit`` instead of once per test case.
- Split out KUnit resource API from ``<kunit/test.h>`` into ``<kunit/resource.h>``.
- Including ``<kunit/test.h>`` will still automatically ``#include <kunit/resource.h>`` for now.
- ```` no longer colorizes output when stdout is not a tty.
- Add a manually curated ``tools/testing/kunit/configs/all_tests_uml.config``
- This attempts to enable as many tests as possible on UML.
- Change quoting requirements for qemu_config files (`commit
- E.g. instead of ``['-m 256']``, do ``['-m', '256']``
.. _v6.0:
- Overhaul support for building tests as modules
- Before, it wouldn't work in modules that already had their own
``module_init``/``module_exit``. Now this restriction is gone.
- Add ``--qemu_args`` flag, e.g. use ``--qemu_args='-m 2048'`` to give tests more RAM.
- Allow ``--kunitconfig`` to be repeated.
- This will just concatenante the contents together.
- One use case for this is appending the options for code coverage:
.. code-block:: shell
$ ./tools/testing/kunit/ run --kunitconfig=my/dir --kunitconfig=tools/testing/kunit/configs/coverage_uml.config
- Taint the kernel if any KUnit tests run.
- Add a new ``TAINT_TEST`` flag, shows up as ``'N'``.
- Add support for KASAN to run on 64-bit UML.
- By default, enable PCI support under UML.
.. _v6.1:
- `` run --alltests`` now works.
- Renamed ``all_tests_uml.config`` (added in :ref:`v5.19`) to ``all_tests.config``.
- Changed the ``--alltests`` flag to use this manually curated file instead of
trying to get ``allyesconfig`` to work.
- Removed Virtio and PCI UML specific options from ``all_tests_uml.config``.
- ``Documentation/`` updates and fixes.
- Further reduction in the stack consumption of ``KUNIT_EXPECT_`` macros.
- For context, see :ref:`v5.18`.
- On UML, now ``KUNIT_FAIL()`` uses 0 bytes (was 8), and ``KUNIT_EXPECT_EQ()``
uses 24 (was 32).
- In edge cases, ``kunit_kfree()`` matches ``kfree()``:
- ``kunit_kfree(NULL)`` is a no-op.
- ``kunit_kfree(invalid_ptr)`` will no longer crash, but it will now fail the test.
- Introduce new ``kunit.enable`` kernel command-line argument.
- This is a niche feature. If you don't want to run tests, prefer setting
- However, if you need to compile KUnit, you can use this to forcibly prevent
tests from running.
- You can set the default value of the option via ``CONFIG_KUNIT_DEFAULT_ENABLED``.
.. _v6.2:
- Added new ``KUNIT_EXPECT_MEMEQ(test, a, b, size)`` macro for comparing memory blocks.
- This is an alternative to ``KUNIT_EXPECT_EQ(test, memcmp(a, b, size), 0)``.
- On failure, KUnit will print out the bytes and highlight where the regions differ.
- Failure logs are available in decimal and hexadecimal output.
- Use a static key to check if tests are currently running.
- This reduces the performance overhead of `kunit_get_current_test()` and
`kunit_fail_current_test()` when no test is running.
- For more details, see `kunit: Provide a static key to check if KUnit is actively running tests <>`__.
- Make KUnit output and the parser compliant with KTAP version 1 specification.
- In KUnit output, subtests are now indicated using "KTAP version 1" headers.
- Added two macros, `VISIBLE_IF_KUNIT` and `EXPORT_SYMBOL_IF_KUNIT`, that let
tests to conditionally expose static symbols.
- For an example of how to use these macros, see `apparmor: test: make static symbols visible during kunit testing < linux/kernel/git/torvalds/linux.git/commit/?id=b11e51dd70947107fa4076c6286dce301671afc1>`__.
- Made the following KUnit documentation changes:
- Updated the "architecture.rst" page for style and grammatical issues.
- Made "usage.rst" a superset of "tips.rst" and removed "tips.rst" page.
- Reworded assertion description.
- Fix "How Do I Use This" / "Next Steps" section.
- Updated the requirements for test result description in KTAP specification.
.. _v6.3:
- Added "hooks" to call into KUnit when it's built as a module.
- These allow some KUnit features to be used in built-in code, or in other
modules, without introducing a hard dependency on KUnit.
- Introduced a "static_stub" feature to add redirection support for KUnit tests.
Any function that wants to be intercepted adds a call to a macro which checks
if a test has redirected calls to it, and then calls the corresponding replacement.
.. _v6.4:
- now supports building / running tests via QEMU on the m68k and SuperH (sh)
- Several fixes to debugfs log support, particularly with parameterized tests.
- The default log size has been increased to 2048 bytes.
.. _v6.5:
- Added a new 'deferred actions' API, based on the devm\_/devres APIs, to call a function
automatically on test exit. This should simplify resource management an cleanup.
- See:
- Test cleanup is now always run from a KUnit-managed test thread.
- Previously, if a test aborted early, its cleanup (deferred actions,
resources, and test / suite exit functions) would be run from the main
KUnit executor thread.
- This could cause problems if an assertion or crash occurred during cleanup.
- Now, if the original test thred exits, a new cleanup thread is spawned.
- Improved documentation and examples for test / suite init()/exit() functions.