Documentation: kunit: added mocking documentation Add documentation for KUnit's function class and platform mocking libraries. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Change-Id: I4603bbc420cfa6c0e076ab633b6a76c544b2a88d
diff --git a/Documentation/kunit/api/class-and-function-mocking.rst b/Documentation/kunit/api/class-and-function-mocking.rst new file mode 100644 index 0000000..15fdf88 --- /dev/null +++ b/Documentation/kunit/api/class-and-function-mocking.rst
@@ -0,0 +1,68 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +Class and Function Mocking +========================== + +This file documents class and function mocking features. + +.. note:: + If possible, prefer class mocking over arbitrary function mocking. Class + mocking has a much more limited scope and provides more control. + This file documents class mocking and most mocking features that do not + depend on function or platform mocking. + +Readability Macros +------------------ +When defining and declaring mock stubs, use these readability macros. + +.. code-block:: c + + #define CLASS(struct_name) struct_name + #define HANDLE_INDEX(index) index + #define METHOD(method_name) method_name + #define RETURNS(return_type) return_type + #define PARAMS(...) __VA_ARGS__ + +Consider a ``struct Foo`` with a member function +``int add(struct Foo*, int a, int b);`` + +When generating a mock stub with :c:func:`DEFINE_STRUCT_CLASS_MOCK`, which +takes a method name, struct name, return type, and method parameters, the +arguments should be passed in with the readability macros. + +.. code-block:: c + + DEFINE_STRUCT_CLASS_MOCK( + METHOD(add), + CLASS(Foo), + RETURNS(int), + PARAMS(struct Foo *, int, int) + ); + +For a more detailed example of this, take a look at the example in +:doc:`../start` + +These macros should only be used in the context of the mock stub generators. + + +Built in Matchers +----------------- + +.. kernel-doc:: include/kunit/mock.h + :doc: Built In Matchers + +Mock Returns +------------ +These functions can be used to specify a value to be returned (``ret``) when a +mocked function is intercepted via :c:func:`EXPECT_CALL`. + +.. code-block:: c + + struct mock_action *test_int_return(struct test *test, int ret); + struct mock_action *test_u32_return(struct test *test, u32 ret); + +API +--- +.. kernel-doc:: include/kunit/mock.h + :internal:
diff --git a/Documentation/kunit/api/index.rst b/Documentation/kunit/api/index.rst index c31c530..a4fdc35 100644 --- a/Documentation/kunit/api/index.rst +++ b/Documentation/kunit/api/index.rst
@@ -6,6 +6,8 @@ .. toctree:: test + class-and-function-mocking + platform-mocking This section documents the KUnit kernel testing API. It is divided into 3 sections: @@ -13,4 +15,7 @@ ================================= ============================================== :doc:`test` documents all of the standard testing API excluding mocking or mocking related features. +:doc:`class-and-function-mocking` documents class and function mocking features. +:doc:`platform-mocking` documents mocking libraries that mock out + platform specific features. ================================= ==============================================
diff --git a/Documentation/kunit/api/platform-mocking.rst b/Documentation/kunit/api/platform-mocking.rst new file mode 100644 index 0000000..72555eb --- /dev/null +++ b/Documentation/kunit/api/platform-mocking.rst
@@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================ +Platform Mocking +================ + +This file documents *platform mocking*, mocking libraries that mock out platform +specific features and aid in writing mocks for platform drivers and other low +level kernel code. + +Enable Platform Mocking +----------------------- +``CONFIG_PLATFORM_MOCK`` needs to be added to the .config (or kunitconfig) to +enable platform mocking. + +Mocked IO Functions +------------------- +The following functions have been mocked for convenience. + +.. code-block:: c + + u8 readb(const volatile void __iomem *); + u16 readw(const volatile void __iomem *); + u32 readl(const volatile void __iomem *); + u64 readq(const volatile void __iomem *); + void writeb(u8, const volatile void __iomem *); + void writew(u16, const volatile void __iomem *); + void writel(u32, const volatile void __iomem *); + void writeq(u64, const volatile void __iomem *); + +.. note:: These functions do not have any non-mocked behaviour in UML. + +API +--- +.. kernel-doc:: include/linux/platform_device_mock.h + :internal:
diff --git a/Documentation/kunit/index.rst b/Documentation/kunit/index.rst index c671021..08c0ebb 100644 --- a/Documentation/kunit/index.rst +++ b/Documentation/kunit/index.rst
@@ -22,7 +22,7 @@ KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common -infrastructure for running tests, and much more. +infrastructure for running tests, mocking, spying, and much more. Get started now: :doc:`start`
diff --git a/Documentation/kunit/usage.rst b/Documentation/kunit/usage.rst index 491466d..5b80560 100644 --- a/Documentation/kunit/usage.rst +++ b/Documentation/kunit/usage.rst
@@ -445,3 +445,432 @@ destroy_eeprom_buffer(ctx->eeprom_buffer); } +Mocking Classes +~~~~~~~~~~~~~~~ + +Sometimes the easiest way to make assertions about behavior is to verify +certain methods or functions were called with appropriate arguments. KUnit +allows classes to be *mocked* which means that it generates subclasses whose +behavior can be specified in a test case. KUnit accomplishes this with two sets +of macros: the mock generation macros and the ``TEST_EXPECT_CALL`` macro. + +For example, let's go back to the EEPROM example; instead of faking the EEPROM, +we could have *mocked it out* with the following code: + +.. code-block:: c + + DECLARE_STRUCT_CLASS_MOCK_PREREQS(eeprom); + + DEFINE_STRUCT_CLASS_MOCK(METHOD(read), CLASS(eeprom), + RETURNS(ssize_t), + PARAMS(struct eeprom *, size_t, char *, size_t)); + + DEFINE_STRUCT_CLASS_MOCK(METHOD(write), CLASS(eeprom), + RETURNS(ssize_t), + PARAMS(struct eeprom *, size_t, const char *, size_t)); + + static int eeprom_init(struct MOCK(eeprom) *mock_eeprom) + { + struct eeprom *eeprom = mock_get_trgt(mock_eeprom); + + eeprom->read = read; + eeprom->write = write; + + return 0; + } + + DEFINE_STRUCT_CLASS_MOCK_INIT(eeprom, eeprom); + +We could use the mock in a test as follows: + +.. code-block:: c + + struct eeprom_buffer_test { + struct MOCK(eeprom) *mock_eeprom; + struct eeprom_buffer *eeprom_buffer; + }; + + static void eeprom_buffer_test_does_not_write_until_flush(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + struct eeprom_buffer *eeprom_buffer = ctx->eeprom_buffer; + struct MOCK(eeprom) *mock_eeprom = ctx->mock_eeprom; + struct mock_expectation *expectation; + char buffer[] = {0xff, 0xff}; + + eeprom_buffer->flush_count = SIZE_MAX; + + expectation = TEST_EXPECT_CALL(write(mock_get_ctrl(mock_eeprom), + test_any(test), + test_any(test), + test_any(test))); + expectation->max_calls_expected = 0; + expectation->min_calls_expected = 0; + + eeprom_buffer->write(eeprom_buffer, buffer, 1); + eeprom_buffer->write(eeprom_buffer, buffer, 1); + + mock_validate_expectations(mock_get_ctrl(mock_eeprom)); + + expectation = TEST_EXPECT_CALL(write(mock_get_ctrl(mock_eeprom), + test_any(test), + test_memeq(test, + buffer, + ARRAY_SIZE(buffer)), + test_ulong_eq(test, 2))); + expectation->max_calls_expected = 1; + expectation->min_calls_expected = 1; + expectation->action = test_long_return(test, 2); + + eeprom_buffer->flush(eeprom_buffer); + } + + static int eeprom_buffer_test_init(struct test *test) + { + struct eeprom_buffer_test *ctx; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + ASSERT_NOT_ERR_OR_NULL(test, ctx); + + ctx->mock_eeprom = CONSTRUCT_MOCK(eeprom, test); + ASSERT_NOT_ERR_OR_NULL(test, ctx->fake_eeprom); + + ctx->eeprom_buffer = new_eeprom_buffer(mock_get_trgt(ctx->mock_eeprom)); + ASSERT_NOT_ERR_OR_NULL(test, ctx->eeprom_buffer); + + test->priv = ctx; + + return 0; + } + + static void eeprom_buffer_test_exit(struct test *test) + { + struct eeprom_buffer_test *ctx = test->priv; + + destroy_eeprom_buffer(ctx->eeprom_buffer); + } + +This test case tests the same thing as the +``eeprom_buffer_test_does_not_write_until_flush`` test case from the example in +the faking section. Observe that in this test case you specify how you expect +the mock to be called (technically this is both stubbing and mocking `which are +different things +<https://martinfowler.com/articles/mocksArentStubs.html#TheDifferenceBetweenMocksAndStubs>`_, +but KUnit combines them as many other xUnit testing libraries do) and also how +the mock should behave when those expectations are met (see +``test_long_return``). + +Mocks are extremely powerful as they allow you the finest possible granularity +for verifying how units interact, and allows the injection of arbitrary +behavior. But as Uncle Ben said, "Great power comes with great responsibility." +Mocks are not to be used lightly; they make it possible to test things which are +otherwise difficult or impossible to test, but when used improperly they have a +much higher maintenance burden than using the real thing or even a high quality +fake. + +Compare the ``eeprom_buffer_test_does_not_write_until_flush`` in the faking +example to the above version that uses mocking. It is pretty clear that the +version that uses faking is easier to read. It is also pretty clear that common +behavior between test cases would have to be duplicated with the mocking +version; the fake has the advantage of implementing desired behavior in a single +place. Finally, it is pretty clear that the fake would be much easier to +maintain. Of course what's even easier than having to maintain a fake is not +not having to maintain anything at all. Thus, + +.. important:: + Always prefer high quality fakes over mocks, and always prefer "real" code to + fakes. + +Fakes should generally be used when there is an external dependency that there +is no way around; in the kernel that usually means hardware. If you write a fake +you have to make sure it can be maintained; consequently, it is just as +important as real code and it should get its own tests to verify it works as +expected. Yes, we are telling you to write tests for your fakes. + +Of course sometimes faking something out is infeasible, or there is some code +that is just otherwise impossible to reach; generally this means that your code +should be refactored, but not always. Either way, well tested code in need of +refactoring is better than code that needs refactoring but has no tests. This +leads to the single most important testing principle that overrides all others: + +.. important:: + **Always prefer tests over no tests, no matter what!** + +For more information on class mocking see :doc:`api/class-and-function-mocking`. + +Mocking Arbitrary Functions +--------------------------- + +.. important:: + Always prefer class mocking over arbitrary function mocking where possible. + Class mocking has a much more limited scope and provides more control. + +Sometimes it is necessary to mock a function that does not use any class style +indirection. First and foremost, if you encounter this in your own code, please +rewrite it so that uses class style indirection discussed above, but if this is +in some code that is outside of your control you may use KUnit's function +mocking features. + +KUnit provides macros to allow arbitrary functions to be overridden so that the +original definition is replaced with a mock stub. For most functions, all you +have to do is label the function ``__mockable``: + +.. code-block:: c + + int __mockable example(int arg) {...} + +If a function is ``__mockable`` and a mock is defined: + +.. code-block:: c + + DEFINE_FUNCTION_MOCK(example, RETURNS(int), PARAMS(int)); + +When the function is called, the mock stub will actually be called. + +.. note:: + There is no performance penalty or potential side effects from doing this. + When not compiling for testing, ``__mockable`` compiles away. + +.. note:: + ``__mockable`` does not work on inlined functions. + +Spying +~~~~~~ + +Sometimes it is desirable to have a mock function that delegates to the original +definition in some or all circumstances. This is called *spying*: + +.. code-block:: c + + DEFINE_SPYABLE(i2c_add_adapter, RETURNS(int), PARAMS(struct i2c_adapter *)); + int REAL_ID(i2c_add_adapter)(struct i2c_adapter *adapter) + { + ... + } + +This allows the function to be overridden by a mock as with ``__mockable``; +however, it associates the original definition of the function with an alternate +symbol that KUnit can still reference. This makes it possible to mock the +function and then have the mock delegate to the original function definition +with the ``INVOKE_REAL(...)`` action: + +.. code-block:: c + + static int aspeed_i2c_test_init(struct test *test) + { + struct mock_param_capturer *adap_capturer; + struct mock_expectation *handle; + struct aspeed_i2c_test *ctx; + int ret; + + ctx = test_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + test->priv = ctx; + + handle = TEST_EXPECT_CALL( + i2c_add_adapter(capturer_to_matcher(adap_capturer))); + handle->action = INVOKE_REAL(test, i2c_add_adapter); + ret = of_fake_probe_platform_by_name(test, + "aspeed-i2c-bus", + "test-i2c-bus"); + if (ret < 0) + return ret; + + ASSERT_PARAM_CAPTURED(test, adap_capturer); + ctx->adap = mock_capturer_get(adap_capturer, struct i2c_adapter *); + + return 0; + } + +For more information on function mocking see +:doc:`api/class-and-function-mocking`. + +Platform Mocking +---------------- +The Linux kernel generally forbids normal code from accessing architecture +specific features. Instead, low level hardware features are usually abstracted +so that architecture specific code can live in the ``arch/`` directory and all +other code relies on APIs exposed by it. + +KUnit provides a mock architecture that currently allows mocking basic IO memory +accessors and in the future will provide even more. A major use case for +platform mocking is unit testing platform drivers, so KUnit also provides +helpers for this as well. + +In order to use platform mocking, ``CONFIG_PLATFORM_MOCK`` must be enabled in +your ``kunitconfig``. + +For more information on platform mocking see :doc:`api/platform-mocking`. + +Method Call Expectations +======================== +Once we have classes and methods mocked, we can place more advanced +expectations. Previously, we could only place expectations on simple return +values. With the :c:func:`TEST_EXPECT_CALL` macro, which allows you to make +assertions that a certain mocked function is called with specific arguments +given some code to be run. + +Basic Usage +----------- +Imagine we had some kind of dependency like this: + +.. code-block:: c + + struct Printer { + void (*print)(int arg); + }; + + // Printer's print + void printer_print(int arg) + { + do_something_to_print_to_screen(arg); + } + + struct Foo { + struct Printer *internal_printer; + void (*print_add_two)(struct Foo*, int); + }; + + // Foo's print_add_two: + void foo_print_add_two(struct Foo *this, int arg) + { + internal_printer->print(arg + 2); + } + +and we wanted to test ``struct Foo``'s behaviour, that ``foo->print_add_two`` +actually adds 2 to the argument passed. To properly unit test this, we create +mocks for all of ``struct Foo``'s dependencies, like ``struct Printer``. +We first setup stubs for ``MOCK(Printer)`` and its ``print`` function. + +In the real code, we'd assign a real ``struct Printer`` to the +``internal_printer`` variable in our ``struct Foo`` object, but in the +test, we'd construct a ``struct Foo`` with our ``MOCK(Printer)``. + +Finally, we can place expectations on the ``MOCK(Printer)``. + +For example: + +.. code-block:: c + + static int test_foo_add_two(struct test *test) + { + struct MOCK(Printer) *mock_printer = get_mocked_printer(); + struct Foo *foo = initialize_foo(mock_printer); + + // print() is a mocked method stub + TEST_EXPECT_CALL(print(test_any(test), test_int_eq(test, 12))); + + foo->print_add_two(foo, 10); + } + +Here, we expect that the printer's print function will be called (by default, +once), and that it will be called with the argument ``12``. Once we've placed +expectations, we can call the function we want to test to see that it behaves +as we expected. + +Matchers +-------- +Above, we see ``test_any`` and ``test_int_eq``, which are matchers. A matcher +simply asserts that the argument passed to that function call fulfills some +condition. In this case, ``test_any()`` matches any argument, and +``test_int_eq(12)`` asserts that the argument passed to that function must +equal 12. If we had called: ``foo->print_add_two(foo, 9)`` instead, the +expectation would not have been fulfilled. There are a variety of built-in +matchers: :doc:`api/class-and-function-mocking` has a section about these +matchers. + +.. note:: + :c:func:`TEST_EXPECT_CALL` only works with mocked functions and methods. + Matchers may only be used within the function inside the + :c:func:`TEST_EXPECT_CALL`. + +Additional :c:func:`EXPECT_CALL` Properties +------------------------------------------- + +The return value of :c:func:`TEST_EXPECT_CALL` is a ``struct +mock_expectation``. We can capture the value and add extra properties to it as +defined by the ``struct mock_expectation`` interface. + +Times Called +~~~~~~~~~~~~ +In the previous example, if we wanted assert that the method is never called, +we could write: + +.. code-block:: c + + ... + struct mock_expectation* handle = TEST_EXPECT_CALL(...); + handle->min_calls_expected = 0; + handle->max_calls_expected = 0; + ... + +Both those fields are set to 1 by default and can be changed to assert a range +of times that the method or function is called. + +Mocked Actions +~~~~~~~~~~~~~~ +Because ``mock_printer`` is a mock, it doesn't actually perform any task. If +the function had some side effect that ``struct Foo`` requires to have been +done, such as modifying some state, we could mock that as well. + +Each expectation has an associated ``struct mock_action`` which can be set with +``handle->action``. By default, there are two actions that mock return values. +Those can also be found in :doc:`api/class-and-function-mocking`. + +Custom actions can be defined by simply creating a ``struct mock_action`` and +assigning the appropriate function to ``do_action``. Mocked actions have access +to the parameters passed to the mocked function, as well as have the ability to +change / set the return value. + + +The Nice, the Strict, and the Naggy +=================================== +KUnit has three different mock types that can be set on a mocked class: nice +mocks, strict mocks, and naggy mocks. These are set via the corresponding macros +:c:func:`NICE_MOCK`, :c:func:`STRICT_MOCK`, and :c:func:`NAGGY_MOCK`, with naggy +mocks being the default. + +The type of mock simply dictates the behaviour the mock exhibits when +expectations are placed on it. + ++-----------------------+------------+--------------------+--------------------+ +| | **Nice** | **Naggy (default)**| **Strict** | ++-----------------------+------------+--------------------+--------------------+ +| Method called with no | Do nothing | Prints warning for | Fails test, prints | +| expectations on it | | uninteresting call | warning | +| | | | uninteresting call | ++-----------------------+------------+--------------------+--------------------+ +| Method called with no | Fails test, prints warnings, prints tried | +| matching expectations | expectations | +| on it | | ++-----------------------+------------------------------------------------------+ +| Test ends with an | Fail test, print warning | +| unfulfilled | | +| expectation | | ++-----------------------+------------------------------------------------------+ + +These macros take a ``MOCK(struct_name)`` and so should be used when retrieving +the mocked object. Following the example in :doc:`start`, there was this test +case: + +.. code-block:: c + + static void misc_example_bar_test_success(struct test *test) + { + struct MOCK(misc_example) *mock_example = test->priv; + struct misc_example *example = mock_get_trgt(mock_example); + struct mock_expectation *handle; + + handle = TEST_EXPECT_CALL(misc_example_foo(mock_get_ctrl(mock_example), + test_int_eq(test, 5))); + handle->action = int_return(test, 0); + + TEST_EXPECT_EQ(test, 0, misc_example_bar(example)); + } + +If we wanted ``mock_example`` to be a nice mock instead, we would simply write: + +.. code-block:: c + + struct MOCK(misc_example) *mock_example = NICE_MOCK(test->priv);