136 lines
5.1 KiB
Markdown
136 lines
5.1 KiB
Markdown
|
# Testing the CHRE Framework
|
|||
|
|
|
|||
|
|
[TOC]
|
|||
|
|
|
|||
|
|
The CHRE framework can be tested at various levels to ensure that
|
|||
|
|
components/modules components are working correctly and API contracts are being
|
|||
|
|
met. Below are some examples of what the team currently does to verify new
|
|||
|
|
changes.
|
|||
|
|
|
|||
|
|
## Unit tests
|
|||
|
|
|
|||
|
|
### Tests run on a host machine
|
|||
|
|
|
|||
|
|
Currently, unit tests exist for various core components and utilities capable
|
|||
|
|
of running on a Linux host machine. Since
|
|||
|
|
platform-specific components likely aren’t compilable/available on a host
|
|||
|
|
machine, only components that are OS independent can be tested via this path.
|
|||
|
|
|
|||
|
|
In order to write new tests, add a test source file under the test directory in
|
|||
|
|
the appropriate subdirectory. e.g. `util/tests`. Then, add the file to the
|
|||
|
|
`GOOGLETEST_SRCS` variable in the appropriate .mk file for that subdir,
|
|||
|
|
`util/util.mk` for example.
|
|||
|
|
|
|||
|
|
Unit tests can be built and executed using `run_tests.sh`.
|
|||
|
|
|
|||
|
|
|
|||
|
|
### On-device unit tests
|
|||
|
|
|
|||
|
|
#### Background
|
|||
|
|
|
|||
|
|
The framework aims to provide an environment to test CHRE (and its users) code
|
|||
|
|
on-device, using [Pigweed's][PW_URL] Unit Test [Framework][PW_UT_URL]. Test
|
|||
|
|
instantiations are syntactically identical to [Googletest][GT_URL], so
|
|||
|
|
modifications to on-host unit tests to run on-device are easier.
|
|||
|
|
|
|||
|
|
CHRE recommends running the same host-side gtests on-device using this
|
|||
|
|
framework, to catch subtle bugs. For example, the target CPU may raise an
|
|||
|
|
exception on unaligned access, when the same code would run without any
|
|||
|
|
problems on a local x86-based machine.
|
|||
|
|
|
|||
|
|
#### Use Cases
|
|||
|
|
|
|||
|
|
Example use cases of the framework include:
|
|||
|
|
|
|||
|
|
* In continuous integration frameworks with device farms
|
|||
|
|
* As a superior alternative to logging and/or flag based debugging to quickly test a feature
|
|||
|
|
* As a modular set of tests to test feature or peripheral functioning (eg: a system timer implementation) during device bringup.
|
|||
|
|
|
|||
|
|
###### Note
|
|||
|
|
|
|||
|
|
One key difference is to run the tests via a call to `chre::runAllTests` in
|
|||
|
|
_chre/test/common/unit_test.h_, which basically wraps the gtest `RUN_ALL_TESTS`
|
|||
|
|
macro, and implements CHRE specific event handlers for Pigweed's UT Framework.
|
|||
|
|
|
|||
|
|
#### Running Tests
|
|||
|
|
|
|||
|
|
Under the current incarnation of the CHRE Unit Test Framework, the following
|
|||
|
|
steps need to be taken to run the on-device tests:
|
|||
|
|
* Set to true and export an environment variable called `CHRE_ON_DEVICE_TESTS_ENABLED`
|
|||
|
|
from your Makefile invocation before CHRE is built.
|
|||
|
|
* Ensure that this flag is not always set to avoid codesize bloat.
|
|||
|
|
* Append your test source file to `COMMON_SRCS` either in _test/test.mk_ or in
|
|||
|
|
your own Makefile.
|
|||
|
|
* Call `chre::runAllTests` from somewhere in your code.
|
|||
|
|
|
|||
|
|
##### Sample code
|
|||
|
|
|
|||
|
|
In _math_test.cc_
|
|||
|
|
```cpp
|
|||
|
|
#include <gtest/gtest.h>
|
|||
|
|
|
|||
|
|
TEST(MyMath, Add) {
|
|||
|
|
int x = 1, y = 2;
|
|||
|
|
int result = myAdd(x, y);
|
|||
|
|
EXPECT_EQ(result, 3);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
In _some_source.cc_
|
|||
|
|
```cpp
|
|||
|
|
#include "chre/test/common/unit_test.h"
|
|||
|
|
|
|||
|
|
void utEntryFunc() {
|
|||
|
|
chre::runAllTests();
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
#### Caveats
|
|||
|
|
|
|||
|
|
Some advanced features of gtest (SCOPED_TRACE, etc.) are unsupported by Pigweed.
|
|||
|
|
|
|||
|
|
#### Compatibility
|
|||
|
|
|
|||
|
|
The framework has been tested with Pigweed's git revision ee460238b8a7ec0a6b4f61fe7e67a12231db6d3e.
|
|||
|
|
|
|||
|
|
## PAL implementation tests
|
|||
|
|
|
|||
|
|
PAL implementation tests verify implementations of PAL interfaces adhere to the
|
|||
|
|
requirements of that interface, and are intended primarily to support
|
|||
|
|
development of PAL implementations, typically done by a chip vendor partner.
|
|||
|
|
Additional setup may be required to integrate with the PAL under test and supply
|
|||
|
|
necessary dependencies. The source code is in the files under `pal/tests/src`
|
|||
|
|
and follows the naming scheme `*_pal_impl_test.cc`.
|
|||
|
|
|
|||
|
|
In order to run PAL tests, run: `run_pal_impl_tests.sh`. Note that there are
|
|||
|
|
also PAL unit tests in the same directory. The unit tests are added to the
|
|||
|
|
`GOOGLETEST_SRCS` target while PAL tests are added to the
|
|||
|
|
`GOOGLETEST_PAL_IMPL_SRCS` target.
|
|||
|
|
|
|||
|
|
## FeatureWorld nanoapps
|
|||
|
|
|
|||
|
|
Located under the `apps/` directory, FeatureWorld nanoapps interact with the set
|
|||
|
|
of CHRE APIs that they are named after, and can be useful during framework
|
|||
|
|
development for manual verification of a feature area. For example, SensorWorld
|
|||
|
|
attempts to samples from sensors and outputs to the log. It also offers a
|
|||
|
|
break-it mode that randomly changes which sensors are being sampled in an
|
|||
|
|
attempt to point out stress points in the framework or platform implementation.
|
|||
|
|
|
|||
|
|
These apps are usually built into the CHRE framework binary as static nanoapps
|
|||
|
|
to facilitate easy development. See the Deploying Nanoapps section for more
|
|||
|
|
information on static nanoapps.
|
|||
|
|
|
|||
|
|
## CHQTS
|
|||
|
|
|
|||
|
|
The Context Hub Qualification Test Suite (CHQTS) tests perform end-to-end
|
|||
|
|
validation of a CHRE implementation, by using the Java APIs in Android to load
|
|||
|
|
and interact with test nanoapps which then exercise the CHRE API. While this
|
|||
|
|
code is nominally integrated in another test suite, the source code is available
|
|||
|
|
under `java/test/chqts/` for the Java side code and `apps/test/chqts/` for the
|
|||
|
|
CHQTS-only nanoapp code and `apps/test/common/` for the nanoapp code shared by
|
|||
|
|
CHQTS and other test suites.
|
|||
|
|
|
|||
|
|
[PW_URL]: https://pigweed.dev
|
|||
|
|
[PW_UT_URL]: https://pigweed.googlesource.com/pigweed/pigweed/+/refs/heads/master/pw_unit_test
|
|||
|
|
[GT_URL]: https://github.com/google/googletest
|