136 lines
5.1 KiB
Markdown
136 lines
5.1 KiB
Markdown
# Testing the CHRE Framework
|
||
|
||
[TOC]
|
||
|
||
The CHRE framework can be tested at various levels to ensure that
|
||
components/modules components are working correctly and API contracts are being
|
||
met. Below are some examples of what the team currently does to verify new
|
||
changes.
|
||
|
||
## Unit tests
|
||
|
||
### Tests run on a host machine
|
||
|
||
Currently, unit tests exist for various core components and utilities capable
|
||
of running on a Linux host machine. Since
|
||
platform-specific components likely aren’t compilable/available on a host
|
||
machine, only components that are OS independent can be tested via this path.
|
||
|
||
In order to write new tests, add a test source file under the test directory in
|
||
the appropriate subdirectory. e.g. `util/tests`. Then, add the file to the
|
||
`GOOGLETEST_SRCS` variable in the appropriate .mk file for that subdir,
|
||
`util/util.mk` for example.
|
||
|
||
Unit tests can be built and executed using `run_tests.sh`.
|
||
|
||
|
||
### On-device unit tests
|
||
|
||
#### Background
|
||
|
||
The framework aims to provide an environment to test CHRE (and its users) code
|
||
on-device, using [Pigweed's][PW_URL] Unit Test [Framework][PW_UT_URL]. Test
|
||
instantiations are syntactically identical to [Googletest][GT_URL], so
|
||
modifications to on-host unit tests to run on-device are easier.
|
||
|
||
CHRE recommends running the same host-side gtests on-device using this
|
||
framework, to catch subtle bugs. For example, the target CPU may raise an
|
||
exception on unaligned access, when the same code would run without any
|
||
problems on a local x86-based machine.
|
||
|
||
#### Use Cases
|
||
|
||
Example use cases of the framework include:
|
||
|
||
* In continuous integration frameworks with device farms
|
||
* As a superior alternative to logging and/or flag based debugging to quickly test a feature
|
||
* As a modular set of tests to test feature or peripheral functioning (eg: a system timer implementation) during device bringup.
|
||
|
||
###### Note
|
||
|
||
One key difference is to run the tests via a call to `chre::runAllTests` in
|
||
_chre/test/common/unit_test.h_, which basically wraps the gtest `RUN_ALL_TESTS`
|
||
macro, and implements CHRE specific event handlers for Pigweed's UT Framework.
|
||
|
||
#### Running Tests
|
||
|
||
Under the current incarnation of the CHRE Unit Test Framework, the following
|
||
steps need to be taken to run the on-device tests:
|
||
* Set to true and export an environment variable called `CHRE_ON_DEVICE_TESTS_ENABLED`
|
||
from your Makefile invocation before CHRE is built.
|
||
* Ensure that this flag is not always set to avoid codesize bloat.
|
||
* Append your test source file to `COMMON_SRCS` either in _test/test.mk_ or in
|
||
your own Makefile.
|
||
* Call `chre::runAllTests` from somewhere in your code.
|
||
|
||
##### Sample code
|
||
|
||
In _math_test.cc_
|
||
```cpp
|
||
#include <gtest/gtest.h>
|
||
|
||
TEST(MyMath, Add) {
|
||
int x = 1, y = 2;
|
||
int result = myAdd(x, y);
|
||
EXPECT_EQ(result, 3);
|
||
}
|
||
```
|
||
|
||
In _some_source.cc_
|
||
```cpp
|
||
#include "chre/test/common/unit_test.h"
|
||
|
||
void utEntryFunc() {
|
||
chre::runAllTests();
|
||
}
|
||
```
|
||
|
||
#### Caveats
|
||
|
||
Some advanced features of gtest (SCOPED_TRACE, etc.) are unsupported by Pigweed.
|
||
|
||
#### Compatibility
|
||
|
||
The framework has been tested with Pigweed's git revision ee460238b8a7ec0a6b4f61fe7e67a12231db6d3e.
|
||
|
||
## PAL implementation tests
|
||
|
||
PAL implementation tests verify implementations of PAL interfaces adhere to the
|
||
requirements of that interface, and are intended primarily to support
|
||
development of PAL implementations, typically done by a chip vendor partner.
|
||
Additional setup may be required to integrate with the PAL under test and supply
|
||
necessary dependencies. The source code is in the files under `pal/tests/src`
|
||
and follows the naming scheme `*_pal_impl_test.cc`.
|
||
|
||
In order to run PAL tests, run: `run_pal_impl_tests.sh`. Note that there are
|
||
also PAL unit tests in the same directory. The unit tests are added to the
|
||
`GOOGLETEST_SRCS` target while PAL tests are added to the
|
||
`GOOGLETEST_PAL_IMPL_SRCS` target.
|
||
|
||
## FeatureWorld nanoapps
|
||
|
||
Located under the `apps/` directory, FeatureWorld nanoapps interact with the set
|
||
of CHRE APIs that they are named after, and can be useful during framework
|
||
development for manual verification of a feature area. For example, SensorWorld
|
||
attempts to samples from sensors and outputs to the log. It also offers a
|
||
break-it mode that randomly changes which sensors are being sampled in an
|
||
attempt to point out stress points in the framework or platform implementation.
|
||
|
||
These apps are usually built into the CHRE framework binary as static nanoapps
|
||
to facilitate easy development. See the Deploying Nanoapps section for more
|
||
information on static nanoapps.
|
||
|
||
## CHQTS
|
||
|
||
The Context Hub Qualification Test Suite (CHQTS) tests perform end-to-end
|
||
validation of a CHRE implementation, by using the Java APIs in Android to load
|
||
and interact with test nanoapps which then exercise the CHRE API. While this
|
||
code is nominally integrated in another test suite, the source code is available
|
||
under `java/test/chqts/` for the Java side code and `apps/test/chqts/` for the
|
||
CHQTS-only nanoapp code and `apps/test/common/` for the nanoapp code shared by
|
||
CHQTS and other test suites.
|
||
|
||
[PW_URL]: https://pigweed.dev
|
||
[PW_UT_URL]: https://pigweed.googlesource.com/pigweed/pigweed/+/refs/heads/master/pw_unit_test
|
||
[GT_URL]: https://github.com/google/googletest
|