https://git.lavasoftware.org/lava/lava/issues/179
If your Lava-Test Test Definition 1.0 YAML files explicitly use a parse: block (like: https://git.linaro.org/qa/test-definitions.git/tree/automated/linux/ltp/ltp....) then this will remain supported in Definition 1.0.
If you use the monitors or interactive test actions, this does not affect you at all.
If you rely on LAVA to create a TestCase based on a command in the Lava-Test Test Definition just echoing "pass" or "fail", then this is the Default Pattern and this change will directly affect those test jobs.
The current Default Pattern and Fixup are lifted directly from V1 (https://git.lavasoftware.org/lava/lava/blob/master/lava_common/constants.py#...):
# V1 compatibility DEFAULT_V1_PATTERN = "(?P<test_case_id>.*-*)\s+:\s+(?P<result>(PASS|pass|FAIL|fail|SKIP|skip|UNKNOWN|unknown))" DEFAULT_V1_FIXUP = { "PASS": "pass", "FAIL": "fail", "SKIP": "skip", "UNKNOWN": "unknown", }
We've recently updated the documentation to drop mention of the default pattern support for the following reasons:
* It has always been problematic to encode a Python regular expression in YAML. Failures are difficult to debug and patterns are global for the entire test operation.
* The move towards more portable test definitions puts the emphasis on parsing the test output locally on the DUT using a customised parser. This has further advantages:
* The pattern does not have to be mangled into YAML * The pattern can be implemented by a language other than Python * The pattern can change during the operation of the test shell, e.g. a different pattern may be required for setup than for the test itself.
We are now starting to plan for Lava-Test Test Definition 2.0 with an emphasis on requiring portable test scripts and removing more of the lava_test_shell Test Helper scripts. Full information on 2.0 will be available early in 2019.
As a first step, the generally unhelpful Default Pattern and Default Fixup dict are likely to be removed. If you need this support, the pattern can be added to your Lava-Test Test Definition 1.0 YAML files. In the next release, it is proposed that unless an explicit pattern is specified in the Lava-Test Test Definition 1.0 YAML file, then no pattern will be implemented. Processes which echo "pass" or "fail" would be ignored and no TestCase would be created.
Let us know if there are any thoughts or problems on this proposal.
--
Neil Williams ============= neil.williams@linaro.org http://www.linux.codehelp.co.uk/