I am seeking assistance with a performance regression observed on an i.MX6SOLO rev 1.4 (996 MHz) board in our LAVA farm.
The Issue: A test suite containing 90 test cases, which previously took 10 minutes to complete, is now consistently taking 20 minutes. This issue is unique to this one specific board; other boards of the same type are still completing the run in 10 minutes using the same build/firmware.
Observations:
I'm not using any test_character_delay externally in my YAML.
Identical Logs: The "Fast" and "Slow" jobs produce identical serial output.
Recent Hangs: The test run has recently started to get stuck/hang mid-run, even though CPU usage is not at 100%.
Debugging Performed:
Verified that CPU thermal throttling is not active.
Network latency and DNS resolution times appear normal.
No hardware errors are reported in dmesg.
Swapped power supplies and network cables with no improvement.
Questions:
What is the best way to debug "dead time" between lava-test-case signals?
Could the LAVA worker be experiencing serial buffer delays on this specific node that don't appear in the logs?