On 25 April 2016 at 14:56, Daniel Thompson daniel.thompson@linaro.org wrote:
On 25/04/16 14:14, Dave Pigott wrote:
I’ve compared the “working” and “non-working” kernel logs and there are only two differences I can find.
- In the non working one it has to run an fsck:
https://validation.linaro.org/scheduler/job/845472/log_file#L_349_223
- I can’t imagine that this is causing the problem
- A little later on, you see it tries to bring the network interfaces up:
I was curious about this.
I looked at the kernel traces and was interested to see that the working version has an smsc911x message coming out as soon as the network adapter is used. On close inspection it looks to me like in the working case the smsc911x driver compiled into the kernel, whilst for the non-working version smsc911x is loaded as a module. Worse the module is not loaded until well after we have started trying to bring up the network.
- Working one:
https://validation.linaro.org/scheduler/job/845552/log_file#L_285_9
[ 0.000000] Linux version 3.10.63.0-1-linaro-lt-vexpress64 (buildslave@x86-64-07) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #1ubuntu1~ci+150113192220 SMP Tue Jan 13 19:25:14 UTC 2015
Wrong link, should be:
https://validation.linaro.org/scheduler/job/845552/log_file#L_281_8
Linux version 4.4.0-rc2+ (leoy@leoy-linaro) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #73 SMP PREEMPT Sat Apr 23 23:11:46 CST 2016
Always check the kernel boot prior to the message: INFO: System is in test image now, performing basic user space tests.
- Non working one:
https://validation.linaro.org/scheduler/job/845472/log_file#L_351_10
[ 0.000000] Linux version 4.4.0-rc2+ (leoy@leoy-linaro) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #73 SMP PREEMPT Sat Apr 23 23:11:46 CST 2016
Wrong link, should be: https://validation.linaro.org/scheduler/job/845472/log_file#L_279_8
Linux version 4.4.0-rc2+ (leoy@leoy-linaro) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #73 SMP PREEMPT Sat Apr 23 23:11:46 CST 2016
So it is the same kernel booting each time and the logs also include the checksums of the downloaded files, those match in each job.
It's one of the problems of V1 - there are a lot of kernel boots involved in a juno testjob. You also need to check the reported firmware versions and other messages. It can be best to download both log files, strip out the other kernel boot logs and compare the results that way.