[yocto] 1.3 M3 Full Pass test results

[Xu, Jiajun]

We used to get 115m and 127m for 1.3 M1 and M2 on our test machine.

> Hello guys,
> 
> I am sorry I did not notice the last line in the e-mail.
> 
> When we ran the performance testing we pre-downloaded all the sources,
> because last week had a lot of connectivity issues. I mentioned in the
> e-mail that the build time refers to an image built after all the
> sources were downloaded.
> The machine on which the test was ran is the same as for M1.
> 
> I will ask Stefan to re-run the performance tests for the different
> milestones so far, M1 and M2, in the same conditions as for M3 so we
> would have a clear view.
> 
> I add Jiajun in the loop so he can help us if he can with a test to
> see if this is related to an artifact as Richard said, but also there
> were some improvements made by Beth on filesystem generation.
> 
> Br,
> Laurentiu
> 
> -----Original Message-----
> From: Liu, Song
> Sent: Monday, August 27, 2012 8:50 PM
> To: Serban, Laurentiu; Purdie, Richard
> Cc: yocto at yoctoproject.org; Stewart, David C; Wold, Saul
> Subject: RE: 1.3 M3 Full Pass test results
> 
> Hi Laurentiu,
> 
> Do you have any comment on the performance question Richard asked? I
> know your team is using a machine with different configuration from
> the one used by the Shanghai team. The performance figure from the
> Shanghai team has been hovering around 110 mins. That's the case for
> 1.2 release and 1.3 M2 milestone report. 1.3 M1 milestone report has
> the build time as 95 minutes, which I believe is from your team. So my
> question is whether you used the same machine for M3 performance
> testing as for M1. Another factor that might have caused the
> difference (between 95 and 83 minutes) is your testing
> procedure/environment such as other processes running at the same
> time, memory usage, sstate cache, etc. If you used the same machine
> and same testing procedure/environment, then we have some improvement in M3 compared with M1. Please let me know your thoughts.
> 
> Thanks,
> Song
> 
> -----Original Message-----
> From: Serban, Laurentiu
> Sent: Friday, August 24, 2012 6:34 AM
> To: Purdie, Richard
> Cc: yocto at yoctoproject.org; Liu, Song; Stewart, David C; Wold, Saul
> Subject: RE: 1.3 M3 Full Pass test results
> 
> Hello Richard,
> 
> Even if the installer is used in the default mode, issues still occur
> (see comment 7). I think the root cause for these is the same, so I did not submit a new bug.
> 
> Thank you,
> Laurentiu
> 
> -----Original Message-----
> From: Purdie, Richard
> Sent: Friday, August 24, 2012 1:22 PM
> To: Serban, Laurentiu
> Cc: yocto at yoctoproject.org; Liu, Song; Stewart, David C; Wold, Saul
> Subject: Re: 1.3 M3 Full Pass test results
> 
> On Thu, 2012-08-23 at 20:13 +0100, Serban, Laurentiu wrote:
> 
>> Here are the results for the full pas tests on 1.3 M3 RC2. The
>> commit used for testing is 8b8748c8f963900b83dc0fdd7757556f917fe4fd.
>> 
>> Some details about the encountered issues below:
>> 
>> BSP – Sudoku-savant project build issue (2878)
>> 
>> ADT – the relocatable sdk issue (2980) causes 13 test cases to be on
>> faile/blocked state
> 
> I thought it worked as long as you didn't have to relocate it so no
> tests should have been blocked, we just have the relocation issue?
> 
>>  , also the Clutter C template issue is unsolved (2577)
>> Core Build System – x32 is still an issue (2888), cleaning sstate
>> issue is still not solved (2897), incremental RPM image generation
>> (2969), source archiving (2619), the kvm issue was reproduced by
>> another colleague (2790) Yocto BSP creation via JSON (2693) or for
>> qemu (2991) fails, multilib issue (2918 – this requires a little
>> more investigation from QA),
>> 
>> HOB - all seems ok for RC2
>> 
>> Self-hosted-image  - cannot start on Virtual Box (X issue), it is
>> very slow on qemu and it has a m4 package build (3005) issue on
>> VMWare. If the self-hosted-image is used on machine with internet
>> connectivity via proxy there will be an initial sanity check
>> failure, but this is not a blocking issue.
>> 
>> A mention for the performance testing: on a Ubbuntu 12.04  i7
>> machine using 8 threads the build time was 83 minutes (with prior fetching).
> 
> How does this compare with our other performance numbers. From what I
> remember, we used to hover around the 105-115 minute mark. Did we have
> some significant speed gains or is this just an artefact of changing
> the test machine?
> 
> Cheers,
> 
> Richard
> 
>

Best Regards,
Jiajun