[poky] Processing extra field requested by bitbake -u in Cache

[Joshua Lock]

On Wed, 2011-04-27 at 17:34 +0800, Ke, Liping wrote:
> Hi, all
> 
> I am looking @ bug 770 Add mechanism to enable UI's to request extra
> data be stored in the cache. We are now thinking about the possible
> solutions.

For people wishing to follow:
http://bugzilla.pokylinux.org/show_bug.cgi?id=770

> 
> There are two choices:
> 1. Store extra cache fields in a separated small cache file, so non-ui mode will only fetch the base cache file, if in ui-mode, it will load the extra small cache (with the extra 3 fields, such as summary)
>   Advantage: 1) Will 'not' change the cache invalidation mechanism. For non-ui mode, it will go the old path. If extra fields are required (ui-mode), we need to load the extra field into the memory.
>             2) disk file size is optimized
>   Disadvantage: cache code change will be big, since all data load/sync/ methods need to be modified.

Why must we change the invalidation mechanisms when having a separate
cache file per UI.

My thought is that we'll have a cache file name which encodes the UI and
then run the usual invalidation tests against that cache.

Say that the UI name is known by the cache and stored in variable
currentui we can achieve this by:

-        self.cachefile = os.path.join(self.cachedir, "bb_cache.dat")
+        self.cachefile = os.path.join(self.cachedir, "bb_cache_%s.dat"
% currentui)

Also, surely we *do* need alter the cache invalidation mechanisms for
multiple files, at least insofar as we must test more than one cache
file some times.

> 
> 2. Have two big cache files, one is for ui mode, one is for non-ui mode
>   Advantage: Code change will be a little smaller. When loading, if it's non-ui mode, load smaller cache file. Otherwise, load bigger cache file.  
>   Disadvantage: 1) Cache invalidation mechanism (timestamp) is broken. When mode switching, we need to invalidate all caches, reparsing, reloading time 30s.
>               s) disk file size is bigger. 2*original_size (2*6M)

I'm not sure that 6MB disk overhead is worth the code complexity
increase by having multiple cache files stitched together.

I am definitely keen to hear others opinions here.

Cheers,
Joshua
-- 
Joshua Lock
        Yocto Build System Monkey
        Intel Open Source Technology Centre