[poky] [PATCH 7/8] Fetcher: break the "SRCREVINACTION" deadlock

[Yu Ke]

On Jan 06, 11:56, Richard Purdie wrote:
> On Thu, 2011-01-06 at 11:06 +0000, Richard Purdie wrote:
> > On Mon, 2010-12-27 at 21:58 +0800, Yu Ke wrote:
> > > Current fetcher has annoying "SRCREVINACTION" deadlock,
> > > which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev():
> > > get_srcrev()->setup_localpath()->srcrev_internal_helper()
> > > ->evaluate SRCREV->get_srcrev()
> > > 
> > > current fetcher resolve the deadlock by introducing a
> > > "SRCREVINACTION" condition check. Althoguh it works, it is
> > > indeed not clean.
> > > 
> > > This patch use antoehr idea to break the deadlock: break
> > > the dependency among SRCREV and get_srcrev(), i.e. assign
> > > a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet
> > > this keyword, it will check and set the latest revision to
> > > urldata.revision. get_srcrev later can use the urldata.revision
> > > for value evaluation(SRCPV etc). In this case, SRCREV no longer
> > > depends on get_srcrev, and there is not deadlock anymore.
> > > 
> > > In implementation side, several things are done:
> > > a) set the revision in FetchData:__init__, so that
> > >    urldata.revision is ready when get_srcrev() need it
> > > b) init fetch method specific data in Fetch:supports, so that
> > >    Fetch.latest_revision() can be conducted when FetchData:
> > >    __init__ need it to achieve a)
> > 
> > I do think we need to make this change. 
> > 
> > Since we're not doing this in a fetcher "version 2" currently there is
> > the backwards compatibility question though. What does this code do if
> > the bitbake.conf change isn't made? Can we detect the old situation and
> > error at least? This change is going to need a little discussion on the
> > bitbake-dev list...
> > 
> > The answer may be that we need to start a fetch2 codebase to carry on
> > with this kind of cleanup.
> 
> I've given this further thought and lets go with the original plan,
> create a fetch2 and then go and really clean it up.

Ok, I like the idea of having a seperate fetch2. This allow me not to worry about breaking the old fetch code.

> 
> We therefore need to copy the fetch directory to fetch2, then start
> applying patches against fetch2. We can update poky to use bb.fetch2
> instead of bb.fetch. This should also give OE a way to transition too.
> 
> I made a quick grep over bitbake itself and there are some fetch
> references we'll need to resolve to switch between the two fetch
> modules:
> 
> lib/bb/cooker.py:        bb.fetch.persistent_database_connection = {}
> lib/bb/cooker.py:        bb.fetch.persistent_database_connection = {}
> 
> The above two references no longer do anything as the
> persistent_database code was removed in bitbake upstream and for now
> poky has followed that although I would like to revisit it.
> 
> lib/bb/parse/parse_py/BBHandler.py:import bb.fetch, bb.build, bb.utils
> 
> I doubt BBHandler still needs this reference
> 
> lib/bb/cooker.py:            bb.fetch.fetcher_init(self.configuration.data)
> lib/bb/command.py:        if bb.fetch.fetcher_compare_revisions(command.cooker.configuration.data):
> 
> These two are important and we'll need to have a way to switch between
> the fetchers for these.

There are also some bbclase like base.bbclass, distrodata.bbclass, sstate.bbclass, patch.bbclass, utility-tasks.bbclass, utils.bbclass need to take care of. I will search and handle them case by case.

Regards
Ke

> 
> Cheers,
> 
> Richard
> 
> _______________________________________________
> poky mailing list
> poky at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky