[Toaster] [review-request] 7414-cut the prefix from layer file paths for bitbake variables
Damian, Alexandru
alexandru.damian at intel.com
Wed Mar 25 06:04:31 PDT 2015
I've submitted your patch upstream after testing.
Please wait until it's merged, and then mark Resolved/Fixed.
Alex
On Tue, Mar 24, 2015 at 7:44 PM, Lerner, David M (Wind River) <
dave.lerner at windriver.com> wrote:
> Hi Alex,
> I'm not sure if you want me to take back ownership and mark it
> Resolved/Fixed or if you are just suggesting a different alternative than
> my patch and will handle it.
> Dave
>
> > -----Original Message-----
> > From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
> > Sent: Tuesday, March 24, 2015 10:29 AM
> > To: Lerner, Dave
> > Cc: BARROS PENA, BELEN; toaster at yoctoproject.org
> > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer
> file paths for
> > bitbake variables
> >
> > Hi,
> >
> >
> > After thinking about the issue, if we mangle at build time, we'll have
> trouble
> > distinguishing between paths originating in the layer directories and
> paths in the build
> > directory. So the good solution is that we mangle the path at display
> time, and I will
> > be taking this patch.
> >
> >
> > I have some more comments below,
> >
> >
> > Cheers,
> >
> > Alex
> >
> >
> > On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River)
> > <dave.lerner at windriver.com> wrote:
> >
> >
> > Alex,
> > I've thought about this some more... see below.
> > Dave
> >
> > > -----Original Message-----
> > > From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
> > > Sent: Tuesday, March 17, 2015 10:19 AM
> > > To: BARROS PENA, BELEN
> > > Cc: Lerner, Dave; toaster at yoctoproject.org
> > > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from
> layer file paths
> > for
> > > bitbake variables
> > >
> > > Hi,
> > >
> > >
> > > Sorry for the significant delay in addressing this patch - I've
> been a bit
> > overwhelmed.
> >
> > It's not a significant delay and the delay is not a problem.
> >
> > >
> > >
> > > I am NOT taking this patch because of the way the layer matching
> is done.
> > >
> > >
> > > We already have local paths for layers present in the build in
> the Layer object;
> > we also
> > > know _where_ the build took place, for we have
> > > "build.buildrequest.buildenvironment.builddir" variable that
> holds the path to
> > the build
> > > directory.
> >
> > Either there is a bug, or your claim isn't quite right regarding
> > orm_layer.local_paths which are all empty or null for a yocto release
> build. That was
> > my first choice examining the database, but not possible since data was
> missing.
> >
> >
> >
> > I should've been more clear than this. The layers which HAVE BEEN
> built have local_path
> > set to a correct value. The layers that we know about because they are
> imported manually
> > or from Layer Index, the local_path is NULL. The variables are always
> attached to a
> > build, so the layers on which the variables are modified will have
> local_path set to
> > something else than NULL.
> >
> >
> >
> >
> >
> >
> > > We must verify that the file path starts with one of the layer
> paths or the build
> > > environment builddir path, and strip that leading path from the
> full file path.
> > >
> >
> > I agree that would make more sense, but that data isn't in the
> table, and I believe
> > due to performance reasons, that it should be done at display-a-page
> time.
> >
> > > I would also expect this to be done at data logging time instead
> of data display
> > time
> > > (i.e. in the buildinfohelper.py), with the distinct advantage of
> speeding up page
> > > processing time.
> >
> >
> > Since build-time layer path extraction for the given database
> structures penalizes
> > all build times for the low likelihood that some user wants the path to
> a layer, I would
> > recommend adding just a little more processing time to the data logging
> as follows:
> >
> > a) Constraining Layer.local_path to non-null, and always
> populating that field,
> >
> >
> >
> > This is impossible due to the fact that we keep info about layers that
> have not been
> > checked-out in that table.
> >
> >
> >
> >
> > b) normalizing things a bit by creating a new Model Config which
> has:
> > b1) the full path that was formerly in the (non-normalized)
> > VariableHistory.file_name and
> > b2) a foreign key to from Config to it's Layer, calculated once
> > c) modifying (normalizing) orm_variablehistory
> > c1) removing orm_variablehistory.file_name and
> > c2) replacing that with a foreign key to it's defining Config
> model.
> >
> >
> >
> >
> > Yes, this makes sense. Still it wouldn't work 100% because some
> variables are modified
> > outside any Layer checkout (i.e. in the build directory). Having foreign
> keys to
> > different objects to re-compute the path in the "Config" model would be
> unworkable in
> > the queries sense.
> >
> >
> >
> >
> > Then display time could just join on demand to find the config
> file's layer's local
> > path whenever the last bitbake path or history is required.
> >
> > Or else provide the display-time matching against a non-null
> Layer.file_name, which
> > would be much faster since the set of layers is of order 10^ and the
> list to match is at
> > most 150 variables(per page), still my first choice.
> >
> > Dave
> >
> > >
> > > Thank you,
> > >
> > > Alex
> > >
> >
> > My recommendations are:
> > 1) to populate the Layer model with local_path in all cases,
> changing the
> > local_path to NOT NULL
> > 2) perform the trim of variable names at display time, which
> should be faster than
> > penalizing builds with Config model updates, or variable defined file
> stripping for
> > every new config file read.
> > OR)
> > 3) add the Config model, update each variable log with the key to
> the Config model.
> >
> >
> >
> > >
> > > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <
> belen.barros.pena at intel.com>
> > wrote:
> > >
> > >
> > >
> > >
> > > On 16/03/2015 14:14, "Lerner, Dave" <
> dave.lerner at windriver.com> wrote:
> > >
> > > >Hi Belen,
> > > >
> > > >> -----Original Message-----
> > > >> From: Barros Pena, Belen [mailto:
> belen.barros.pena at intel.com]
> > > >> Sent: Friday, March 13, 2015 9:17 AM
> > > >> To: Lerner, Dave
> > > >> Cc: toaster at yoctoproject.org
> > > >> Subject: Re: [review-request] 7414-cut the prefix from
> layer file paths
> > > >>for bitbake
> > > >> variables
> > > >>
> > > >> Hi Dave,
> > > >>
> > > >> This is looking good. But I've realised I forgot about
> something. Not
> > > >>all
> > > >> files setting variables are inside a layer: some of
> them are
> > > >>configuration
> > > >> files inside the /build or /bitbake directories.
> > > >>
> > > >> I guess we should also cut those, showing only the path
> inside the root
> > > >> source directory. So something like
> > > >>
> > > >> /home/user/poky/build/conf/local.conf
> > > >>
> > > >> Would show as
> > > >>
> > > >> build/conf/local.conf
> > > >>
> > > >> I am not fully sure if this is the right thing to do,
> so I'd be
> > > >>interested
> > > >> in hearing what you and the others think.
> > > >>
> > > >> I also realise this was not explicitly explained in the
> original bug. If
> > > >> you want me to open a separate issue in Bugzilla, I can
> do so: just let
> > > >>me
> > > >> know.
> > > >
> > > >Please open a separate bug and give Alex the ok for
> 7414. Layer relative
> > > >makes sense to me, active-build-directory-relative does
> not, too many
> > > >questions "what is the relative to?" are raised.
> > >
> > >
> > >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v
> >
> > > arpath_7414 <
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-
> > > contrib/log/?h=lerner/trim_v arpath_7414>
> >
> > >
> > > is good to go, then. I've opened this
> > >
> > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463
> > >
> > > for the configuration file paths.
> > >
> > > Cheers
> > >
> > > Belén
> > >
> > >
> > >
> > > >
> > > >Dave
> > > >
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Belén
> > > >>
> > > >> On 12/03/2015 15:48, "Lerner, Dave" <
> dave.lerner at windriver.com> wrote:
> > > >>
> > > >> >rebased.
> > > >> >Dave
> > > >> >
> > > >> >> -----Original Message-----
> > > >> >> From: Lerner, Dave
> > > >> >> Sent: Thursday, March 12, 2015 10:46 AM
> > > >> >> To: Barros Pena, Belen (belen.barros.pena at intel.com)
> > > >> >> Subject: RE: [review-request] 7414-cut the prefix
> from layer file
> > > >>paths
> > > >> >>for bitbake
> > > >> >> variables
> > > >> >>
> > > >> >> I don't think that I rebased (since Tuesday) before
> I pushed. Can
> > > >>you
> > > >> >>hold off pulling
> > > >> >> for 15 minutes? I'll send a note when rebased.
> > > >> >> Dave
> > > >> >>
> > > >> >> > -----Original Message-----
> > > >> >> > From: Lerner, Dave
> > > >> >> > Sent: Thursday, March 12, 2015 10:29 AM
> > > >> >> > To: Barros Pena, Belen (
> belen.barros.pena at intel.com); Damian,
> > > >> >>Alexandru
> > > >> >> > (alexandru.damian at intel.com)
> > > >> >> > Cc: toaster at yoctoproject.org
> > > >> >> > Subject: [review-request] 7414-cut the prefix from
> layer file paths
> > > >> >>for bitbake
> > > >> >> > variables
> > > >> >> >
> > > >> >> >
> > > >>
> > > >>>>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr
> > > >>>>im
> > > >> >>_varpath_7414
> > > >> >> >
> > > >> >> >
> > > >> >> > Page Testing:
> > > >> >> > Create project using yocto master which includes
> openembedded-core,
> > > >> >>meta-yocto and
> > > >> >> meta-
> > > >> >> > yocto-bsp, then add layers meta-oe, meta-games.
> > > >> >> > Build the project.
> > > >> >> >
> > > >> >> > Open the build, configuration page, bitbake
> variables.
> > > >> >> > Note the following types of variable path names in
> the "Set in
> > > >>file"
> > > >> >>column and when
> > > >> >> > clicking the variable in the History.
> > > >> >> >
> > > >> >> > Variable: B
> > > >> >> > Set in file/history: meta/conf/bitbake.conf
> > > >> >> >
> > > >> >> > Variable: BBFILE_PATTERN_core
> > > >> >> > Set in file: full path to configure
> > > >> >> > History: first meta/conf/layer.conf, then
> > > >> >> >
> /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py
> > > >> >> >
> > > >> >> > Variable: BBFILE_PATTTERN_games-layers
> > > >> >> > Set in file and History:
> > > >> >>meta-games_master.toaster_cloned/conf/layer.conf
> > > >> >> >
> > > >> >> >
> > > >> >> > HTML Validation: Pass
> > > >> >> > Dave Lerner
> > > >>
> > > >
> > >
> > > --
> > > _______________________________________________
> > > toaster mailing list
> > > toaster at yoctoproject.org
> > > https://lists.yoctoproject.org/listinfo/toaster
> > >
> > >
> > >
> > >
> > >
> > > --
> > >
> > > Alex Damian
> > > Yocto Project
> > >
> > > SSG / OTC
> >
> >
> >
> >
> >
> > --
> >
> > Alex Damian
> > Yocto Project
> >
> > SSG / OTC
>
--
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20150325/06a98b20/attachment-0001.html>
More information about the toaster
mailing list