[Toaster] [review-request] 7414-cut the prefix from layer file paths for bitbake variables
Damian, Alexandru
alexandru.damian at intel.com
Tue Mar 24 07:28:51 PDT 2015
Hi,
After thinking about the issue, if we mangle at build time, we'll have
trouble distinguishing between paths originating in the layer directories
and paths in the build directory. So the good solution is that we mangle
the path at display time, and I will be taking this patch.
I have some more comments below,
Cheers,
Alex
On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River) <
dave.lerner at windriver.com> wrote:
> Alex,
> I've thought about this some more... see below.
> Dave
>
> > -----Original Message-----
> > From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
> > Sent: Tuesday, March 17, 2015 10:19 AM
> > To: BARROS PENA, BELEN
> > Cc: Lerner, Dave; toaster at yoctoproject.org
> > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer
> file paths for
> > bitbake variables
> >
> > Hi,
> >
> >
> > Sorry for the significant delay in addressing this patch - I've been a
> bit overwhelmed.
>
> It's not a significant delay and the delay is not a problem.
>
> >
> >
> > I am NOT taking this patch because of the way the layer matching is done.
> >
> >
> > We already have local paths for layers present in the build in the Layer
> object; we also
> > know _where_ the build took place, for we have
> > "build.buildrequest.buildenvironment.builddir" variable that holds the
> path to the build
> > directory.
>
> Either there is a bug, or your claim isn't quite right regarding
> orm_layer.local_paths which are all empty or null for a yocto release
> build. That was my first choice examining the database, but not possible
> since data was missing.
>
I should've been more clear than this. The layers which HAVE BEEN built
have local_path set to a correct value. The layers that we know about
because they are imported manually or from Layer Index, the local_path is
NULL. The variables are always attached to a build, so the layers on which
the variables are modified will have local_path set to something else than
NULL.
>
>
> > We must verify that the file path starts with one of the layer paths or
> the build
> > environment builddir path, and strip that leading path from the full
> file path.
> >
>
> I agree that would make more sense, but that data isn't in the table, and
> I believe due to performance reasons, that it should be done at
> display-a-page time.
>
> > I would also expect this to be done at data logging time instead of data
> display time
> > (i.e. in the buildinfohelper.py), with the distinct advantage of
> speeding up page
> > processing time.
>
>
> Since build-time layer path extraction for the given database structures
> penalizes all build times for the low likelihood that some user wants the
> path to a layer, I would recommend adding just a little more processing
> time to the data logging as follows:
>
> a) Constraining Layer.local_path to non-null, and always populating that
> field,
>
This is impossible due to the fact that we keep info about layers that
have not been checked-out in that table.
> b) normalizing things a bit by creating a new Model Config which has:
> b1) the full path that was formerly in the (non-normalized)
> VariableHistory.file_name and
> b2) a foreign key to from Config to it's Layer, calculated once
> c) modifying (normalizing) orm_variablehistory
> c1) removing orm_variablehistory.file_name and
> c2) replacing that with a foreign key to it's defining Config model.
>
Yes, this makes sense. Still it wouldn't work 100% because some variables
are modified outside any Layer checkout (i.e. in the build directory).
Having foreign keys to different objects to re-compute the path in the
"Config" model would be unworkable in the queries sense.
>
> Then display time could just join on demand to find the config file's
> layer's local path whenever the last bitbake path or history is required.
>
> Or else provide the display-time matching against a non-null
> Layer.file_name, which would be much faster since the set of layers is of
> order 10^ and the list to match is at most 150 variables(per page), still
> my first choice.
>
> Dave
>
> >
> > Thank you,
> >
> > Alex
> >
>
> My recommendations are:
> 1) to populate the Layer model with local_path in all cases, changing the
> local_path to NOT NULL
> 2) perform the trim of variable names at display time, which should be
> faster than penalizing builds with Config model updates, or variable
> defined file stripping for every new config file read.
> OR)
> 3) add the Config model, update each variable log with the key to the
> Config model.
>
>
> >
> > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <
> belen.barros.pena at intel.com> wrote:
> >
> >
> >
> >
> > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner at windriver.com>
> wrote:
> >
> > >Hi Belen,
> > >
> > >> -----Original Message-----
> > >> From: Barros Pena, Belen [mailto:belen.barros.pena at intel.com]
> > >> Sent: Friday, March 13, 2015 9:17 AM
> > >> To: Lerner, Dave
> > >> Cc: toaster at yoctoproject.org
> > >> Subject: Re: [review-request] 7414-cut the prefix from layer
> file paths
> > >>for bitbake
> > >> variables
> > >>
> > >> Hi Dave,
> > >>
> > >> This is looking good. But I've realised I forgot about
> something. Not
> > >>all
> > >> files setting variables are inside a layer: some of them are
> > >>configuration
> > >> files inside the /build or /bitbake directories.
> > >>
> > >> I guess we should also cut those, showing only the path inside
> the root
> > >> source directory. So something like
> > >>
> > >> /home/user/poky/build/conf/local.conf
> > >>
> > >> Would show as
> > >>
> > >> build/conf/local.conf
> > >>
> > >> I am not fully sure if this is the right thing to do, so I'd be
> > >>interested
> > >> in hearing what you and the others think.
> > >>
> > >> I also realise this was not explicitly explained in the
> original bug. If
> > >> you want me to open a separate issue in Bugzilla, I can do so:
> just let
> > >>me
> > >> know.
> > >
> > >Please open a separate bug and give Alex the ok for 7414. Layer
> relative
> > >makes sense to me, active-build-directory-relative does not, too
> many
> > >questions "what is the relative to?" are raised.
> >
> >
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v
> > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky-
> > contrib/log/?h=lerner/trim_v arpath_7414>
> >
> > is good to go, then. I've opened this
> >
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463
> >
> > for the configuration file paths.
> >
> > Cheers
> >
> > Belén
> >
> >
> >
> > >
> > >Dave
> > >
> > >>
> > >> Thanks!
> > >>
> > >> Belén
> > >>
> > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner at windriver.com>
> wrote:
> > >>
> > >> >rebased.
> > >> >Dave
> > >> >
> > >> >> -----Original Message-----
> > >> >> From: Lerner, Dave
> > >> >> Sent: Thursday, March 12, 2015 10:46 AM
> > >> >> To: Barros Pena, Belen (belen.barros.pena at intel.com)
> > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer
> file
> > >>paths
> > >> >>for bitbake
> > >> >> variables
> > >> >>
> > >> >> I don't think that I rebased (since Tuesday) before I
> pushed. Can
> > >>you
> > >> >>hold off pulling
> > >> >> for 15 minutes? I'll send a note when rebased.
> > >> >> Dave
> > >> >>
> > >> >> > -----Original Message-----
> > >> >> > From: Lerner, Dave
> > >> >> > Sent: Thursday, March 12, 2015 10:29 AM
> > >> >> > To: Barros Pena, Belen (belen.barros.pena at intel.com);
> Damian,
> > >> >>Alexandru
> > >> >> > (alexandru.damian at intel.com)
> > >> >> > Cc: toaster at yoctoproject.org
> > >> >> > Subject: [review-request] 7414-cut the prefix from layer
> file paths
> > >> >>for bitbake
> > >> >> > variables
> > >> >> >
> > >> >> >
> > >>
> > >>>>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr
> > >>>>im
> > >> >>_varpath_7414
> > >> >> >
> > >> >> >
> > >> >> > Page Testing:
> > >> >> > Create project using yocto master which includes
> openembedded-core,
> > >> >>meta-yocto and
> > >> >> meta-
> > >> >> > yocto-bsp, then add layers meta-oe, meta-games.
> > >> >> > Build the project.
> > >> >> >
> > >> >> > Open the build, configuration page, bitbake variables.
> > >> >> > Note the following types of variable path names in the
> "Set in
> > >>file"
> > >> >>column and when
> > >> >> > clicking the variable in the History.
> > >> >> >
> > >> >> > Variable: B
> > >> >> > Set in file/history: meta/conf/bitbake.conf
> > >> >> >
> > >> >> > Variable: BBFILE_PATTERN_core
> > >> >> > Set in file: full path to configure
> > >> >> > History: first meta/conf/layer.conf, then
> > >> >> >
> /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py
> > >> >> >
> > >> >> > Variable: BBFILE_PATTTERN_games-layers
> > >> >> > Set in file and History:
> > >> >>meta-games_master.toaster_cloned/conf/layer.conf
> > >> >> >
> > >> >> >
> > >> >> > HTML Validation: Pass
> > >> >> > Dave Lerner
> > >>
> > >
> >
> > --
> > _______________________________________________
> > toaster mailing list
> > toaster at yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/toaster
> >
> >
> >
> >
> >
> > --
> >
> > Alex Damian
> > Yocto Project
> >
> > SSG / OTC
>
--
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20150324/6b74e280/attachment-0001.html>
More information about the toaster
mailing list