[Toaster] [review-request] 7414-cut the prefix from layer file paths for bitbake variables
Lerner, Dave
dave.lerner at windriver.com
Tue Mar 24 12:44:40 PDT 2015
Hi Alex,
I'm not sure if you want me to take back ownership and mark it Resolved/Fixed or if you are just suggesting a different alternative than my patch and will handle it.
Dave
> -----Original Message-----
> From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
> Sent: Tuesday, March 24, 2015 10:29 AM
> To: Lerner, Dave
> Cc: BARROS PENA, BELEN; toaster at yoctoproject.org
> Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths for
> bitbake variables
>
> Hi,
>
>
> After thinking about the issue, if we mangle at build time, we'll have trouble
> distinguishing between paths originating in the layer directories and paths in the build
> directory. So the good solution is that we mangle the path at display time, and I will
> be taking this patch.
>
>
> I have some more comments below,
>
>
> Cheers,
>
> Alex
>
>
> On Wed, Mar 18, 2015 at 3:48 PM, Lerner, David M (Wind River)
> <dave.lerner at windriver.com> wrote:
>
>
> Alex,
> I've thought about this some more... see below.
> Dave
>
> > -----Original Message-----
> > From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
> > Sent: Tuesday, March 17, 2015 10:19 AM
> > To: BARROS PENA, BELEN
> > Cc: Lerner, Dave; toaster at yoctoproject.org
> > Subject: Re: [Toaster] [review-request] 7414-cut the prefix from layer file paths
> for
> > bitbake variables
> >
> > Hi,
> >
> >
> > Sorry for the significant delay in addressing this patch - I've been a bit
> overwhelmed.
>
> It's not a significant delay and the delay is not a problem.
>
> >
> >
> > I am NOT taking this patch because of the way the layer matching is done.
> >
> >
> > We already have local paths for layers present in the build in the Layer object;
> we also
> > know _where_ the build took place, for we have
> > "build.buildrequest.buildenvironment.builddir" variable that holds the path to
> the build
> > directory.
>
> Either there is a bug, or your claim isn't quite right regarding
> orm_layer.local_paths which are all empty or null for a yocto release build. That was
> my first choice examining the database, but not possible since data was missing.
>
>
>
> I should've been more clear than this. The layers which HAVE BEEN built have local_path
> set to a correct value. The layers that we know about because they are imported manually
> or from Layer Index, the local_path is NULL. The variables are always attached to a
> build, so the layers on which the variables are modified will have local_path set to
> something else than NULL.
>
>
>
>
>
>
> > We must verify that the file path starts with one of the layer paths or the build
> > environment builddir path, and strip that leading path from the full file path.
> >
>
> I agree that would make more sense, but that data isn't in the table, and I believe
> due to performance reasons, that it should be done at display-a-page time.
>
> > I would also expect this to be done at data logging time instead of data display
> time
> > (i.e. in the buildinfohelper.py), with the distinct advantage of speeding up page
> > processing time.
>
>
> Since build-time layer path extraction for the given database structures penalizes
> all build times for the low likelihood that some user wants the path to a layer, I would
> recommend adding just a little more processing time to the data logging as follows:
>
> a) Constraining Layer.local_path to non-null, and always populating that field,
>
>
>
> This is impossible due to the fact that we keep info about layers that have not been
> checked-out in that table.
>
>
>
>
> b) normalizing things a bit by creating a new Model Config which has:
> b1) the full path that was formerly in the (non-normalized)
> VariableHistory.file_name and
> b2) a foreign key to from Config to it's Layer, calculated once
> c) modifying (normalizing) orm_variablehistory
> c1) removing orm_variablehistory.file_name and
> c2) replacing that with a foreign key to it's defining Config model.
>
>
>
>
> Yes, this makes sense. Still it wouldn't work 100% because some variables are modified
> outside any Layer checkout (i.e. in the build directory). Having foreign keys to
> different objects to re-compute the path in the "Config" model would be unworkable in
> the queries sense.
>
>
>
>
> Then display time could just join on demand to find the config file's layer's local
> path whenever the last bitbake path or history is required.
>
> Or else provide the display-time matching against a non-null Layer.file_name, which
> would be much faster since the set of layers is of order 10^ and the list to match is at
> most 150 variables(per page), still my first choice.
>
> Dave
>
> >
> > Thank you,
> >
> > Alex
> >
>
> My recommendations are:
> 1) to populate the Layer model with local_path in all cases, changing the
> local_path to NOT NULL
> 2) perform the trim of variable names at display time, which should be faster than
> penalizing builds with Config model updates, or variable defined file stripping for
> every new config file read.
> OR)
> 3) add the Config model, update each variable log with the key to the Config model.
>
>
>
> >
> > On Mon, Mar 16, 2015 at 3:41 PM, Barros Pena, Belen <belen.barros.pena at intel.com>
> wrote:
> >
> >
> >
> >
> > On 16/03/2015 14:14, "Lerner, Dave" <dave.lerner at windriver.com> wrote:
> >
> > >Hi Belen,
> > >
> > >> -----Original Message-----
> > >> From: Barros Pena, Belen [mailto:belen.barros.pena at intel.com]
> > >> Sent: Friday, March 13, 2015 9:17 AM
> > >> To: Lerner, Dave
> > >> Cc: toaster at yoctoproject.org
> > >> Subject: Re: [review-request] 7414-cut the prefix from layer file paths
> > >>for bitbake
> > >> variables
> > >>
> > >> Hi Dave,
> > >>
> > >> This is looking good. But I've realised I forgot about something. Not
> > >>all
> > >> files setting variables are inside a layer: some of them are
> > >>configuration
> > >> files inside the /build or /bitbake directories.
> > >>
> > >> I guess we should also cut those, showing only the path inside the root
> > >> source directory. So something like
> > >>
> > >> /home/user/poky/build/conf/local.conf
> > >>
> > >> Would show as
> > >>
> > >> build/conf/local.conf
> > >>
> > >> I am not fully sure if this is the right thing to do, so I'd be
> > >>interested
> > >> in hearing what you and the others think.
> > >>
> > >> I also realise this was not explicitly explained in the original bug. If
> > >> you want me to open a separate issue in Bugzilla, I can do so: just let
> > >>me
> > >> know.
> > >
> > >Please open a separate bug and give Alex the ok for 7414. Layer relative
> > >makes sense to me, active-build-directory-relative does not, too many
> > >questions "what is the relative to?" are raised.
> >
> >
> > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/trim_v
>
> > arpath_7414 <http://git.yoctoproject.org/cgit/cgit.cgi/poky-
> > contrib/log/?h=lerner/trim_v arpath_7414>
>
> >
> > is good to go, then. I've opened this
> >
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=7463
> >
> > for the configuration file paths.
> >
> > Cheers
> >
> > Belén
> >
> >
> >
> > >
> > >Dave
> > >
> > >>
> > >> Thanks!
> > >>
> > >> Belén
> > >>
> > >> On 12/03/2015 15:48, "Lerner, Dave" <dave.lerner at windriver.com> wrote:
> > >>
> > >> >rebased.
> > >> >Dave
> > >> >
> > >> >> -----Original Message-----
> > >> >> From: Lerner, Dave
> > >> >> Sent: Thursday, March 12, 2015 10:46 AM
> > >> >> To: Barros Pena, Belen (belen.barros.pena at intel.com)
> > >> >> Subject: RE: [review-request] 7414-cut the prefix from layer file
> > >>paths
> > >> >>for bitbake
> > >> >> variables
> > >> >>
> > >> >> I don't think that I rebased (since Tuesday) before I pushed. Can
> > >>you
> > >> >>hold off pulling
> > >> >> for 15 minutes? I'll send a note when rebased.
> > >> >> Dave
> > >> >>
> > >> >> > -----Original Message-----
> > >> >> > From: Lerner, Dave
> > >> >> > Sent: Thursday, March 12, 2015 10:29 AM
> > >> >> > To: Barros Pena, Belen (belen.barros.pena at intel.com); Damian,
> > >> >>Alexandru
> > >> >> > (alexandru.damian at intel.com)
> > >> >> > Cc: toaster at yoctoproject.org
> > >> >> > Subject: [review-request] 7414-cut the prefix from layer file paths
> > >> >>for bitbake
> > >> >> > variables
> > >> >> >
> > >> >> >
> > >>
> > >>>>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=lerner/tr
> > >>>>im
> > >> >>_varpath_7414
> > >> >> >
> > >> >> >
> > >> >> > Page Testing:
> > >> >> > Create project using yocto master which includes openembedded-core,
> > >> >>meta-yocto and
> > >> >> meta-
> > >> >> > yocto-bsp, then add layers meta-oe, meta-games.
> > >> >> > Build the project.
> > >> >> >
> > >> >> > Open the build, configuration page, bitbake variables.
> > >> >> > Note the following types of variable path names in the "Set in
> > >>file"
> > >> >>column and when
> > >> >> > clicking the variable in the History.
> > >> >> >
> > >> >> > Variable: B
> > >> >> > Set in file/history: meta/conf/bitbake.conf
> > >> >> >
> > >> >> > Variable: BBFILE_PATTERN_core
> > >> >> > Set in file: full path to configure
> > >> >> > History: first meta/conf/layer.conf, then
> > >> >> > /path/_poky_master.toaster_cloned/bitbake/lib/bb/data_smart.py
> > >> >> >
> > >> >> > Variable: BBFILE_PATTTERN_games-layers
> > >> >> > Set in file and History:
> > >> >>meta-games_master.toaster_cloned/conf/layer.conf
> > >> >> >
> > >> >> >
> > >> >> > HTML Validation: Pass
> > >> >> > Dave Lerner
> > >>
> > >
> >
> > --
> > _______________________________________________
> > toaster mailing list
> > toaster at yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/toaster
> >
> >
> >
> >
> >
> > --
> >
> > Alex Damian
> > Yocto Project
> >
> > SSG / OTC
>
>
>
>
>
> --
>
> Alex Damian
> Yocto Project
>
> SSG / OTC
More information about the toaster
mailing list