[poky] [PATCH 1/2] module: build and clean hostprogs for each module

Mon Mar 7 14:16:53 PST 2011

On 03/06/2011 08:21 AM, Gary Thomas wrote:
> On 03/04/11 14:31, Darren Hart wrote:
>> On 03/04/2011 12:33 PM, Darren Hart wrote:
>>> On 03/04/2011 03:44 AM, Gary Thomas wrote:
>>>> On 03/03/2011 09:39 PM, Darren Hart wrote:
>>>>> On 03/03/2011 08:59 AM, Gary Thomas wrote:
>>>>>> On 03/03/2011 09:50 AM, Gary Thomas wrote:
>>>>>>> On 03/02/2011 11:00 AM, Darren Hart wrote:
>>>>>>>> From: Darren Hart<dvhart at linux.intel.com>
>>>>>>>>
>>>>>>>> This fixes [BUGID #241]
>>>>>>>>
>>>>>>>> The kernel hostprogs are built for the host architecture. They
>>>>>>>> should
>>>>>>>> not
>>>>>>>> be deployed with to the target, and they should not be included
>>>>>>>> in an
>>>>>>>> sstate
>>>>>>>> package which might get reused on a host of a different
>>>>>>>> architecture.
>>>>>>>>
>>>>>>>> As we don't build many out-of-tree modules, this patch takes the
>>>>>>>> approach
>>>>>>>> of building the hostprogs as part of the module compile process
>>>>>>>> with a
>>>>>>>> do_compile_prepend() routine in module.bbclass. To ensure the
>>>>>>>> hostprogs
>>>>>>>> don't contaminate the build, they are removed in
>>>>>>>> do_install_append().
>>>>>>>>
>>>>>>>> Signed-off-by: Darren Hart<dvhart at linux.intel.com>
>>>>>>>> CC: Gary Thomas<gary at mlbassoc.com>
>>>>>>>
>>>>>>> Sadly, this doesn't seem to work for me. I don't see any
>>>>>>> indication in
>>>>>>> run.do_compile that the extra steps were added at all.
>>>>>>>
>>>>>>> Will it matter if my recipe overrides the do_compile() method?
>>>>>>>
>>>>>>
>>>>>> Also, when you tested this, what was your target MACHINE (in
>>>>>> particular,
>>>>>> was the target a different architecture than the build host?) I ask
>>>>>> because
>>>>>> I tried to just manually insert the compile_prepend() functions
>>>>>> into my
>>>>>> recipe and it ended up trying to build host tools (that's what the
>>>>>> fuss
>>>>>> is all about) using the target toolchain.
>>>>>
>>>>> I made some modifications to avoid a Race that RP pointed out (just
>>>>> removed the cleans from the module.bbclass since they weren't
>>>>> necessary anyway). With this, I set my MACHINE to
>>>>> beagleboard and built minimal. You can see from the following that
>>>>> hostprogs were build for the host and the module for the target:
>>>>>
>>>>> dvhart at rage:build-beagleboard$ file
>>>>> ./tmp/work/beagleboard-poky-linux-gnueabi/linux-yocto-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+2b412826bbeb4a16abe2ea74f2456ab880c6e3c1-r16/linux-beagleboard-standard-build/scripts/basic/fixdep
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ./tmp/work/beagleboard-poky-linux-gnueabi/linux-yocto-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+2b412826bbeb4a16abe2ea74f2456ab880c6e3c1-r16/linux-beagleboard-standard-build/scripts/basic/fixdep:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically
>>>>> linked (uses shared libs), for GNU/Linux 2.6.15, not stripped
>>>>
>>>> Why are you looking in ${KERNEL_SRC}? It may not exist (think rm_work)
>>>> when your module gets built.
>>>>
>>>> The example recipe I have uses ${STAGING_KERNEL_DIR} something like
>>>> this:
>>>> make -C ${STAGING_KERNEL_DIR} M=src
>>>>
>>>
>>> module.bbclass passes KERNEL_SRC=${STAGING_KERNEL_DIR} to the module's
>>> Makefile.
>>>
>>> I don't use STAGING_KERNEL_DIR in the hello-mod Makefile as I wanted the
>>> Makefile to exemplary of external modules, and not poky specific.
>>>
>>>
>>>> Also, if you look at the run.do_compile script for your module
>>>> recipe, is
>>>> there any indication that the compile_prepend() stuff was even added?
>>>
>>> I do see the compile_prepend() code prepended to the do_compile code:
>>>
>>> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
>>> oe_runmake CC="arm-poky-linux-gnueabi-gcc "
>>> LD="arm-poky-linux-gnueabi-ld
>>> --sysroot=/home/dvhart/source/poky.git/build-beagleboard/tmp/sysroots/beagleboard
>>>
>>> " AR="arm-poky-linux-gnueabi-ar " \
>>> -C
>>> /home/dvhart/source/poky.git/build-beagleboard/tmp/sysroots/beagleboard/kernel
>>>
>>> scripts
>>>
>>> Hrm.... but looking at that, that's the target compiler... not the host
>>> compiler... so why on earth did this work.... grumble. Some more
>>> investigation remains it seems...
>>
>>
>> Actually, this isn't a problem. For the compilation of hostprogs, the
>> kernel Make system uses HOSTCC, not CC. From log.do_compile:
>>
>> make -C
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/linux-yocto-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+2b412826bbeb4a16abe2ea74f2456ab880c6e3c1-r16/linux
>>
>> O=/home/dvhart/source/poky.git/build-beagleboard/tmp/sysroots/beagleboard/kernel/.
>> scripts
>
> This sure looks like it's using the kernel source tree, not the staging
> tree. If you run with rm_work
> and/or sstate caching, that directory may not exist! My example (which
> was lifted fro OE) uses the
> kernel _staging_ directory, which should exist.

Ah, the problem is that while I pass STAGING_KERNEL_DIR, the Makefile 
there just redirects the make to the kernel source directory. What we 
appear to need is something along the lines of:

$ make O=${PN}-modules_prepare modules_prepare

which runs 'prepare' and 'scripts'. Unfortunately, this still has a 
symlink to the source dir in the output dir - probably for things like 
vmlinux, etc. I'm looking into how we can best accomplish this without 
the recipe becoming too fragile to maintain from kernel version to 
kernel version.

--
Darren

>
> Recipe attached - I'm not sure how much use it is as it as a boat-load
> of dependencies
> which are not germane to this issue.
>
>> make[1]: Entering directory
>> `/home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/linux-yocto-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+2b412826bbeb4a16abe2ea74f2456ab880c6e3c1-r16/linux'
>>
>>
>> HOSTCC scripts/basic/fixdep
>> HOSTCC scripts/basic/docproc
>> GEN
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/sysroots/beagleboard/kernel/Makefile
>>
>> HOSTCC scripts/kconfig/conf.o
>> HOSTCC scripts/kconfig/kxgettext.o
>> HOSTCC scripts/kconfig/zconf.tab.o
>> HOSTLD scripts/kconfig/conf
>>
>> While the module itself builds with:
>>
>> make -C
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/linux-yocto-2.6.37+git1+e2737075b79e4fc682e41051cf1c0bc47a47d502_1+2b412826bbeb4a16abe2ea74f2456ab880c6e3c1-r16/linux
>>
>> O=/home/dvhart/source/poky.git/build-beagleboard/tmp/sysroots/beagleboard/kernel/.
>>
>> LD
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/hello-mod-0.1-r0/built-in.o
>>
>> CC [M]
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/hello-mod-0.1-r0/hello.o
>>
>> Building modules, stage 2.
>> MODPOST 1 modules
>> CC
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/hello-mod-0.1-r0/hello.mod.o
>>
>> LD [M]
>> /home/dvhart/source/poky.git/build-beagleboard/tmp/work/beagleboard-poky-linux-gnueabi/hello-mod-0.1-r0/hello.ko
>>
>>
>>
>> Things are working as intended for the hello-mod recipe.
>>
>> --
>> Darren
>>
>>>
>>>
>>> > I don't see it in mine (yes, it inherits module)
>>>
>>> Even though there is something up with the script build above, you
>>> should be seeing it in run.do_compile. Is this recipe something you can
>>> share? Also, have you tried from a clean build? I ran into various
>>> (strange) problems because I wasn't cleaning the right
>>> tasks/recipes/etc. I think your problem is different, but might have a
>>> similar cause.
>>>
>>> --
>>> Darren
>>>
>>>>
>>>>>
>>>>> dvhart at rage:build-beagleboard$ file
>>>>> ./tmp/work/beagleboard-poky-linux-gnueabi/hello-world-mod-1.0-r0/package/lib/modules/2.6.37.2-yocto-standard+/extra/hello_world.ko
>>>>>
>>>>>
>>>>>
>>>>> ./tmp/work/beagleboard-poky-linux-gnueabi/hello-world-mod-1.0-r0/package/lib/modules/2.6.37.2-yocto-standard+/extra/hello_world.ko:
>>>>>
>>>>>
>>>>> ELF 32-bit LSB relocatable, ARM, version 1
>>>>> (SYSV), not stripped
>>>>>
>>>>> I'm going to do a beagleboard boot test, but if that works I'll be
>>>>> resending these changes to close BUG 241 as they work on both qemux86
>>>>> and beagleboard using hello-world-mod which
>>>>> uses a very typical out-of-tree kernel module Make setup.
>>>>>
>>>>
>>>
>>>
>>
>>
>

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel