[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linaro-validation
Subject:    [Linaro-validation] what's the open-embedded platform bringup story
From:       michael.hudson () linaro ! org (Michael Hudson-Doyle)
Date:       2012-08-06 23:28:16
Message-ID: 878vdrsir3.fsf () canonical ! com
[Download RAW message or body]

Ricardo Salveti <ricardo.salveti at linaro.org> writes:

>>> The first idea I had was to basically have a way at lava-tests to skip
>>> the step which it installs the test dependencies. At least with that
>>> approach, we could already have the dependencies in place for the test
>>> cases we care about.
>>
>> I think I'm coming around to the android team's POV here: tests don't
>> have dependencies -- you test things in the base system (for a while we
>> were running the ubuntu-leb-graphics tests on a nano image, which
>> installed X and everything else first -- madness!).
>
> It can have, but if they are part of the image itself, we can at least
> cover that by default.

I don't quite understand here, I'm afraid.

> The problem is that supporting more test cases is directly connected
> on producing new kinds of images (with extra packages and such), which
> can be painful.

Maybe we should make that easier then?

>>> Otherwise, we'd need a way to dynamically install the dependencies,
>>> which can be quite complicated with OE as we'd need to create and
>>> export a repository.
>>>
>>>> One of the things I've raised in the past that we should look at is
>>>> consolidation of lava-test and lava-android-test by means of a host driven
>>>> test framework, and this is one reason why.
>>>
>>> Can you explain more about how this host driven test framework would work?
>>
>> So the idea is that "test installation" arranges for the test to run on
>> next boot of the test system, putting the result output in a known
>> location.  Then we boot the device, wait for the test to complete (hand
>> waving here), fetch the results from the known location and parse them
>> into a bundle.  There will be a system dependent part around how the
>> tests are started -- for ubuntu you might create an upstart job, for OE
>> I guess you'll do something else -- but that feels like a small amount
>> of variation.
>>
>> How you get the result output after the tests have run will also vary by
>> deployment approach, but we already handle this today to get the bundles
>> off the device.
>
> Sounds fine, while you don't have total progress in real time, it's a
> lot simpler to follow up the execution and parsing.

We could spam the output to the serial console as well while the test is
running -- probably a good idea, in fact.

> While it can help depending on the environment you have, it'll not
> improve the current situation a lot (it'll probably just make the
> maintenance a bit easier).

One of the nice things is that it means that we don't have to install
(and have working) things like lava-test on the DUT, which is nice for
things like fast models of new architectures :-)

>>>> Certainly the dispatcher will have to support another means of installation,
>>>> just like we have to for Android images.  If linaro-media-create is used,
>>>> then this makes things slightly better, but I'm sure it'll need at least a
>>>> bit of special handling in the dispatcher, though probably not as bad as
>>>> android was. I don't foresee a big problem here.
>>>
>>> I believe we could even use the hwpacks we produce for Ubuntu here,
>>> but only dealing with the kernel and bootloader (skipping all other
>>> packages). As we don't care much about upgradability, we could have a
>>> logic at linaro-media-create to just extract the debian packages
>>> related with the kernel and bootloader (something that happens in some
>>> way anyway).
>>
>> I guess.  I don't think the way it happens today is at all reusable
>> fwiw: it looks like it looks it depends on linaro-hwpack-install
>> installing the a package that creates a path such as
>> boot/vmlinuz-*-linaro-omap.  I presume it wouldn't be very hard to make
>> a guess at which deb creates this file and use ar/tar to "install" it,
>> but it would be new code I think.
>>
>>> With that logic in place, we could support any rootfs, even Android
>>> (and share the same basic set of hwpacks we also use at Ubuntu).
>
> The code for such usage is mostly done at
> https://code.launchpad.net/~rsalveti/linaro-image-tools/generic-oe-support/,
> and I was able to test with the current OE images I produced locally.
> I'll be testing it properly in different use cases in the next
> following days, but should be ready to merge/review soon.

Ah, cool.

> The idea is basically to use linaro-media-create the same way as it's
> done with Ubuntu's images, so we avoid changing much on the interface
> of LAVA or any other tool that relies on lmc.
>
> The only problem I see in the future, which will require a bit of
> re-work again (on lmc), is when we start bootstrapping a new
> architecture, as we'll not have any qemu support to run native code
> (post installs or similar).

Yeah, that's another reason to do blackbox-style/host-side testing...

> While it's not that complicated, it can get messy because of the
> current way lmc is flashing up the images.

Cheers,
mwh


[prev in list] [next in list] [prev in thread] [next in thread]