[Toaster] package view detail test plan - as executed with last push
Lerner, Dave
dave.lerner at windriver.com
Sat Jan 25 12:29:04 PST 2014
Hi,
The following is the test plan that has been executed with the commit pushed to
lerner/package-detail-views as noted in the email to toaster at yoctoproject.org
on 1/23/2014.
The test plan closely follows the specs:
'Design 1.1.1 Included package details' : https://bugzilla.yoctoproject.org/attachment.cgi?id=1683
and
'Design 1.5.1 Package details;
https://bugzilla.yoctoproject.org/attachment.cgi?id=1684
Regards,
Dave Lerner
====================
Test Case Setup
====================
The following build configuration generated packages to satisfy the test
cases implied by the two specs. The equivalence classes were constructed to
create:
* 2 builds
* 2 similar images, different machines
* 1 build two different images
OUT OF SCOPE ISSUE: In order to generate packages (such as tzdata) that
were 'built but not installed' for the 1.5.1 spec, you must delete your
sstate-cache directory. An older sstate-cache with an empty sqlite directory
generates NO BUILT BUT NOT INSTALLED pacakgs.
After running
source YOUR-POKY-INSTALL/oe-init-build-env
start toaster
source toaster start
For build 1 set conf/local.conf variable
MACHINE = "qemux86"
run
bitbake core-image-minimal
For build 2 set conf/local.conf variable
MACHINE = "qemuarm"
run
bitbake core-image-minimal core-image-basic
Wait for builds to complete.
Launch browser, point to 127.0.0.1:8000.
OUT OF SCOPE ISSUE: As of 1/24/14 the test plan requires a small patch to enable
package searches on the Packages screen:
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -145,6 +145,7 @@ class Task_Dependency(models.Model):
class Package(models.Model):
+ search_allowed_fields = ['name', 'installed_name', 'section', 'summary']
build = models.ForeignKey('Build')
recipe = models.ForeignKey('Recipe', null=True)
name = models.CharField(max_length=100)
----------------------------------------------
Test Plan 1.1.1 Included package details spec
----------------------------------------------
Navigation from all image packages.
Click on the checkbox icon in the qemuarm row, then in the left panel, under BUILD, click Packages.
In the Search box, enter Busybox, then click Search
OUT OF SCOPE ISSUE: Navigation from build summary IMAGE | packages is non responsive today
Click on busybox(files).
Click on core-image-basic
Page 4 and 5.
Note that the title reads busybox-<version>-<revision>.
Note that some package page titles will only show package name,
without "-version-revision" if the version fields are empty.
Note that the 'breadcrumbs' under the "yocto Toaster" title bar are visible
and that all active links work, followed by the package-version-revision name.
Note on left side
3 tabs labeled as per spec.
File size count on tab line in brackets matches the count in the
table.
Under the tabs that if files are present, a table show with
two columns labeled as per spec
with size and units as per spec (actual values in spec violate the
xxx.x specification)
file sizes in B, KB, MB with ".0" right trimmed (change to eglibc to
view all 4 unit conditions above)
sorted alpha on filename
IN SCOPE BUG: file links are not active - waiting for
information directory views.
SPEC ISSUE: The link to the fading blue row doesn't fade.
Note on the right side
Package Information contains
package size
% size of total: OUT OF SCOPE ISSUE: depends on orm_target.file_size set
to non-zero value. If 0 (which seems to be a bug) then that % is not
shown.
License Files: OUT OF SCOPE ISSUES: blank due to bugzilla entry: ????
Recipe
Layer: OUT OF SCOPE ISSUE: the layer path is truncated to a single path
directory element and the logic to link to the oe website has not
been established.
Layer Branch
Layer Commit
Layer Directory
And note that each label has a help bubble describing per example on
yoctoproject.org, the meaning of the label
Note that files in root file system are sorted in alphabetical order.
Note that abbreviations are MB, KB, B (not shown but ".0" values are trimmed
Page 6 Empty state no root files
Click on Other runtime relationships, then click on package update-rc.d.
Note that the update-rc.d Files in root file system bracketed count is 0
Note the message update-rc.d does not generate any files.
IN SCOPE BUG - Title shows // update-rc.d-- // with two trailing dashes.
Fixed in subsequent push commit summary with "v2".
Page 7 Runtime dependencies
Navigation: Return to busybox page, click on runtime dependencies.
Note that runtime dependency tab bracketed count matches rows count in
table (3).
OUT OF SCOPE ISSUE: The orm_package_dependency for busybox includes
a dependency on itself as shown by:
sqlite> select * from orm_package_dependency where
target_id=3 and package_id=5131 and dep_type <= 1;
id|package_id|depends_on_id|dep_type|target_id
23355|5131|5131|1|3
which shows the busybox id 5131 = package_id and depends_on_id.
Note that the bracketed number in the Files in root file system matches
the count of files in the File/Size table.
Page 7 Other Dependencies
Note that relationship type string is capitalized.
Note that hover help appears to the right of relationship type (spec in email).
Note that packages without versions are shown as package name only.
Sorting by grouped Other Dependencies
TESTING ISSUE: setup builds and a sato build did not generate any
included package's dependencies that had more than one non-runtime
dependency type and in fact all non *RDEPENDS types were RRECOMMENDS.
So other dependency type group ordering untestable with dataset.
--------
sqlite> select distinct IPD.dep_type
...> from orm_package_dependency IPD
...> where IPD.dep_type > 1
...> and IPD.target_id not null
...> order by IPD.dep_type;
dep_type
3
sqlite> (only TRECOMMENDS)
--------
Page 8
Note that in top list, clicking on eglibc goes to the eglibc included page.
Backarrow to busybox.
Note that clicking on busybox-syslog in Other list goes to the busybox
included page.
TESTING ISSUE: setup builds and a sato build did not generate any
included package's dependencies that were not included.
So muted uninstalled package dependencies untestable with data set.
---------
select P.build_id, PD.target_id, P.id, P.name
from orm_package P, orm_package_dependency PD
where
PD.dep_type > 1
AND P.id = PD.package_id
AND
(select OIP.package_id
from orm_target_installed_package OIP
where OIP.target_id = PD.target_id
and OIP.package_id = PD.package_id) IS NOT NULL
AND
(select IIP.package_id
from orm_target_installed_package IIP
where IIP.target_id = PD.target_id
and IIP.package_id = PD.depends_on_id) IS NULL;
sqlite> (empty set)
---------
Page 9 runtime vs other dependency counts
Runtime deps > 0, other depends = 0
Navigation: Assuming at the page of the last step busybox-syslog detail, if not
go to that page.
Note that with the "Other dependencies" messsage and table is suppressed.
Runtime deps = 0, other deps =0
Navigation: IN build 2, search for ncurses-terminfo-base, select it, click
on Runtime dependencies.
Note that a message appears per spec stating no runtime dependencies.
Runtime deps = 0, other deps > 0
TESTING ISSUE: Such entries don't exist in database including sato
-------------
sqlite> select P.build_id, PD.target_id, P.id
...> from orm_package_dependency PD, orm_package P
...> where PD.target_id not null and P.id = PD.package_id and PD.target_id < 4
...> and
...> (SELECT IPD.package_id from orm_package_dependency IPD
...> where IPD.dep_type not in (0,1)) is null;
s
sqlite> (no entries)
-------------
page 10 Reverse Runtime Dependencies
Navigation: go to build 2, Packages, search for Busybox,
click on core-image-minimal.
Note reverse runtime count in brackets is (23), click on that tab.
NOte that table count shows 23 entries.
Note that package column doesnt show version/version.
NOte that version column shows version/revision.
NOte that size shows filtered unit symbols.
Note that each cell links to the included details page.
Note that table is sorted alpha.
Special case - 0 dependencies.
Navigation: Search for update-rc.d and click to the Reverse Runtime Depenencies page.
Note the message "no reverse runtime dependencies"
OUT OF SCOPE ISSUE: The contents for update-rc.d has null installed_size,
version, license, description, summary and revision.
----------------------------------------------
1. Test Plan 1.5.1 Built but not included package details spec
----------------------------------------------
Page 4
Navigation: From All Builds, select the 2nd build, select packages
Search for 'tzdata', then click on it.
Note that it is included in the image core-image-basic.
Go back to All builds, select the 1st build, packages, and search for tzdata.
Click on tzdata and note a different page appears comparable to the layout on
page 4.
Note that the breadcrumb links are all active, terminating in the
package-name'-'version'-'revision string.
NOte that the title includes the package version and revision string.
On the right side in the Package Information area
verify the same as in 1.1.1
Page 5
Refer to previous test with busybox to confirm layout when package in
one or more images.
Page 6
Reset to tzdata and the first build with just core-image-minimal and qemux86.
Note the alert message matches the spec alert.
Note the the Files table has a files and size column and
the row count matches the value in brackets in the "Generated Files Tab".
OUT or maybe IN SCOPE ISSUE: There is not constraint on the number of files
shown in the table. If that number is a 1000, then the page would hold a
thousand rows.
Note that the table is sorted alpha.
Page 7 no genereated files
Go back to build 1, packages, search for timeones, and select it.
Note that the message "does not genearte any files" appears.
Page 8 Runtime dependencies
Go back and select build 1, Packages, eglibc-dev and select it, then select
runtime dependencies tab.
Upper DEPENDS table:
Note tha the upper page has a table of files and files sizes.
Note that the table is sorted ascending alpha.
LOWER OTHER table:
Note that the lower other table heading matches the spec.
Note that each relationship type (Conflicts in this case) is capitalized.
Note the hover help bubble describes the relationship type.
NOte the package, version, size columns match formatting in spec.
Note that the packages not built are muted without interactivity.
Page 10
Additional cases for runtime count vs other dep count
runtime count=0 othder dep count = 0
using build 1( core-image-min. qemux86), timezones
Note that the alert re no runtime dependencies appears without
the files table
Note that the other dependency list is absent
runtime dep count > 0, other dep count =0
using build 1( core-image-min. qemux86), dbus-ptest
Note that the alert re no runtime dependencies appears with the files table
Note that the other dependency list is absent
runtime dep count = 0, other dep count > 0
using build 1( core-image-min. qemux86), tzdata
Note that the alert re no runtime dependencies appears without
the files table
Note that the other dependency list appears
SPECIAL CASE empty version/revision:
Select from build 1 'timeones' package.
Note that only the package name version appear in breadcrumbs, title, and
the "Does not generate any files" line
SPECIAL CASE empty version suppresses 0 B size:
Select from build 1 'timeones' package.
Note that all fields in the Package Information well are empty, including size.
More information about the toaster
mailing list