[Toaster] Reference browser for testing purposes

[Barros Pena, Belen]

On 19/03/2014 18:14, "Damian, Alexandru" <alexandru.damian at intel.com>
wrote:

>
>
>Our goal is not "decent" but complete HTML5 compatibility.
>
>
>
>The target is that our HTML output is to be validated by HTML5 validators
>with no errors shown. We already selected the industry-standard HTML5
>validators to verify this.

Validating is a great and necessary thing, but does not address
compatibility. The markup might be valid, but the browser rendering it
might not support some of its features


>
>Specifically, we are using in development
>http://validator.w3.org/ through a browser extension. This MUST be
>automated at a certain point.
>
>
>What I'm trying to avoid here is coding specifically for a target browser
>or platform. I suggest to not restrict testing to a certain
>browser/platform/version, but use what ever the tester uses in real life.

I am not sure this is the right approach, to be honest. What the tester
uses might be completely out of sync with the technology and goals of the
application we are developing.

>In case of presentation bugs are discovered, first we have to rule out an
>issue with the browser of choice by testing visual reproducibility with
>another browser on the same page and verifying browser's HTML5
>compatibility.

In order to address compatibility, we need to clarify what we mean by it.
Compatibility with what: just with the markup language, or with all or
some of the technologies that make html5? If we decide that with the
markup language only (like so http://caniuse.com/#cats=HTML5), and
cross-checking with the traffic data from yoctoproject.org, we should
probably be testing with Firefox 26 and Chrome 31. But bearing in mind
that Toaster might be used with oldish distros (Yuan's point about Ubuntu
12.04 I think is a valid one), maybe we should push down the versions a
bit farther.

>
>
>For visual reference, widgets in the page change across different
>platforms and browsers. Do we have test cases for the appearance ?

I think the criteria here is not the presentation, but if the UI
components are functional (for example, can I select and deselect columns
using the Edit columns widget, or can I toggle the errors and warnings
content, not if the labels are not correctly aligned).

>
>
>The test cases should not be dependent or executed with a specified
>platform/browser version.

I am sorry, but I disagree: I believe we need some kind of reference.

>
>
>Cheers,
>Alex
> 
>
>
>
>
>On Wed, Mar 19, 2014 at 4:23 PM, Barros Pena, Belen
><belen.barros.pena at intel.com> wrote:
>
>We should probably have raised this question earlier and had a plan in
>place, but hey, better late ... The question is: which browsers should we
>be using as a reference for QA purposes? Our guideline here is decent
>HTML5 compatibility, but we never qualified what 'decent' means.
>
>The other reference we could use is traffic to the Yocto Project website.
>Visits are mainly coming from Chrome 32 and 33 on Windows, and Firefox 26
>and 27 on Linux. I can put together more detailed numbers if anybody wants
>to see them.
>
>Those might be a bit too cutting edge, but could guide our decision
>somehow. QA is currently testing with Firefox 11: that is probably too
>old.
>
>In light of the above, any suggestions about which browsers we should use
>for testing?
>
>Thanks!
>
>Belén
>
>
>
>
>
>
>
>
>-- 
>Alex Damian
>Yocto Project
>
>SSG / OTC 
>
>