-
Notifications
You must be signed in to change notification settings - Fork 15
WPSS Validator Tool Test Cases
ABOUT THE PWGSC WPSS VALIDATION TOOL Limitations Risks WEB DOCUMENT LIST Web Site Crawler Password Protected Applications Web Server Password Protection MARKUP VALIDATION HTML/XHTML Cascading Style Sheets JavaScript Robots.txt LINK CHECKING Broken Links Cross Language Links Redirected Links Bad Network Scope Links Broken Anchors Firewall Blocked Links IPV4 Links METADATA CHECKING Metadata Checking Profiles Required Metadata Tags Required Metadata Content Metadata Content and Format Date Values Language Code Subject Value Email Address Scheme PDF File Properties ACCESSIBILITY CHECKING Accessibility Checking Profiles WCAG 2.0 WCAG 2.0 Techniques Reported Under Other Techniques LAYOUT AND DESIGN CHECKING Layout and Design Checking Profiles PWGSC SWU – Standard on Web Usability Layout and Design test cases WEB ANALYTICS Web Analytics Profiles TBS Web Analytics – Standard on Privacy and Web Analytics test cases WEB INTEROPERABILITY Web Interoperability Profiles SWI – Standard on Web Interoperability test cases CONTENT CHECKING Content Checking Profiles PWGSC SWU – PWGSC Standard on Web Usability Content Check Test Cases HTML DOCUMENT FEATURES HTML Features Profiles ALL – All HTML Features CONTENT REPORTS Image Details Report Sample report Heading Outline Report Sample Report
The PWGSC WPSS Validation Tool combines a number of individual analysis and validation functions into a single tool. This tool provides web developers and quality assurance testers the ability to perform a number of Web site and Web page analysis and validation tasks at the same time. The analysis and validation functions applied to Web sites and Web documents include:
- markup validation
- link violation checking
- metadata checking
- accessibility checking
- content checking
- reporting HTML document features, such as forms and tables.
Each of these analysis or validation functions is described in this document.
There are limitations within the Validation Tool that may affect the analysis or validation results, these are:
- The Validation Tool does not support JavaScript. Sites that rely on JavaScript may not be crawled or analysed accurately.
- The Validation Tool does not use a standard Web browser User Agent name. The Validation Tool may not crawl or analyse sites that rely on the User Agent name accurately.
- The Validation Tool’s default behaviour is to respect robots directives. If a site has a robots.txt file to prohibit crawlers, then no validation of the site’s documents are performed. You can configure the Validation Tool to ignore robots.txt directives. See the PWGSC WPSS Validation Tool User Guide for more information.
- The output of the individual Validation Tools is in English only. It is using 3rd-party software components. The source of these components is available only in the language that it was authored.
- Markup validation cannot be performed on HTML 5 documents.
The Validation Tool includes a crawler that follows links and retrieves web documents from sites. Care should be taken with this tool to:
- ensure entry page URLs for the sites are accurate so the crawler does not go beyond the site being analysed.
- ensure that document retrieval from the site being analysed, and any links to other sites, do not impose excessive loads on web servers and the network.
The list of Web documents or URLs that the WPSS Validation Tool analyses can be supplied either in a list or by performing a crawl of a site.
A Web site crawler module is included with the Validation Tool. It crawls a site to get the list of documents to analyze. The crawler uses the site directory URL and the entry page address for the English and French entry points to perform the crawl.
The site directory URL is the URL of the top level directory or folder containing the site documents. The entry page address is the file name and URL arguments portion of the entry page. The crawler starts at the entry page and extracts any URL references from that page. These can be links, images or any other HTML tag containing a URL. If the URLs are references to other documents in the site, they are added to the set of URLs to analyze. They are in turn retrieved and analyzed for URL references. This process of retrieving and analyzing documents continues until either a the Validation Tool reaches the preset crawl limit, or there are no more URLs to analyze. After analyzing each document for URLs, the other analysis modules of the Validation Tool use that URL information.
A URL is deemed to be part of the site being crawled if the URL matches the site directory URL pattern. If it does not match the pattern, it is assumed to be a document outside the site. This is true for all URL references except those related to JavaScript files and style sheets (CSS files). References to these file types that appear in either a <script> tag or a <link> tag are also considered to be part of the site. This is due to the nature in which these files affect the display and performance of the site.
Web applications typically have a login or authentication page that limits many automated testing tools to only access Web pages outside the login. The Validation Tool has the capability to handle sites or application that employ a single login page with a form for entering credentials, and login pages to get at Web pages behind the login. By providing the Validation Tool with the URL of the login page, it will:
- retrieve the login page.
- locate the login form and form fields.
- prompt the user for credentials.
- submit the form to accomplish application login.
Once these steps are complete, the Validation Tool can proceed with crawling for Web documents behind the login. This ensures a more complete site/application analysis. See the PWGSC WPSS Validation Tool User Guide for more information.
A Web server can be configured to password protect portions of a site. When a user tries to access a protected portion, the Web server returns an “HTTP 401 (Unauthorized)” message. Web browsers prompt the user for credentials to retrieve the Web document. When the Validation Tool receives this message, it can also prompt the user for credentials to retrieve the document.
The Markup Validation module of the WPSS Validation Tool checks the mark-up, or syntax, of various file types. It checks that the files conform to the appropriate standard, such as HTML. The Validation Tool checks the following content types:
- HTML/XHTML
- Cascading Style Sheets
- JavaScript
- Robots.txt
The Validation Tools checks all content with a mime-type of text/html, which includes HTML and XHTML content, using the Web Design Group (WDG) validator. This is an open source validator based on the W3C validator. The markup is validated to the DTD specified in the content.
NOTE: This tool does not validate HTML5 mark-up.
The Validation Tools checks CSS markup using the open source W3C CSS validator. CSS content is validated from the following sources:
-
All content with a mime-type “text/css”.
-
Files listed in tags with a type attribute value of “text/css” found in HTML content. For example:
<link href="/comm/css/internet.css" rel="stylesheet" type="text/css" /> -
Content found in
<style>tags with a type attribute value of text/css. For example:<style type="text/css" > body { color: purple; background-color: #d8da3d } </style>
The Validation Tools checks JavaScript markup using the open source JavaScript Lint program. JavaScript content is validated from the following sources:
- All content with a mime-type application/x-javascript.
- URLs with a .js suffix.
- Files listed in
<script>tags with a type attribute value of text/javascript found in HTML content. For example,<script src="/source/scripts/pe-ap.js" type="text/javascript"></script>
The Validation Tool does not validate inline JavaScript in HTML content that uses the <script> tag. This content may not conform to proper JavaScript syntax due to the way browsers read and interpret inline scripts.
All robots.txt files found at the root of a web server domain, for example, http:///robots.txt, is validated using the PWGSC Robots.txt validator. Any robots.txt file that exists somewhere other than the top level is not validated as it would not be a valid location for a robots.txt file.
The Link Checker module of the WPSS Validation Tool performs a number of link violation checks in addition to the standard broken link check. These checks include:
- broken links
- cross language links
- redirected links
- bad network scope links
- broken anchors
- firewall blocked links
- IPV4 links.
Link checking is applied to all URL references found in any tag. This includes, but is not limited to:
- anchor tags
<a> - image tags
<img> - link tags
<link> - script tags
<script>
Link violations are described in the following sections.
Broken links are links or references to URLs that do not exist. This is the most basic form of link checking. When trying to retrieve a document, if it does not exist, the web server returns a “404 (Not Found)” response.
A cross language link occurs when a document in one language references a document in another. For example an English web page contains a link to a French document.
The language of a document is determined by one of the following techniques:
-
File name suffix – use the characters before the file type extension to determine language.
English: index-eng.html, report_eng.pdf, sign_e.gif, sign-e.gif
French: index-fra.html, rapport_fra.pdf, sign_f.gif, sign-f.gif
-
Language variable in URL string – check for presence of the lang variable in URL string. English: index.cfm?lang=eng French: index.cfm?lang=fra
-
Content – determine language of content for HTML, PDF or text files.
If content is available in only one language, such as a document in English only, then any links from French documents would appear as cross language links. You can avoid errors from the Validation Tool by including a language attribute within the HTML tag.
For example, a link from a French web page would look like:
<html xmlns="http://www.w3.org/1999/xhtml" lang="fr" xml:lang="fr">
<a href="http://www.tpsgc-pwgsc.gc.ca/comm/index-eng.html" lang="en" xml:lang="en">English</a>
The _lang _and xml:lang attributes indicate the link references an English document, which is a different language than the main French document. If the Validation Tool cannot determine the language of either the source document or the target document, no cross language link violations are reported.
A redirected link is one in which the URL of a link is different from the final location of the document. The final URL may differ by the domain name and or the file name. Redirects may be used translate vanity domains into official domains or to direct users to new file names for documents. There is a risk that redirects may be removed on the target site resulting in broken links.
Domain redirect : http://www.pwgsc.gc.ca/ > http://www.tpsgc-pwgsc.gc.ca/
File redirect : http://www.canada.gc.ca/main_e.html > http://www.canada.gc.ca/home.html
The Validation Tool only reports redirect link violations for permanent redirects; not for temporary redirects.
Network scope refers to a level of the network; PWGSC Intranet, Government of Canada Intranet or Internet. A bad network scope link occurs when a document at a higher scope references a document at a lower scope. For example, an Internet document has a link to a Government of Canada Intranet document. Some users may experience a broken link when selecting the link while others may get the referenced document. For example, public users versus users on the Government of Canada network.
A named anchor is a link marker on an HTML page. If an anchor is referenced in a URL, the browser positions the user at the anchor in the document. This allows for linking to particular section in a larger web document. If the named anchor does not exist in the target document, the browser placees the user at the top of the document.
A firewall blocked link is a link to a site or document that is prohibited by the departmental firewall. This is essentially a broken link as the user cannot get to the target document.
An IPV4 link is a link that has an IP address rather than using a domain name. For example, http://10.11.12.13/index.html.
The metadata checking of the WPSS Validation Tool analyses metadata within HTML/XHTML and PDF documents. It checks HTML/XHTML documents for:
- the presence of required metadata tags.
- that non-white space content is provided where required.
- the content value where the value set is known. For example, language values.
- the content format where it must have a specific pattern. For example, date values.
The metadata check also checks PDF documents for the presence of required file properties. Details on the PWGSC HTML/XHTML metadata tags and content is detailed at http://source.tpsgc-pwgsc.gc.ca/comm/web/w2/inter-meta-eng.html
The Validation Tool uses a profile to check for the required metadata tags or PDF file properties. The profile specifies the metadata tag name or PDF file property name, whether or not they need content and the content type. The metadata profiles used are:
- PWGSC SWU – Standard on Web Usability metadata tags
- TBS SWU – Treasury Board Standard on Web Usability metadata tags
- PWGSC – CLF 2.0 tags plus additional PWGSC tags
- TBS CLF 2.0 – Treasury Board CLF 2.0 list of metadata tags
- None – No metadata tags are required.
The PDF file property profiles used are:
- PWGSC –PWGSC properties (Title)
- None – No properties required.
The PWGSC SWU metadata profile requires:
| dcterms.subject | dcterms.creator | dcterms.modified | description | title |
| dcterms.language | dcterms.issued | dcterms.description | keywords |
The Treasury Board SWU metadata profile requires:
| dcterms.title | dcterms.language | dcterms.issued | description |
| dcterms.subject | dcterms.creator | dcterms.modified | title |
The PWGSC metadata profile requires:
| dc.title | dc.language | dc.publisher | dcterms.modified | keywords |
| dc.subject | dc.creator | dcterms.issued | description | pwgsc.contact.email |
The Treasury Board CLF 2.0 (TBS CLF 2.0) metadata profile requires:
| dc.title | dc.language | dc.publisher | dcterms.modified | keywords |
| dc.subject | dc.creator | dcterms.issued | description |
The PWGSC PDF file property set includes the “Title” property.
While the metadata profiles are required, not all the items require content. Those items that do require content, those metadata tags and PDF file properties must contain meaningful text. The Validation Tool checks for the absence of required text, or if the text consists of white space only (spaces, tabs, etc). The set of metadata items that require content is specified in a profile supplied to the Validation Tool.
The PWGSC SWU metadata items that require content are:
| dcterms.title | dcterms.subject | dcterms.creator | description | title |
| dcterms.language | dcterms.issued | dcterms.modified | dcterms.description | keywords |
The PWGSC metadata items that require content are:
| dcterms.title | dc.language | dc.publisher | dcterms.modified | keywords |
| dc.subject | dc.creator | dcterms.issued | description | pwgsc.contact.email |
The Treasury Board CLF 2.0 (TBS CLF 2.0) metadata items that require content are:
| dcterms.title | dc.language | dc.publisher | dcterms.modified | keywords |
| dc.subject | dc.creator | dcterms.issued | description |
Some metadata tags must contain specific values or values in a specific format. The Validation Tool checks for a specific content formats, such as date values, or that content comes from a controlled vocabulary. It cannot check that the actual value is appropriate. Specific content formats and values are further specified below.
The content of the dcterms.issued and dcterms.modified metadata items is a date value that must be expressed in YYYY-MM-DD format. The Validation Tool checks that the content conforms to this format and that the month is in the range 01 to 12 and that the date is in the range 01 to 31. For example:
<meta name="dcterms.issued" scheme="W3CDTF" content="2005-11-21" />
The Validation Tool also checks that dates are consistent; dcterms.modified is not before dcterms.issued.
The content of the dc.language metadata item is a three character language code. For example:
<meta name="dc.language" scheme="ISO639" content="eng" />
The content of the dc.subject metadata item on PWGSC web pages is a list of terms that describe the topic of the web page. These terms must be selected from a controlled vocabulary, the Government of Canada Core Subject Thesaurus. The terms must be separated with a semicolon selected from the list of preferred terms, not the non-preferred terms. The Validation Tool verifies that the dc.subject terms are taken from the Thesaurus matching the language of the page. For example:
<meta name="dc.subject" scheme="gccore" content="Newsletters, Information, Information sources" />
The content of the pwgsc.contact.email item is a single email address. The Validation Tool verifies that the content is a well formed e-mail address. It does not verifies that it’s a valid address. For example:
<meta name="pwgsc.contact.email" content="[email protected]"/>
The scheme attribute of some metadata tags must contain specific values. The TBS CLF 2.0 profile requires the following scheme value for the specified metadata tags:
| Tag Name | Scheme Attribute Value |
|---|---|
| dcterms.issued | W3CDTF |
| dcterms.modified | W3CDTF |
| dc.language | ISO639-2/T |
| dc.subject | gccore |
The PWGSC SWU profile requires the following scheme value for the specified metadata tags:
| Tag Name | Scheme Attribute Value |
|---|---|
| dcterms.issued | W3CDTF |
| dcterms.modified | W3CDTF |
| dcterms.language | ISO639-2 |
| dcterms.subject | gccore |
The Validation Tool reports invalid content errors for the metadata tags that do not have these scheme values.
The Validation Tool checks for the set of required PDF file properties specified by a profile. The PWGSC PDF File Properties profile contains the following item:
- Title
The accessibility checking module of the WPSS Validation Tool, also referred to Technical Quality Assurance (TQA) performs a number of accessibility tests on Web documents. A profile of test cases is used to determine which checks to perform and report on.
The Validation Tool uses the WCAG 2.0 – WCAG 2.0 test cases profile to check and create the report.
A list of test cases in each profile is outlined below. Each test case details what the Validation Tool checks for, and if it does not fully implement the checkpoint, it details what is not checked.
Accessibility checking is limited to verifying that appropriate mark-up is used and/or text is supplied where needed. There is no checking on the value of the content. A test case can check if alternative text is provided, but it cannot check if the text is appropriate. For example, it can verify there is a description for an image, but cannot verify that the description is valid for that image.
This profile contains WCAG 2.0 test cases.
| Description | Failure of Success Criterion 1.3.1 due to using changes in text presentation to convey information without using the appropriate markup or text. |
| Checks For | Text styled to look like a heading using HTML mark-up (e.g. <strong>). |
| Does Not Check | Text styled to look like a heading using CSS. |
| Images used to convey structural information. |
| Description | Failure of success criterion 1.1.1 due to using CSS to include images that convey important information. |
| Images are deemed non-decorative if they are specified through tool configuration (non-decorative image list) or if the first occurrence of an image in a site includes alt text. | |
| Checks For | If non-decorative images are loaded using CSS. |
| Does Not Check | If the information is provided to assistive technologies and is also available when the CSS image is not displayed. |
| Description | Failure of success criterion 2.2.2 due to using text-decoration:blink without a mechanism to sop it in less than five seconds. |
| Checks For | All styles for the text-decoration property with a value of blink. |
| Does Not Check | If the styles are actually used within the document. |
| Description | Failure of Success Criterion 2.2.2 due to including scrolling content where movement is not essential to the activity without also including a mechanism to pause and restart the content. |
| Checks For | The presence of a <marquee> tag. |
| Does Not Check | Any other mechanism, such as scripting, generates moving content. |
| Description | Failure of Success Criterion 1.3.1 and 4.1.1 due to insufficient information in DOM to determine one-to-one relationships, for example, between labels with same id in HTML. |
| Checks For | The accesskey and for attribute values are unique. |
| The attribute values that have an idref value have a corresponding id value. | |
| The accesskey values consist of a single character only. | |
| Note: Duplicate id values are reported under failure F77. | |
| Does Not Check | For tables that use the axis attribute, check that all values listed in the axis attribute have a corresponding id value in a table header cell in the same table. |
| For client-side image maps, check that the value of the usemap attribute has a corresponding id value if the usemap attribute is not a URI. |
| Description | Failure of Success Criterion 2.4.2 due to the title of a page not identifying the contents. |
| Checks For | If title is missing. |
| If the title is the same as the file name in the document URL. | |
| If the title matches any invalid values, such as “untitled”. | |
| Does Not Check | Other invalid title values. |
| Description | Failure of Success Criterion 1.1.1 and 1.2.1 due to using text alternatives that are not alternatives. |
| Checks For | If the alt attribute matches the src attribute for <img> tags. |
| If the alt attributes matches any invalid values in items such as photos, images, or graphs. | |
| Does Not Check | Other text alternatives that are not text alternatives. |
| Description | Failure of Success Criterion 1.3.2 due to using white space characters to control spacing within a word. |
| Checks For | Whitespace inside words. |
| Description | Failure of Success Criterion 1.1.1 by omitting the alt attribute for non-text content used for decorative purposes only in HTML. |
| Images or applets are deemed decorative if they are specified through tool configuration (decorative image list) or if the first occurrence of an image or applet in a site has no alt text. | |
| Checks For | If the alt attribute is missing on decorative images or applets. |
| Description | Failure of Success Criterion 1.1.1 due to providing a text alternative that is not null (alt="spacer" or alt="image") for images that should be ignored by assistive technology. |
| Images are deemed decorative if they are specified through tool configuration (decorative image list) or if the first occurrence of an image in a site has no alt text. | |
| Checks For | If the alt attribute is an empty string for images that are decorative. |
| Description | Failure of Success Criterion 2.2.1 and 2.2.4 due to using meta redirect with a time limit. |
| Checks For |
<meta http-equiv=”refresh”> with a timeout and a URL |
| Description | Failure of Success Criterion 2.2.1, 2.2.4 and 3.2.5 due to using meta refresh with a time limit. |
| Checks For |
<meta http-equiv=”refresh”> with a timeout and a URL. |
| Description | Failure of Success Criterion 1.3.1 and 2.1.1 due to using scripting events to emulate links in a way that is not programmatically determinable. |
| Checks For |
onclick or onkeypress attributes on tags other than <a> <area> <button> <input> or <select>. |
| Description | Failure of Success Criterion 1.3.1 due to using structural markup in a way that does not represent relationships in the content. |
| Checks For |
<fieldset> used outside a <form>. |
<hr> used as decoration (<hr> followed by another <hr>, or heading followed by <hr>). |
|
| Does Not Check | The semantic meaning of other tags. |
| Description | Failure of Success Criterion 2.2.2 due to using the blink element. |
| Checks For | The <blink> tag. |
| Description | Failure of Success Criterion 2.1.1 due to using only pointing-device-specific event handlers (including gesture) for a function. |
| Checks For | Mouse only event handlers. |
| Description | Failure of Success Criteria 2.1.1, 2.4.7, and 3.2.1 due to using script to remove focus when focus is received. |
| Checks For | JavaScript this.blur function. |
| Description | Failure of Success Criterion 2.2.1 due to using server-side techniques to automatically redirect pages after a time-out. |
| Checks For | HTTP header for a “refresh” field with a time out greater than zero. |
| Description | Failure of Success Criterion 1.1.1 due to omitting the alt attribute on img elements, area elements, and input elements of type “image”. |
| Checks For |
alt attribute on <img>, <area> and <input type=“image”> tags. |
| Description | Failure of Success Criterion 3.2.3 due to presenting navigation links in a different relative order on different pages. |
| Checks For | The relative order of navigation links of all pages. The order of the navigation links on a site’s entry page is used as the expected relative order. |
| Does Not Check | If there are multiple site navigation links with the same anchor text. These links are not checked. |