Contact and help

The Australian Government's study into the Accessibility of the Portable Document Format for people with a disability

Study approach and methodology

The Study used a qualitative research framework with multiple methods of data collection including desk research, focus groups, technical analysis and participatory approaches. As the term ‘accessibility’ is highly subjective, the Study relies upon phenomenological research; that is, the primacy of the lived experience, to support a user perspective of accessibility.

The research elements used in the Study involved:

Focus groups

Vision Australia conducted three focus groups between 28 September and 1 October 2009 in New South Wales, Queensland and Victoria. They included 22 people who are blind or have low vision and use assistive technologies. Most participants use multiple adaptive strategies, depending on their situational context. These are outlined in Table 9: Adaptive strategies and assistive technologies used. Participants for each session were recruited against predetermined demographic profiles which included their gender, age, type of disability, the assistive technology they used and their level of skill in using their assistive technologies. A Vision Australia consultant facilitated the 80 minute focus groups. A scribe and digital recording equipment were used to capture the feedback for further analysis.

Table 9: Adaptive strategies and assistive technologies used

Adaptive Strategies AT Device Total Users
Braille Notetaker BrailleNote 4
PAC Mate 1
Daisy Player VictorReader Stream 2
Screen Reader JAWS 19
System Access (SATOGO) 3
VoiceOver 1
Window-Eyes 3
Screen Magnifier MAGic 3
ZoomText 4

Telephone interviews and email correspondence

Telephone interviews and email correspondence were undertaken with disability organisations supporting people who are blind or have low vision between April 2009 and February 2010 to establish the most common assistive technologies, and their versions, used by clients.

Contact with clients was facilitated through: Vision Australia; the Association for the Blind of WA – Guide Dogs WA; Royal Society for the Blind of South Australia; Royal Institute for Deaf and Blind Children; Retina Australia; Women with Disabilities Australia; Network of Disability Officers ; Blind Citizens Australia (email list); and Vision Impaired People’s Information List in Australasia (email list).

Online consultation

The public consultation ran from 16 September until 26 October 2009. Consultation Guidelines were published on the Web Publishing Guide External Site. The consultation was promoted through the AGIMO web manager community, an external email distribution list “Web Standards Group”, with peak disability bodies, web developers and accessibility consultations and on the Department of Finance and Deregulation website.

In total, 38 submissions were made. Of those, four were conducted via a telephone interview, three were made anonymously, and the remaining 31 were written submissions either emailed or sent via post. Approximately half of the submissions were made on behalf of businesses or organisations (such as Peak Disability Bodies), the others were personal accounts.

Technical evaluation

Vendor interviews

Interviews with vendors or Australian resellers were conducted between October 2009 and February 2010. Organisations participating included HumanWare, Quantum Technology (on behalf of Freedom Scientific), NV-Access, Serotek, Apple, GW Micro and Ai Squared.

Technical testing

The technical test suite used was created by Adobe. It included 43 specific tests pertaining to the WCAG 2.0 Success Criteria. Success Criteria that were considered to have no impact on the support of the PDF format for accessibility were excluded from the tests. For example, Success Criteria 1.4.3 Contrast (minimum) was considered, by Adobe, to be a requirement of the document author not the format itself.

Adobe determined the characteristics within a specific Success Criteria that warranted testing, as there is no defined list yet available for PDF. For example, for Success Criteria 1.3.1 (Info and Relationships) paragraphs, headings, controls and input elements are tested, but special presentations of text (bold, italicised) and mark-up of tables are not tested.

A test suite containing 22 PDF files was created to ensure only the specified set of behaviours of the 43 tests was evaluated. Each test file replicated specific behaviours or elements of a PDF file requiring testing. Some test files applied to more than one test, for example Test File 6 contained a list of textual headings (h1 through to h6) with appropriate PDF tags applied. Test File 6 was therefore used to conduct the following tests as it presented the required behaviours of each:

The test suite does not take into account usability functions of assistive technologies not covered by WCAG 2.0. For example, identification of paragraphs is included (Success Criteria 1.3.1), but navigating by paragraph is not a requirement under a WCAG 2.0 Success Criteria and therefore is not included.

Testing was conducted by Adobe, replicating the methodology used in the original 2008 and 2009 tests. It is summarised as:

User experience evaluations

Twenty-three user experience evaluations were undertaken on eight PDF files by people using different adaptive strategies. Evaluations were conducted at either a Vision Australia office, or in the participant’s home or place of work, due to their disability, technical requirements or availability.

A pilot evaluation was conducted on 19 February 2010 to test and refine the methodology. Formal evaluations were conducted between 22 February and 23 March 2010.

Evaluations took up to two hours: 30 minutes provided for pre- and post-evaluation interviews and up to 90 minutes for user testing. A sample of PDF documents was prepared (Collection A & B) and six tasks were presented to the participants using Collection A, followed by six tasks in Collection B. Tasks were presented to the participants in a randomised order to reduce the impact of any learning curve as a result of specific document types following one another.

Data recorded

An evaluator from Vision Australia facilitated each of the sessions and recorded both qualitative and quantitative data including:

Profiles of participants included:

Users with different skill/experience level were also recruited, particularly for screen reader users. Experience or skill level had been highlighted as a factor that impacted on participant’s ability to use PDF files during the user consultation sessions. Participants were selected so as to ensure a mix of age and gender. A full listing of the participants is included in the Supplementary Report.


Contact for information on this page:

Back to top

Last Modified: 25 November, 2010