Postgraduate Research Experience Survey: Chris Park Andria Hanbury Malgorzata Kulej

Postgraduate Research Experience Survey Chris Park Andria Hanbury Malgorzata Kulej
Lee Harvey
FINAL REPORT
Postgraduate Research Experience Survey
Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1. The Postgraduate Research Experience Survey . . . . . . . . . . . . . . . . . . . . . 6 2. Operation of the 2007 PRES survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3. Overview results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4. Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 5. Skills development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 6. Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 7. Intellectual climate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 8. Goals and standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 9. Thesis examination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 10. Personal factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 11. Condence for completing on schedule . . . . . . . . . . . . . . . . . . . . . . . . . 39 12. QAA Code of Practice indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 13. Process review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Appendix 1: The 2007 PRES questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . 44 Appendix 2: Demographic prole of respondents . . . . . . . . . . . . . . . . . . . . 54 Appendix 3: Multiple regression analyses for students overall evaluation ratings and condence for completing on schedule . . . . . . . . . . . 59 Appendix 4: Multilevel modelling: Scales broken down by demographic variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
The Higher Education Academy November 2007 1
Executive summary
Introduction
The rst national survey in the UK of what postgraduate research students think about their experiences provides a useful snapshot of the research student experience with implications for policy and practice within the sector. The Postgraduate Research Experience Survey (PRES), which will be available annually and supported by the Higher Education Academy, offers higher education institutions an opportunity to nd out what research students think about their experiences and to benchmark their student views against the much larger aggregate sample. No institutions are named in the Higher Education Academys aggregate report and aggregate data is stored anonymously, making it impossible to construct league tables. The participating institutions could choose which students they would like to take part in the survey; and most invited all of them. Each institution remains the owner of its own institutional data and can publish its own PRES results internally and externally after the national aggregate data has been published.
Institutional response rates

58 institutions took part and a total of 10,544 replies were received: institutional response rates varied from 10% to 65%. The overall prole of PRES respondents is similar to the sector prole recorded by HESA. PRES closely matches disability, domicile and ethnicity in the sector, but has relatively more female respondents (54%) than the sector average (45%), and more full-time students (79% compared to 51%). The latter is partly accounted for by the status of writing-up students who are often recorded as part-time by institutions while the students see themselves as full-time.
Overview results
Respondents considered supervision to be the most important aspect in successfully completing their research degree programme. Intellectual climate was also an important factor in overall satisfaction. Research degree students were positive about their overall experience: 81% indicated that the programme as a whole met or exceeded their expectations. Furthermore, 65% of research degree students expected to complete their programme more or less on schedule.
2 The Higher Education Academy November 2007
Disciplinary differences were found for some aspects, but no discipline received consistently more or less positive ratings. The PRES ndings are similar to ndings from the Australian Postgraduate Research Experience Questionnaire (PREQ) on which PRES was based.
Supervision
Although supervision was rated as important and rated highly overall, a fth of respondents thought supervision did not meet expectations. There were some concerns about guidance with literature searches. Arts and humanities, and social sciences tended to be more positive about aspects of supervision than the sciences and medicine areas. More positive demographic groups were: males; ethnic minority students and rst-year students, the latter is reected in reservations among students writing up the thesis or amending the thesis after the viva voce examination.
Intellectual climate
Although intellectual climate was a major factor, and regarded as important, it was not as highly rated as other areas. A quarter of students were disappointed with the intellectual climate. Research ambience and a feeling of being integrated into the department were areas where half the respondents were negative. Science and medicine students were most positive about intellectual climate. Disengagement from the department seemed to play a part: part-timers, distance learners and those doing an MPhil were least positive, as were non-EU students.
Skills development
Development of research skills was more positively viewed, and regarded as more important, than the development of transferable skills. Respondents indicated improvement in learning independently and analytical skills. MPhil and masters students were less positive about skills development than PhD students. Only two-fths of respondents thought that they had received adequate support and guidance in developing their teaching.
Infrastructure
Infrastructure was rated as important, although not a signicant factor in overall experience. Most had appropriate equipment, library and computing resources, although fewer than half thought nancial support for research activities was appropriate. Arts and humanities, and social science students were less positive about infrastructure than other disciplines. Less positive demographic groups were: part-time, distance, MPhil, non-EU and students with a declared disability. Students in the second year were also less positive about infrastructure than rst-year students.
Goals and standards

The closer respondents were to the end of their study, the more positive they became about goals and standards. Fewer than two-thirds claimed that they understood the requirements for the thesis examination, but this may be a matter of students in the early years not focusing on this issue. Nonetheless, some work to alert students earlier might enhance completion rates.
Thesis examination
Only a small sub-sample had progressed to thesis examination. They were positive about the process, in particular its fairness. The main concern was the adequacy of the guidance and support for the viva voce examination.
Personal factors
Respondents agreed that partners, friends and family, and employers were supportive of their programmes, although they did not always understand the demands placed upon respondents by the research degree programme. The main problem area was the ability to support their study from personal nances.
Completion on schedule
Two-thirds of the respondents expected to complete their programme more or less on schedule. There was less condence among physical sciences, and engineering and
computer studies students than other disciplines. Funding apparently did not impact on condence to complete to schedule.
QAA Code of Practice

PRES includes questions relating specically to the UK context, informed by the precepts of the QAA Code of practice for the assurance of academic quality and standards in higher education and the Research Councils skills training requirements. Career and personal development planning appears to be an aspect of the postgraduate research experience that is poorly addressed; only a third of respondents indicated any encouragement to reect on career opportunities and development needs. Fewer than half the respondents were aware of the institutional responsibilities towards students, or indicated that their institution valued and responded to feedback from research degree students.
1. The Postgraduate Research Experience Survey

1.1 Introduction
This report summarises the development, operation and ndings of the rst national survey in the UK of what postgraduate research students think about their experiences. Participating institutions are able to analyse their own results and compare them with aggregate results from the whole survey; the focus in this report is on analysis of the aggregate results. Chapters 1 and 2 describe the survey, and Chapters 312 summarise the main ndings. The results from this rst PRES survey provide a very useful snapshot of the research student experience, highlighting areas where students rated their experiences very positively and other areas where they rated them much less positively. The results have implications for policy and practice, both within individual higher education institutions (HEIs) and across the sector.
1.2 Objective
PRES is an online survey tool designed to collect feedback from current research students in a systematic, user-friendly and comparative way. The main objective of PRES is to help HEIs to enhance the quality of their postgraduate research degree provision, informed by evidence-based decision-making. PRES is not a student satisfaction survey; it offers HEIs an opportunity to nd out what research students think about their experiences, and the ability to benchmark their students views against the much larger aggregate sample which is representative of the sector across the UK. PRES is designed to be made available annually to any HEIs that wish to use it. The static snapshot offered by this rst survey will, through time, be replaced with more of a moving picture, by tracking and comparing changes from year to year. Longitudinal comparisons will enable monitoring of the speed and direction of change, both within individual HEIs and across the sector, in response to initiatives designed to enhance the research student experience. The need for a survey like PRES is clear, given four considerations. First, the nature of research degree programmes has changed a great deal over
the last decade, for example in response to the QAA Code of practice for the assurance of academic quality and standards in higher education (section 1 covers research degree programmes, and was revised in 2004); the 2006 QAA Special Review of research degree programmes; publication in 2002 of the Roberts Review SET for Success, which shaped a new skills training and development agenda and provided additional funding through the Research Councils; and publication in 2001 of the Research Councils Joint Statement of the Research Councils Skills Training Requirement for Research Students, which dened the transferable skills that research students should be acquiring. Second, there has been no systematic way of collecting information about research students and what they think; many HEIs have developed their own internal surveys, but these dont allow comparisons to be made between HEIs and they are often not run annually. Third, there is growing interest across the UK in the overall nature of the student experience and it is important that the student voice is heard and taken seriously. The National Student Survey operates at undergraduate level and PRES provides a similar opportunity for research students. Fourth, there is growing commitment across the UK towards enhancing the student experience, and PRES allows evidence-based decision-making for enhancing the research student experience.
1.3 Development
PRES is based on a survey that was originally developed in Australia in 1998, the Postgraduate Research Experience Questionnaire (PREQ), which has been found to be a reliable and robust survey instrument. In recent years the University of Oxford has used a modied version of PREQ (locally called OPREQ ) to collect feedback on the views of its research students. Although PRES uses many of the core questions from PREQ, and has adopted the same scales approach, there are some basic differences between the two surveys: PRES asks current students for their views whereas PREQ asks recent graduates; PRES is voluntary for HEIs whereas PREQ is compulsory; and PRES includes questions relating specically to the UK context, informed by the precepts of the QAA Code of practice for the assurance of academic quality and standards in higher education and the Research Councils skills training requirements. One catalyst for the development of PRES as a national survey in the UK was a recommendation by the so-called Rugby Team, the sector working
group on the evaluation of skills development of early career researchers, which was set up by the UK GRAD programme in 2005 . Recommendation 5 of the Rugby Teams 2006 strategy report states that: The Academy should consult with the other national stakeholders, particularly the funding councils, research councils and QAA to inform the development of their proposed survey on the impact of the research environment on the PGR learning experience. The development of PRES has been informed by wide consultation across the sector, including HEIs and bodies such as the National Postgraduate Committee (NPC), the Quality Assurance Agency (QAA), the funding council (HEFCE) and research councils (RC-UK), and the UK GRAD Regional Hubs. The development and testing of PRES was overseen by a steering group, chaired by Chris Park (a Higher Education Academy Senior Associate), that includes representatives of the main stakeholder groups outlined above, along with Gosia Kulej (Survey Co-ordinator) and Andria Hanbury (Adviser: Quantitative Research Methods), who have provided guidance, support and advice on behalf of the Academy. The work of the Steering Group was informed by feedback from the PRES Ofcers in individual HEIs, who were responsible for running the survey within their institution. The PRES website contains information on the background and development of PRES, and contact details for HEIs interested in taking part. A number of design principles informed the development of PRES (Table 1).
Table 1. Design principles The survey should be: a. Student-centred: it must listen to the student voice, and focus on enhancement of the student experience. b. Easy to use: from the students perspective, it must be in an accessible online format, easy to understand, quick to complete; from the institutions perspective, it must be easy to set up and administer, and easy to analyse and interpret the results. Voluntary: institutions and their research students must be allowed and encouraged, but not required, to take part. Flexible: while for comparative purposes it must have an agreed standard set of core questions, it must be possible for HEIs to add their own questions if they wish to. Useful: it must provide information that is useful to HEIs and national bodies, and this includes a focus on the student experience and the opportunity for comparative analysis (benchmarking and longitudinal tracking). Cost-effective: it must be economical for HEIs to run [the Academy met all central development and support costs]; the survey itself is free to users; participating HEIs need a BOS site licence (see page 13). Anonymous: the anonymity of student respondents and institutions taking part must be protected: all student responses are anonymous; a list of participating HEIs is not published. Secure: participating HEIs must be condent that their institutional results will not be made available to any third party. Aggregate results are held on the BOS server; even the Academy cannot identify individual institutions by name in the aggregate results; aggregate results will not be released or sold to any third parties.
c. d.
e.
f.
g.
h.
1.4 Methodology
The PRES questionnaire, format and method of delivery were piloted in August 2006 with eight volunteer HEIs. Detailed analysis of the 1202 responses (14% response rate) included student views on the questionnaire, its relevance and ease of use. The pilot results, coupled with results from a focus group of institutional PRES ofcers, informed revision of some questions, addition of questions relating to teaching opportunities and personal factors, and preparation of appropriate guidance for participating institutions for the main 2007 PRES survey.
10
1.4.1 Approach PRES used the same method as the Careers in Research Online Survey (CROS), which has been used successfully by HEIs across the UK in recent years for collecting feedback from research staff . Both surveys are based on online questionnaires that are delivered via the Bristol Online Surveys (BOS) website. Participating institutions used their own BOS accounts to run PRES. Many HEIs already had BOS site licences because they were participating in CROS and using it for other online surveys. Those who did not have a BOS site licence needed to buy one to use PRES. Information about the site licence can be obtained from BOS. Each institution was given an electronic template of the PRES questionnaire before PRES went live, which they could modify by adding institutionspecic questions (75% of institutions used this option). One question (question 24 in Appendix 1) in the demographic section asked students which school or department they belonged to. This question could be edited so that institutions were able to map their own departmental structure into the survey, which allowed detailed breakdown of results in meaningful ways. All other parts of the survey were locked and could not be changed in any way, to make sure that results would be comparable across institutions. The participating institutions were responsible for contacting their research students by email to invite them to take part in PRES. Institutions could choose which students they would like to take part in the survey, and most invited all of them. The BOS website allowed the PRES Ofcers to monitor their own institutional results and the aggregate results at any time while the survey was open and after it closed. 1.4.2 The questionnaire PRES was based on a standard set of questions seeking students views on a range of aspects of their student experience. The questionnaire also included free text boxes, and participating institutions were able to add as many of their own specic questions as they wished (for example, to collect feedback on particular services, initiatives, policies or practices). The whole survey ran to six web pages: a welcome page, data protection information, the core questionnaire, institutional questions, the demographic section and the nal Thank you page. This was felt to be an optimum size to allow students to complete it in a reasonable period of
11
time. The core PRES questionnaire is included in Appendix 1, and a copy can be downloaded in PDF format from the PRES website. A Welsh version of the questionnaire was also created. The questions in PRES were structured in eight sections. Sections one, two and three comprised 38 randomly ordered items relating to different dimensions of the research student experience. Twenty-eight of these items were grouped together to form six scales: supervision skills development infrastructure intellectual climate goals and standards thesis examination.
These were also used in the Australian PREQ and the Oxford OPREQ surveys; the remaining two scales were stand-alone groups of items that related to teaching opportunities and personal factors (sections 4 and 5 in Appendix 1). The remaining non-scale items in these sections covered additional aspects of the research student experience, informed by precepts from section 1 of the QAA Code of Practice: career/personal development progress and review arrangements student representations, complaints and appeals feedback mechanisms selection, admission and induction of students skills training.
Section six of the questionnaire asked how important students considered six broad aspects of their experience to be, with regard to completing their research degree programme. The aspects were: supervisory support and guidance access to appropriate facilities opportunities to develop a range of research skills the research environment opportunities to develop a range of transferable skills provision of guidance on institutional standards and expectations for the research degree programme.
They were also asked how well their experience of these six broad aspects
12
met with their expectations, and how they rated their experience overall. One question asked students to rate their condence for completing their programme more or less within the planned timescale. Section seven contained one open question that invited students to add further comments on their experience of their research degree programme. The demographic section consisted of 14 items, allowing analysis of patterns of responses for different types of student: age gender disability ethnicity full-time/part-time mode of study (face-to-face, distant) year of study fee status (UK, EU, overseas) previous activity (year before started) current activity discipline department (optional for institutions to use) funding source degree registered for.
The response categories for disability and ethnicity reected the categories that HESA uses, to make the PRES results comparable with published national data. In sections four to twelve, responses to questions in the scales have been broken down by discipline. In future reports the focus will be on different dimensions of the demographic prole of respondents.
1.5 Ownership and anonymity

All data collected in the PRES survey was collected on an anonymous basis. In a few institutions where students were invited to take part in a prize draw, they were asked to provide their email address if they wished to enter. Institutional results remain condential to the institution. Aggregated results, in which the identity of each institution is not recorded, remain condential to the participating institutions and to the Academy. The list of institutions participating in PRES remains condential to these institutions and the Academy, although the latter does not see the free text comments, which are exclusive to the institution.
13
Two documents were issued by the Academy to control ownership and use of the PRES data: a PRES Code of Practice and a PRES Collaborative Agreement (which was signed by all participating institutions). This was to make it possible for institutions to benchmark themselves against the whole cohort or with chosen institutions (with the agreement of those institutions), while minimising the risk of any league tables being constructed using PRES data (which could inhibit institutions from taking part in the future, given that participation in PRES is voluntary). Each institution remains the owner of its own institutional data and it can publish its own PRES results internally and externally. The Steering Group decided that institutions should not publish aggregated PRES data for their institution, until after the Higher Education Academy has published the national data. The Academy has access to the aggregate dataset with individual institutionlevel data. Institution names and free text replies have been removed from this dataset on purpose, to make it impossible for the Academy to identify particular institutions (and thus protect the anonymity of institutions).
14
2. Operation of the 2007 PRES survey

The survey was open from mid-March to mid-May 2007. Participating institutions launched their PRES surveys between 16 and 31 March, and the survey closed on 18 May. The Academy offered various forms of support to institutions during and after the survey, including documentation (How to set up PRES; How to analyse PRES data; Eight things to avoid when analysing PRES data; and How to set up a PRES benchmarking club), a regular PRES email bulletin, a Frequently Asked Questions sheet, the dedicated website, and ad hoc support by telephone and email whenever necessary. The Academy also arranged technical support from the BOS team, and ran a PRES Ofcers meeting in London in July 2007 to share good practice and collect feedback from institutions. There is good reason to believe that the results are broadly representative of the sector (the overall research student population); both of institutions represented and students who participated. The dataset is suitable for statistical analysis, which allows meaningful general conclusions to be drawn which have broad relevance.
2.1 Participating higher education institutions

All HEIs in the UK were invited to take part on a voluntary basis, so the sample of participating institutions was self-selecting. A total of 65 HEIs expressed initial interest in running PRES, and of these 59 launched the survey and useful results were collected at 58.The sample included 44 HEIs from England, eight from Scotland, four from Wales and two from Northern Ireland. By type, the sample comprised 25 post-92 institutions, 19 pre-92 institutions, eight Russell Group and six small and specialist colleges.The number of institutions, the geographical coverage, and the range of types were positive signals of how useful institutions thought PRES would be for their own internal purposes, particularly in the rst year that PRES has run nationally.
2.2 Institutional response rates

A total of 10,544 replies were received out of the 42,000 students that were invited to take part by the 58 participating institutions, producing an overall response rate of 25.2%. According to feedback offered by PRES Ofcers after the close of the survey, institutional response rates varied from around 10% to over 60%.
15
Two HEIs achieved response rates of more than 60%, but most were in the range 1030%. The BOS website allowed PRES Ofcers to monitor their institutional response rates while the survey was open, in order to make informed decisions about follow-up to encourage students to take part. Most PRES Ofcers sent between three and six email reminders. Some institutions introduced prize draws in an effort to maximise response rates.
2.3 Demographic prole of respondents

The demographic prole of the respondents can be compared against that of the total population of research degree students in the UK, in order to establish how representative the PRES sample was. Higher Education Statistics Agency (HESA) data on higher degree (research) students covering the period 200405 were used for this comparison, the details of which are given in Appendix 2. PRES included more demographic variables than the HESA dataset. The overall prole of PRES respondents was, for most factors, similar to the sector prole recorded by HESA. PRES had relatively more female respondents (54.3%) than the sector average (45.3%) (Table A2.1 in Appendix 2), but closely matched disability (Table A2.2), domicile (Table A2.3) and ethnicity (Table A2.4) in the sector. The biggest difference was in mode of study (Table A2.5); PRES had relatively many more full-time students (79.3%) than the sector average (51.2%). The Steering Group believes that this probably reects the fact that many students who are writing up see themselves as full-time students, whereas HESA (perhaps based on the way institutions record them) classies them as part-time.
16
3. Overview results
Detailed results from the survey are summarised in the following chapters under the main areas: supervision, skills development, infrastructure, intellectual climate, goals and standards, thesis examination, personal factors, completion and QAA Code of Practice precepts. The analysis is for the national data and no comparisons at an institutional level are provided in this report. The following briey summarises the overall results and the detail is in the subsequent sections. Respondents considered supervision to be the most important aspect in successfully completing their research degree programme. Research degree students were positive about how their overall experience of their research degree programme had met with their expectations: 59% agreed that their overall experience had exceeded their expectations and a further 22% rated it as having met their expectations. Therefore, in total, 81% of research degree students rated their experience of their research degree programme as a whole, as having met or exceeded their expectations (Table 2). Furthermore, 65% of research degree students expected to complete their programme more or less on schedule. Across all of the scales, research degree students were positive about how their experience had met with their expectations, particularly with regards to the opportunities provided to develop research skills. For the full sample, personal factors (4.09) had the highest average level of agreement with the composite scale items, although this was not directly assessing institutional provision. Of the scales that examined institutional provision, supervision (mean 3.93) had the highest mean agreement, followed by skills development (3.86), goals and standards (3.80), infrastructure (3.62), intellectual climate (3.40) and teaching opportunities (3.11). Thesis examination, which only affected a sub-sample of respondents, was also highly rated (mean 3.96) (Table 2). Disciplinary differences were found across the different aspects of research degree students experiences; however, there were no consistent ndings, with no one particular discipline receiving consistently more or less positive ratings.
17
Table 2. Scale scores, importance, experience and expectations ratings

% who rated experience as having failed to meet expectations (score of 1 to 3)* 22.6 16.5 18.1 21.8 25.9 26.9 19.3 % who rated experience as having met or exceeded expectations (score of 0 to +3)* 77.3 83.6 81.9 78.2 74.2 73.1 80.7 % who rated this aspect as not important (score of 1 or 2)** % who rated this aspect as important (score of 4 or 5)** 95.3 88.1 67.6 89.1 83.7 74.2 -
Mean 3.93 3.86
Scale
Supervision (N=10489) Skill development (N=10455) Research skills Transferable skills Infrastructure (N=10454) Intellectual climate/ research environment (N=10439) Goals and standards (N=10493) Thesis examination (N=408) Teaching opportunities (N=10248) Personal factors (N=9863) Overall experience of your research degree programme (N=10423)
3.62 3.40
3.80 3.96 3.11 4.09 -
* The question asked was: Please rate the following broad aspects of your research degree programme in terms of how your experience of them has met with your expectations (-3 = it is much more negative, 0 = it has met my expectations, +3 = it is much more positive) ** The question asked was: For the following items, please rate how important, in terms of successfully completing your research degree programme, you consider them to be (1 = Not at all important and 5 = Very Important)
SD 0.798 -
0.984 2.1
3.2 9.8 0.850 2.6 0.971 3.7
0.883 5.8 1.057 1.317 0.783 -
18
A multiple regression and multilevel modelling analysis (detailed in Appendices 3 and 4) show that the supervision and intellectual climate were the two aspects that had the most inuence on whether the overall experience of the research programme met expectations and on students perceptions of their condence in completing on schedule. Demographic differences were found in the students perceptions of their experience. Although there were no clear distinctions, students of black ethnicity and those receiving some form of funding were more positive about certain aspects of their experience. On the other hand, students with a declared disability, registered as doing an MPhil, or having come from a non-research background, were examples of groups found to be less positive about some aspects of their experience.
3.1 Supervision
Although supervision was rated as important and the composite scale was rated highly, there were some concerns about guidance with literature searches and a fth of respondents thought supervision did not meet expectations. The arts and humanities, and social sciences tended to be more positive about aspects of supervision than the sciences and medicine areas. Males and ethnic minority students were more positive than females and White students, respectively, when it came to supervision and, it seems, respondents in their second-year and above were less positive than rstyears, reected in some reservations among students writing up the thesis or amending the thesis after the viva voce examination.
3.2 Intellectual climate

Although intellectual climate was a major factor, and regarded as important, it was not as highly rated as other areas, such as supervision and skills development. Indeed a quarter of students were disappointed with intellectual climate; research ambience and a feeling of being integrated into the department were only endorsed by half the respondents. Science and medicine students were most positive about intellectual climate. Being disengaged from the department seemed to play a part: part-timers and distance learners, as well as MPhil students, were least positive, as were non-EU students.
3.3 Skills development

Development of research skills was more positively viewed, and regarded as more important, than development of transferable skills. Learning independently and analytical skills were areas of improvement endorsed
19
most positively. MPhil and masters students were less positive about skills development than PhD students. Only two-fths of respondents thought that they had received adequate support and guidance in developing their teaching.
3.4 Infrastructure
Infrastructure was rated as important although not a signicant factor in overall experience. Two-thirds of respondents indicated they had appropriate equipment, library and computing resources, although fewer than half the respondents thought nancial support for research activities was appropriate. Arts and humanities, and social science students had more reservations about infrastructure than other disciplines. There were various sub-groups of students who were less positive about infrastructure including part-time, distance, MPhil, non-EU and students with a declared disability. Students in the second year were also less positive about infrastructure than rst-year students.
3.5 Goals and standards

Although goals and standards were reasonably highly rated, there were more than a quarter of students for whom expectations had failed to be met. However, it seems that the closer respondents were to the end of their study the more positive they became about goals and standards. Fewer than two-thirds claimed that they understood the requirements for the thesis examination, but this may be a matter of students in the early years not focusing on this issue. Nonetheless, some work to alert students earlier might enhance completion rates.
3.6 Thesis examination

Only a small sub-sample had progressed to thesis examination. They were positive about the process, in particular its fairness. The main concern was the adequacy of the guidance and support for the viva voce examination.
3.7 Personal factors

Respondents agreed that partners, friends and family, and employers were supportive of their programmes, although they did not always understand the demands placed upon respondents by the research degree programme. The main problem area was the ability to support their study from personal nances.
20
3.8 Completion on schedule

Two-thirds of the respondents expected to complete their programme more or less on schedule. There was less condence among physical sciences, and engineering and computer sciences students than other disciplines. Funding apparently does not impact on condence to complete to schedule.
3.9 QAA Code of Practice

Career and personal development planning appears to be an aspect of the postgraduate research experience that is poorly addressed, as around a third of respondents indicated any encouragement to reect on career opportunities and development needs. Furthermore, fewer than half the respondents were aware of the institutional responsibilities towards students or indicated that their institution valued and responded to feedback from research degree students.
21
4. Supervision
Supervision was an area of major concern for postgraduate research students. As noted above, it was the area considered most important: 95.3% of students rated supervision as important (a score of four or ve on question 9, see Appendix 1). Overall, 77.3% responded that their experience of supervisory support and guidance had met with or exceeded their expectations, although over a fth (22.6%) felt that it had failed to meet their expectations (Table 2). Most (82.2%) respondents agreed that their supervisors have the necessary skills and subject knowledge to adequately support their research (8.4% disagreed). The statement with the lowest endorsement was I have received good guidance in my literature search from my supervisors (62.1% agreed; 19.5% disagreed) (Table 3). Although scores were consistently high across the supervision items, there was a tendency for students from arts and humanities, and to a lesser extent social sciences, to rate supervision more highly, and students from biomedical and veterinary sciences to rate supervision less highly. The greatest disciplinary variation was for my supervisors make a real effort to understand any difculties I face (Table 4). Multilevel modelling (Appendix 4) suggests that arts and humanities students were, overall, more positive about supervision than social sciences, whereas the science and engineering disciplines were less positive. Multiple regression analysis shows that supervision was the main factor inuencing the overall evaluation of the research programme: the rating (from 3 to +3) of the extent to which the overall experience of the research programme has met expectations (see Appendix 3 for details). Multilevel modelling (Appendix 4) suggests that demographic variables affect perceptions of supervision. Males were more positive than females. Black, Asian and Chinese ethnicity students were more positive than White respondents. The youngest and oldest age groups were more positive than the reference group (2630 years). Distance learners were more positive about supervision than face-to-face respondents. Compared to respondents who had undertaken postgraduate studies, those who came from undergraduate study or those who worked in a nonresearch job were less positive about supervision. Students writing up the thesis or amending the thesis after the viva voce examination were less positive about supervision than those planning or doing their research.
22
Different forms of funding appear to have no bearing on perceptions of supervision. There was little difference in perception of supervision by registered status, respondents with MPhil with transfer status were more positive than PhD registered respondents. An issue for concern is that students were less positive about supervision after the rst year. Perhaps this suggests that a lot of effort goes into registering students and initiating the research. Table 3. Supervision items
Item My supervisor/s have the skills and necessary subject knowledge to adequately support my research (N=10502) My supervisor/s make a real effort to understand any difculties I face (N=10490) I have been given good guidance in topic selection and renement by my supervisor/s (N=10456) I have received good guidance in my literature search from my supervisor/s (N=10449) My supervisor/s provide helpful feedback on my progress (N=10434) My supervisor/s are available when I need them (N=10445) Mean 4.27 % Disagree % Neutral % Agree 8.4 9.4 82.2
4.00
12.9
14.1
73.0
3.83
15.3
16.3
68.4
3.66
18.4
19.5
62.1
3.90
14.2
15.0
70.8
3.92
13.9
14.4
71.7
23
Table 4. Supervision items by discipline

My supervisor/s make a real effort to understand any difculties I face My supervisor/s are available when I need them 3.84 3.71 3.61 3.53 3.43 3.56 3.60 3.87 3.92 My supervisor/s have the skills and necessary subject knowledge to adequately support my research I have been given good guidance in topic selection and renement by my supervisor/s I have received good guidance in my literature search from my supervisor/s 3.98 3.97 3.81 3.90 3.82 3.85 3.95 4.07 3.66 My supervisor/s provide helpful feedback on my progress 3.96 3.88 3.77 3.73 3.76 3.73 3.86 3.87 3.90
Arts and humanities Social sciences Physical sciences Biological sciences Biomedical and veterinary sciences Engineering and computer sciences Medicine, dentistry and other healthrelated disciplines Other Mean
4.30 4.26 4.40 4.30 4.26
4.12 4.04 3.93 3.87 3.82
4.06 4.00 3.71 3.74 3.67
4.16
3.97
3.82
4.25
4.04
3.95
4.67 4.27
4.20 4.00
4.07 3.83
24
5. Skills development
This chapter examines the items making up the skills development scale, as well as items from the separate section in the survey on teaching opportunities, as this can be viewed as contributing to research degree students development of skills.Two stand-alone items, developed as indicators of the skills training precept from the QAA Code of Practice, are also summarised. Overall, skills development was rated reasonably highly (Table 2). The development of transferable skills was less positively rated than the development of research skills. In all, 83.6% of respondents indicated that their experience had met or exceeded their expectations to develop a range of research skills (16.5% said it had failed to meet their expectations) (Table 2), and 81.9% indicated that opportunities to develop a range of transferable skills had met or exceeded their expectations (18.1% that their expectations had not been met). Similarly, 88.1% indicated that developing research skills was important, whereas 67.6% rated development of transferable skills as important (Table 2). Improvement of independent learning and analytical skills received most endorsement: 80.1% of respondents agreed that as a result of my experience so far I have improved my ability to learn independently (6.6% disagreed), and 74.2% agreed that my experience so far has improved my analytical skills. They were least positive about improving communication and having opportunities to develop transferable skills: 63.8% agreed that my experience so far has helped me to develop a range of communication skills (11.9% disagreed), and only 57.9% agreed there are adequate opportunities available for me to further develop my transferable skills (14.3% disagreed) (Table 5). There were few disciplinary differences for the skills items (Table 6); however, there were higher levels of agreement by respondents from the sciences and medicine than from social sciences, and arts and humanities that there are adequate opportunities to further develop transferable skills, and from medicine and related areas, biomedical and biological sciences about their development of a range of communication skills. The multilevel modelling (Appendix 4) suggests that respondents in biological sciences were the only group signicantly more positive about skills development overall than social science students. Experience of teaching opportunities appear varied; respondents were most positive in their perception of whether experience gained through teaching has been worthwhile (61.1% agreed that it had; 22.4% disagreed). They were
25
least positive in their perceptions of whether they had received adequate support and guidance in their teaching (40.4% agreed that they had; 32.5% disagreed) (Table 5). Table 5. Skills development and related items
Item As a result of my experience so far I feel condent about managing a research project (N=10456) My experience so far has improved my analytical skills (N=10459) My experience so far has helped me to develop a range of communication skills (N=10424) As a result of my experience so far I have improved my ability to learn independently (N=10420) There are adequate opportunities for me to further develop my research skills (N=10438) (QAA Precept: skills training) There are adequate opportunities available for me to further develop my transferable skills (N=10364) (QAA precept: skills training) I have been given adequate support and guidance for my teaching (N=7381) I think that the experience that I have gained through teaching has been worthwhile (N=7097) Mean 3.75 % Disagree 12.1 %Neutral % Agree 21.5 66.5
3.92
8.6
17.1
74.2
3.71
11.9
24.3
63.8
4.08
6.6
13.3
80.1
3.62
14.6
24.6
60.8
3.59
14.3
27.8
57.9
3.05
32.5
27.0
40.4
3.63
22.4
16.5
61.1
26
Multilevel modelling (appendix 4) suggests that perceptions of skills development did not vary by age or gender. Black students were more positive than White respondents (other groups do not vary signicantly). EU students were more positive than UK-based respondents about skills development. Students with a declared disability were less positive about skills development than students with no declared disability. Compared to respondents who had undertaken postgraduate studies, those who took a gap year or worked in a non-research job were less positive about skills development. Students writing up the thesis, submitting the thesis or awaiting an award were more positive about goals and standards than those planning or doing their research. All those with some form of funding (except funding from overseas and from UK industry) were more positive about skill development than self-funders. MPhil and masters students were less positive about skills development than PhD registered respondents.
27
Table 6. Skills development and related items by discipline

As a result of my experience so far I have improved my ability to learn independently 4.08 4.08 4.03 4.15 4.05 4.11 4.10 3.73 4.08 There are adequate opportunities available for me to further develop my research skills My experience so far has helped me to develop a range of communication skills 3.65 3.67 3.68 3.81 3.84 3.69 3.81 3.93 3.71 My experience so far has improved my analytical skills 3.90 3.89 3.92 4.02 3.93 3.86 4.03 4.00 3.92 As a result of my experience so far I feel condent about managing a research project 3.78 3.78 3.66 3.77 3.69 3.71 3.76 3.67 3.75
3.61 3.57 3.71 3.69
3.44 3.48 3.74 3.79
3.62
3.78
3.57
3.60
3.70
3.67
3.40 3.62
3.47 3.59
There are adequate opportunities available for me to further develop my transferable skills
28
6. Infrastructure
Infrastructure was, overall, a positively rated area, although over a fth of respondents (21.8%) indicated that their research experience had failed to meet expectations. This was also seen as an important aspect for potential successful completion (Table 2). This was rated as an important area; 89.1% rated it as important, higher than the percentage who rated research and transferable skills, intellectual climate and goals and standards as important. However, the multiple regression analysis suggests that infrastructure was not a major inuence on perceptions of overall experience of the research degree programme (Appendix 3). Respondents were most positive about having access to necessary equipment (67.8% agreed they had; 13.7% disagreed). Similar proportions agreed that there is adequate provision of computing resources and facilities (66.0%) and library facilities (66.3%). Fewer than a half, though, agreed that there is appropriate nancial support for research activities (44.6% agreed they did; 31.5% disagreed) (Table 7). Students from arts and humanities, and social sciences disciplines were less positive about infrastructure than students from other disciplines, in particular, physical sciences and biological sciences (Table 8). (This is also reected in the multilevel modelling in Appendix 4.) Table 7. Infrastructure items
Item I have had adequate access to the equipment necessary for my research (N=10430) I have a suitable working space (N=10392) There is appropriate nancial support for research activities (N=10404) There is adequate provision of computing resources and facilities (N=10398) There is adequate provision of library facilities (N=10445) I have the technical support I need (N=10377) Mean 3.80 % Disagree % Neutral 13.7 18.4 % Agree 67.8
3.70 3.16
19.0 31.5
16.8 23.8
64.1 44.6
3.73
16.4
17.7
66.0
3.73 3.60
15.5 16.2
18.2 24.7
66.3 59.1
29
Table 8. Infrastructure items by discipline I have adequate access to the equipment necessary for my research I have a suitable working space There is adequate provision of computing resources and facilities
There is appropriate nancial support for research activities
3.72 3.73 3.86 3.91 3.87
3.29 3.56 3.96 3.95 4.00
2.74 2.84 3.64 3.58 3.53
3.52 3.67 3.88 3.74 3.85
3.58 3.70 3.87 3.75 3.80
3.47 3.54 3.74 3.70 3.70
3.82
3.97
3.39
3.94
3.84
3.68
3.90
3.87
3.30
3.86
3.88
3.65
3.73 3.80
3.87 3.70
3.20 3.16
4.07 3.73
3.60 3.73
3.80 3.60
Multilevel modelling suggests that perceptions of infrastructure provision did have some small variation by institution, as well as by a range of demographic variables (see Appendix 4 for details). Males were more positive than females. The youngest student age cohort (25) was more positive about infrastructure than the comparison group (2630), and those in the age range 3135 were less positive. Black students were more
I have the technical support I need
There is adequate provision of library facilities
30
positive than White students (whereas Asian, Chinese and mixed ethnicity students did not vary signicantly from the White respondents). Non-EU students were less positive than UK-based respondents; part-timers less positive than full-timers; distance learners less positive than face-to-face learners; and respondents with a declared disability were less positive about infrastructure than students with no declared disability. Compared to respondents who had undertaken postgraduate studies, those who had come from undergraduate study, those who had worked as a researcher and those who worked in the same organisation they currently worked in, were all positive about infrastructure. Taking a gap year and having worked as a non-researcher did not affect perceptions of infrastructure, compared to those who had previously undertaken postgraduate study. Students at the end of the process writing up the thesis or amending the thesis following the viva tended to be more negative about infrastructure compared to those planning or doing their research. All respondents with some form of funding were more positive about infrastructure than those who were self-funded. MPhil students were less positive about infrastructure than PhD registered students. Students in the second year were less positive about infrastructure than rst-year students, which perhaps reveals shortcomings once the study has progressed beyond the initial registration and start-up period.
31
7. Intellectual climate
Intellectual climate was a poorly rated area. Overall, a quarter of students (25.9%) indicated that the intellectual climate had failed to meet their expectations (Table 2). It was regarded as important in successful completion for 83.7% of respondents, and the multiple regression analysis suggested that it was second to supervision as the main inuence on perception of overall experience of the research degree programme (Appendix 3). The aspect that had most endorsement from respondents was whether their department provides a good seminar programme for research degree students (57.2% agreed; 19.5% disagreed). Barely half the respondents agreed that the research ambience in my department or faculty stimulates my work (49.3%). Least agreement was on whether respondents feel integrated into their departments community (49.0% agreed; 26.7% disagreed) (Table 9). Students from arts and humanities, social sciences and to a lesser degree engineering were, on average, less likely, than other disciplines, to agree that they felt integrated into their departments community and that the research ambience in their department or faculty stimulates their work. Social scientists were also least positive about their department providing a good seminar programme for research students (Table 10).This general pattern is conrmed by the multilevel modelling for intellectual climate as a whole (Appendix 4). Table 9. Intellectual climate items
Items My department provides opportunities for social contact with other research students (N=10403) My department provides opportunities for me to become involved in the broader research culture (N=10426) The research ambience in my department or faculty stimulates my work (N=10399) I feel integrated into my departments community (N=10372) My department provides a good seminar programme for research students (N=10390) Mean % Disagree % Neutral % Agree 3.43 23.2 23.4 53.4
3.40
22.7
25.3
52.0
3.33
24.4
26.3
49.3
3.31 3.54
26.7 19.5
24.3 23.2
49.0 57.2
32
Table 10. Intellectual climate items by discipline

The research ambience in my department or faculty stimulates my work My department provides opportunities for me to become involved in the broader research culture My department provides a good seminar programme for research students 3.52 3.40 3.66 3.76 3.60 3.47 3.58 3.40 3.54 My department provides opportunities for social contact with other research students
3.41 3.41 3.53 3.50 3.35
3.42 3.34 3.50 3.45 3.41
3.24 3.19 3.50 3.48 3.47
3.14 3.15 3.49 3.51 3.63
3.34
3.32
3.38
3.35
3.56
3.59
3.44
3.49
3.60 3.43
3.60 3.40
3.40 3.33
3.67 3.31
Multilevel modelling suggests that perceptions of intellectual climate provision varied to a small extent by institution as well as by a range of demographic variables (Appendix 4). The youngest student age cohort (25) was more positive about intellectual climate than the comparison group (2630). Mixed ethnicity students were less positive than White students (whereas Asian, Chinese and Black ethnicity students did not vary signicantly from the White respondents). Non-EU students were less positive than UK-based respondents; part-timers less positive than full-timers; distance learners less positive than face-to-face learners; and
I feel integrated into my departments community
33
respondents with a declared disability were less positive about intellectual climate than students with no declared disability. Compared to respondents who had undertaken postgraduate studies, those who had come from undergraduate study, those who the previous year worked in same organisation they currently work in, were all positive about intellectual climate. Some students at the end of the process writing up the thesis or amending the thesis following the viva tended to be more negative about intellectual climate compared to those planning or doing their research. All respondents with some form of funding were more positive about intellectual climate than those who were self-funded, except for students with funding from overseas. MPhil students were less positive about intellectual climate than PhD registered students. Given the importance of research degree students perceptions of intellectual climate, the role this appears to play in overall satisfaction and the reservations that there appear to be about intellectual climate, identifying which groups of students were more and less positive about intellectual climate may provide clues for improvement. As noted, younger students, those from the sciences and with some form of funding were more positive. Less positive were respondents of mixed ethnicity, non-EU, those with a declared disability, and part-time and distance learners, as well as those registered as MPhil students. This suggests intellectual climate is a function of regular time spent on-site working at a high level. The concern of non-EU students might be the result of unfullled expectations or some other overseas benchmarks.
34
8. Goals and standards

The goals and standards scale was rated reasonably highly (mean of 3.8). Overall, 73.1% of respondents indicated that their expectations in respect of goals and standards had been met or exceeded. However, 26.9% reported that their expectations had failed to be met, making goals and standards the aspect most likely rated as having failed to meet their expectations (Table 2). Given that this, inter alia, relates to examination requirements, this is not an encouraging nding. Goals and standards received the second lowest rating for importance, with 74.2% rating this as important for successful completion, compared, for example, with 95.3% for supervision and 89.1% for infrastructure (Table 2). Three-quarters of the respondents (75.8%) agreed that they understand the standard of work expected (8.0% disagreed); almost as many (69.2%) agreed that they understand the required standard for the thesis. Rather fewer agreed that they understand the requirements of thesis examination (61.4%), while one in seven (14.7%) disagreed (Table 11). There is little disciplinary variation: physical science students tended to rate the three goals and standards items less positively, and arts and humanities students more positively (Table 12) (See also Appendix 4). Table 11. Goals and standards items
Items I understand the required standard for the thesis (N=10471) I understand the standard of work expected (N=10443) I understand the requirements of thesis examination (N=10440) Mean 3.81 SD 1.004 % Disagree % Neutral 11.4 19.4 % Agree 69.2
3.95
0.936
8.0
16.2
75.8
3.65
1.057
14.7
23.9
61.4
Multilevel modelling suggests that perceptions of goals and standards varied by demographic variables (see Appendix 4 for details). The youngest student
35
age cohort (25) were less positive about goals and standards than the comparison group (2630), and those in most of the older groups were more positive. All groups of non-White students (except mixed ethnicity students) were more positive than White respondents. EU and non-EU students were more positive than UK-based respondents. Students with a declared disability were less positive about goals and standards than students with no declared disability. Compared to respondents who had undertaken postgraduate studies, those who had come from undergraduate study, took a gap year or worked in a non-research job were less positive about goals and standards. Students writing up the thesis or submitting the thesis were more positive about goals and standards than those planning or doing their research. Funding sources had no bearing on views about goals and standards. Students in the third year and beyond were more positive about goals and standards than rst-year students, suggesting more of a focus on the outcomes of the research process. Table 12. Goals and standards items by discipline
I understand the required standard for the thesis Arts and humanities 3.94 Social sciences 3.84 Physical sciences 3.71 Biological sciences 3.75 Biomedical and 3.72 veterinary sciences Engineering and 3.79 computer sciences Medicine, dentistry 3.76 and other healthrelated Other 3.60 I understand the standard of work expected 3.97 3.91 3.92 4.03 3.98 3.92 3.99 I understand the requirements of thesis examination 3.68 3.64 3.62 3.66 3.59 3.68 3.67
3.87
3.47
36
9. Thesis examination
Only 408 of the 10544 respondents had sat their thesis examination. Of those who had, perceptions of the examination process were positive: the thesis examination scale rated second only to personal factors. Research degree students were most likely to agree about the fairness of the thesis examination process (82.2% agreed; 8.6% disagreed) and least inclined to agree that they were given adequate guidance and support in preparation for their viva (61.8% agreed; 23.7% disagreed). That a quarter of respondents felt guidance for the viva voce examination was inadequate is a cause for concern (Table 13). The most notable disciplinary differences were less positive ratings for biomedical and veterinary sciences students, and more positive ratings for engineering and computer science students. However, it should be borne in mind that the sample sizes when broken down by discipline are small; for example, there were only 14 biomedical and veterinary science students in the sample who had completed their thesis examination (Table 14). The multilevel modelling (Appendix 4) only shows that, overall for thesis examination, engineering respondents were more positive than the reference group of social science students. Table 13. Thesis examination items
Item The thesis examination process was fair (N=443) The examination of my thesis was completed in a reasonable time scale (N=445) I was given adequate support and guidance in preparation for my viva voce (N=447) I was given adequate support and guidance to make any changes to my thesis following my viva voce (N=414) Mean 4.30 4.13 % Disagree % Neutral % Agree 8.6 9.3 82.2 13.7 9.2 77.1
3.62
23.7
14.5
61.8
3.84
18.4
10.9
70.7
The multilevel modelling suggests that, for the sub-sample to whom this applied, age, gender, ethnicity and disability had no bearing on the perception
37
of the thesis examination process. Nor did it matter whether the respondent was part- or full-time; a face-to-face or distance learner. Those who in the previous year worked in same organisation they currently work in were more positive than students who had come from postgraduate study. Students with funding from the institution or from EU/EC sources were more positive about thesis examination than self-funders. Those registered on a professional doctorate were more positive about thesis examination than PhD registered respondents. Table 14. Thesis examination items by discipline
I was given adequate support and guidance in preparation for my viva voce 3.55 3.63 3.50 3.53 3.20 3.80 4.07 3.62 3.62 I was given adequate support and guidance to make any changes to my thesis following my viva voce 3.77 3.55 3.71 4.06 3.29 4.22 4.22 3.84 3.84 The examination of my thesis was completed in a reasonable time scale 4.01 3.96 4.11 4.34 3.67 4.38 4.33 4.13 4.13
Arts and humanities (N=73) Social sciences (N=108) Physical sciences (N=49) Biological sciences (N=65) Biomedical and veterinary sciences (N=14) Engineering and computing (N=67) Medicine, dentistry and other health-related (N=27) Other Mean
4.08 4.26 4.36 4.40 4.07
4.57 4.37
4.31 4.30
The thesis examination process was fair
38
10. Personal factors

Responses to personal factor items were particularly positive: respondents agreed that partners (89.4%) friends and family (87.5%), and employers (where applicable) (71.9%) were supportive of their programmes. However, partners, and friends and family, while being supportive were somewhat less understanding of the demands placed upon respondents by the research degree programme. The main area of disagreement related to nances: 28.1% disagreed that their personal nances are supportive of their research degree programme (48.5% agreement) (Table 15). Overall, personal factors was the area with the highest mean agreement with the items (4.09) and least variation (SD=0.783), although this set of items was not directly addressing institutional provision. It is for this reason that further analyses, including breakdown by discipline, are not performed on these items (Table 2). Table 15. Personal factors
Items My friends and family are supportive of my research degree programme (N=10243) My partner is supportive of my research degree programme (N=7429) My friends and family are understanding of any demands placed upon me by my research degree programme (N=10162) My partner is understanding of any demands placed upon me by my research degree programme (N=7400) My employer is supportive of my research degree programme (N=4324) My personal nances are supportive of my research degree programme (N=9193) Mean % Disagree % Neutral % Agree 4.45 4.6 8.0 87.5
4.53
4.8
5.8
89.4
4.04
10.1
15.6
74.3
4.30
7.2
10.4
82.4
3.97
12.8
15.3
71.9
3.32
28.1
23.4
48.5
39
11. Condence for completing on schedule

Two-thirds of the research degree students (65.2%) agreed that they expected to complete their programme more or less on schedule (Table 16). There were minor variations between disciplines. Students from arts and humanities, biomedical and veterinary sciences, and medicine, dentistry and other related disciplines were more condent. Students from physical sciences and from engineering and computer sciences were less condent (Table 17). Table 16. Condence for completing on schedule Mean SD % Disagree % Neutral % Agree 3.74 1.161 15.2 19.6 65.2 Table 17. Condence for completing on schedule items by discipline Condence for completing on schedule Arts and humanities 3.85 Social sciences 3.75 Physical sciences 3.61 Biological sciences 3.71 Biomedical and veterinary sciences 3.83 Engineering and computer sciences 3.58 Medicine, dentistry and other 3.83 health-related Other 3.54 Mean 3.74 Multilevel modelling suggests that condence in completion was unaffected by gender and ethnicity (see Appendix 4). Some age cohorts were more positive than others, but with no discernible pattern. NonEU students were more condent than home students about completion, as were students with a declared disability. Mode of study did not affect condence in completion although those coming from undergraduate study were less condent than respondents who had undertaken postgraduate studies.
40
Compared to social science students who are the reference group, arts and humanities respondents were more positive about completion, and physical sciences and engineering less positive. Perhaps surprisingly, funding apparently did not impact on condence to complete (other than those with multiple sources of funding, who were less positive about completion). Students on professional doctorates, undertaking PhD by publication and registered as masters in research were more positive than PhD registered students about completing in the planned timescale.
41
12. QAA Code of Practice indicators

The following sets of tables summarise the ndings for the items developed as indicators of the QAA Code of Practice; they are presented grouped together in their respective precepts. Breakdowns of the items by discipline are not provided as the items are considered more indicative of institutional arrangements. Career and personal development planning (precepts 18-20) appears to be an aspect of the postgraduate research experience that is poorly addressed. Only 37.6% of respondents agreed that I am encouraged to think about the range of career opportunities that are available to me and only 38.3% agreed that I am encouraged to reect on my career development needs. Only slightly better was encouragement to reect on professional development needs (47.5% agreed) (Table 18). Progress and review arrangements (precepts 15-17) were most positively rated; 73.9% agreed that they understand the requirements and deadlines for formal monitoring of my progress. However, clarity regarding institutions responsibilities towards the students (precepts 6-10) was less positive: only 47.3% of respondents agreed that I am aware of my institutions responsibilities towards me as a research degree student. This is also reected in the low percentage (47.8%) of respondents who agreed that my institution values and responds to feedback from research degree students (precept 21) (Table 18).
42
Table 18. Code of Practice items

Items (and precept/s from QAA) Career and personal development planning (18-20) I am encouraged to think about the range of career opportunities that are available to me I am encouraged to reect on my professional development needs I am encouraged to reect on my career development needs Student representations, complaints and appeals (2527) I know who to approach, or where to nd this out, if I am dissatised with any element of my research degree programme Feedback mechanisms (21) My institution values and responds to feedback from research degree students Progress and review arrangements (1517) I understand the requirements and deadlines for formal monitoring of my progress Selection, admission and induction of students (6-10) I understand my responsibilities as a research degree student I am aware of my institutions responsibilities towards me as a research degree student Mean % disagree % neutral % agree
3.10
29.1
33.3
37.6
3.33
22.9
29.6
47.5
3.11
29.3
32.5
38.3
3.56
19.7
21.4
58.9
3.39
17.9
34.2
47.8
3.92
10.5
15.7
73.9
4.01
7.5
14.9
77.6
3.32
23.3
29.5
47.3
43
13. Process review

One of the key design principles (Table 1) that underpinned the development of PRES was that the survey should be useful to HEIs and national bodies, and opportunities have been taken to collect feedback from users in various ways to inform improvement of the questionnaire and the ways in which the Academy supports institutions in using PRES. The pilot survey provided invaluable feedback, as a result of which the questionnaire was improved. Some questions were slightly reworded; the skills development scale was revised; new sections were added (teaching opportunities and personal factors); section three had a not applicable option added; questions about students satisfaction with aspects of their research degree programme were removed; and a new question was added that allows institutions to categorise students responses according to their own departmental or school structure. Additional useful feedback was collected at a meeting with PRES ofcers from pilot institutions in October 2006. They broadly liked the questionnaire and methodology, and valued the Academy support. Only two concerns were raised: ownership of the data and the amount of workload needed to analyse PRES data. The collaborative agreement and code of practice helped resolve the rst problem. A survey of PRES ofcers from the institutions that used PRES in 2007 was conducted in June 2007, seeking feedback. Forty-seven institutions (80%) replied, and the feedback was extremely positive. Some areas for improvement were suggested. These included: having an alternative paper questionnaire for use by students without access to the internet; having more exible dates for the survey, to better t in with institutional processes; and providing a longer period of notice before the survey goes live so that institutions have more time to prepare. Not all of the institutions that took part in PRES in 2007 plan to run the survey in 2008, either because of concerns about survey fatigue among students, or because they expect it to take longer than a year for major changes in institutional policies or practices to yield tangible results. Conversely, there are several institutions that did not take part in PRES in 2007 who are planning to do so in 2008. The PRES Steering Group have agreed that PRES will run again in 2008 at the same time as in 2007, and the Academy has agreed to provide support at a similar level and in similar ways as it did to the 2007 survey.
44
Appendix 1: The 2007 PRES questionnaire

Data Protection
All data collected in this survey will be held anonymously and securely. Each institution will be able to see its own survey results and an overall report will be available to the Higher Education Academy. All results will be presented in an aggregated and anonymised form. Demographic data collected at the end of the survey will only be used for the purposes of this survey and will not be used to identify any individuals.
Section 1
1. For each statement, please rate the extent of your agreement or disagreement. (1 = Strongly Disagree and 5 = Strongly Agree) 1 a. My supervisor/s have the skills and subject knowledge to adequately support my research My supervisor/s make a real effort to understand any difculties I face My institution values and responds to feedback from research degree students I am encouraged to think about the range of career opportunities that are available to me. I understand the required standard for the thesis I have adequate access to the equipment necessary for my research I have been given good guidance in topic selection and renement by my supervisor/s 2 3 4 5
b. c. d.
e. f. g.
45
h. i. j.
I have a suitable working space I am encouraged to reect on my professional development needs My department provides opportunities for social contact with other research students My department provides opportunities for me to become involved in the broader research culture I know who to approach, or where to nd this out, if I am dissatised with any element of my research degree programme There are adequate opportunities available for me to further develop my research skills I have received good guidance in my literature search from my supervisor/s I understand my responsibilities as a research degree student There are adequate opportunities available for me to further develop my transferable skills As a result of my experience so far I feel condent about managing a research project
k.
l.
m.
n. o. p.
q.
46
Section 2
2. For each statement, please rate the extent of your agreement or disagreement. (1 = Strongly Disagree and 5 = Strongly Agree) 1 a. b. There is appropriate nancial support for research activities The research ambience in my department or faculty stimulates my work My experience so far has improved my analytical skills I understand the standard of work expected There is adequate provision of computing resources and facilities I understand the requirements and deadlines for formal monitoring of my progress I understand the requirements of thesis examination My experience so far has helped me to develop a range of communication skills There is adequate provision of library facilities My supervisor/s provide helpful feedback on my progress I am encouraged to reect on my career development needs As a result of my experience so far I have improved my ability to learn independently My supervisor/s are available when I need them 2 3 4 5
c. d. e. f.
g. h.
i. j. k. l.
m.
47
n. o. p.
I have the technical support I need I feel integrated into my departments community My department provides a good seminar programme for research students I am aware of my institutions responsibilities towards me as a research degree student
q.
Section 3
3. Have you sat your nal viva examination? No (If No, please go to section 4) Yes (If Yes, please respond to the following statements:) For each of the following, show the extent of your agreement or disagreement. (1 = Strongly Disagree and 5 = Strongly Agree) 1 The thesis examination process was fair The examination of my thesis was completed in a reasonable time scale I was given adequate support and guidance in preparation for my viva voce I was given adequate support and guidance to make any changes to my thesis following my viva voce 2 3 4 5 NA
48
Section 4: Teaching opportunities

4. I have had adequate opportunity to gain experience of teaching [e.g., lectures, seminars or workshops] whilst doing my research degree programme (1 = Strongly Disagree and 5 = Strongly Agree) 1 5. 2 3 4 5
I have been given adequate support and guidance for my teaching (1 = Strongly Disagree and 5 = Strongly Agree) 1 2 3 4 5 Not Applicable
6.
I think the experience that I have gained through teaching has been a worthwhile aspect of my research degree programme (1 = Strongly Disagree and 5 = Strongly Agree) 1 2 3 4 5 Not Applicable
7.
Please provide further information regarding your teaching experience
............................................................................................................................................ ............................................................................................................................................ ............................................................................................................................................
Section 5: Personal factors

8. Please state to what extent you agree with the following statements (1 = Strongly Disagree and 5 = Strongly Agree) 1 a. My friends and family are supportive of my research degree programme My partner is supportive of my research degree programme My friends and family are understanding of any demands placed upon me by my research degree programme 2 3 4 5 NA
b.
c.
49
d.
My partner is understanding of any demands placed upon me by my research degree programme My employer is supportive of my research degree programme My personal nances are supportive of my research degree programme
e.
f.
Section 6
9. For the following items, please rate how important, in terms of successfully completing your research degree programme, you consider them to be (1 = Not at all important and 5 = Very Important) 1 a. Supervisory support and guidance Opportunity to develop a range of research skills Opportunities to develop a range of transferable skills Access to appropriate facilities The research environment Provision of guidance on institutional standards and expectations for your research degree programme 2 3 4 5 Comment
b. c.
d. e. f.
50
10.
Please rate the following broad aspects of your research degree programme in terms of how your experience of them has met with your expectations (-3 = it is much more negative, 0 = it has met my expectations, +3 = it is much more positive) -3 -2 -1 0 1 2 3
a. b. c. d. e. f.
Supervisory support and guidance Opportunities to develop a range of research skills Opportunities to develop a range of transferable skills Access to appropriate facilities The research environment Provision of guidance on institutional standards and expectations for your research degree programme Overall experience of my research programme I am condent that I will complete my research degree programme more or less within the planned timescale teaching (1 = Strongly Disagree and 5 = Strongly Agree) 1 2 3 4 5
g.
11.
Section 7
12. Please provide further information about your experience of your research degree programme. For example, what would further improve your experience?
............................................................................................................................................ ............................................................................................................................................ ............................................................................................................................................
51
Demographics:
13. I am: 25 years old or younger 26-30 years old 31-35 years old 36-40 years old 41-45 years old 46-50 years old 51-55 years old 56 years old or older 14. I am: Male Female 15. Do you consider yourself to have a disability? Yes No 15. a If yes, please choose one from the following options: (as a drop down list) Dyslexia Blind/visually impaired Deaf/hard of hearing Wheelchair-user/mobility impairments Mental health difculties Autistic spectrum disorder Unseen disability Multiple disabilities Other (If other, please specify..) 16. For fees purposes, is your normal place of residence registered as: Home Other EU Non-EU 17. I class myself as: White: British/Irish/Any other white background Mixed: White and Black Caribbean/White and Black/White
52
and Asian/Any other mixed background Asian or Asian British: Indian/Pakistani/Bangladeshi/Any other West or South Asian background Black or Black British: Caribbean/African/Any other Black background Chinese: Chinese/Any other East Asian background Other (Please specify) 18. I am currently registered as studying: Full-time Part-time 19. I am: Primarily a face-to-face learner [e.g., based at my institution] Primarily a distance learner 20. What year of your research degree programme are you in? Drop down list from 1 to 9 years Other. 21. In the year before starting my research degree programme I: Completed my undergraduate studies Completed my postgraduate studies [for example, MSc, MA] Took a gap year Worked in the same organisation that I currently work in Worked as a researcher Worked in a non-research role Other (please specify..) 22. I currently: am planning or doing my research am writing up my thesis have submitted my thesis and I am awaiting my viva am making amendments to my thesis following my viva am awaiting my doctoral award following my viva Other 23. My discipline can be broadly categorised as: Arts and Humanities [including performing arts, languages, and education]
53
Social Sciences [including law, economics, accounting, business and management, and psychology] Physical Sciences [including mathematics and earth sciences] Biological Sciences Biomedical and Veterinary Sciences Engineering and Computer sciences Medicine, Dentistry and other health related disciplines 24. (Here institutions may insert their own question about faculties/ departments. This question will not be suitable for cross-institutional comparisons.) You are (select all that apply): Self-funded Research Council funded Charity Institution funded UK industry funded UK Government funded EU/EC funded Funded overseas Other 26. I am registered as doing a: PhD Professional doctorate PhD by published work New Route PhD MPhil with transfer to PhD MPhil Master in research Other
25.
54
Appendix 2: Demographic prole of respondents

PRES data is compared below with HESA data for the period 200405. Some groupings used in PRES are different to those used by HESA, and in such cases the HESA data is aggregated or disaggregated where possible, to provide more comparable groupings. For those PRES demographics where there is no comparable HESA data, the PRES data is summarised. Missing data and use of different groupings in the two datasets result in some percentages not adding up to 100 percent. Table A2.1 Prole of respondents by gender Gender Male PRES HESA Female PRES HESA Modal group: PRES (Female), HESA (Male) Table A2.2 Prole of respondents by disability Disability Yes PRES HESA No PRES HESA Modal group: PRES and HESA (no declared disability) Table A2.3 Prole of respondents by domicile Domicile Home PRES HESA Other EU PRES HESA Non-EU PRES HESA
% 45.7 54.7 54.3 45.3
% 5.1 5.8 93.8 94.2
% 60.5 59.4 13.8 12.3 25.3 28.3
55
Modal group: PRES and HESA (Home) Table A2.4 Prole of respondents by ethnicity Ethnicity White PRES HESA Mixed PRES HESA Asian PRES HESA Black PRES HESA Chinese PRES HESA Other PRES HESA Modal group: PRES and HESA (White) Table A2.5 Prole of respondents by mode of study Mode of study % Full-time PRES 79.3 HESA 51.2 Part-time PRES 20.7 HESA 48.1 Modal group: PRES and HESA (Full-time), although HESA suggests mode is more equally split between full and part-time than PRES sample
% 74.4 87.87 2.3 (2.9) 6.7 5.4 2.2 2.2 7.8 1.5 6.6 2.9
56
Table A2.6 Prole of respondents by discipline Discipline PRES (Arts and humanities, including performing arts, languages and education) HESA (Languages, historical and philosophical studies, creative arts and design and education) PRES (Social sciences, including law, economics, accounting, business and management and psychology) HESA (Social studies, law, business and admin, mass communications) PRES (Physical sciences, including mathematics and earth sciences) HESA (Physical sciences, mathematical sciences) PRES (Biological sciences) HESA (Biological sciences) PRES (Biomedical and veterinary sciences) HESA (Veterinary sciences) PRES (Engineering and computer sciences) HESA (Computer science and engineering and technology) PRES (Medicine, dentistry and other health related disciplines) HESA (Medicine and dentistry and subjects allied to medicine) PRES (Other) HESA (Agriculture and related subjects, architecture, building and planning, and combined)
% 21.3 22.7 27.0 17.2 12.0 13.3 13.8 11.9 4.5 0.5 14.4 17.7 6.9 13.9 0.1 2.8
Modal group: PRES (Social sciences, including law, economics, accounting, business and management and psychology); HESA (Social studies, law, business and admin, mass communications). Table A2.7 Prole of respondents by age Age 25 and younger 2630 3135 3640 4145 4650 5155 56 and older
% 28.0 32.3 14.4 8.4 5.9 4.1 3.0 3.1
57
Modal group: PRES (26-30) Table A2.8 Prole of respondents by mode of delivery Mode of delivery % Face-to-face 84.3 Distance learner 15.7 Modal group: PRES (face-to-face) Table A2.9 Prole of respondents by year of study Current year of study % 1 31.6 2 25.8 3 22.5 4 12.7 5 3.9 6 1.8 7 1.1 8 0.3 9+ 0.2 Modal group: PRES (1) Table A2.10 Prole of respondents by previous activity Previous activity % Completed undergraduate studies 19.8 Completed postgraduate studies 33.1 Gap year 4.3 Worked in the same organisation that I currently work in 8.8 Worked as a researcher 9.5 Worked in a non research role 16.3 Other 8.3 Modal group: PRES (completed postgraduate studies)
58
Table A2.11 Prole of respondents by current activity Current activity % Am planning or doing my research 68.2 Am writing up my thesis 22.7 Have submitted my thesis and am awaiting my viva 2.8 Am making amendments to my thesis following my viva 1.5 Am awaiting my doctoral award following my viva 1.8 Other Modal group: PRES (Am planning or doing my research) Table A2.12 Prole of respondents by source of funding Funding % Self-funded 23.3 Research council funded 21.2 Charity 2.5 Institution funded 16.3 UK industry funded 1.8 UK government funded 3.1 EU/EC funded 2.3 Funded overseas 6.7 Mixed 11.0 Other 11.9 Modal group: PRES (Self-funded) Table A2.13 Prole of respondents by degree registered for Registered for % PhD 76.0 Professional doctorate 2.0 PhD by published work 0.2 New route PhD 0.4 MPhil with transfer to PhD 13.6 MPhil 2.1 Master in research 2.7 Other 3.0 Modal group: PRES (PhD)
59
Appendix 3: Multiple regression analyses for students overall evaluation ratings and condence for completing on schedule
Multiple regression was used to examine the extent to which the scales combined account for students ratings of their condence for completing on schedule, and their overall evaluation of their research degree programme. Multiple regression was also used to identify which scales, independently, are the main factors inuencing condence for completion and overall evaluation. The analysis was intended to help identify possible areas for quality enhancement activities. The conventional multiple regression equation is: Y=a+b1X1+b2X2+b3X3bnXn Where: Y is the dependent variable and X1, X2, X3Xn are the independent variables that are expected to have some effect on Y. b1, b2, b3bn are the corresponding coefcients (or weightings) for the independent variables and indicate, in practice, which of the independent variables are most important. What this does, then, is provide an indication of how important each of the posited independent variables are in determining the dependent variable. In the rst analysis the independent variable is the respondents rating on the item Overall experience of my research programme. In the second analysis the independent variable is the respondents rating on the item I am condent that I will complete my research degree programme more or less within the planned timescale teaching. As the thesis examination scale only affected a subset of the respondents, the analysis was done twice in each case, once including thesis examination as an independent variable and once excluding it.
60
Overall evaluation ratings

For both analyses (with and without thesis examination), the scales combined account for 41% of the variance in research degree students overall evaluation, a statistically signicant amount and a medium effect size. Tables A3.1 and A3.2 below summarise the beta weights (b1, b2, b3) of the scales for the analysis. The higher the beta weight the more important the independent variable is as a factor in determining condence in completion (Y). Table A3.1. Beta weights and signicance of scales for dependent variable overall evaluation Thesis scale included Scale Supervision Intellectual climate Goals and standards Thesis examination Infrastructure Skills development Teaching Personal factors Beta 0.353 0.246 0.067 0.052 0.042 0.037 0.029 -0.094 Rank 1 2 3 4 5 6 7 8 Yes (0.0001) Yes (0.0001) No No No No No 0.059 0.162 0.011 Thesis scale excluded Rank 1 2 5 Signicant Yes (0.0001) Yes (0.0001) Yes (0.01) Signicant Beta 0.295 0.249 0.032
EXCLUDED 4 3 6 7 Yes (0.0001) Yes (0.0001) No No
Yes (0.05) -0.005
More of the scales in the analysis that excluded the thesis examination scale were found to signicantly predict overall evaluation. This potentially reects the larger sample size for this analysis, making it more powerful. Across both analyses, supervision and intellectual climate are statistically signicant predictors of research degree students overall evaluations, and have the highest beta weights. This suggests these two aspects of research degree students experience have the strongest bearing upon their overall evaluation of their research degree programme.
61
Condence for completing on schedule

This analysis excluded those respondents who had already sat their thesis examination as this would be expected to inuence their ratings of condence for completing on schedule. The scales combined account for only 19% of the variance in research degree students ratings of their condence for completing on schedule. The survey was not designed to specically explore factors inuencing condence for completion; therefore, the low variance accounted for likely reects this, with there being other more important factors that were not measured in the survey. Table A3.2 summarises the beta weights of the scales, with higher beta weights indicating scales that have a greater inuence on perceptions of condence for completing on schedule. Table A3.2. Beta weights and signicance of scales for independent variable condence in completing on time Scale Supervision Intellectual climate Skill development Infrastructure Goals and standards Teaching Personal factors Beta 0.295 0.249 0.162 0.059 0.032 0.011 -0.005 Signicant Yes (0.0001) Yes (0.0001) Yes (0.0001) Yes (0.0001) Yes (0.01) No No Rank 1 2 3 4 5 6 7
Supervision and intellectual climate again have the highest beta weights, suggesting these have the most bearing on students ratings of their condence for completing on schedule.
62
Appendix 4: Multilevel modelling: Scales broken down by demographic variables

Testing for demographic differences in responses is valuable in identifying areas to focus enhancement activities. The scales were rst examined to see whether they signicantly discriminate between institutions, after controlling for the demographic prole of students at the institutions. For example, if it was found that males respond more positively than females and a particular institution had a higher proportion of males in their sample, if not controlled for this would bias the results for that particular institution. The importance of examining whether a scale discriminates between institutions is that if a scale is found not to discriminate between institutions, any differences in scores on that scale can be assumed to be due to differences in students characteristics, not due to differences between the institutions. If, on the other hand, a scale is found to discriminate between institutions, differences in scores on that scale can be assumed to be due to differences between the institutions and between the students. For scales found to discriminate between institutions, multilevel modelling was used. This controlled for the inuence of the different student, course and institutional characteristics, as well as the clustering, within the data. Single level modelling (ANOVA) was used for the other scales. The intellectual climate and infrastructure scales differentiated between institutions after controlling for the demographic prole of students. However, the proportion of variance accounted for by differences between institutions was low in both cases (2.27% for intellectual climate and 1.76% for infrastructure). Eight institutions had scores on the intellectual climate scale that were signicantly above what would be expected given the prole of their students, three had signicantly lower scores. The rest had scores as expected, given the prole of their students. Four institutions had signicantly higher scores on the infrastructure scale and three were signicantly lower than expected.
63
The other scales did not reveal institutional differences, suggesting that differences in scores can be assumed to be due to differences in student characteristics, not due to differences between the institutions. Multilevel modelling uses reference groups against which to compare the responses of the different demographic groups. For example, when testing the effect of a student having a declared disability upon the scale scores, students with no declared disability are the reference group that their scores are compared against.This is fairly self-evident and unproblematic for dichotomous variables. It is more problematic for non-dichotomies because a reference group has to be selected in a somewhat articial manner. Nonetheless it provides a relatively straightforward way to make some comparisons. In the summary table below, the reference group for each demographic variable is presented in brackets. Each demographic variable has one reference group that it is compared against. Due to the large sample size, a more conservative statistical signicance level of 0.01 was used for the analyses.The results of the analyses for all of the scales are summarised in the tables. A plus (+) sign indicates that students scores in that demographic group were signicantly higher than the reference group, a minus () sign indicates that students scores in that demographic group were signicantly lower than the reference group, and ns indicates that they were not statistically signicant. NA indicates that the analysis was not run/was not considered feasible. So, taking gender as an example, males were signicantly more positive than females about infrastructure, indicated by the positive sign in that cell of the table. The results are presented separately for the different demographic variables for clarity.
64
Table A4.1. Multilevel modelling results for demographic age

Thesis (N=408) ns ns ns ns ns ns ns Thesis (N=408) ns ns ns ns Thesis (N=408) ns Age (reference group: 2630 years) Infrastructure Skills development
Supervision
25 3135 3640 4145 4650 5155 56
+ ns ns ns ns ns ns
+ ns ns ns ns ns
ns + + ns + +
ns ns ns ns ns ns ns
+ ns ns ns ns ns +
* I am condent that I will complete my research degree programme more or less within the planned timescale Table A4.2. Multilevel modelling results for demographic gender
Gender (reference group: female) Infrastructure Skills development Supervision Intellectual climate
Male
ns
ns
ns
Table A4.3. Multilevel modelling results for demographic ethnicity

Infrastructure Ethnicity (reference group: white) Skills development Supervision Intellectual climate
Asian Black Chinese Mixed
ns ns ns -
ns + ns ns
+ + + ns
ns + ns ns
+ + + ns
Condent* ns ns ns ns
Goals and standards
Condent* ns
Goals and standards
Condent* + ns ns ns + ns +
Goals and standards
Declared disability
Other EU Non-EU
Table A4.6. Multilevel modelling results for demographic mode of study

Disability (reference group: no declared disability)
Table A4.5. Multilevel modelling results for demographic disability
Table A4.4. Multilevel modelling results for demographic domicile
Part-time ns Intellectual climate ns Infrastructure + + Goals and standards + ns Skills development ns ns ns Supervision + ns + Condent* ns ns ns Thesis (N=408)
Mode (reference group: full-time)
Domicile (reference group: home)
Infrastructure
Infrastructure
ns
Goals and standards
Goals and standards Skills development Supervision
ns
Skills development
ns
Supervision
ns
Condent*
Condent*
ns
Thesis (N=408)
Thesis (N=408)
65
66
Table A4.7. Multilevel modelling results for demographic method of delivery

Goals and standards Delivery (reference group: face-to-face) Intellectual climate Skills development
Distance learner
ns
ns
ns
Table A4.8. Multilevel modelling results for demographic before starting my research degree
Before starting (reference group: postgraduate studies)
Goals and standards
Skills development
Undergraduate studies Took a gap year In the last year worked in same organisation I currently work in Worked as a researcher Worked in a nonresearch role
+ ns + ns ns
+ ns + + ns
ns ns -
nd ns ns -
ns ns ns -
ns ns ns ns
Thesis (N=408) ns ns + ns ns
Infrastructure
Supervision
Condent*
Thesis (N=408) ns
Infrastructure
Supervision
Condent*
67
Table A4.9. Multilevel modelling results for demographic, current activity

Current activity (reference group: planning or doing my research)
Goals and standards
Skills development
Writing up thesis Submitted thesis Amending thesis following viva Awaiting award
ns ns
ns ns
+ + ns ns
+ + ns +
na na
NA NA NA NA
NA NA NA NA
Table A4.10. Multilevel modelling results for demographic discipline

Discipline (reference group: social science) Goals and standards Intellectual climate Skills development
Arts and humanities Physical sciences Biological sciences Biomedical and veterinary Engineering and computing Medical, dentistry and other health related
ns + + + + +
+ + + + +
+ ns ns ns ns ns
ns ns + ns ns ns
+ ns
+ ns ns ns
Thesis (N=408) ns ns ns ns + ns
Infrastructure
Supervision
Condent*
Thesis (N=408)
Infrastructure
Supervision
Condent*
68
Table A4.11. Multilevel modelling results for demographic funding

Goals and standards Intellectual climate Funding (reference group: self-funded) Skills development
Research Charity Institution UK Industry UK Government EU/EC Overseas Mixed (more than one source of funding)
+ + + + + + ns +
+ + + + + + + +
ns ns ns ns ns ns ns ns
+ + + ns + + ns +
ns ns ns ns ns ns ns ns
ns ns ns ns ns ns ns -
Table A4.12. Multilevel modelling results for demographic, registered as

Thesis (N=408) + ns ns ns ns ns Infrastructure Skills development Registered as (reference group: PhD)
Supervision
Professional doctorate PhD by publication New route PhD MPhil with transfer status MPhil Masters in research
ns ns ns ns ns
ns ns ns ns ns
ns ns ns ns
ns ns ns -
ns ns ns + ns ns
Condent* + + ns ns ns +
Goals and standards
Thesis (N=408) ns ns + ns ns + ns ns
Infrastructure
Supervision
Condent*
69
Table A4.13. Multilevel modelling results for demographic, year of study

Intellectual climate Skills development Year of study (reference group: one)
2 3 4 5
NA NA NA NA
ns + + +
NA NA NA NA
NA NA NA NA
NA NA NA NA
There were few clear differences by demographic feature, but those who were more likely to rate certain aspects of their experience more positively include Black students (compared to White), and those with some form of funding (compared to self-funded). Those who were more likely to rate certain aspects of their experience less positively include: those registered as MPhil students (compared to PhD); those with a declared disability (compared to no declared disability); part-time and distance delivery students (compared to full-time and face-to-face students); and those who had come from a non-research background. Other ndings were more mixed across the demographic groups and scales. For example, non-EU students rated goals and standards, and condence for completing on time more positively than home students, and intellectual climate and infrastructure less positively than home students. Students from science-based disciplines typically rated intellectual climate and infrastructure more positively than those from the social science-based disciplines, yet tended to rate supervision less positively. Finally, those who had come straight from undergraduate studies rated intellectual climate and infrastructure more positively, and goals and standards, supervision and condence for completing on schedule less positively (compared to those who had come straight from previous postgraduate studies). Given the importance of research degree students perceptions of intellectual climate found in this study, identifying which groups of students were more and less positive about intellectual climate may be
Thesis (N=408)
Infrastructure
Supervision
Condent*
Goals and standards
70
valuable for further investigation. Those more positive were students aged 25 years and under (compared with those aged 2630 years), those from science based disciplines (compared to social sciences), and those with some form of funding (compared to self-funded). Those less positive were students of mixed ethnicity, those whose domicile is non-EU (compared to home), those with a declared disability (compared to no declared disability), part-time and distance learners (compared to full-time and face-to-face learners), and those registered as MPhil students (compared to PhD). Due to the smaller sample size for the thesis examination scale and the non-signicant ndings for this scale, some of the demographic groupings were combined. Those studying sciences (collapsed together as one group) rated thesis examination more positively than those studying arts and humanities, and social sciences. Those whose research degree was UKfunded (comprising charity, UK government and UK industry funded) rated thesis examination more positively than those whose research degree was self-funded. No other differences were found. Cluster analysis was also used to group together research degree students who responded similarly across the three thesis examination items. Three groups emerged: a high-scoring group (who rated thesis examination highly, mean composite score 4.90, N=154); a middle-scoring group (mean composite score 3.86, N=186); and a low-scoring group (mean composite score 2.13, N=68). Cluster membership was crosstabulated with demographic variables and the statistical signicance of any differences tested (using the chi-square statistic). This was to establish if any demographic groups were disproportionately represented, for example, in the low- or high-scoring group. Of the demographic variables tested (domicile, gender, disability, mode of study and discipline) only discipline showed any statistical signicance. The ndings show that a higher proportion of biomedical and veterinary science students were clustered into the low-scoring group, suggesting they rate their experience of thesis examination less positively. A higher proportion of students from engineering and computer sciences, and medicine, dentistry and other health-related disciplines were clustered into the high-scoring group, with low representation in the low-scoring group. This suggests students from these disciplines rate their experience of thesis examination more positively.
71
72
73
Postgraduate Research Experience Survey Final Report

Published by: The Higher Education Academy Innovation Way York Science Park Heslington York YO10 5BR United Kingdom Tel: +44 (0)1904 717500 Fax: +44 (0)1904 717505 enquiries@heacademy.ac.uk www.heacademy.ac.uk ISBN 978-1-905788-54-5 The Higher Education Academy November 2007 All rights reserved. Apart from any fair dealing for the purposes of research or private study, criticism or review, no part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any other form or by any other means, graphic, electronic, mechanical, photocopying, recording, taping or otherwise, without the prior permission in writing of the publishers.
The Higher Education Academy

Our mission is to help institutions, discipline groups and all staff to provide the best possible learning experience for their students. We provide an authoritative and independent voice on policies that inuence student learning experiences, support institutions, lead and support the professional development and recognition of staff in higher education, and lead the development of research and evaluation to improve the quality of the student learning experience. The Higher Education Academy is an independent organisation funded by grants from the four UK higher education funding bodies, subscriptions from higher education institutions, and grant and contract income for specic initiatives.
74

Postgraduate Research Experience Survey: Chris Park Andria Hanbury Malgorzata Kulej

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Postgraduate Research Experience Survey: Chris Park Andria Hanbury Malgorzata Kulej

Transféré par

Droits d'auteur :

Formats disponibles

Postgraduate Research Experience Survey Chris Park Andria Hanbury Malgorzata Kulej

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 1

Postgraduate Research Experience Survey

Institutional response rates

2 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 3

Postgraduate Research Experience Survey

Goals and standards

4 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

QAA Code of Practice

The Higher Education Academy November 2007 5

Postgraduate Research Experience Survey

1. The Postgraduate Research Experience Survey

6 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 7

Postgraduate Research Experience Survey

8 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 9

Postgraduate Research Experience Survey

10 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 11

Postgraduate Research Experience Survey

1.5 Ownership and anonymity

12 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

The Higher Education Academy November 2007 13

Postgraduate Research Experience Survey

2. Operation of the 2007 PRES survey

2.1 Participating higher education institutions

2.2 Institutional response rates

14 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

2.3 Demographic prole of respondents

The Higher Education Academy November 2007 15

Postgraduate Research Experience Survey

16 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

Table 2. Scale scores, importance, experience and expectations ratings

Mean 3.93 3.86

3.80 3.96 3.11 4.09 -

The Higher Education Academy November 2007 17

3.2 9.8 0.850 2.6 0.971 3.7

0.883 5.8 1.057 1.317 0.783 -

Postgraduate Research Experience Survey

3.2 Intellectual climate

3.3 Skills development

18 The Higher Education Academy November 2007

Postgraduate Research Experience Survey

3.5 Goals and standards

3.6 Thesis examination

3.7 Personal factors

The Higher Education Academy November 2007 19

Postgraduate Research Experience Survey

3.8 Completion on schedule

3.9 QAA Code of Practice

20 The Higher Education Academy November 2007