BAILII is celebrating 24 years of free online access to the law! Would you consider making a contribution?
No donation is too small. If every visitor before 31 December gives just £1, it will have a significant impact on BAILII's ability to continue providing free access to the law.
Thank you very much for your support!
 |
[Home] [Databases] [World Law] [Multidatabase Search] [Help] [Feedback] | |
United Kingdom Journals |
||
|
You are here: BAILII >> Databases >> United Kingdom Journals >> Crofts and Sneddon, 'Equality in Legal Assessment: A Scoping Study' URL: http://www.bailii.org/uk/other/journals/WebJCLI/2009/issue3/crofts3.html Cite as: Crofts and Sneddon, 'Equality in Legal Assessment: A Scoping Study' |
||
[New search] [Help]
 |
 |
[2009] 3 Web JCLI |
 |
 |
 |
Senior Lecturer,
Senior Lecturer, University Teaching Fellow,
University of Northampton
Park Campus
Boughton Green Road
Northampton, NN2 7AL
Copyright © 2009 Melanie Crofts and Simon Sneddon
First published in the Web Journal of Current Legal Issues.
This paper represents a summary of findings from a 12-month project, funded by the University of Northampton Office of Educational Partnerships and Lifelong Learning (OEPLL), which looked at the effect of the assessment strategies used in the compulsory LL.B modules on the different equality strands.
The project contained two elements which employed both quantitative and qualitative methods. For the purposes of this article, the qualitative results relating specifically to race/ethnicity will be highlighted as this is where the largest discrepancy with regards to achievement in assessment lay. However, the full report is available by contacting the authors.
The main body of the quantitative research concentrated on the results of students enrolled at all stages of the LL.B course at the University of Northampton between October 2002 and July 2007, with a minor section on the results achieved by the cohort in the 2007-2008 academic year. The grades for each student in each element of assessment on each compulsory module were compared and analysed on the basis of the student's ethnicity, gender, age at enrolment and declared disability. The results of these analyses were taken on a longitudinal basis and emerging trends investigated. The results suggested that there was little significant statistical difference in these cohorts between students on the basis of gender, age at enrolment or declared disability.
The results further suggested that there was a difference in attainment when the ethnicity of students was used as the comparator, and that this difference occurred in many different types of assessment. Tentative conclusions were drawn as to possible reasons for this difference, although considerable further work would be required in order to draw any deeper conclusions. Areas for discussion were raised as to possible ways of amending the overall assessment strategy to deal with the inevitable juggling act caused by ensuring that:
Having identified that, statistically, some groups of student achieved better results in certain types of assessment, the qualitative section of the research investigated reasons as to why this may be the case. This was done through conducting focus groups and inviting written submissions from students within the Law Division. The aim was to encourage a cross section of views across the equality variables analysed in the quantitative data.
Results from the qualitative data seemed to demonstrate that some students, for example mature students, employed differing strategies which seemed to give them an advantage over younger, school-leaver students, for example. Other interesting comments were made regarding support for disabled students, international students and students from ethnic minority backgrounds.
These comments led to areas for discussion which aim to highlight, and further debate, issues regarding the support of students and therefore potentially raise the achievement of disadvantaged groups in particular. Many of the comments made by participants regarding assessments could not be directly linked to specific equality variables but led to some interesting areas for consideration. Further, more extensive, research is required to see if any more specific interventions, targeting lower achieving groups, could close the 'achievement gap'.
This research stemmed from past and continuing research into issues of achievement and the interrelationship with different equality strands, primarily race and gender.
"Research during the 1980s and 1990s and into then 2000s has continued to find the same groups [African-Caribbean, Pakistani and Bangladeshi] making less progress than other pupils.. [I]nternational studies have confirmed the importance of examining the intersection of ethnicity and gender in order to understand achievement." (Tomlinson, S, 2005, p161)
In August 2007 the REACH Group's report 'Raising the Aspirations and Attainment of Black Boys and Young Black Men' was published and presented to Government. The report, which related primarily to Secondary education and the knock on effects of under achievement in school, highlighted the fact that:
"[b]lack boys are among the least likely to obtain 5 A* to C GCSEs, good A levels, and entry to the more established universities i.e. pre-1992 Institutions of Higher Education." (REACH, 2007, p15)
Other research into HE has reached similar conclusions. In January 2008 the Higher Education Academy (HEA) and the Equality Challenge Unit (ECU) published a report 'Ethnicity, Gender and Degree Attainment'. This report confirmed the findings of previous research into degree attainment which suggested that:
"even after controlling for the majority of contributory factors, being from an ethnic minority group. was still found to have statistically significant and negative effect on degree attainment. The research also showed that females are more likely to obtain higher degree classification than males, except when it comes to obtaining a first." (HEA & ECU, 2008, p2)
One of the key recommendations of this report was that:
"[Higher Education Institutions] should ensure that their systematic review processes include consideration of equalities issues and a robust evaluation of learning, teaching and assessment practices in light of any discovered attainment variation." (HEA & ECU, 2008, p29)
University of Northampton data has reflected the overall UK scene in relation to ethnicity and gender and the attainment of "good" degrees. In the statistical report of "good" degrees at the University of Northampton it was stated that:
"[o]ver the four year period studied ethnic minority students (except students of Chinese origin) are consistently gaining fewer good degrees than White students across the university . [and] men are consistently achieving fewer good degrees that women . [T]he adverse difference ranges from 5% in 2005 and 2006, to 10% in 2004 and 2007." (EDU, 2008, p7)
Although it has been accepted that the factors contributing to these differences in achievement are complex, with the growing importance of Widening Participation (see for example Hadfield, D, 2006, Coffield, F & Vignoles, A, 1997, and Torgerson et al, 2007) and an increase in the number of international students, issues relating to student support and retention are increasingly important for Higher Education Institutions. Consideration of these issues and the interlinking issue of different and fair methods of assessment will not just fall within the statutory general and specific duties on Public Authorities to promote equality under the Race Relations (Amendment) Act 2001, Disability Discrimination Act 2005 and Equality Act 2006, for example, but could also improve institutions' competitiveness within the HE market. As the HEA and ECU report suggested, a vital part of looking into the issue of differential adverse impact in relation to degree attainment is consideration of assessment strategies. (HEA & ECU, 2008, p29)
The research suggests a disparity between different groups/strands of students and their achievements, and it could thus be questioned whether it is the assessment strategies themselves that are having this differential impact. Improving the learning experience for a group of students of one type (i.e. based on ethnicity, age, gender or declared disability) should help to improve the learning experience for all students, even those who are not within the category being targeted. In other words, to paraphrase Healey et al., (2006, p41) the main beneficiaries of adjustments made for one group of students may be those from outside that group.(1)
Although much of the previous research relates primarily to race and gender, our full report also considered declared disability and age, and identified that there was varying impact on students based both on types of assessment, and on considerations of gender, ethnicity, age, or declared disability. The results have led to suggestions being made for modifying teaching methods, designing new programmes and revising the existing curriculum (Peterson, M., & Augustine, C, 2000)
For this project, students were assessed by:
All the results were converted into alphabetical grades(3) to make comparison more straightforward. The nine categories of assessments used in the University of Northampton LL.B programme over the period being investigate were:
Whilst it is acknowledged that this is in many ways an artificial distinction, the low numbers of students from some ethnic backgrounds (e.g. Bangladeshi) were not statistically viable on their own and, rather than discount any particular group of students, the results of all BME students were aggregated. Future, larger-scale work may be able to disaggregate the BME students and explore any differences therein. The average number of white students at Level 4 was larger than BME students, but only by a ratio of 1.3:1, in other words BME students made up about 45 per cent of the LL.B cohort. This figure was not static, and the percentage of BME students at Level 4 increased from 37 per cent in 2002-3 to 52 per cent in 2006-7.(4)
According to Universities UK, ethnic minority student make up 16 per cent of overall entrants to HE institutions (UUK, 2006, p38), and calculations based on data available from HESA(5) suggest that, of those students who declared their ethnicity on their HE application forms, 14.2 per cent were from a BME background. (HESA, 2007a)
Bearing in mind these figures, the proportion of BME students on the LL.B course was comparatively high. Even taking into account the dip in 2005-6, the LL.B figures back up the contention that in post 1992-universities (Kukhareva, M., et al 2007):
"there is an increasing concentration of students from ethnic minority groups in a limited number of institutions" (UUK, 2006, p37-8)
Over the five year period, white students consistently outperformed BME for all bar one of the assessment types. The margin of difference varied year-on-year between a C- and a C for 2005-6 in the SHORT ASS, and a C- to B- for 2005-6 in the TCA. The average over the period was a D+ for BME students and C for white students.
The only assessment type in which white students did not, on average, perform better was SEMINAR, where there was no difference between the performance of white and BME students. To put this into perspective, female students generally outperformed male by D to C- in this type of assessment. No analysis has yet been carried out to see whether there is a distinction between the achievement of students separated simultaneously by ethnicity and gender, but it remains an area of clear focus for future work.
Of all of the equality strands considered in the study, it was ethnicity which constantly provided the largest disparity between groups, although there was a gradual reduction in difference over time. As with year-on-year number of BME students, this disparity was not constant, falling more-or-less steadily from C/D+ in 2002-3 to C/C- in 2006-7.(6) There was a sudden rise in the disparity in the 2004-5 year, and this remains the largest gap between the two groups.
In terms of the difference between white and BME students, Level 5 follows a similar pattern to Level 4 - on average over the five years, white students outperformed in every one of the assessment types. This cohort was also largely the same group which formed the 2004-5 Level 4 students, who had had one of the highest proportions of BME students, so the drop was doubly apparent. Much of this drop in student numbers could be explained by the comparatively poor performance of the Level 4 BME students in 2004-5 - that the worst-performing students simply did not make it to Level 5.
The difference in average grades for the year, whilst it consistently favoured the white students, fell steadily between 2002-3 and 2005-6, from C/D+ to C/C- before jumping unexpectedly in 2006-7 to C+/D+. It is hoped that this rise represents nothing more than a poor performance by a single cohort (particularly in the examination, where the BME students' average grade was D-).
According to the only QAA Subject Overview Report for Law (which was carried out in 1993-4), "once students passed the first year, wastage was relatively small." (QAA, 1995, Para 47)
If this were still to be true, one would expect that, since there are not so many weaker students failing to make the transition between Levels 5 and 6 as there were between Levels 4 and 5, any difference between white and BME students should be reduced. Equally, given the average advantage that Level 5 white students seem to have over BME students, one could reasonably expect the proportion of BME students at Level 6 to be consistently lower than it was at Level 5. However, this was not the case in two of the three cohorts studied. This could be explained by the fact that entry onto the LL.B is not 'frozen' at Level 4, and students are able to transfer from other courses or from other institutions right up to the beginning of Level 6. Any future research would have to take into account what impact that transfers into (and out of) the LL.B might have had.
The proportion of BME students at Level 6 rose sharply from 24 per cent in 2003-4 to 45 per cent in 2005-6, before falling back again in 2006-7 to 34 per cent. Whilst this suggests a certain level of sensitivity in the figures, the average proportion of BME students at Level 6 was 34 per cent, which was still more than double the national average figure. (UUK, 2006)
Level 6 provided a more interesting set of statistics than either of the previous levels. In those, a clear trend could be extracted whereby the difference in achievement between white and BME students was steadily reducing (albeit with an occasional statistical anomaly). At Level 6, however, the difference appeared to be considerably more volatile, rising from C+/C+ in 2003-4 to B-/C in 2004-5 and C+/C- in 2005-6 before falling again to C+/C in 2006-7. Closer inspection of those statistics revealed that, whilst the average white student's achievement over the four years had been C+ for three of the four years, and B- for the remainder, that of the BME student fell rapidly from C+ to C- over three years, before recovering slightly to a C in 2006-7.
Over the three years that a student will normally spend on a full time LL.B, it would always be hoped and expected that their performance in particular types of assessment would improve, as the students are able to hone their skills through practise. The Division of Law at the University of Northampton was objectively assessed by the Guardian newspaper as having a "Value-Added score of 7/10" in their University League Tables(7) in 2008, which supports the argument that all students will perform better over the course of their degree.
For the results studied here it can be seen that there was an average increase in the overall achievement of white and BME students at each level. It can clearly be seen that all students increased their overall performance between Levels 4 and 6, but the improvement is considerably more marked for the BME students. It should also be noted that all the scores fell within the general 2:ii classification for degrees so, although there was clearly a difference in how well students were doing, it may not make a difference to their future employment or study prospects, as potential employers will not be able to differentiate easily between a "good 2:ii" and a "bad 2:ii".
That there exists an adverse impact on BME students across all three years of the LL.B compulsory modules cannot, in any conscience, be disputed. The narrowing of the difference over the three years of assessment could be an encouraging trend, although it may also mean that the weaker students are not completing the course, which is less encouraging.
It is possible that parental pressure, which is more "evident among Pakistani, Bangladeshi, Indian and Black African groups" (Connor, H, et al, 2003, p13) has led to a higher proportion of BME students taking the LL.B, with a view to a future career in the legal profession, than taking some of the less vocational courses offered elsewhere at the University of Northampton.
Indeed, one report in 2007 suggested that "some BME students. are pressured into taking courses they do not want" (HERO, 2007, p2) which may go some way to explaining the disparity in results between the two groups.
In order to complement the quantitative data and to provide further understanding of its results, it was decided to carry out a number of focus groups with students in Law in order to gain some rich data regarding their views of various pieces of assessment. It was hoped that the qualitative data would provide some insight as to possible reasons why some groups of students performed better in some pieces of assessment than others. The use of focus groups also enabled those students who were merely 'statistics' with regards to the quantitative data to comment on their own experiences and feelings, as well as those of their colleagues.
The research was primarily concerned with a diverse student population, so a number of equality variables were considered, and it was important to try and encourage a diverse focus group composition so that a range of views could be expressed. There was broad representation from across the equality variables over the course of all the focus groups. There was representation from under 21, over 21, white, BME, disclosed disability and male and female participants as categorised by the qualitative data analysis.
The focus group data was analysed using thematic analysis. Thematic analysis "offers an accessible and theoretically flexible approach to analysing qualitative data. [and is] a useful and flexible method for qualitative research" (Braun, V & Clarke, V., 2006, p77).
A definition of thematic analysis might be considered as;
"analysis based on the identification of themes in qualitative material, often identified by means of a coding scheme. A widely used approach to qualitative analysis, generally treating accounts as a resource for finding out about the reality or experiences to which they refer." (Seale, C (ed), 2004, p7)
All the data was analysed by identifying themes within the data items, and these themes form the headings and sub headings below. Some of the more interesting themes in terms of possible equality issues raised by the students have been highlighted and commented on in light of the quantitative data analysis and overall aims of the research.
The likes and dislikes of participants in relation to assessment types appeared to be fairly consistent and backed up what has been suspected anecdotally, that students prefer written assignments over exams, and particularly end of year exams. Participants felt that the amount of information which is required to be retained for an exam means that they, on the whole, do not do well.
"I think I am good at coursework because I'm in control and know where I'm going with it and I know the demands of it. In terms of exams I don't think I do well in exams because it's in my head but when I get into the exam it's just totally blank."
However, it might be said that the feelings of underachieving in exams primarily applied to students who were traditional and disabled.
"[t]he problem with exams is that people with certain learning difficulties are not able to achieve in an exam environment."
Both the non-disabled mature students viewed exams slightly differently, with some seeing them as a positive challenge, and others achieving well in them despite their dislike for them. One notable exception was a participant who had English as their second language who had had previous experiences of exam based assessments, which greatly influenced their success.
"I do well in exams . because that's the way it used to be. I am used to remembering events and dates accurately so actually most of my exams I can remember the dates. It's the way my memory is trained. .I read it once or twice and then I commit it to memory. .It's the way we were trained. You were given something to read, if you missed something out you had to do it until you got it right."
The view that overseas students might be exposed more to exam based assessments was backed up by another participant. Whether this is a one off or a common experience of other international students, or students who experienced education systems in other countries, is unknown. Due to the intense system of testing which students undergo throughout their educational career in the UK, it may also be said that traditional students also have a lot of exposure to exams and therefore, in theory, should be better at them than more mature students (which is reflected in some years in the quantitative data).
The level of difficulty, whether the work is challenging, motivation and enjoyment the student gets from particular pieces of assessment and their perceived success were not necessarily always linked, as some of the "easy" assignments (e.g. TCA) require little motivation but success in terms of grades was seen to be achieved, possibly unsurprisingly. Whereas those which the students thought they would find challenging were not always spoken of positively in relation to achievement (e.g. EXAM). However, the students appeared to make a distinction between assessments which were "challenging" and "difficult". The former appeared to be a positive aspect of the assessment which motivated the students and in some cases may have been linked to the success of the students. The latter, however, appeared to present a negative view of the assessment type, rather than being viewed as a challenge. This in turn appears to have a knock on effect in terms of the student's perceived success.
What was clear was that students felt a mix of assessment types was important and should remain. One student stated that:
"There is a good mix, I do like the mix. There's some that encourage you, I suppose to put a lot of effort into if you want to get the grades. I think it's a pretty good mix of stuff so you don't get too bored."
A mix of different assessments was viewed as important by all participants as it allowed the students to excel in the areas they were good at. Discussions in the focus groups highlighted the need to consider the support given to students who do not have English as their first language and were seen to struggle more with oral presentations and written assessments.
". if you have a student which doesn't have English as their first language, they might find it [presentations] intimidating. that might be a problem."
Work/life experiences:
Participants made reference a number of times to previous work experiences, and the connection with the role of Personal Development Planning (PDP) in assessments. This seemed to connect strongly to the participants' mature student status and appeared to have an impact on the participants in relation to their skills development, and thus a knock-on effect on their achievement in assessments. In relation to the PDP it was stated:
"it's not a concept that's completely alien to me, from the job I used to be in before, PDPs were an essential part of my job and you're assessed. each year and that's part of your pay review. I do sort of embody a lot of that stuff in what I do anyway when I'm looking at my own work."
The implication therefore is that the students' previous experiences of skills development, whether through knowledge of PDP and self-reflection had an effect on the improvement and success of the student in assessments. A longer period of work experience prior to entering HE may apply more to mature students thereby increasing their ability to reflect on their work.
However, other participants also expressed the view that it was generally the life experiences that mature students had gained which were important factors in assignment achievement.
"[t]hey [mature students] have had that slight more experience." "I think there's more to it. I think it's more of what you've been exposed to, not just in university life, but outside of university. . When it comes to writing an essay it brings that kind of knowledge. those skills are already there."
The mature students view was that life experiences in general, not just work experience, helped when it came to being successful in assessments. One traditional student also took the view that home/family experiences (as part of life experience) also played a role in assessment success and seemed to indicate that these were factors when it came to understanding the context of certain assessment questions and that mature students might have the advantage when it came to some of these issues due to their more 'rounded experiences'.
The predominant view from participants was that mature students somehow have an advantage in assessments due to their previous experience. The quantitative results in the full report did not really bear this assertion out, however. If follow-up research does show that this is the case, it would be difficult to replicate life experience for school leaver students and more emphasis would need to be placed on issues of peer support and mentoring.
Assessments and transferrable skills:
Another sub-theme which was identified was the link between the development of skills and future career needs. This view was very pronounced with the traditional students and was particularly concerned with exams as a form of assessment which did not build the skills needed for the future.
"I don't know where the skills for exams would be helpful in a future career..[assessments] relate to future needs. I can see the point of presentations and assignments, but not exams."
Some students viewed assessments in terms of the transferrable skills that they could acquire from them. It might therefore be said that there needs to be further emphasis in the information provided to students as to which skills various pieces of assessment aim to develop and link these into how the development of these key skills can be transferred into future careers. It seemed that this was particularly necessary with regards to exams as the point of them was questioned throughout by all students. This was not to say that exams did not necessarily play a role with regards to assessment, but merely that they may need further explanation or justification.
This theme was the most emphasised by participants, who felt that support mechanisms were key to the success of students in assessments. It was also the area which made most reference to the equality variables and seemed to suggest that certain factors did influence the achievement in assessments of particular groups of students.
Peer Support:
Peer support was most emphasised by the mature students.
"[m]ature students did have that advantage of getting together and everything else. sitting around the table like this and bouncing ideas off each other."
The support of like-minded students in discussing a variety of issues, not necessarily just assessment-related, was very important to mature students. Peer support and help appeared to be vital in sharing ideas and having discussions, comparing answers for assignments, and running revision sessions within their group. However, it was clear that the peer support did not or could not come from just any colleagues, but those with whom the participants had identified and believed were "like-minded". Some comments which were made are as follows:
"I think one of the driving factors for forming a group is a mutual focus."
"The first port of call [for help] is going to be the people I'm around and the friendships I have built up because we're all quite similar. I can trust the answers I'm getting from those sorts of people."
"I find it quite difficult, if I don't know the people. so I'd rather discuss it with people who know me and how I work because I feel I get better responses and answers from them, purely because they know me."
"I wouldn't necessarily get up and pick up with people I didn't know to do that kind of thing. I need to feel that sort of connection with people."
This type of support appeared to be vital in providing help for all types of assessment. However, trust and familiarity with those who were providing this support were essential in order for the participants to feel that the help they were receiving would be beneficial.
However, some participants did not feel that being a mature student was the only 'mutual focus' which had the potential for forming such a support group and acknowledged that other groups of students gravitate towards each other.
"You see it with international students.they band together."
It was clear that this type of self-support mechanism was strong amongst mature students at all levels, but another form of peer support which was highlighted was that of more formal student mentors. Student mentors are a formal support mechanism which are set up and encouraged within the University. Some of the participants (particularly those who were mentors) felt that this type of peer support was important and could benefit those students who did not have the benefit of belonging to an informal support group in the way the mature students were. Despite the participants viewing this as an important mechanism, they expressed their frustration at the lack of use made of it by their colleagues.
"I've done it and basically I've had like two people coming." "I find that quite frustrating. . it wasn't the people who weren't doing well who needed to come and see us."
An issue which was not raised specifically with regards to student mentors but which was raised regarding informal peer support groups was that of 'like mindedness' or 'mutual focus.' This is interesting given it was stressed by all the mature students that these factors were important in feeling secure about the information received/discussed and feeling comfortable within the support group. However, these aspects were not considered as pivotal factors in whether students approached the student mentors for support. Although there is no direct evidence that students do not approach student mentors because they do not feel that they are 'like minded', it might be considered as a factor given its importance in peer support groups. In addition, comments made by some participants regarding email support, may corroborate the theory that students are reluctant to make face to face contact for support with people with whom they do not feel an affinity;
"Last year we had a lot of success with [the email] because it was anonymous and people would just email a problem and we would have a look at it, give them back an answer and that worked really, really well and I thought that would really take off this year. .I think that is really important anonymising it so that people can have the confidence that it is in confidence and the anonymity of it. .most of those were international students or ethnic minority students because. trying to grasp legal terms in a language which isn't your first language [is a] problem I think."
It seems that anecdotal evidence has been used here to suggest that students whose first language is not English, and BME students are more likely to use an anonymous email system for peer support. Whether this is actually the case cannot be verified, but it may be said to add to the argument that face to face contact is sought from people (peers or tutors) with whom the student identifies with in some way.
Other support mechanisms:
Other support mechanisms, particularly tutor support, were mentioned more by the traditional students. The approachability of the tutors and affinity with them was seen as key in accessing such support;
".obviously it depends on the person whether you get on with them. Obviously that does affect your learning experience quite a lot if you get on with that person. . If you don't get on with a lecturer or seminar tutor, you just, not even as in you don't like them, just that you don't understand where they're coming from, I think that's very difficult and obviously that's to the pupil's detriment."
Some participants related this back to anecdotal evidence regarding international and BME students,
"[a] lot of international students, or even ethnic minority students here, don't seem to feel at ease talking to them [tutors], where the majority of them are white. In fact they're all white in the Law Division, so you have that kind of barrier to start with. A lot of them don't feel particularly comfortable about it."
And similarly in relation to younger students;
"For traditional students the experience is a very different relationship with a lecturer than it is seeing a teacher or head teacher. It's a very different relationship and again, a lot of mature students feel more comfortable going to speak to their lecturers because there isn't this kind of feeling of being above you."
As discussed above in relation to informal peer groups and student mentors, having an affinity to the person the student is approaching for help seems to be an important factor in accessing that help. Some areas may be easier to address than others, for example in supporting and encouraging informal support groups for particular groups of students, or persuading students from more diverse backgrounds to become involved in the student mentoring scheme. The make-up of the teaching staff is more difficult to address and is down to complex factors not controlled by the Law Division.
Support for disabled students was also highlighted by some of the participants. Some participants felt that certain disabled students appeared not to be given the support, either by the University as a whole, or from tutors, that they needed in order to successfully complete assessments. The provision of lecture notes or PowerPoint slides in advance of lectures was the main area, relating to tutor support for disabled students, which was mentioned;
" .If people with mental disabilities, if they had full notes, they would be able to read through them first, have questions and go to the tutor afterwards or to someone who can help them with it." "There doesn't seem to be enough done. I'm talking about dyslexic students. .many times those notes [advanced lecture notes] were not there for whatever reason, even after asking for them they were not there. I think it's disadvantageous. .There were several times where I've gone to the lecture, not had lecture notes, not had PowerPoint slides and I feel at a disadvantage because I'm not able to follow or even get to read what's going on before the lecture."
There seemed to be a mixed picture regarding the issue of support for disabled students and this could be argued to be down to the individual student's experience of a particular tutor as well as the type of disability the student has. In addition, not all experiences of disabled students were negative.
".the university was extremely accommodating with regards to providing a separate room and breaks during the TCAs if I needed them and more significantly the tutors bent over backwards to ensure that we were not only well prepared for the TCAs, but were fully aware of the subject areas that would be tested."
What was clear, however, was there seemed to be a lack of awareness on the part of students (and possibly also staff) regarding the support which was available and which could be accessed at the institution. This information may need to be made more readily available in order to tackle any misconceptions about the support for disabled students, as well as to ensure that students who are disabled are accessing the support they need. With regards to access of teaching material, again, there seemed to be a mixed picture and not wholly negative. Support provided to disabled students should, however, be consistent and lecturers should ensure that their teaching materials are accessible.
This theme created the most discussion and some disagreement amongst the participants. This was due to the fact that two of the participants viewed the comments from another participant as attacking the integrity of the tutors and became quite defensive. However, some important issues regarding the procedures and perceptions of the marking processes were raised. There were two main sub-themes which were identified from the discussions, that of transparency of marking processes and the potential for bias in marking.
Transparency:
One participant seemed to be of the view that the way that marks were awarded for assessments was not always clear and rationale for the grades was not obvious.
"I think if we're talking about the way grades are awarded, for example for an assignment, how, if you are marking, how do you arrive at how I got an 'A' or a 'B' or 'D'? To me it looks arbitrary. Sometimes there is no consistency, it might not be because of ethnicity or someone's age but. sometimes we discuss among ourselves. when you know what to expect, with me I know what to expect, so sometimes if you don't get what you expect and you try to find out why."
The participant in this instance also made reference for the potential for abuse if the marking criterion was not sufficiently transparent.
"I always thought that the piece of work that shows me 10 points there and 5 points there is easier to see how you've got them. What I always think is that the current method is open to abuse, because you may not like Scottish people and someone can just have that in the back of their mind. How do we guard against that?"
The overall marking criteria are available on the University website, individual module VLE sites, and in the student handbook, this may not be apparent to many students and therefore it could be highlighted more effectively. In addition a useful exercise to encourage students to think about how their own work fits within the marking criteria might be to ask them to reflect on their work using the marking criteria and award themselves a grade on that basis which can then be compared with the tutor's mark and discussed. It is acknowledged that such a process is time consuming and dependant on student numbers but may make the marking processes clearer to students.
Anonymity:
Another issue raised by one participant was in relation to eliminating unintended bias, or the appearance of bias, within the marking procedures. The double marking and external sampling of assessments goes some way to addressing this. However, it might be said that there was still the view that the potential for bias could exist. It is interesting to note that when looking at the quantitative data the largest differences in achievement for groups, such as BME groups, were in those assessments where the making was anonymous (e.g. exams). However, this is not to say that there is never any unintentional bias when it comes to marking and this could potentially affect a range of students for a variety of reasons. The issue of anonymous marking, where possible, was raised.
"The whole point of just putting your student number in is that it's anonymous and the tutor isn't going to know your student number and they are going to mark the work on the piece of work that's in front of them, not on the personality of the person that's produced that work."
This area was a contentious area and was viewed by some participants as attacking the integrity of the tutors and the marking systems. However, a consideration of anonymous marking might be beneficial in two ways, to protect tutors from any accusations of bias and to tackle any perceptions of potential bias in the system.
The qualitative data provided some interesting insights into the views of students on assessments. There are clearly some issues which have arisen which have shown that some groups of students employ strategies which put them at an advantage over other groups of students, for example mature students. These strategies can be borne in mind and potentially applied to other student groups to hopefully improve achievement.
Additionally some participants in the focus groups made important contributions which related directly to some of the equality variables, specifically disabled students and students whose first language is not English. Although it cannot be said that these views apply to all students in these categories, some of the comments made were certainly important and should not be dismissed lightly. This is particularly the case with regards to the theme of 'Marking'.
There were, however, some notable absences in terms of representation in the focus groups, for example, there were no traditional male students and no BME female students. Further research would also be required to investigate the views of students in terms of multiple identities (for example do traditional male students have differing views of assessment than mature male students, or traditional BME male students and so on. This was beyond the scope of this research.
It needs to be recognised that broad generalisations have been made regarding the categories of particular students and the achievement of those groups. It has to be recognised that although not all students within these groups are underachieving, a significant minority are and that this significant minority are affecting the overall data relating to some assessment types. It is worth emphasising that further research and analysis is required in order to investigate whether particular BME students are underachieving in various assessment types. This would need to be done over a longer period of time or including other Divisions so that the numbers within each ethnic group are significant. Further research might also consider other factors, such as social economic background, as well as looking at multiple identities as mentioned previously.
The discussion points below aim to stimulate dialogue within Divisions regarding the issues raised in the report. It is important to work towards improving grades (and thus retention) at all levels amongst all students and therefore some of the discussion points are aimed at all students and not particular groups of students. However, at the same time it must be recognised that there is a significant minority of students underachieving. Some of the discussion points are therefore targeted more specifically at stimulating debate regarding raising the achievement of this significant minority. Some of the points made can be dealt with relatively easily, both conceptually and practically, but others may require deeper discussion.
Some of the points below have already been considered and implemented within the Division of Law at the University of Northampton following the preliminary results of this research and, in the first iteration of the new regime, initial results show a slight improvement in attainment across the board. However, they may still be useful areas of debate for other subject areas within the institution or other Law departments at other institutions. Follow up research is required to see if some of the initiatives adopted by the Law Division are improving grades and tackling disadvantage.
Aimhigher (2008a) 'Jobs for the Girls' Aimhigher / Higher Education Funding Council for England. Available at http://www.aimhigher.ac.uk/careers___guidance/jobs_for_the_girls.cfm, accessed on 28/05/08
Aimhigher (2008b) 'Mature Students' Aimhigher / Higher Education Funding Council for England. Available at http://www.aimhigher.ac.uk/courses/getting_back_into_education/mature_students.cfm, accessed on 28/05/08
Bone, A., (2004) 'The impact of formative assessment on student learning' University of Brighton, unpublished.
Braun, V and Clarke, V 2006 Using Thematic Analysis in Psychology Qualitative Research in Psychology 2006 Vol 3 pp77 - 101 accessed via http://science.uwe.ac.uk/psychology/drvictoriaclarke_files/thematicanalysis%20.pdf
Carson, C. (2004) 'The Lawyer Statistical Report: The U.S. Legal Profession in 2000' American Bar Foundation, Chicago, IL.
Coffield, F and Vignoles, A, 'Widening Participation in Higher Education by Ethnic Minorities, Women and Alternative Students - Report 5' Dept of Education, University of Newcastle, www.leeds.ac.uk/educol/ncihe/report5.htm
Connor, H., Tyers, C., Davis, S., Tackey, N., and Modood, T. (2003) 'Research Report RR-448: Minority Ethnic Students in Higher Education: Interim Report' Institute for Employment Studies / Department for Education and Skills, London
Dunn, S. (2003) 'Return to SENDA? Implementing accessibility for disabled students in virtual learning environments in UK further and higher education' Sarah Dunn / City University, London. Available at http://www.saradunn.net/VLEreport/documents/VLEreport.pdf, accessed on 18/06/08.
EDU (2008) 'A Statistical Overview of "Good" Degrees awarded 2004-7 and Recommendations for Future Action, Report 19' University of Northampton
Equality and Diversity Unit, Northampton
European Commission, Eurydice and Eurostat, (2007), 'Key data on higher education in Europe. 2007 edition', Brussels: Eurydice, http://www.eurydice.org/ressources/Eurydice/pdf/0_integral/088EN.pdf, accessed on 15/04/08.
Guardian (2008) 'University Guide' The Guardian. Available at http://education.guardian.co.uk/universityguide2008/0,,2027789,00.html, accessed on 15/04/08
Hadfield, D (2006) 'Boys into Higher Education' Aimhigher East Midlands
HEA and ECU (January 2008) 'Ethnicity, Gender and Degree Attainment Project' Higher Education Academy and Equality Challenge Unit, York & London.
Healey, M., Bradley, A., Fuller, M. & Hall, T. (2006) 'Listening to students: the experiences of disabled students of learning at university' in M. Adams and S. Brown (eds.) 'Towards Inclusive Learning in Higher Education.
Developing curricula for disabled students' Routledge, London.
HERO (2007) 'Where are the black students?' Higher Education and Research Opportunities, Newcastle-Upon-Tyne, Tyne & Wear.
HESA (2007a) 'Students and Qualifiers Data Tables' Higher Education Statistics Agency, Cheltenham, Gloucs. Available online at http://www.hesa.ac.uk/index.php?option=com_datatables&Itemid=121&task=show_category&catdex=3, accessed on 17/06/08.
HESA (2007b) 'Students in higher education institutions 2005/06' Higher Education Statistics Agency, Cheltenham, Gloucs.
HMSO (June 2008) 'Framework for a Fairer Future - The Equality Bill' CM7431
Houston, M., Knox, H., and Rimmer, R., 'Wider access and progression among full-time students' Higher Education (2007) 53: 107-146
ISCED (1997) 'International Standard Classification of Education 1997' UNESCO, http://www.unesco.org/education/information/nfsunesco/doc/isced_1997.htm, accessed on 15/04/08
Kukhareva, M., Eames, J., Sinclair, T., Kendall, S., Chakravorty, M., and Watts, M., (2007) 'Ethnic Minority Participation in Higher Education in the East of England' Association of Universities in the East of England, Cambridge.
Leitch, S. (2006) 'Prosperity for all in the global economy - world class skills: Final Report' HM Treasury, London.
Martin, S., & Jurik, N. (2006) 'Doing Justice, Doing Gender: Women in Legal and Criminal Justice Occupations' Sage Publications Inc., Thousand Oaks, CA.
Peterson, M., & Augustine, C. (2000) 'Organizational Practices Enhancing the Influence of Student Assessment Information in Academic Decisions' RiHE 41 (1) 21 - 52.
QAA (1995) 'QO 1/95 Subject Overview Report - Law: Quality Assessment of Law 1993-94' Quality Assurance Agency, Gloucester.
QAA (2006) 'Code of practice for the assurance of academic quality and standards in higher education' Quality Assurance Agency, Gloucester.
REACH (August 2007) 'Raising the Aspirations and Attainment of Black Boys and Young Black Men' Department for Communities and Local Government, London
Richardson, J. and Woodley, A. (2003) 'Another look at the role of age, gender and subject as predictors of academic attainment in higher education' Studies in Higher Education 28(4), 475-490
Seale, C (Ed) (2004) 'Researching Society and Culture: Glossary' 2nd Edition Sage Publishing accessed via http://people.brunel.ac.uk/~hsstcfs/glossary.htm (accessed 15/09/08)
SE (2007) 'The Age Participation Index for Scotland 2005-6', Scottish Executive National Statistics, Glasgow
Smith, J. and Naylor, R. (2001) 'Dropping out of university: a statistical analysis of the probability of withdrawal for UK university students' Journal of the Royal Statistical Society A 164(2), 389-405.
Tomlinson, S. (2005) 'Race, Ethnicity and Education under New Labour'. The Oxford Review of Education, 31 (1) 153 - 171.
Torgerson, C.J. et al (2007) 'Strategies that are Effective in Encouraging Participation of Minority Ethnic Groups in Post 16 Education: A Systematic Review of North American Literature' Department for Children, Schools and Families Research Brief
TSG (2005) 'The Trainee Solicitor's Group of England and Wales: Response to the paper "Women in the Legal Profession and the Judicial Appointments Process"' Trainee Solicitor's Group, London. Available at http://www.tsg.org/women-in-the-legal-profession.html, accessed on 28/05/08
UUK (2006) 'Patterns of higher education institutions in the UK - Sixth Report' Universities UK, London.
UUK (2007) 'Beyond the honours degree classification: The Burgess Group final report' Universities UK, London.
Wakeling P., & Johnson, J. (2006) 'Ethnicity and postgraduate study in politics: a look at the evidence' Paper given at the Political Studies Association Conference in Reading, April 2006.
Woodley, A., Thompson, M. and Cowan, J. (1992) 'Factors Affecting Non-completion Rates in Scottish Universities' Milton Keynes: Institute of Educational Technology, The Open University.
(1) The actual quote is "the main beneficiaries of disability legislation and the need to make suitable adjustments in advance are the nondisabled students."
(2) Levels 4, 5 & 6 equate to Years 1, 2 & 3 of the full-time LLB Course.
(3) A = First, B = 2:i, C = 2:ii, D = Third
(4) The percentage of BME students was 43 per cent in 2003-4, and increased to 48 per cent in 2004-5, before dipping to 39 per cent in 2005-6.
(5) The HESA figures for 2002-3 to 2006-7 have a total of 804,120 students declaring their ethnicity, of whom 690,075 (85.8 per cent) were white, and 114,045 (14.2 per cent) from a BME background.
(6) The white students' average grade rose from C in 2002-3 to C+ in 2003-4 and 2004-5, before dropping back to C in 2005-6 and 2006-7. The average grade of the BME students, however, rose from D+ in 2002-3 to C- in 2003-4 where it remained.
(7) The categorisation of Law was "Study of criminal and civil legal systems - includes criminology, jurisprudence" so some of the 7/10 ranking is due to the contribution of the Field of Criminology, which was not assessed here. UN ranked 23= (out of 90 institutions) in terms of the Value Added in Law.
