Evaluation of effectiveness of Peer tutoring

To measure the effectiveness of the Peer Tutoring, two groups were used, a tutored group, and a control group who were individually matched in points level, and subject expertise with individuals from the tutored group. Both quantitative and qualitative measures of effectiveness were used.

Summary

1. Peer tutoring significantly reduced the number of students who dropped out the course in the first semester.
2. Peer tutoring significantly improved the student performance in Chemistry.
3. Attendance at Mathematics in-house tests was significantly better.
4. The scores on an index of student study strategies, the LASSI test (Learning and Study Strategy Inventory) were significantly higher for the tutored group for motivation, time-management, information processing and test-strategies.
5. Students liked the peer tutor sessions, would have liked more of them, and would recommend them to a friend.
6. The tutors enjoyed their role as peer tutors, and visibly matured over the course of the Semester.
7. The Journals that the tutors kept were a fantastic source of feedback about the peer tutoring generally, and about the course as a whole.
8. The effectiveness of a variety of approaches for tutoring was assessed and a large number of recommendations made for improvements.

Quantitative measures of the effectiveness of peer tutoring

Quantitative measures of the effectiveness of peer tutoring were based on the following:

• "Attrition rate"

  The proportion of students who dropped out of the course in the first semester in the tutored and control group.

• Performance in Chemistry Module (CS101)

  The scores from the Semester I examination were examined for both groups, and factorial analysis of variance used to adjust the responses for the level of chemistry (Leaving Cert. Or not) and for overall entry standard (points).

• Performance in Mathematics (MS125)

  The scores from the initial diagnostic test in mathematics and the scores from the 3 in-house tests taken in the first semester were used.

• Study Strategies (LASSI test)

  The Learning and Study Strategy Inventory test (LASSI) was used with the tutored and control groups at the start of the semester prior to tutoring, and again at the start of Semester II after the tutoring had finished. This provides an index of the student's learning strategies.

Qualitative feedback

Both tutees and the tutors were interviewed.

• Qualitative feedback from tutees

• Discussion group / comments

At the end of the first Semester, a discussion was carried out with tutees in each tutorial group, and the main points summarised.

One tutor also asked the students a number of questions about the tutoring.

At the start of Semester II, the students were also asked for their individual comments.

• Qualitative feedback from tutors

• Discussion group /Journals/ presentations

Two formal discussion sessions were run with the tutors at 4 and 8 weeks into the Semester.

Each tutor kept a journal where they wrote down their impressions of the tutoring process after each tutorial. This was integrated in their BT2 course, and the students were assessed on the breadth and clarity of their comments and recommendations.

Early in Semester II, the students were asked to give a presentation where they were asked to cover their impressions of the peer tutoring, and to highlight the successes, failures, and ways to improve the process.

Quantitative Feedback

"Attrition rates"

Four of the 24 individuals in the control group dropped out of the course in the first Semester while no-one in the tutored group dropped out of the course.

Probability of dropping out of the course = 4/24 = 0.167.

Probability of having zero dropouts in a sample of 24 = 0.0125.

• Significantly fewer people in the tutored group dropped out the course than might have been expected based on the control group, and on knowledge of the proportion of the 1999/2000 class who dropped out of the course in Semester I.

Semester I examination results in CS101, Inorganic and Organic Chemistry

Factorial ANOVA and correlation and regression analysis were used to tease out the factors which might affect examination performance

• Whether the students had a good standard in Chemistry at Leaving Cert.
• What points level the students had entering DCU
• Tutored versus Non-tutored

Students who had Higher Chemistry at Leaving cert. did significantly better than those without (p=0.022).

The higher the points at Leaving cert., the better the mark in Chemistry (p=0.002).

• Students who were tutored did significantly better than those in the control group (p=0.032).

Performance in Mathematics

In the control group, 16 of the 20 students still in the course presented for Maths tests 2 and 3.

In the tutored group, 23 of the 24 still in the course presented.

• A significantly greater proportion of the tutored students presented for Maths tests than the non-tutored (p=0.009).

Study Strategies

Qualitative feedback from tutees

In tutorials in the last 2 weeks of the first semester, students were asked for qualitative feedback on the Peer Tutoring sessions. Students were asked what they would like us to stop, start and continue, and to give us an overall impression of the tutoring. 5 of the 6 groups gave back feedback.

Areas the students would like us to stop or have less of.

1. The one area consistently identified by all students was the timing of the meetings on Monday mornings.
2. One student felt that the search for feedback was taking too much time from the tutorials.

Areas the students would like us to start

3. To cover all the subjects and not just Chemistry and Mathematics.
4. More sessions per week.
5. Calling a roll of students, making attendance mandatory, and replacing those students who fail to attend on a regular basis.
6. Allowing tutors to photocopy materials for use in tutorials.
7. One student would like more explanation of reasons why answers are incorrect as well as being shown the correct way to do things.
8. All students reported that they would like worked solutions to examination papers.

Areas the students would like us to continue

9. Peer Tutorials. Tutees liked the tutorials a lot, would like more of them, and would like them to continue into Semester II.
10. The focus on areas of difficulty.
11. The use of the Mentoring Suite for tutorials.
12. Covering examination papers and tutorial questions.

Overall impression

Tutorials are very useful. All students who consistently attended tutorials were very enthusiastic about them. The one major flaw brought out by all students is the timing of tutorials on Monday mornings.

Individual Student responses to the peer tutoring

• At the start of Semester II, the students were asked for their frank opinions about the peer tutoring and told that I would appreciate a response from every student as every comment would help us to improve the programme. The students were also told that the responses would be included in a report exactly as they were sent to me.
  

  The responses are detailed below with one page for each student who responded.

  I am writing in response to the peer-tutoring that took place last semester.

  Personally I thought it was a great success. It helped me a lot in maths and it saved me in chemistry. It was very hard to find motivation to do extra study outside my lectures but the peer-tutoring provided me with this motivation.

  Anything that was perhaps unclear during a lecture would be discussed within the peer-tutoring group. By discussing it with the rest of the members of the group you actually developed a proper understanding of it for yourself. In a way you could say that four heads are better than one - each member has different ways at looking at a thing and when they are shared everyone sees things in a new light.

  Another thing I liked about peer-tutoring was that you got to know three people properly that were doing the same course as you. Within lectures and labs you don't really get to know people that well.

  I have to say that there were some peer-tutoring sessions that I didn't bother going to but I regret it now. I realise now that the extra hour a week spent at peer-tutoring was well worth it. I hope that peer-tutoring will continue and that it will be extended to involve all class members as I believe that everyone will benefit from it.

  Remembering when I first signed up for the peer tutoring, I wondered what the program would entail, and after being selected to participate in the program, I even questioned if the experience would prove beneficial, or whether I was just wasting my time...

  Now, 13 weeks on, after first experiencing the programme first hand, I'm happy to say that I don't regret signing up, not for a minute, and I also realise how lucky I was to be picked for the programme.

  As we were in effect the Lab Rats for the programme, in that it was the first run of the idea, everyone, including the tutors were adapting to the new programme, which entailed 12 1hour sessions over the course of the 1st semester. Our tutors were 2nd year students of BT, who had a good knowledge of the 1st year material we needed to cover.

  Although the programme was originally designed to deal with Mathematics and Chemistry, we did in fact cover some Biology and Physics as well, thus encompassing all aspects of the BT course. This was helpful to everyone, as each member of the group had their needs met. Having never studied Chemistry before 3rd level, I was having some trouble getting on top of the subject, and I had hoped that the Peer Tutoring would help me to understand it a bit better.

  Needless to say, I was pleased with the outcome.

• Over the course of the program, I learned how to correctly use formulae, apply equations to problems, solve tutorial problems, and among other things I got a better understanding for the principles of the subject in general.

  Looking back, I think the Peer Tutoring as a whole was a great success, as each student got something out of the programme. I don't think it would have been quite as successful were it not for the tutors themselves, who not only helped us in every way they could, but also worked hard in their own time to prepare the material for each tutoring session. They were excellently prepared for the programme, and were perfectly suited to the job. Never did it seem that they were unhappy with the programme, or that they were forced into it... they were always willing to help, and do what was necessary to aid our learning.

  To sum up, I feel that the Peer Tutoring was an excellent bonus for all who got to participate, and I would not be disappointed to see the programme up and running again in the near future. I hope very much that when it does return, I will be lucky enough to be selected for it again.

  I would like to conclude with both congratulations and thanks not only to Roisin Noone, our tutor, who worked very hard to help us along; but also to Dr. Michael Parkinson, without whom the programme could not have been a success.

  Thank You.

  I TRIED TO DROWN MY SORROWS, BUT MY SORROWS LEARNED TO SWIM!

• Dr. Parkinson,

  My name is ***, and I am writing to you in response to your request for feed-back on last semesters peer-tutoring scheme.

  As one of the first year Biotechnology students chosen to take part in the programme, my overall few on the topic is one of enthusiasm, and optimism.

  On commencing the programme I was rather apprehensive with regards its benefits, primarily as I tend to adopt a rather private approach to my studies. In the past I found study groups to be more of a hindrance than a help with the majority of time wasted getting started, not to mention the tendency of examination panic to spread as people feed upon each others anxiety. I soon found however that with this scheme there was one fundamental and very important difference, the groups were supervised. Not by means of one member exerting his/ her authority over the others, but by acting instead as a guide for the group, helping to facilitate a calm and relaxed atmosphere. This 'gentle guidance' helped to keep the group focused, and everyone was aware of the direction and approach our sessions would follow, since everyone was involved in deciding what that approach would be. This element of equality was very important in promoting respect and trust within the group, and became the foundation from which our relationship developed.

  Supervision within the groups came from a second year biotechnology student, and this was of immense help. The fact that our tutor was of similar age to us meant that it was far easier to approach him. He too had gone through the transition from secondary school to college, and had to learn how to adapt, and change his approach to a great many things, not just study. This I think was one of the main advantages of the scheme. Knowing that it was actually possible to succeed, despite all the statistics said about drop-out rates within the course (a fact which was presented to us from day one!) really gave us reassurance, and incentive.

  The sessions themselves were a great way of reviewing work.

  While the time allowed was insufficient for any in depth study, or discussion, it did help in putting shape and direction on topics covered within the lectures. It can often be hard to decipher relevant information from lecture notes etc., but within the peer-study groups, the use of examination papers as a reference point meant we were more aware of what exactly was expected of us. Students can often become paralysed by fear, and uncertainty if they are not aware of what must be covered, or if they do not know how to approach a topic. In this way a negative attitude toward study begins to develop making it impossible to sit down and undertake the task at hand. Peer-tutoring helped foster a positive attitude in relation to study, showing us that by working together we could share our knowledge in ways that would benefit us all. And this spirit of teamwork really became evident as the semester progressed. Everyone within the group was always willing to help each other out, not just during the sessions themselves. If a problem arose with laboratory work, or notes for missed lectures there was always someone to go to for help. So in this way the group acted as a much needed support network in a campus which can appear, at times, to be dauntingly cold and impersonal.

  In conclusion I feel that the peer-tutoring scheme is an extremely worthwhile and beneficial project. It helped give me the confidence I needed to adapt to college life in a balanced way, developing, from the beginning positive study skills that I hope to call on and develop within the coming year.

Overview of the tutors comments

The two meetings with the tutors, the Journals and the presentations were a rich source of feedback about the processes involved in peer tutoring. The comments fell into a number of areas, and included recommendations to improve the tutoring. The journals in particular provided a very rich source of feedback.

• Group Dynamics
• Thing that worked
• Things that didn't work

In addition, there were a number of comments and recommendations about the course generally, about the modules in Chemistry, and about the module in Mathematics. The tutoring therefore also acted as an excellent source of feedback about the course.

Group Dynamics

• The initial team building phase is critical for success. It is critical that students feel comfortable in the group and feel that they can come up with any ideas without being negatively criticised. The development of a level of trust within the group is therefore vital.
• Good attendance is highly desirable. A group of 4 is ideal for discussions, and anything smaller will inhibit the range of discussion, and the processes of information sharing. Students who fail to attend one week but turn up the next will be behind the rest of the group and slow the whole group down leading to frustration on the part of the rest of the group.
• Good preparation from the students helps. It was very difficult to get the students to be prepared, but familiarity with the lecture notes and the tutorial questions were enormously helpful. The tutors were sometimes frustrated when the students themselves did not seem to be putting in any effort to help themselves.
• Tutors need to be able to detect the roles that students will occupy so that the group can be helped to work together effectively. Within each group, students quickly take on roles, such as leader or being submissive, and these need to be gently tackled so that all members of the group can contribute.
• With a good group dynamic the group can be independent very quickly. All groups were working almost independently of the tutor by the end of the semester, but with committed students, it was possible for the group to be working well within as little as 3 tutorial sessions.

Effectiveness of differing strategies

• Mind-mapping and brain-storming are very effective.
• Getting the whole group to work on a single A4 sheet worked very well.
• Short exam questions are excellent for recapping on problems in the tutorial.
• Long exam questions are excellent for working through problem areas.
• Providing a structure / routine to the tutorials worked very well. After a few sessions, the students just got stuck in.
• Get group to split into pairs, work on a short problem, and then get one group to explain to the other how they solved it.
• Look at problems from many different angles.

How can we do it better next time?

• Get the group together initially in a workshop with team-building exercises to build confidence and team spirit and to get students to see the benefits of problem solving through group work.
• Worked solutions to exam. Papers and tutorial problems.
• Develop tutorial groups with individuals who are at the same level of competence in the subjects of difficulty allows all the members of a group to progress at the same pace.
• Allow the students and the tutor to arrange a time for the meetings that suits themselves.
• Find ways to improve attendance.