MirandaNet & TAG Learning Ltd.

[divide margin_top=”10″ margin_bottom=”10″ color=”#a0a0a0″]

Abstract

This evaluation has been produced by MirandaNet for TAGLearning. The aim of the evaluation was to gauge the views of teacher and pupil users of its MAPS software, and the ways in which it has impacted on their teaching, learning and work management. This research collected data from the MAPS user base provided by TAG Learning to investigate the perceptions of teachers and students of the impact of MAPS on learning, teaching and workflow. Data was collected through email contact, online questionnaires and telephone interviews.

Author: John Cuthell

Publication Date: 2007

[s2If !is_user_logged_in()]More… To see the complete Case Study, please Login or Join.[/s2If]

[divide margin_top=”10″ margin_bottom=”10″ color=”#a0a0a0″]

[s2If is_user_logged_in()]

[two_third_last]

Overview

This research collected data from the MAPS user base provided by TAG Learning to investigate the perceptions of teachers and students of the impact of MAPS on learning, teaching and workflow.

Data was collected through email contact, online questionnaires and telephone interviews.

Terms of Reference

This report uses evidence collected from the data to determine the perceptions of the two key bodies of stakeholders: teachers and students.

Key benefits for both of these groups from the use of MAPS are analysed in terms of teaching, learning, workflow, and home-school links.

Of particular interests are the ways in which teachers maximise the impact of MAPS, whether through CPD, resources, ways in which they share any newly developed ‘best practice’, the impact of training versus non-training and transition across Key Stages.

The evaluation includes a number of comments from teachers, and extensive comments from pupils, on the impact of the MAPS on teaching, and for pupils on how it has changed their lessons.

The contribution of MAPS to the Personalised Learning agenda is briefly examined.

Executive summary

During June and July 2007 a number of schools across England were surveyed to investigate the ways in which they use TAG MAPS software, and the ways in which it has impacted on their teaching, learning and work management.

Key Stage Coverage

All Key Stages were represented in participant schools: the majority, however, were Key Stages 3 & 4.

Curriculum subjects for which MAPS is used

All the schools used MAPS for ICT. One school in the sample used MAPS for English at Key Stage 1: another school used it for Geography and Music at Key Stage 3. Other schools were working to extend the use of MAPS into other curriculum areas. Feedback from pupils suggested that they used it to store work in a number of subjects, whether or not their teachers used it.

MAPS effectiveness: Teacher ratings.

Teachers were asked to rate the effectiveness of MAPS for a range of tasks: managing assignments, sharing resources with colleagues, tracking pupil progress, whole-school moderation and communicating with pupils. 70% of teachers rated it as being invaluable or valuable for managing assignments; 80% as invaluable or valuable for tracking pupil progress; 60% as valuable for communicating with pupils.

Range of tasks for which MAPS is used

The majority of teachers (80%) use the program for reviewing, commenting on and evaluating pupil work

ICT and KS3 Strategy tasks

Teachers were asked about the contribution of the ICT and Key Stage 3 Strategy tasks included in MAPS. 70% of the teachers rated these as having an impact on their teaching and pupil learning. Three out of ten teachers rates the resources as valuable.

Ease of use

70% of teachers said that it took less than half a day to become familiar with and use MAPS: not only that, 70% were able to use it without any training. They found the program intuitive. However, the manual, online tutorials, training, support from TAG and the online help topics all contributed to an understanding of the system.

Impact on teaching

TAG MAPS contributes to their management of the ICT curriculum and is rated as valuable or invaluable. Most useful was the ability to allocate tasks to individuals and integrate resources within the work.

Maps has improved submission rates for work with a significant number of students and eliminated ‘lost’ work.

The overall impact has been in terms of raised standards: students have a snapshot of where they are, and their parents can review progress.

It has changed our entire structure and method of teaching ICT. This has had a knock on effect, where KS3 results have increased significantly, to GCSE where results have almost doubled in the years we have been using MAPS.

Implementing MAPS

As far as MAPS goes I cannot praise the facility or the support staff too highly. Whenever we’ve asked for help the response has been almost immediate. When we’ve suggested a facility we would like added it’s been done. I think it should be a compulsory purchase for all schools, particularly with DIDA and e-portfolios.

Impact on learning: pupil perspectives

Extent of use

The majority of pupils used MAPS once a week as part of their ICT lesson: a minority use it more frequently for other subjects.

36% of pupils use MAPS to send messages to their teachers about their work.

69% of teachers use MAPS to send messages to pupils about their work.

35% of pupils use MAPS at home.

40% assess their own work when they upload it?.

53% do online quizzes or tests.

Impact on learning

A significant number of pupils consider that MAPS helps them to do work that was described as Hard, or Difficult. Others say that it helped them to do work that was challenging or complicated.

Many pupils say that MAPS helps them to hand in their work on time, and that it helps them to move it from home to school; work I need to send to school.

Pupils say that with the program there was no forgetting important pieces of paper. One student wrote that MAPS helped him to do work that was cool.

Pupils say that using MAPS makes them more organised: they feel more in charge of their own work. They have the ability to improve their own work, and engage in personal dialogue with their teachers.

“It has made me take more time in my work because it is more fun.

Changing approaches to work

Pupils enjoy using MAPS: they regard it as ‘fun and easy’. There is a strong sense of personal involvement in using the program because of the element of personalisation it affords them. The work itself is personalised, and teacher comments on their work increase the sense of pupil involvement.

There is a strong desire to use MAPS across the curriculum in what many pupils regard as a virtual learning environment.

“I think that MAPS is really good and I think we should be able to use it more through out school.

Data analysis

Between June 15 and July 15 some 60 individuals whose schools were subscribers to MAPS were invited to provide feedback on their uses of the program. Online questionnaires for both teachers and colleagues were hosted on the MirandaNet website. There were fourteen respondents: ten teachers completed the online questionnaire: four teachers provided feedback by telephone or email: all were ICT teachers or co-ordinators. Of non-respondents, 31 were from Sheffield Local Authority and 8 from Worcestershire. There would appear to be issues with schools’ uptake of learning platforms in each authority.

In real terms, therefore, the response rate was 14 teachers from 20 schools: 70%

Meaningful responses were received from 298 pupils in Years 7, 8 & 9 in various schools.

School background

Virtual Learning Environment or Learning Platform

More than half of the schools had, and used, a learning platform in addition to MAPS. 40% of the schools used MAPS as their learning platform. There was no uniformity of provision: Microsoft Learning Gateway, Worcestershire Virtual Workspace, My Internet, Kaleidos, Assimilate and Moodle were all used by schools.

The most commonly used learning platform in the schools surveyed, therefore, was MAPS.

Key Stages in which MAPS is used

All Key Stages were represented in participant schools: the majority, however, were Key Stages 3 & 4.

KS1	KS2	KS3	KS4	Post-16
1	2	8	6	2

Curriculum subjects for which MAPS is used

All the schools used MAPS for ICT. One school also used MAPS for English at Key Stage 1: another school used it for Geography and Music at Key Stage 3. In a range of schools, however, there were attempts to involve curriculum areas other than ICT in the use of MAPS. Feedback from pupils suggested that they used it to store work in a number of subjects, whether or not their teachers used it.

Implementing the program: MAPS and ICT

How effective is MAPS? Teacher perceptions.

Teachers were asked to rate the effectiveness of MAPS on a five-point Lickert-type scale for a range of tasks: managing assignments, sharing resources with colleagues, tracking pupil progress, whole-school moderation and communicating with pupils. For all of these tasks MAPS had made an impact; in many cases a significant impact. 70% of teachers rated it as being invaluable or valuable for managing assignments; 80% as invaluable or valuable for tracking pupil progress; 60% as valuable for communicating with pupils.

	Invaluable	Valuable	Of some impact	Of little impact	Of no impact
For managing assignments	30%	40%	20%	10%	0
Sharing resources with colleagues	20%	30%	30%	2%	0
Tracking pupil progress	20%	60%	20%	0	0
Whole-school moderation	10%	10%	30%	20%	30%
Communicating with pupils	0	60%	20%	20%	0

Range of tasks for which MAPS is used

The majority of teachers (80%) use the program for reviewing, commenting on and evaluating pupil work

I use MAPS for:
Creating curriculum links	Uploading images	Storing reference documents	Distributing work templates	Managing website links	Planning	Reviewing, commenting on and evaluating work	Other
30%	40%	50%	50%	10%	50%	80%	Setting & assessment

ICT and KS3 Strategy tasks

70% of teachers rated these as ‘valuable or of some impact.

The ICT and KS3 Strategy tasks are:	Invaluable	Valuable	Of some impact	Of little impact	Of no impact
The ICT and KS3 Strategy tasks are:	0	30%	40%	30%	0

No teachers reported use of the discussion forums, although some pupils used them.

No response

I use the discussion forums for because

How long did it take to learn how to use MAPS?	An hour or less	Less than half a day	About a day	More than a day	I still haven’t learned to use it properly.
How long did it take to learn how to use MAPS?	30%	40%	20%	10%	0

70% of teachers said that it took less than half a day to become familiar with and use MAPS. 70% were able to use it without any training.

How did you learn how to use and apply MAPS?:
It’s an intuitive system that tells you how to use it	I worked it out myself	Colleagues and I set up a working group	We had a demonstration in a CPD session	It’s part of the whole-school CPD programme
20%	50%	0	30%	0

A significant number of teachers were able to implement and use the software quickly – in less than half a day – and were self-taught. They found the program intuitive. However, the manual, online tutorials, training, support from TAG and the online help topics all contributed to an understanding of the system.

Which of the following resources did you use to assist your understanding of MAPS?
Training	Manual	Online tutorials	TAG Support	Online help topics
50%	50%	50%	30%	20%

Almost all of the users found that it contributed to their management of the ICT curriculum, Eighty percent said that it was valuable or invaluable. The two facilities that were rated as most useful were allocating tasks to individuals (80%) and integrating resources (20%). Data output to (and input from) SIMS proved the most problematic for a number of schools. It would seem to be that control of SIMS lies outside the responsibility of the department that implements MAPS, and that data interchange is less than seamless.

Now that I am using it I find it	Invaluable	Valuable	Of some impact	Of little impact	Of no impact
Now that I am using it I find it	40%	40%	10%	10%	0

The tool I find most useful is	Automated marking	Quiz tools	Allocating tasks to individuals	Integrating resources	Data output to SIMS
The tool I find most useful is	0	0	80%	20%

Overall impact

The major benefit has been the reduction in pieces of paper. Nothing gets lost and everything is accessible in school and at home. Pupils now get more or less instant feedback on their work via the comments in MAPS.

Mark Harrison, Sinfin Community School, Derby

Maps has improved submission rates for work with a significant number of students and eliminated ‘lost’ work. Students are more at home with electronic submission than with hard copy and expect support to be available on line 24/7.

George Rouse, King Edward IV Camp Hill Boys School, Birmingham

The overall impact has been in terms of raised standards: students have a snapshot of where they are, and their parents can review progress. MAPS helps with setting.

I feel that MAPS has dramatically reduced the amount of paper work that we have to keep and made lesson planning a lot easier. We are able to recycle materials and improve them every year. The students enjoy using MAPS and find it convenient to be able to access to their work at home.

Chris Love, Grammar School for Girls, Wilmington

Most use is made of the uploading feature. The time taken to upload resources is minimal, but marking work ‘on screen’ has proved unpopular with the staff, so little has been done to develop our use of MAPS.

It is popular with staff and students. As a result it provides a way of ensuring improved consistency in formative and summative assessment tasks.

Adam King, Newlands Girls School

I am able to consolidate, amend and review allocated tasks. Pupils are able to see progression through topics.

MAPS is a good motivator for pupils because it’s their personal learning space, and KS1 children have said they feel ‘grown up’ sending their work to the Internet.

Feedback from telephone interviews

Although the school uses Kaleidos and a portal, MAPS is currently being used as the school learning platform because it is robust and stable. When it has been proved that Kaleidos (and the portal) are stable the school plan is then for pupil work to be transferred to the RM product. It is anticipated that skills developed by pupils and staff can be transferred to the new system. A perceived limitation of MAPS is in the way in which pupils try to re-submit work. The teacher feels that it is important to have a series of drafts available for formative assessment. (GSGW)

One training day was used to introduce all of the school staff to MAPS, and it is used as a learning platform in conjunction with My Internet. This school commented on the need for pupils to be able to store various drafts of work, together with the ability to delete a whole set of assessments through the management system of the task that had been set proved inappropriate. This school thought that the product was extremely good value for money. (KE IV CH)

One user commented on the excellent support and service from the company, and could not praise them enough. Integration with data from SIMS had proved problematic, however. (OL&SB)

A school in Derby had suffered a serious fire the previous year, when the school had been burned down. It was now in temporary accommodation, and the teacher responsible for ICT said that MAPS had been a lifeline and meant that what was a crisis had not become a disaster. The program was very stable, and the company provided good support. The messaging system had proved very effective, especially with feedback from pupils. (SCS)

All of the schools that responded ran post-school sessions for those pupils without ICT or broadband access at home. All of the teachers commented on the program’s value for money.

Feedback from a former user

The school’s Internet access goes through a caching service provided by NTL (Hertfordshire Grid).

Because most of MAPS web pages are dynamic the caching system could not provide them fast enough. When lots of students (i.e. a whole class) request a dynamic page the system becomes unusable and this has a big impact on the internet connection speed across the school. This was even evident when a TAG representative came to visit the school and look at the problem.

MAPS does not work with the SIMS database directly. New students often took a while to add to the system.

Many of the SIMS codes were not understood by MAPS, which meant that from September only a few subjects worked. Making the system fully integrated by all subjects was difficult.

This was an expensive mistake for one year because the system could not be used by members of staff across the school.

We were very annoyed that MAPS wanted to charge the school £500 for providing data which belongs to us at the end of the contract. The school paid £3000 for a service that was barely usable in the first place.

It would be a very good bit of kit, if it worked effectively for us.

Mr J A Dent Head of Information Technology Nicholas Breakspear School, Colney Heath Lane, St.Albans, Herts, AL4 OTT Tel: 01727 860079 / Fax: 01727 848912 www.nicholasbreakspearschool.co.uk Email: dentj@nicholasbreakspear.herts.sch.uk

[/two_third_last]

[divide margin_top=”10″ margin_bottom=”10″ color=”#a0a0a0″]