The success of your inquiry project relies on your ability to collect evidence or data in order to determine the impact or influence your project is having. Data and evidence can take many forms, however, one of the defining features of collecting evidence in an inquiry project is that data collection is planned and systematic. Earl and Timperley (2014) suggest that evidence must be ‘fit-for-purpose, of sufficient quality to form an accurate representation of the situation being evaluated and be available when decisions are being made’ (p. 17).
There are a myriad of methods that can be used to collect evidence. Some possible methods include:
- Document analysis (this may include examples of students’ work)
- Surveys
- Focus groups and/or interviews
- Student assessment data (this could include both formative and summative assessment).
- Pre and post test data (this could be used if you are developing an intervention designed specifically to raise student achievement in a particular area, e.g. spelling or math. You could give students a pre test to ascertain their current level and then give them a similar or the same test after the intervention to determine whether there has been improvement)
- Students’ just-in-time responses using digital technologies and social media
- Classroom (or other locations) observations
When deciding on the methods you will use you need to think about the purposes of your inquiry and the constraints and opportunities in your specific context. You also need to think about how many different methods you should employ. By using multiple methods you will get richer data, which may be more reliable (if different methods indicate the same findings you can be more confident in these findings). However, you also need to make sure that you don’t try to do too much. You are a single person/researcher and if you spread yourself too thin you run the risk of not being able to analyse your data as thoroughly as you need to.
Earl and Timperley (2014) provide two examples of different types of evidence that were collected as part of two separate inquiry projects.
Just In Time Evidence
An innovative arts programme focused on increasing student engagement in their learning, partnered artists with teachers in classrooms to deliver elements of the curriculum through arts-related activities. To gather data related to the students’ engagement with their learning during these activities, the evaluators designed surveys to be completed by the students. These were delivered to the students through an “experience sampling” process (Hektner, Schmidt & Csikszentmihalyi, 2007) in which students received the survey via their cellphones at selected times during their classroom activities, coinciding with their participation in the arts-related activities and in their other classroom experiences, several times during the school year. This produced “just in time” data rather than asking students to give global statements related to their engagement, after the fact.
Some helpful online resources
The Ministry of Education has created some very helpful resources to support teachers in collecting and using data in their practice. The following links might provide you with some
TKI provides some useful guidance on how teachers can use a range of assessment methods. (http://assessment.tki.org.nz/Using-evidence-for-learning/Gathering-evidence/Topics/Assessment-methods).
The Ministry of Education has also produced some useful information to support teachers and school leaders in using evidence.
Also on the MoE and TKI websites are a number of case studies of how schools around NZ have used evidence and data to inform their teaching and learning.
Using self-report measures
It is probably fairly likely that at least some of the evidence you collect will be based on self-report measures (that is participants – i.e. students, teachers, whanau – providing their point of view). The document entitled Self-report Methods (chapter 6 in Barker, Pistrang, and Elliott's 2016 book), which has been uploaded to the platform has some useful information on how you can use self-report methods.
Some things that would be useful to think about.
Interviews: Interviews can take a variety of forms. They can be structured, meaning they follow a set of pre-determined questions and do not deviate from these. They can be unstructured, which means that you do not establish any questions before the interview. The final form of interviews is semi-structure, where you create an interview schedule but you can deviate from it to follow up any interesting points that the interviewee might raise.
Something to think about when creating an interview schedule (the questions you will be asking) is whether you ask open or closed questions. Open questions are going to provide you with potentially much deep and more detailed information.
Surveys: There are several things to consider when designing a survey. The first is the form that the questions will take. Do you want to use short answer questions, long answer questions, yes/no questions or a likert scale (where participants rank their responses on a scale, for example from 1 to 5). Each type of question will provide different types of data.
You also need to think about the length of your survey. You don’t want it to be too long, as participants will loose focus. However, at the same time you want to make sure that you collect all the information that you will need. Try to think about what information you are trying to gain from your survey and what questions you need to ask in order to get this information.
Also think about your mode of surveying. Do you want to use an online survey builder or do you want people to complete the survey by hand? The benefit of an online survey is that the programme will collate the data for you. However, the online format may make it harder for some participants to access it.
Observations: You might want to conduct observations of your students (or possibly other staff members or community groups) as part of your data collection. Before you start your observation it can often help if you think of some questions or topic areas that will help to structure or guide you and remind you of things to focus on or think about. You also need to think about how you will record your observations. Do you want to be taking notes or videoing students while you are observing them? Or do you want to do the observations and then write up your thoughts afterwards? Do you need to collect any artefacts (for example students’ work) as part of your observation?
Further Recommended Reading
Two short documents have been uploaded to the platform to provide additional reading on using data.
Easton, C. (ND). Data for self-evaluation. Slough, NFER. Retrieved from http://www.nfer.ac.uk/schools/data-for-self-evaluation-seced.pdf
Ministry of Education (2014) Using Data: Transforming Potential into Practice. Ideas into Action, 5, 1-32. Retrieved fromhttp://www.edu.gov.on.ca/eng/policyfunding/leadership/IdeasIntoActionFall11.pdf
Tasks for this week
1. Read the class notes and explore the links to videos that have been developed by the Ministry of Education on how teachers can use evidence and data (these are in the class notes).
2. Have a read through the two short documents uploaded to the platform:
- Data for self-evaluation (also available at http://www.nfer.ac.uk/schools/data-for-self-evaluation-seced.pdf)
- Using Data: Transforming Potential into Practice (also available athttp://www.edu.gov.on.ca/eng/policyfunding/leadership/IdeasIntoActionFall11.pdf)
3. Think about the methods that will best enable you to collect the evidence you need for your inquiry project.
If you are going to use interviews, focus groups or surveys it would be a good idea to start thinking about how you are going to design these. See the class notes and the reading on self-report methods for more information about this.
References
Barker, L., Pistrang, N., & Elliott, R. (2016). Research Methods in Clinical Psychology An Introduction for Students and Practitioners. West Sussex, UK: John Wiley & Sons.
Earl, L. & Timperley, H. (2014). Evaluative Thinking for Successful Educational Innovation. Report prepared for CERI/OECD. Unpublished.
No comments:
Post a Comment