Evaluation of the intervention in terms of the causal chain I theorised

The Intervention:
The collaborative process for development and creation of 30 second to 1 minute movie clips.

The causal chain:


The Data
















1. Pre-implementation Data for Intervention Group:



2. Post-implementation Data for Intervention Group:



3. Post-implementation Data for Comparison Group:



4. Quotes from cybersmart clips showing interpersonal skills and/or critical thinking:

PASSWORDS:

"Your password must be atleast 8 characters long. Why is that you may be wondering? Because it will be trickier and longer so it will be harder to guess."

WORKING TOGETHER:

"Being cybersmart looks like helping the person that's beside you but you can help the people from a different country and if people want help you can help them."

"...but that doesn't mean that you can't disagree."

"...looking out for people and helping them if they are stuck on something on the internet."

THE WORDS WE USE:

"Is that what you really meant?!"
"...I didn't mean it to sound that way. Now that I think about it, can you guys help me?"

"When you write something like a blog comment, you want it to make sense and come across to the reader as you mean it to."

"Think about what you have written and how it will make the other person feel."

"Write appropriate and sensible words."

"Looking at your writing before you publish your post because you might hurt other people's feelings without meaning to."

GIVING AND RECEIVING FEEDBACK:

5. Other data:

This graph shows the level of interpersonal skills and critical thinking found in the collaborative plans, cybersmart clips and blog comments. This data was collected from student work throughout the year.

A graph of individual student levels of critical thinking across collaborative plans, cybersmart clips and blog comments.

This graph shows the relationship between reading level and Interpersonal skills/ Critical thinking. There is a positive correlation between reading level and level of interpersonal skills/ critical thinking.

This graph shows the relationship between writing level and Interpersonal skills/ Critical thinking. There is a positive correlation between writing level and level of interpersonal skills/ critical thinking.

Summary of the evidence about key changes in teaching and other factors that influence student learning.

This year, I entirely changed the way I approached and thought about cybersmart learning in the classroom. Prior to this year, my cybersmart teaching approach and beliefs were very superficial, with a focus on 'right and wrong' rather than teaching skills which would allow the children to make smart decisions online. Learning activities tended to stem from the Manaiakalani Cybersmart Curriculum which I feel were highly beneficial for the children to learn certain skills.

Examples of what my previous cybersmart teaching looked like can be seen on the 2018 Team 4 Site. Teaching notes for my class from the year can be accessed at the bottom of the page.

Here is a comparison of the cybersmart topics covered between 2018 and 2019:

NB: The grey areas did not get covered, however with working through these 10 interpersonal skill/critical thinking topics from the beginning of the year, I believe it would be possible to cover all the areas on the right of this table.

I feel that going forward, a mix of what I was previously doing (before 2019) and what I am now doing would provide the best benefit for the children. Also, a stronger focus on critical thinking and interpersonal skills across the curriculum would also be extremely beneficial.

Summary of evidence about key shifts in the problem of student learning.

When you look at the quotes above which were made by year 5 children in the cybersmart clips, there is definitely evidence of interpersonal skills and critical thinking being present. However, when the data was analysed in terms of blog comments, there did not appear to be a strong improvement in level of critical thinking. In fact, most of the data showed that children were unable to show the same level of critical thinking that they demonstrated in their collaborative planning and cybersmart clips in their blog comments. Graphs comparing reading and writing level and level of interpersonal skills and critical thinking showed a trend that those who were stronger readers or writers also tended to be stronger at using interpersonal skills and critical thinking. This is where my reflection that a focus on interpersonal skills and critical thinking across the curriculum was necessary.

In the graph called Collaborative plan, Movie and blog comments, the pattern of the level of interpersonal skills/ critical thinking in blog comments (yellow) is totally opposite to the pattern shown of the level of interpersonal skills/ critical thinking in the collaborative plan (blue). A more positive trend was evident for the collaborative plans than for the blog comments. This shows that although the collaborative plan was a good support for students showing critical thinking and interpersonal skills, the skills learned did not appear to translate to a more applied setting (that is, blog commenting).

What does this mean for student learning?

- Focus on interpersonal skills/ critical thinking across the curriculum (a common language?)
- Focus more on developing blog comments which demonstrate stronger interpersonal skills and levels of critical thinking - give the children more practice at this.

Overall evaluation of the intervention in terms of the causal chain I had theorised:
- To what extent was the intervention successful in changing factors such as teaching?
- To what extent were those changes in teaching effective in changing patterns of student learning?




The blue elements of this causal chain diagram, suggesting a connection between interpersonal skills and critical thinking, were consistently reinforced throughout the year.

The pink elements of the causal chain: I taught deeper interpersonal skills and critical thinking whilst connecting with others online.

The red elements of the causal chain: I did not get to this separate part of the causal chain. This could be a focus going into next year.

My teaching of cybersmart skills changed considerably this year. My focus was on much deeper skills (based around the key competencies) which could be used both in and out of the digital world. Going into the future, I feel that I need to continue with this focus however with continuing some of the more superficial skills required when working in a digital environment.

I feel that the intervention was successful in terms of changing student engagement in learning interpersonal skills and critical thinking in a digital context. Children made attempts to use the new skills learned when communicating with others online. For example, when commenting on someone's blog, some children made a conscious effort to make a connection back to the blog post and add supporting detail. However, more regular blog commenting will be necessary to see any progress.

Reflection on my own professional learning through this inquiry cycle.

Throughout this inquiry, I have made a connection between the research and literature, my teaching and the children's learning. I believe that the reason the data (above) did not show an obvious shift in use of interpersonal skills and critical thinking included the factor of time. I did not get the intervention into full swing until about term 2. With this, a large focus was on learning the routine of planning and then creating (and completing) our movies. A regular focus on blog commenting was still missing. From next year, I aim to begin a more regular programme from the beginning of the year.

End of year findings: comparing findings of the end of year cybersmart survey across classes

The data in these slides compares to different classes - one who had an intervention of creating cybersmart clips in class and with a focus on interpersonal skills and critical thinking online; the other who did not have the intervention.

MIT19: Final Summary and Impact Story

Today is our final day at KPMG as the MIT19 group. We have had a fantastic year with many opportunities to collaborate together and be challenged outside of our comfort zones within a strong support network.

Our year has consisted of:
- A beginning of year hui where we began the design thinking process and were challenged to think outside of the box.
- 4 days at KPMG in Auckland City (1 day per term). These days were fantastic opportunities to work together, challenge each other's thinking and learn from each other.
- Attending and presenting at the 2019 EdTech Summit in Sydney. This was an amazing opportunity to share our knowledge and teach others about an area of digital teaching of our choice in an international context.
- Presenting our inquiry and tool to Principals and Leaders from across New Zealand at the annual Manaiakalani Outreach Principals and Leaders Wananga.
- Meeting and working with an amazing group of teachers from around New Zealand.

MIT19 Teachers and their Blogs:

Sandra Quick
Joanne Ryken
Amber Wing
Eugene Becconsall
Kelsey Parrant
Marc Gibson
Naomi Toland
Nicola Cameron
Santi Vega

Thank you to this amazing group of teachers who I have shared this journey with throughout the year! Thank you to the Manaiakalani Education Trust for allowing it to happen, particularly to Jenny Oxley, Dorothy Burt, Anne Sinclair, Pat Sneddon, Gerhard Vermeulen, Dave Winter and to Pt England School and principal Russell Burt.

Below is a screencast of my presentation from the Manaiakalani Outreach Principals and Leaders Wananga, along with some additions from the recent presentation for my role as a within-school teacher.

As I mention in the screencast, the final data did not show what I was hoping however, I believe that expecting miraculous shift and results after a year which has included the researching, creating AND implementing of the tool would not be realistic. Therefore, I am happy with what I have created and what it means for my thinking, mindset and forward direction to support children in becoming critical thinkers who are competent, confident and responsible in an online environment.

End of year findings: comparing findings from the pre and post cybersmart survey

In this presentation you will see a comparison of graphs between the pre and post cybersmart survey data which I collected.

Things to note:

- More children responded at the end of the year than at the beginning of the year.

- These findings were not based on standardised tests but rather analysis of data collected from in-class cybersmart learning.

Manaiakalani and Outreach Principals Wananga 2019: MIT19 Presentation

Last week I presented at the 2019 Manaiakalani and Outreach Principals Wananga as part of the Manaiakalani Innovative Teachers 2019 programme. We presented our inquiries as a Pecha Kucha (20 slides, 20 seconds per slide). Here is my presentation along with a transcript of my speech:



Slide 1: In this moment, think about the digital technologies you use which allow you to connect with others. Think beyond this room to the digital technologies our children are using in school while we are sitting here today which allow them to connect with others beyond the classroom. Connecting is interpersonal. It requires critical thinking. Can we do more to provide our children with these necessary skills?

Slide 2: We live in a digital world which is much more connected than ever before. There is therefore a need for teaching children the skills to be able to relate to others and be critical thinkers in what is a highly interpersonal digital environment. Through this, we teach through the key competencies and teach life-long skills which can be applied throughout what is and will continue to be an innovative and quick-changing environment.

Slide 3: The challenge I faced was that children living in a digital world, where key competencies are essential, require ubiquitous access to scaffolds and resources which are not currently available. Our children are using digital tools to enhance their learning. Therefore it’s important that we teach the critical thinking and problem solving essential for people to be able to make smart decisions when online.

Slide 4: We have cybersmart resources as part of the Manaiakalani Cybersmart Curriculum however two questions I asked myself were: 

How can we continue to develop the incorporation of the Manaiakalani values into cybersmart learning?

How can we get the children really excited about learning, creating and sharing in cybersmart learning?

Slide 5: Some things stood out for me when exploring cybersmart resources which are being used and shared in our schools:

1. Classes with learning sites mostly had visible cybersmart resources available directly on their site OR had a place on the site ready to be populated with these resources.

2. Most of the cybersmart resources which I have seen being used are teacher-led resources/tasks.

3. We have the teacher-led tasks and student-created DLOs/content...now more student-centred, ubiquitous and eye-capturing resources would be useful to engage our children in cybersmart learning.

Slide 6: This venn diagram from netsafe New Zealand shows the many aspects of ‘Digital Citizenship.’ You can see that Cybersafety (as they call it) is only one section of digital citizenship. Manaiakalani’s use of the word ‘cybersmart’ already emphasises that we want our children to be thoughtful when online. I like to think that Cybersmart content is actually the cybersafety and NZ Curriculum Values and Key Competencies sections of this diagram combined.

Slide 7: In fact, when we look into the NZ curriculum, these are the sorts of words that are used to describe children learning in New Zealand. These definitely describe what we would like our children to be able to do and be when learning and working in a digital environment.

Slide 8: I began the inquiry process by wanting to create a number of resources such as posters and flow diagrams to support the children, like the well known and highly useful quality blog comments poster. However, with much research and data collection, I realised that there was a need that went deeper….

Slide 9: What the children were missing was the ability to think critically online and the connection of this to interpersonal skills. This was evident through blog comments written by the children in my class which were often positive, however not always helpful, thoughtful and connected to the content they had read.

Slide 10: These preliminary findings really drew my attention to the importance of teaching children critical thinking, interpersonal skills and the connection between them. And the research backed this up! I started getting the children creating short movies to share their learning and knowledge around creating strong passwords. The children were mostly engaged. They enjoyed the competitive element which came with working in groups and creating short movie clips.

Slide 11: I then narrowed down the research to 10 areas which would teach the connection between critical thinking and interpersonal skills. The children began creating 30 second to 1 minute cybersmart clips.

Slide 12: For each cybersmart topic, the learning sequence begins with a whole class brainstorm. At this point, I also explicitly teach them important aspects to think about. After this, the children break up into small groups where they begin to collaboratively plan a script for a movie clip. They then film and edit their 30 second to 1 minute movie before sharing it on their blogs. The biggest challenge that arose was that some children had difficulty with the collaborative planning. However this observation really reinforced that this change in teaching and learning was exactly what was needed to support children in development of key competencies.

Slide 13: Much learning comes from this tool for children of all levels and abilities. It supports development of all the key competencies, critical thinking and ability to collaborate with others. It encourages creativity and gets the children excited about being cybersmart! Finally, it develops skills which are important everywhere, not just online.

Slide 14: The cybersmart clips that the children create could be shared through school news, other classes and across schools. The children are learning through concepts such as Tuakana Teina and Ako, where the children are teaching and supporting each other in their learning - they become the teachers.

Slide 15/16: You can see from these cybersmart clips that the children are happy, confident learners! These are all examples of children who worked through the process from the initial brainstorm, through to working as a group to edit and share their cybersmart clips on their blogs. As a result, they have developed their key competencies while learning to be critical thinkers and while using interpersonal skills (both in personal and in a digital environment). They have learned movie making and editing skills and have created some great resources which could be used to teach and support other children. 

Slide 17: And the children’s responses back this up too! The children have told me that they enjoy working together and choosing their own groups. They have told me that they enjoy filming and making their movies and that they like putting the movies on their blog to share with the whole world. There is not a lot that they told me they wanted to do differently. One child responded by telling me that they wanted to continue making the movies but with more props to support their creations.

Slide 18: So...is it making a difference and meeting the original challenge?

If you compare the blue and red lines, there could be a possible correlation between the ability to collaboratively plan and create cybersmart clips which include both critical thinking (CT) and interpersonal skills (IS). 

Those low in IS/CT in the collaborative planning stage are often also low in IS/CT in the cybersmart clips and blog commenting.

Slide 19: Another interesting potential correlation is that between reading and or writing level and ability to write a comment which includes critical thinking and interpersonal skills. In these graphs, each of the dots represents a child.

Slide 20: So what next for this inquiry?

Based on these findings, it is going to be important to focus in on the children who have difficulty collaboratively planning so that I can support them to develop their ability to relate to others and manage themselves so they can then develop their critical thinking skills in an interpersonal environment. I need to begin looking into the connection between reading and writing level and the ability for children to be critical thinkers in an interpersonal environment 

3. EXPLAIN: Reflections & Tweaks and reasons for them

Explain the reflections and tweaks you have made along the way and the reasons why you made these changes. Share your evidence for these decisions.

Through the initial data collection (cybersmart quiz) and research, I found the need for children to be taught interpersonal skills, critical thinking, and the connection between them, in a digital environment. These skills needed to be measurable through the data which I collected throughout the year.

1. What did not work and why?

I found that there was difficulty creating the movies in class due to background noise.

What did I do differently?
As a result, I have supported children more by utilising times when other children in our shared space are out of class and when I am on release. I have also ensured that during filming, the children are utilising external microphones rather than just the internal microphone on the camera/iPad.

2. What did not work and why?

Some groups are having trouble with the process of collaborative planning. There is a real difference in the ability of different groups to successfully plan as a group. This shows a need to focus on the key competencies of managing self, relating to others and participating and contributing. Through observation, I have noted that this inability/difficulty to plan by some groups appears to correlate with an inability/difficulty to be able to explain their understanding and demonstrate strong interpersonal skills and critical thinking when online. I created rubrics to measure the interpersonal skills and  levels of critical thinking when collaboratively planning, in their cybersmart clips and then finally when writing a blog comment. Here are the rubrics I used:

The data found can be seen in the graph below:

I also found an interesting correlation between the children's reading and writing levels and their ability to think critically and use interpersonal skills. You can see in the graphs below that those children with higher reading levels tend to be more able to write a blog comment with a high level of interpersonal skill and critical thinking AND those children with higher writing levels tend to be more able to write a blog comment with a high level of interpersonal skill and critical thinking.

What have I been doing differently?

We create a whole class brainstorm before we move off into our groups so all the children are scaffolded and explicitly taught about the topic. That way, all children have some ideas to get them started. It is really important to ensure children have been grouped in a way that ensures their group mates can support each other. I will continue to reorganise groups to ensure all children get the most out of the learning experiences.

3. What did not work and why?

It has been a challenge to get a completed cybersmart clip per group for each topic we look at. The process of completing the cybersmart clips includes a class brainstorm, group planning, rehearsing, filming, editing and then sharing of the movie on blogs. This has the potential of being a long process, particularly when children are out of class for a variety of reasons. 

What have I been doing differently?
I have been having to be give the children much longer for cybersmart learning per week than I would previously have given. Once this process gets to the editing stage, the learning moves from cybersmart learning to the technical process of completing the movie so it can be shared. I am confident that this process will become quicker as the children become more used to it.

Digital Citizenship Week

This week is digital citizenship week. With this, I have been directed towards several links which are getting people thinking about being cybersmart online.

1. Commonsense Media: Digital Citizenship Week

Is this resource focused on the positive behaviour? Mostly!

Is the link between interpersonal skills and critical thinking touched on? YES!! In fact, 'critical' is the third word in the article!

2. Commonsense Media: Kahoot

This could be a fantastic fun activity to reinforce the cybersmart learning we are doing in class however these public kahoots are more directed at teenagers.

3. NZ Cybersmart Week

This resource reinforces reasons why we are teaching children to be cybersmart from a young age. It is applicable and necessary in the wider world so it is really important the children understand it's importance.