End of year findings: comparing findings of the end of year cybersmart survey across classes

The data in these slides compares to different classes - one who had an intervention of creating cybersmart clips in class and with a focus on interpersonal skills and critical thinking online; the other who did not have the intervention.

MIT19: Final Summary and Impact Story

Today is our final day at KPMG as the MIT19 group. We have had a fantastic year with many opportunities to collaborate together and be challenged outside of our comfort zones within a strong support network.

Our year has consisted of:
- A beginning of year hui where we began the design thinking process and were challenged to think outside of the box.
- 4 days at KPMG in Auckland City (1 day per term). These days were fantastic opportunities to work together, challenge each other's thinking and learn from each other.
- Attending and presenting at the 2019 EdTech Summit in Sydney. This was an amazing opportunity to share our knowledge and teach others about an area of digital teaching of our choice in an international context.
- Presenting our inquiry and tool to Principals and Leaders from across New Zealand at the annual Manaiakalani Outreach Principals and Leaders Wananga.
- Meeting and working with an amazing group of teachers from around New Zealand.

MIT19 Teachers and their Blogs:

Sandra Quick
Joanne Ryken
Amber Wing
Eugene Becconsall
Kelsey Parrant
Marc Gibson
Naomi Toland
Nicola Cameron
Santi Vega

Thank you to this amazing group of teachers who I have shared this journey with throughout the year! Thank you to the Manaiakalani Education Trust for allowing it to happen, particularly to Jenny Oxley, Dorothy Burt, Anne Sinclair, Pat Sneddon, Gerhard Vermeulen, Dave Winter and to Pt England School and principal Russell Burt.

Below is a screencast of my presentation from the Manaiakalani Outreach Principals and Leaders Wananga, along with some additions from the recent presentation for my role as a within-school teacher.

As I mention in the screencast, the final data did not show what I was hoping however, I believe that expecting miraculous shift and results after a year which has included the researching, creating AND implementing of the tool would not be realistic. Therefore, I am happy with what I have created and what it means for my thinking, mindset and forward direction to support children in becoming critical thinkers who are competent, confident and responsible in an online environment.

End of year findings: comparing findings from the pre and post cybersmart survey

In this presentation you will see a comparison of graphs between the pre and post cybersmart survey data which I collected.

Things to note:

- More children responded at the end of the year than at the beginning of the year.

- These findings were not based on standardised tests but rather analysis of data collected from in-class cybersmart learning.

Manaiakalani and Outreach Principals Wananga 2019: MIT19 Presentation

Last week I presented at the 2019 Manaiakalani and Outreach Principals Wananga as part of the Manaiakalani Innovative Teachers 2019 programme. We presented our inquiries as a Pecha Kucha (20 slides, 20 seconds per slide). Here is my presentation along with a transcript of my speech:



Slide 1: In this moment, think about the digital technologies you use which allow you to connect with others. Think beyond this room to the digital technologies our children are using in school while we are sitting here today which allow them to connect with others beyond the classroom. Connecting is interpersonal. It requires critical thinking. Can we do more to provide our children with these necessary skills?

Slide 2: We live in a digital world which is much more connected than ever before. There is therefore a need for teaching children the skills to be able to relate to others and be critical thinkers in what is a highly interpersonal digital environment. Through this, we teach through the key competencies and teach life-long skills which can be applied throughout what is and will continue to be an innovative and quick-changing environment.

Slide 3: The challenge I faced was that children living in a digital world, where key competencies are essential, require ubiquitous access to scaffolds and resources which are not currently available. Our children are using digital tools to enhance their learning. Therefore it’s important that we teach the critical thinking and problem solving essential for people to be able to make smart decisions when online.

Slide 4: We have cybersmart resources as part of the Manaiakalani Cybersmart Curriculum however two questions I asked myself were: 

How can we continue to develop the incorporation of the Manaiakalani values into cybersmart learning?

How can we get the children really excited about learning, creating and sharing in cybersmart learning?

Slide 5: Some things stood out for me when exploring cybersmart resources which are being used and shared in our schools:

1. Classes with learning sites mostly had visible cybersmart resources available directly on their site OR had a place on the site ready to be populated with these resources.

2. Most of the cybersmart resources which I have seen being used are teacher-led resources/tasks.

3. We have the teacher-led tasks and student-created DLOs/content...now more student-centred, ubiquitous and eye-capturing resources would be useful to engage our children in cybersmart learning.

Slide 6: This venn diagram from netsafe New Zealand shows the many aspects of ‘Digital Citizenship.’ You can see that Cybersafety (as they call it) is only one section of digital citizenship. Manaiakalani’s use of the word ‘cybersmart’ already emphasises that we want our children to be thoughtful when online. I like to think that Cybersmart content is actually the cybersafety and NZ Curriculum Values and Key Competencies sections of this diagram combined.

Slide 7: In fact, when we look into the NZ curriculum, these are the sorts of words that are used to describe children learning in New Zealand. These definitely describe what we would like our children to be able to do and be when learning and working in a digital environment.

Slide 8: I began the inquiry process by wanting to create a number of resources such as posters and flow diagrams to support the children, like the well known and highly useful quality blog comments poster. However, with much research and data collection, I realised that there was a need that went deeper….

Slide 9: What the children were missing was the ability to think critically online and the connection of this to interpersonal skills. This was evident through blog comments written by the children in my class which were often positive, however not always helpful, thoughtful and connected to the content they had read.

Slide 10: These preliminary findings really drew my attention to the importance of teaching children critical thinking, interpersonal skills and the connection between them. And the research backed this up! I started getting the children creating short movies to share their learning and knowledge around creating strong passwords. The children were mostly engaged. They enjoyed the competitive element which came with working in groups and creating short movie clips.

Slide 11: I then narrowed down the research to 10 areas which would teach the connection between critical thinking and interpersonal skills. The children began creating 30 second to 1 minute cybersmart clips.

Slide 12: For each cybersmart topic, the learning sequence begins with a whole class brainstorm. At this point, I also explicitly teach them important aspects to think about. After this, the children break up into small groups where they begin to collaboratively plan a script for a movie clip. They then film and edit their 30 second to 1 minute movie before sharing it on their blogs. The biggest challenge that arose was that some children had difficulty with the collaborative planning. However this observation really reinforced that this change in teaching and learning was exactly what was needed to support children in development of key competencies.

Slide 13: Much learning comes from this tool for children of all levels and abilities. It supports development of all the key competencies, critical thinking and ability to collaborate with others. It encourages creativity and gets the children excited about being cybersmart! Finally, it develops skills which are important everywhere, not just online.

Slide 14: The cybersmart clips that the children create could be shared through school news, other classes and across schools. The children are learning through concepts such as Tuakana Teina and Ako, where the children are teaching and supporting each other in their learning - they become the teachers.

Slide 15/16: You can see from these cybersmart clips that the children are happy, confident learners! These are all examples of children who worked through the process from the initial brainstorm, through to working as a group to edit and share their cybersmart clips on their blogs. As a result, they have developed their key competencies while learning to be critical thinkers and while using interpersonal skills (both in personal and in a digital environment). They have learned movie making and editing skills and have created some great resources which could be used to teach and support other children. 

Slide 17: And the children’s responses back this up too! The children have told me that they enjoy working together and choosing their own groups. They have told me that they enjoy filming and making their movies and that they like putting the movies on their blog to share with the whole world. There is not a lot that they told me they wanted to do differently. One child responded by telling me that they wanted to continue making the movies but with more props to support their creations.

Slide 18: So...is it making a difference and meeting the original challenge?

If you compare the blue and red lines, there could be a possible correlation between the ability to collaboratively plan and create cybersmart clips which include both critical thinking (CT) and interpersonal skills (IS). 

Those low in IS/CT in the collaborative planning stage are often also low in IS/CT in the cybersmart clips and blog commenting.

Slide 19: Another interesting potential correlation is that between reading and or writing level and ability to write a comment which includes critical thinking and interpersonal skills. In these graphs, each of the dots represents a child.

Slide 20: So what next for this inquiry?

Based on these findings, it is going to be important to focus in on the children who have difficulty collaboratively planning so that I can support them to develop their ability to relate to others and manage themselves so they can then develop their critical thinking skills in an interpersonal environment. I need to begin looking into the connection between reading and writing level and the ability for children to be critical thinkers in an interpersonal environment 

3. EXPLAIN: Reflections & Tweaks and reasons for them

Explain the reflections and tweaks you have made along the way and the reasons why you made these changes. Share your evidence for these decisions.

Through the initial data collection (cybersmart quiz) and research, I found the need for children to be taught interpersonal skills, critical thinking, and the connection between them, in a digital environment. These skills needed to be measurable through the data which I collected throughout the year.

1. What did not work and why?

I found that there was difficulty creating the movies in class due to background noise.

What did I do differently?
As a result, I have supported children more by utilising times when other children in our shared space are out of class and when I am on release. I have also ensured that during filming, the children are utilising external microphones rather than just the internal microphone on the camera/iPad.

2. What did not work and why?

Some groups are having trouble with the process of collaborative planning. There is a real difference in the ability of different groups to successfully plan as a group. This shows a need to focus on the key competencies of managing self, relating to others and participating and contributing. Through observation, I have noted that this inability/difficulty to plan by some groups appears to correlate with an inability/difficulty to be able to explain their understanding and demonstrate strong interpersonal skills and critical thinking when online. I created rubrics to measure the interpersonal skills and  levels of critical thinking when collaboratively planning, in their cybersmart clips and then finally when writing a blog comment. Here are the rubrics I used:

The data found can be seen in the graph below:

I also found an interesting correlation between the children's reading and writing levels and their ability to think critically and use interpersonal skills. You can see in the graphs below that those children with higher reading levels tend to be more able to write a blog comment with a high level of interpersonal skill and critical thinking AND those children with higher writing levels tend to be more able to write a blog comment with a high level of interpersonal skill and critical thinking.

What have I been doing differently?

We create a whole class brainstorm before we move off into our groups so all the children are scaffolded and explicitly taught about the topic. That way, all children have some ideas to get them started. It is really important to ensure children have been grouped in a way that ensures their group mates can support each other. I will continue to reorganise groups to ensure all children get the most out of the learning experiences.

3. What did not work and why?

It has been a challenge to get a completed cybersmart clip per group for each topic we look at. The process of completing the cybersmart clips includes a class brainstorm, group planning, rehearsing, filming, editing and then sharing of the movie on blogs. This has the potential of being a long process, particularly when children are out of class for a variety of reasons. 

What have I been doing differently?
I have been having to be give the children much longer for cybersmart learning per week than I would previously have given. Once this process gets to the editing stage, the learning moves from cybersmart learning to the technical process of completing the movie so it can be shared. I am confident that this process will become quicker as the children become more used to it.

Digital Citizenship Week

This week is digital citizenship week. With this, I have been directed towards several links which are getting people thinking about being cybersmart online.

1. Commonsense Media: Digital Citizenship Week

Is this resource focused on the positive behaviour? Mostly!

Is the link between interpersonal skills and critical thinking touched on? YES!! In fact, 'critical' is the third word in the article!

2. Commonsense Media: Kahoot

This could be a fantastic fun activity to reinforce the cybersmart learning we are doing in class however these public kahoots are more directed at teenagers.

3. NZ Cybersmart Week

This resource reinforces reasons why we are teaching children to be cybersmart from a young age. It is applicable and necessary in the wider world so it is really important the children understand it's importance.

2. DESCRIBE: Evidence about the effects of my changed practice and intervention on desired learner outcomes.

Link to overview blogpost​

The Tool: Cybersmart Clips

Describe the evidence you have so far about the effects of your changed practices/intervention on desired learner outcomes and how you summarised and recorded these.

What methods have you used to collect information?​

- Start of Year/ End of Year Cybersmart Assessment


(as seen in previous blog post)

- Student collaborative planning/brainstorming docs

(See example on previous blog post)

- Student movie clips

- Student voice

In the last blog post, you will see some word walls which show responses from two other questions asked.

- Regular collection of blog comments/ replies to blog comments

I am currently analysing blog comments by comparing blog comments at the beginning of the year to blog comments now. This qualitative data is difficult to analyse and I am having to think of the best way to measure progress. I am wanting to measure progress in interpersonal skills and critical thinking in blog posts. On an initial look, it is clear that many children are writing more detailed and thoughtful blog comments compared to at the beginning of the year.

How have you been systematic in that collection?​

Each week, I have been collecting and updating data in a spreadsheet which is organised in the above categories. I am working on collating this data into graphs and noticing trends and patterns mainly around interpersonal skills and critical thinking.

What does the data/information tell you?​

Through the process of making movies in learning cybersmart skills and collection of data, I have realised that the creation of the movies and collaborative script-making is just as important as the sharing of the movies to a wider audience. This is where this method of teaching children to be cybersmart is effective in integrating the teaching of the key competencies at the same time. In particular, children are developing their ability to think (about the cybersmart topic given to them), relate to others (through the creation of the script in preparation for movie-making), managing self (to work through the process of creating a movie, from storyboarding to sharing of the movie) and participating and contributing (in creating the script but also in being confident to present/act in the movie or to direct and be responsible for the filming). Overarching the development of these key competencies (NZ Curriculum) is the final key competency of understanding symbols, languages and texts. This overarches the others in this process as the children are learning skills in being able to make sense of the online world and respond responsibly and appropriately.

Data from student voice tells me really clearly that the children are really enjoying using movie-making as the medium to learn how to be cybersmart. I showed this in an answer garden. A higher frequency of the same response is represented by a larger word.

WHAT DO YOU LIKE ABOUT CYBERSMART LEARNING?

I have also been collecting comments that children in my class are writing on other children's blogs. A continuing step in my inquiry is to collate these in a way that will show any development in the quality of the blog comments over time.

I also need to begin collating and making sense of the data which shows development of interpersonal skills and critical thinking through the children's collaborative scripts and movie clips.

What are you going to do with the data/information in terms of ‘tweaking’ your intervention?​

Currently, I am really satisfied with how the intervention/ tool is working. On the whole, the children are engaged, they are learning valuable skills in film making, being given opportunities to develop their critical thinking and interpersonal skills, and being able to share their learning with the wider world through the school news and their public blogs! How cool is that!?

As with anything in teaching though, there is always 'tweaking' that can be done to make it better! At the moment, my focus needs to be on continuing doing what we are doing and now beginning to make sense of what the data shows.

So...what am I going to do with the data to make sense of it?

- I have already analysed the start-of-year cybersmart assessment data (see blog post). This will be followed up later in the year (early in term 4) by children completing the same cybersmart assessment and seeing how the data changed over time.

- Student collaborative brainstorms/planning docs: I need to analyse these for evidence of interpersonal skills and critical thinking. As a result of a range of research (seen in previous blog posts) I plan to make a tally of the following when reading through the docs and watching the cybersmart clips:

• Collaborative plannning (all team members given a role and contributing to ideas; efficient planning in group)
• Communication between group members to produce a coherent and connected script (i.e. not different ideas per child but rather, a script that flows from one child to the next)
• Creativity (have they thought outside the box or just used class brainstorm ideas and copied it?)
• Critical thinking (thought deeply about how to portray the message in an effective way)
• Positive role play
• Perspective taking (each 'character' shares their own ideas)
• Empathy/interpersonal skills (evident through role play - rather than just saying lines without connection to others in the group; thinking about how others feel and how actions can affect others)
• Listening to each others ideas
• Questions asked
• Challenging each others ideas
• Reflecting/ making changes
• Summarising the main ideas