A case study on feedback

Pyke, J. G., & Sherlock, J. J. (2010). A closer look at instructor-student feedback online: A case study analysis of the types and frequency.  Merlot Journal of Online Teaching and Learning, 6(1), 110-121. Retrieved on 2 April, 2012, from http://jolt.merlot.org/vol6no1/pyke_0310.htm

  

This article describes the findings of a case study examining the types and frequency of feedback in an online university course (encompassing 1744 recorded instructor-student feedback interactions).  Students engaged in individual and group activities, asynchronous online discussions, synchronous online chat sessions, instructor-learner emailing, and were provided with assignment feedback online. The course was designed by a group of instructors, although delivered by a single instructor, and based on a constructivist approach.

In a distance learning context is used to help a student engage with learning and understand content.  Feedback is often individualised and ‘the instructor’s feedback helps to personalize the learning and assist the student in learning.’ (p.111)

The authors divided the feedback provided by the instructor into three types; corrective, motivational, and technology.  The authors also divided feedback into individualised or group feedback and when it occurred (during the semester).

Corrective feedback was described a feedback which focuses on the content of learning activities and aims to reinforce or correct learner performance through things such as acknowledging the correct actions or answers of learners, providing guidance to improve learner understanding, providing correct answers or explanations, or suggesting a leaner attempt an activity again.

Motivational feedback was described as feedback used to help motivate a learner to participate in learning and perform.  This feedback incentivise a learner to meet learning goals and face challenges, improving learner resilience, assist a learner to continue with their efforts, or help make learning personally relevant.  It can increase a learner’s motivation and their sense of control and ownership in learning.   

Technology feedback was described as dealing with technological support issues to do with software, hardware, and web-based learning.

The authors found that corrective feedback made up 69% of course feedback, 21% was motivational feedback and 10% was technology feedback.  Additional findings were:  

• Corrective feedback was more often provided to groups than individuals (2:1)
• Corrective feedback increased towards the end of a course 
• Motivational feedback was more often provided to individuals than groups (3:1)
• Motivational feedback was more likely to be provided through online synchronous chat sessions as this provided immediate feedback
• Motivational and technology feedback were higher at the beginning of a course
• Technology feedback was requested by individuals more frequently than teams (2:1)
• Email was the most frequent method of requesting and providing technology support
• Technology feedback, where advice is outside of an instructor’s area of expertise, may require much more time to prepare than is indicated by the proportion of feedback which results. 

While the authors were not looking to pass comment on whether these proportions or other findings reflected appropriate or good practice, they were interested in how instructors would view the results.  

As mentioned earlier, the course was based on a constructivist approach.  The instructor was surprised that such a high proportion of the feedback she provided was corrective, as she thought that ‘motivational feedback, prominent in the constructivist paradigm, would have played a much greater role than corrective feedback’ (p.118) in her teaching.

The constructivist paradigm emphasises the role of feedback as an aid to increase learner’s self-efficacy and to help a learner gain a sense of control over his learning (Jonassen, 1991). Corrective feedback is more aligned with a behaviourist approach where reinforcing correct answers and correcting errors is central to the task of information processing.

The results of this study are too small to be generalisable, however, it does suggest that instructors might want to gather more data on the type of feedback they provide as this may not be consistent with their philosophy of teaching or reflect their beliefs or perceptions about the type and frequency of feedback they provide.

As noted in my last blog, Tyler-Smith (2006) has identified the multiple learning tasks that a first-time eLearner must deal with in the early stages of an online course and which could lead to cognitive overload and drop-out or withdrawal rates in the first weeks of a course.  These tasks apply at any level of study, the tasks are relative to a learners ‘newness’ to online learning rather than their history of learning in other media.

Of the five learning tasks identified by Tyler-Smith, only one task is focused on content, negotiating the leaning content, and might warrant corrective feedback (depending on your teaching philosophy!).  The other four tasks require instructor support more aligned with motivation and technology feedback.  While this is consistent with the Pyke and Sherlock finding that the start of the course has more motivational and technology feedback than later on, the overall proportion of corrective feedback means it is likely that this played a dominating role throughout the course. (The authors do not mention whether any of the learners on the course were new to distance learning so this is not included in the analysis they provide.)

 

Jonassen, D. H. (1991).  Objectivism versus constructivism: Do we need a new philosophical paradigm.  Educational Technology Research and Development, 39(3), 4-14.

Attrition and first time eLearners

In my current work, it is likely that the Web 2.0 resources I develop will be used by distance learners, with little or no face to face with tutor, although a workplace mentor may be involved.  This means I am particularly interested in how to keep distance students engaged in online learning, and how feedback can be used to support engagement and learning.

I was really excited to find this 2006 article by Keith Tyler-Smith on early attrition among first time elearners. http://jolt.merlot.org/vol2no2/tyler-smith.htm .  At the time, Tyler-Smith was leading the elearning programme at the Christchurch Polytechnic Institute of Technology.  This article focuses on “mature adult learners undertaking part-time education by distance eLearning course for the first time” (p 73).  Perfect.

He reviews research carried out in the area of eLearner attrition and concludes that there are many and complex reasons for course attrition, and no simple solutions.  Sociological, psychological, technical and cognitive factors all play a part in attrition.  For example, the degree to which an individual perceives themselves to be influenced by external events affects learner performance and persistence (or attrition).  A learner less influenced by external events, such as technical problems, work pressure or family needs, is more likely to persist in a eLearning course. 

Tyler-Smith has developed a “conceptual model which identifies the multiple learning tasks that a first-time eLearner must deal with immediately and simultaneously on embarking on an eLearning course” (p79).  These multiple tasks have the potential to overwhelm a new eLearner. Tyler-Smith describes this phenomenon as eLearners as experiencing cognitive overload in the early stages of an online course.  This contributes to high drop-out or withdrawal rates in the first weeks of a course.

In summary, the tasks identified in the conceptual model are:

1.       Negotiating the technology: using a new range of computing technologies in new ways.  Stress can be caused by using unfamiliar technology, often with little technical support immediately available. 

2.       Negotiating the Learner Management System (LMS) interface: developing a strong mental model of the structure and content of the LMS which often requires the unfamiliar approach of going ‘down’ through multiple layers within a website.  A learner can become frustrated or unconfident due to navigation confusion.

3.       Negotiating the learning content: engaging with the learning materials, readings, activities and assessments in the learning programme.  Stress can be caused by the unfamiliar content and the challenge of becoming a learner again.

4.       Becoming an eLearner: changing a learner’s mental model of what it is to be a learner in a formal education setting from being teacher-led to learning being self-directed, self-motivatied and physically isolated (from a tutor).

5.       Negotiating CMC interaction: learning to interact with peers through computer mediated communication (CMC). 

As a relatively new eLearner, as I read this article it felt like the author was describing my personal experience!

Tyler-Smith has identified a number of strategies to help reduce eLearner attrition. 

·         A face to face workshop at the start of an online distance course provides an opportunity for learners to meet each other and be introduced to the technology.  Where a workshop is not possible, a guide to getting started could be provided.

·         Using simple online activities in the initial stages of a course and scaffolding into more complex tasks over time. 

·         Using introductory and meaningful discussion board activities to create early engagement and reduce the isolation of learners.

·         Explaining to learners the complexities of the learning tasks at the start can help learners set realistic expectations.

·         Actively supporting and following-up with learners who are struggling or not participating.

·         Keeping the initial content load light and interesting to help learners build confidence and technical capability.

Once again, these were things I recognised in my current eLearning experience.  This was a helpful article that I look forward to discussing with other eLearners.

As an aside, I googled Keith to see if he was still at CPIT and found that in 2010 he set up an eLearning development company Learning@Work Ltd. For more details visit:  http://community.articulate.com/members/KeithTyler_2D00_Smith/default.aspx

The Khan Academy

I have been following the Khan Academy and the work of Salman Khan for a few years now.  He has created over 2200 maths videos and thousands more activities and games to help young people and adults to learn about maths.  He stresses that the Academy is not providing a curriculum but rather, learning resources that allow people to scaffold their learning based on a knowledge map of mathematics.  The videos are free to watch and you don’t have to register to watch them.  Khan thought that the videos might appeal to people who were motivated to learn about math, were looking for supplementary learning, or were homeschooled.  Instead the videos have had widespread use by people who have struggled to learn maths while at school.

Initially I thought of these videos as another way of delivering a lecture and I didn’t see them as particularly interactive, however, I was wrong.  The videos are accompanied by an online forum where people check their understanding of what they have learned and ask questions.  Responses are provided by other learners or watchers in real time.   There are also lots of adaptive activities, games and formative assessment tools that learners can access. In his 2011 presentation for the TED forum http://www.youtube.com/watch?v=gM95HHI4gLk , Kahn describes learners using these videos in ways that put the control in their hands, making this a clear example of Web 2.0 technology.  For example with the videos, he talks about learners pausing and replaying concepts to give them time to review and check anything they don’t understand, going back to earlier videos to review concepts without the embarrassment of asking their tutor, avoiding interruption by the tutor while a student is still concentrating on understanding a new idea, and letting students work in at a time and location that is comfortable - at their own pace and without time limits. 

Khan describes what teachers have called “flipping the classroom” where students watch the videos for homework and then practice using what they have learned in the classroom. This supports peer interaction and helps position a tutor (in a classroom) as providing individualised coaching for students rather than as a ‘one size fits all’ presenter.

Khan has found that the videos have been used by children and adults. 

Having reviewed some of the videos, I think that while they are strong on maths content, they are fairly weak on providing context, or how the maths concepts being taught might apply in the real world or be transferred.  Context is particularly important for adult learners who are often learning maths for wider reasons, such as managing a budget or training for a qualification.  That said, maths is the one subject where people seem more comfortable with learning decontextualised skills and knowledge and transferring these to real situations (although, this is still a challenge).  It is also possible that the Khan material is well aligned with the school assessment system and as such, the testing of decontextualised skills and knowledge is the context. 

Daniel Rezac, a reviewer from edreach, is one of the few public commentators who has identified some limitations with the Kahn videos and activities.  http://edreach.us/2011/03/15/khan-academy-great-idea-with-one-glaring-hole/ .  In particular he thinks that the videos are boring, and that is a there isn’t enough teaching of concepts (especially in the new Khan Application), “The Khan Academy is not teaching concepts and ideas. Khan Academy teaches answers. 1 + 1 = 2. The concepts and the ideas are really what we want our students to understand, not the rote knowledge. We need a good teacher to facilitate the discussion of what the concept of numbers can be.” (p.1).  Rezac does support the idea of online math lessons as it gives students access to their teacher's thinking anytime and anywhere, however, he wants to see more teachers developing their own online resources.  In considering the success of the Khan Academy he states that the “ Khan Academy is a symptom of a teaching profession where too many teachers are too shy or too old-school to jump into the publishing world.” (p.2). 

The popularity of the Khan Academy is huge (over 1 million students per month watch over 200,000 videos a day). He has tapped into a desire and need from learners for improved maths teaching opportunities and the flexibility and reach provided by the internet.  While this popularity may reflect a huge need for flexible, accessable online maths instruction, it may also reflect an absence of choice. At the same time, I congratulate Khan for pioneering an approach that many learners have found enjoyable and that I think will encourage teachers and learners to make wider use of Web 2.0 opportunities.

Here are a couple of Khan videos if you haven’t seen them before:

http://www.khanacademy.org/math/arithmetic/multiplication-division/v/basic-multiplication

http://www.khanacademy.org/math/arithmetic/addition-subtraction/v/subtraction-3---introduction-to-borrowing-or-regrouping

Philosophies of teaching

Following on from my last post, in addition to offering philosophies of technology in this article, Kanuka also describes the philosophies of teaching.  Kanuka explains that both philosophies reflect our underlying view of the world and need to inform the decisions we make (as teachers) about what and how to teach, and how to use technology.  This can help us understand our actions or preferences, explain these to colleagues and learners, and understand the different choices or preferences of others.  It can also help teachers to avoid “mindless activism” (p.111) where teachers get enthused by the latest technology or trend and attempt to use it alongside existing teaching practices which can “lead to incongruence and inconsistency in action between and among instructors...and incongruence between beliefs and actions” (p.111).

Kanuka describes six philosophies of teaching and the role of technology (including the philosophy of technology) for each teaching philosophy:

1.       Liberal /Perennial: instruction in the classics with rigorous debate to bring about an informed, thinking society; promotes strong teacher direction; generally a poor fit for e-learning as the instructor loses some control of the learning environment; aligned with technological determinism.

2.       Progressive: promoting personal growth and a better society; learning through a student/teacher partnership; well aligned with using learner-centred e-learning to encourage learner participation and sharing reflecting uses determinism.

3.       Behaviourist: designed to bring about observable changes in behaviour which will eventually help society improve and survive; uses a subject-centred approach where teachers design an environment which encourages desired behaviours; believe that transformations can occur though using technology reflecting technological determinism.

4.       Humanist: supporting personal growth and self-actualisation; encourages self-directed learning in a safe and cooperative environment; group learning and self reflection/evaluation methods are core activities to help focus on a learner’s experience of learning (rather than subject content); technology offers flexible, convenient and access to meet the needs of each individual learner and support the role of the teacher as a facilitator or guide (at most) reflecting uses determinism.

5.       Radical: supports education for economic, political or social change; promotes instruction through dialogic encounters which lead to changes in belief and practice and avoid traditional lecturing/top-down education as this is oppressive and value-laden; the teacher is seen as an equal (with expertise) who creates the curriculum with students and enables students to reflect on themselves, their reality and beliefs/assumptions; use technology to suit their own purposes reflecting social determinism but avoid technology which is aligned with ‘the establishment’ paradigm.

6.       Analytical: concerned with the development of rationality and transmitting worthwhile knowledge; teachers guide and direct classroom discussion of worthy topics, concepts and arguments; teachers scaffold learning and introduce new knowledge as appropriate; use e-learning to serve the learning process reflecting uses determinism.

 

In different contexts, my philosophy of teaching varies.  As an educator in a health literacy context, I tend towards the progressive philosophy, with elements of an analytical philosophy when the audience is interested in gaining health knowledge or facts from an expert (although the expert may not be the teacher).  In workplace or industry education the behaviourist philosophy is more frequently used due to the opportunity to practice (behaviours) in a real context, which provides activity and safe experimentation.  Once again, expert input is frequently used to scaffold learning either prior to a learner attempting an activity or when reflecting on an activity, showing an analytical philosophy to teaching.

In literacy and numeracy teaching, I have found that it is very important to build learner’s familiarity with any learning technology or this mode of delivery forms yet another barrier to participation in learning.  While this is possible in a face to face teaching situation, such as a workplace, it is much more difficult in distance learning.  This makes it essential to use technology that is as familiar, easy, intuitive, and logical (from a learner’s perspective) as possible.  I find the technology that fits my purpose and use it as needed.  

With the increase in online resources that learners must use, for instance the Literacy and Numeracy for Adults Assessment Tool (https://assess.literacyandnumeracyforadults.com/Login.aspx ), there is an increasing degree of social control through technology (technological determinism) in the NZ tertiary sector.  The Tool is a predetermined and compulsory part of the literacy and numeracy landscape and educators and learners need to make the most out of it.  Widespread behaviour change, in terms of the number of learners being assesses, has already happened.   It is also assumed that formative assessment results will influence teaching practice and content, as well as provider behaviour.  The Tool is based on many assumptions which may not be reflected in an educators beliefs, such as: national testing is appropriate for adult literacy and numeracy learners, the equal relevance all of the learning progressions  in a strand  to every adult and learning programme, that the context for learning does not need to be reflected in assessment, that assessment does not require a trained educator,and that online results provide sufficient immediate feedback for a learner. 

Philospohies of technology

Kanuka, H. (2008). Understanding e-learning technologies-in-practice through philosophies-in-practice.  In T. Anderson (Ed.), The theory and practice of online learning (2nd ed.) (pp.91-112).  Edmonton, Canada: AU Press Athabasca University.

This post looks at the philosophies of technology presented by Kanuka in this chapter in The Theory and Practice of Online Learning (2008).  A practitioner needs to become aware of their philosophical position, that is, what they believe in relation to the purpose and function of education and technology, as this provides the basis for how they use e-learning.  Education results in change. “The desired changes are based on what we believe should happen through education.  This, in turn, will be reflected in how we choose and use e-learning technologies.” (p.111)

In brief, the three philosophies of technology presented by Kanuka are:

Uses determinism: Technology is neutral, how we use it for learning is up to us.  This assumes complete control over technology and how it is used which ignores the social and political origins of technology and the predetermined functions and features of software (although arguably a person could choose not to use this technology)

Catch phrases:  I’m not on Facebook; I have the power.

Social determinism:  Social systems and cultural contexts affect and reflect technology.  This suggests that social structures and change inform and are informed by technological developments.
Catch phrases: The world needed Facebook and Facebook needs the world; What did we do before email?

Technological determinism: Technology determines how we use it and brings about social change.  It can be viewed negatively, as an instrument of social control and oppression, or more positively, as a mechanism for widespread communication, participation and education preventing capture by the elite.
Catch phrases: My iphone made me do it; the computer ate my homework/deleted my posts; I can’t have a thought without tweeting it first.

I believe that social determinism and technical determinism are both at work in the technological world.   To some extent the changes in society have informed the development of technology, for instance email was a logical extension of written communication and a desire for faster delivery and reply, just like postal and fax functions.  At other times, technology seems to be leading us by the nose and redefining our social structures and interactions.  For example, Facebook has redefined the concept of friends as a virtual group of people including life-long friends, family members, casual acquaintances, people never met in person and the occasional stalker - not to mention redefining what is considered appropriate sharing. 

When looking for teaching resources, I try to find sites which are freely accessible and able to be used for the purpose I need.  This could reinforce uses determinism, although now that I examine it more closely, I don't think technology is neutral, in that it encourages specific behaviours within society (like writing texts rather than calling someone or speaking face to face).  I think most of the time I see social determinism - the technology available reflects what the majority of society wants or needs, and often learners are interested in how to use this tecnology because they want to 'fit in'.

It feels like these philosophies relate to the degree of control over technology we believe ourselves (or learners) to have, both in terms of the technology available (do we design and create it or is it given to us by those in power/leadership) and how we might use it (how much flexibility is offered) - and whether we trust the authors/creators of teachnology.  The more marginalised a person (within a society), the less likely their needs, culture and beliefs are reflected within the majority of technology available (suggesting social determinism and control by the social majority, reinforcing marginalisation; or technical determinism, technology built to define and control the masses and monitor those on the fringe).  (At the same time, technology offers some individuals who are marginalised access to an international community (of the marginalised) and endless information if they choose to use it (reflecting uses determinism). )    
The more powerful a person, or the more culturally aligned an individual is with the majority in society, the more they might believe that technology is just a neutral tool (uses determinism) or a reflection of society (social determinism).  I suspect that at different times and in different contexts we feel more or less marginalised and this is likely to affect our beliefs about technology (another whole area of philosophical debate). 

But wait...another opportunity

Happy easter to all.  As an add-on to my last post, I am also working on three health literacy projects this year (gestational diabetes, skin infections and gout).  We are interviewing patients, whanau, health practitioners and managers at this point, with a view to developing professional development (for the health sector) and resources (for the public/patients) which could help improve health literacy (e.g. by improving the interactions between health professionals and patients and building knowledge and skills within families/communities).  The more I read and view, the more potential I see for web 2.0 to be a way of creating online learning communities (maybe by tapping into existing online health communities) to help people manage their own health, prepare for health interactions and make informed choices.

Online LN learning in context

I am studying towards a Masters in Adult Literacy and Numeracy Education with AUT, NZ.  This blog is part of my journey of discovery – exploring the philosophies and realities of online learning and the ways this may be useful for adult literacy and numeracy learners and educators. 
The challengeIn my current role, the closest I get to learners is when preparing education material/resources (usually hardcopy workbooks) for their vocational/industry ‘training’ (a word used in our tertiary system rather than my view of learning). This material is usually part of a distance learning set up where a learner receives face to face (F2F) support from a workplace mentor/trainer (usually not formally) trained as an educator) – and informal support from colleagues/ family members/ friends etc. Some trainees may also get to spend a small amount of F2F time with a trainer.

The learners I write for come with a wide range of skills and prior knowledge (occupational and LN). 
On the up side, we have the opportunity to provide learning through a very meaningful context i.e. relevant to their occupation. We also find that leaners are quite motivated because they see this as relevant learning or 'for work' rather than 'traditional' classroom learning so their attitudes and openness to learning are positive (despite previous negative schooling experiences).

Increasingly, we have an indication of the general literacy and numeracy competencies of the learner group via assessment information from the LANFA Assessment Tool – this could be a LN prior knowledge starting point. We are also able to analyse the LN demands learners' (as a group) need to be able to manage once qualified (the outcomes we seek) and the LN skills and knowledge learners' need to cope with the industry training regime (e.g. reading training material and completing assessments). 

In this scenario, the resource developer (soon to be online resource developer!) is the educator.  However, there is no direct personal contact with the learner – the resource is the only form of communication (i.e. the educator provides the resource but not learning support). 

Ongoing engagement with the learner will need to be a technical feature of an online resource, or perhaps involve colleagues (other learners), mentors, managers and industry experts in an online environment. Hmmmmm. No expert involvement in the F2F with learners.  So this is not ideal for LN teaching and learning - but when do we ever get that luxury in life - I just hope to identify what will work in this very real situation (and what won't).

Working on a collaborative presentation today with another organisation. Realised that I like the powerpoint to be all graphics/pictures rather than words - thanks to all the literacy issues about reading and listening at the same time.

Whoops - that was meant to be a post not a comment.

I suppose it's about time to start a blog.  I've never been that fussed about following other people's - but I need to get with the programme.