Assessment ‘As’ Learning through Project Based Learning

Blended Educator

In putting together the program for our Professional Learning Teams focus on assessment I was thinking about how little we tend to focus on Assessment ‘as’ Learning. The past couple of decades has rightly seen the focus of assessment shifted clearly from solely valuing Assessment ‘of’ Learning, or summative assessment to acknowledging the important place of formative assessment in the learning cycle.

Work from influential learning theorists like Dylan Wiliam has seen a Assessment ‘for’ Learning, or formative assessment, recognised as one of the most important measures of student learning and achievement in our classes.

Despite even this new focus on Assessment for Learning, a quick Google search reveals where the majority of the attention is given:

Screen Shot 2015-05-24 at 7.48.36 pm

Screen Shot 2015-05-24 at 7.49.05 pm

Screen Shot 2015-05-24 at 7.49.22 pm

Lorna Earl‘s 2003 work, Assessment for Learning: Using Classroom assessment to maximise student learning, presented the assessment pyramid as we traditionally identify it:

Screen Shot 2015-05-25 at 9.44.38 pm

However, she suggests that this pyramid…

View original post 823 more words

Advertisements

Equality is Not Enough – Why We Need Justice in Education.

Abstract

Equality in education means treating everybody the same – everybody gets the same shot. But what if your success in later life is not only based on effort, but also on factors that you cannot control?  Were you born into a rich family? Did your parents put you in kindergarten? What was your pre-term birth weight? Shouldn’t we help low performing students more, given that they may experience such disadvantages?

 

Highlights

  • How equality and justice are fundamentally different from one another.
  • Pre-school factors such as pre-term birth weight and early interventions affect a students’ later academic performance.
  • The good become better, the bad become worst –  a.k.a. the Matthew effect.
  • How countries differ with regards to equality and justice.
  • Justice is a question of attitude.

 

Equality and Justice – They Are Not The Same

agvpB66_700bEquality and justice don’t seem very different from each other, but they actually differ fundamentally in how we treat individuals in our society, especially in our education systems. These two words – equality and justice –  can become very polarising in that people either stand behind one or the other, not both. What does it mean to treat everybody equal or just?

Equality means, for example, that all students get the same exam, the same deadline for handing in an essay and the same treatment by teachers. It also means that the state spends the same amount of money on each student. Getting into a prestigious university or landing that high paying job would be 100% based on your effort, which was higher than all other applicants. Pat yourself on the back because you deserved it!

Justice on the other hand acknowledges that there are factors that contribute to later success that lie outside of an individuals’ influence. The idea is that if you get to the top it is not solely due to your effort, but due to clear advantages that you had, but others didn’t. Justice means that teachers spend more time with low performing students and that state funds are distributed unevenly to help the disadvantaged.

Whether you stand with equality of justice depends very much on your belief: Should all humans be treated equal or should we account for dis/advantages that some might experience? I want to show you research findings that exemplify how a students’ ability can be influenced by things that lie outside of their control – factors that affect their success irrespective of their efforts.

 

Pre-School Factors – Pre-Term Birth and Early Interventions

Screen Shot 2015-05-19 at 15.21.53There are factors that influence a childs’ later academic achievement from as early as their birth. For example, pre-term born children showed lower cognitive abilities when in school compared to full-term born children (Source).  The “Cases” in the graph refer to pre-term births, which clearly have a lower birth weight and also lower cognitive test scores than the “Controls” (full-term births). It is astonishing that something as early as your birth weight can influence your later cognitive abilities. Thankfully, our genes have not been found to influence academic success to this point in time (Source).

Early interventions such as kindergarten can also have an effect on students’ later academic performance. A meta-analysis revealed an effect size of 0.47,  which means that 33 out of 100 students show greater academic achievement in school if they experienced a form of early intervention such as kindergarten (Source). It is not vocabularyexactly clear why kindergarten might be beneficial, but we can make pretty reasonable assumptions. For example,  I mentioned in a previous post (here) that parents with a professional background speak much more with their children than parents with a welfare background. As a consequence, the vocabulary of children can vary by approx. 500 and 1,00 words by the age of 3  (see chart). This  can have a dramatic impact on later academic success in schools (Source). Children that go to Kindergarten are more exposed to spoken language on a day-to-day basis compared to children that stay at home. By the time kindergarten kids reach school they may already be able to read.

 

School Factors – The Good Become Better, The Bad Become Worse

Prior achievement is an incredibly good predictor for future academic performance. Think of our kindergarten example from before. These kids will have superior language abilities, start reading books at an earlier stage and get placed in an advanced class where they are challenged more. In contrast, kids that struggle with reading will fall behind, unable to keep up with the rest, eventually repeating a grade. Having said that, the good students will become better and the bad students will become worse – they drift apart and create a wider spread in performance. The accumulative advantage of high performers is also known as the Matthew effect and it’s an effect to be reckoned with.  Prior achievement “…will lead to gains in achievement on 48 percent of the occasions [effect size = 0.67],…” (Source). It is possible for low performing students to become a high performer, but you are much more likely to do so if your parents are wealthy. “…the children of educated or wealthy parents who scored poorly in the early tests, had a tendency to catch up whereas children of worse off parents who scored poorly were extremely unlikely to catch up and are clearly shown to be an at-risk group” (Source). 

 

Pushing For More Justice – Looking at Other Countries

agvpB66_700bFrom pre-term birth weight, to kindergarten and ones’ socio-economic status, a students’ achievement is strongly influenced by factors the they cannot influence. It is hard to argue for an equal society if you know that performance can be dependent on so many different things. In the end, it doesn’t matter WHY students perform better or worse than other students. The only thing that matters is the fact that some students perform worse than others and that this may be due to a disadvantage they experienced in their lives. A more just education system would allocate more resources to the ones that need them the most so that the disadvantages are countered.

Take a look at this striking paragraph from Amanda Ripley’s book “The Smartest Kids in the World”:

“In Finland and all the top countries, spending on education was tied to need, which was only logical. The worse off the students, the more money their school got. In Pennsylvania, Tom’s home state, the opposite was true. The poorest school districts spent 20 percent less per student, around $9,000 compared to around $11,000 in the richest school districts.
That backward math was one of the most obvious differences between the United States and other countries. In almost every other developed country, the schools with the poorest students had more teachers per student; the opposite was true in only four countries: the United States, Israel, Slovenia, and Turkey, where the poorest schools had fewer teachers per student” (Source).

131203035908-final-oecd-pisa-table-story-top

A good education system does not only foster top performers, but also helps low performers with catching up. What needs to change is the attitude we have towards low performing students. They are not simply lazy, but their current abilities are the sum of all factors that influenced them over the years. Giving them extra support would create a more just education system. Treating kids in such a way also sends a powerful message: your success is partly based on advantages that others didn’t have. Support others that haven’t made it as far as you have.


Please subscribe if you’d like to stay updated
Leave a comment.

Homeschooling – A Good Alternative or Simply Crazy?

Abstract

Currently, 2.9% of all US students are not attending school, but rather educated at home and within their community – the homeschoolers. But is homeschooling a real alternative to the traditional school setting or just a movement that will fade with time? Research findings suggest that homeschooling is here to stay. Both academically and in other domains, homeschooled students seem to significantly outperform public school students. The more unstructured homeschooling variant called ‘unschooling’ on the other hand correlates with weaker academic performance.

 

Highlights

  • Homeschooled students significantly outperform public school students in nearly all subjects.
  • Unschooled students lag behind structured homeschoolers and possibly also public school students with regards to academics.
  • Homeschooled students appear to be happier with their jobs and their lives, participate more often in protests and go voting more frequently.

 

Introduction

Screen Shot 2015-04-15 at 15.49.10As a kid, you probably spent around one third of your waking time in school. That is about 15,000 hours (Source), which is a tremendous amount of time if you think about it! A teacher of yours may be the reason you decided to study a certain thing and many of your longstanding friends went to school with you. Now imagine spending those 15,000 at home and in your community rather than going to school. All those teachers who will never have an impact on you, all of your friends who you will never meet. Wouldn’t your social and academic skills suffer greatly?

Despite these potentially harmful outcomes a growing number of parents in the US and other western countries are educating their kids at home. There are many variants of homeschooling and what they all have in common is their attempt to to avoid school. I find homeschooling to be very exciting because of how radically different the lives of homeschoolers can be compared to public school students. Homeschooling is an exciting experiment that could teach us a lot about our own schools to what extent they contribute to our academic and social skills.

In this post, we’ll:

  1. give a general introduction to homeschooling, what homeschooling variants exist and who decides to homeschool their kids in the first place,
  2. look at scientific studies on how homeschoolers perform academically compared to public school students,
  3. look at how they differ from public school students in other aspects of life,
  4. and what these findings might imply for our education system.

Homeschooling is Nothing New – Historically Speaking

For most of human history, homeschooling was actually the rule, not the exception. Knowledge was passed down from one generation to the next, either by the family or the community (i.e. farmers boys became farmers). More theoretical knowledge such as the arts or natural sciences were reserved to the lucky few who were rich enough to pay for private tutoring. The institutionalisation of education only happened much later during the industrialization when more skilled and educated workers were needed. Suddenly, reading, writing and mathematics became a prerequisite for sustaining yourself and your family. School became the only place where you could acquire that knowledge and it often times became mandatory. That was a good thing because it allowed us to create more educated modern societies. Today, schools are nearly impossible to think away. After all, who would argue with 300+ years of gathered knowledge on how to teach and socialize youngsters in the best possible way? School is mandatory, so it must be good for you. And after all, it holds the promise to a better life if you perform well.

The truth is that school is not the only choice for education anymore. Information has become ubiquitous with the internet. You can tune in to the lectures of the very best teachers. School is not necessarily mandatory anymore and many universities are already accepting homeschooled students. And the argument that a critical part of our socialisation happens in school is just an assumption. Therefore, an increasing number of parents have decided to homeschool their children. According to the National Center for Education Statistics there were about 1.5 million students (2.9%) in the US being homeschooled in 2007, while there were only 850,000 students (1.7%) in 1999 (Source). The figure below depicts this trend quite nicely.

Screen Shot 2015-04-06 at 19.15.58

 

Parents who homeschool their children are often seen as religious fundamentalists who don’t agree with the secular nature of public schools. Although this is still a major reason for why parents homeschool their kids, there are a number of other reasons for kids are being homeschooled. For example, parents are dissatisfied with the quality of education or want to protect their child from harmful experiences (i.e. bullying) (see below).Screen Shot 2015-04-14 at 19.36.41

 

What types of homeschooling exist? There are endless variants, but they can roughly be put on a spectrum of how much structure is imposed over the students’ daily routine. On the one end of the spectrum lies the all-in one curriculum, where parents try to basically replicate school at home. Parents act as the instructors, buy the relevant books and follow a class schedule. On the other end of the spectrum lies the unschooling movement, which strips any form of structure from the students’ learning experience. Here is a quote by John Holt, an American educator and author who had a significant impact on the unschooling movement in the 1970s:

“… the human animal is a learning animal; we like to learn; we are good at it; we don’t need to be shown how or made to do it. What kills the processes are the people interfering with it or trying to regulate it or control it.” (Source).

Parents see themselves as facilitators rather than instructors who help children learn what they wish to learn.
Between all-in one curricula and homeschooling, there are endless variations of how homeschooling can be done. Is there an instructor? If yes, is it the parents or a tutor? How much freedom is given to the student with regards to what they learn? Does the student go to school for at least a few hours per week or is there no connection at all?

 

DOCUMENTARY ON UNSCHOOLING


RESEARCH ON HOMESCHOOLING

All of the data that I’ll be presenting refers to the US because there is virtually no research done on homeschooling in other countries. But even in the US there is surprisingly little research available. This is due to a lack of available data (Source). Homeschoolers have been fighting for minimal regulation of their childrens’ education since the 1960s. This includes opting out of general state and nation wide statistics that could have been valuable to assessing the success of homeschooling.  I can imagine that the homeschooling community feared that collected data could be used to reinforce regulation over their childrens’ education.The only available data that can be used to compare homeschoolers to public school students are standardized test scores (SAT) that both have to take for getting into universities.

I’ll present 2 studies that use standardized test scores to compare homeschoolers with public school students. The first one is the largest study on homeschooling and it was conducted by the Home School Legal Defense Association (HSLDA), which was founded in 1983 to “…defend and advance the constitutional right of parents to direct the education of their children and to protect family freedoms.” It goes without saying that the following results should be taken with a grain of salt due to their biased stance towards homeschooling. Furthermore, the following results are based on the HSLDA 2009 Progress Report, which is not a scientific paper. Therefore, I had very little insight into how the study was actually conducted and what results may have been left out!

Study 1
The study (Source) was conducted in 2007 and is based on 11,739 participants from all 50 US states.The result is quite startling:

“In the study, homeschoolers scored 34–39 percentile points higher than the norm on standardized achievement tests. The homeschool national average ranged from the 84th percentile for Language, Math, and Social Studies to the 89th percentile for Reading.”

Screen Shot 2015-04-06 at 20.01.30

  1. Parent education hardly had an effect on the performance of homeschooled kids. Irrespective of whether one, both or none of the parents of the homeschooled student had a college degree, their kids always performed better than the average public school student.
  2. Teacher certification didn’t matter as well. Students with parents who didn’t have a teaching certification performed equally well to students with certified teacher parents.
  3. Family income hardly made a difference between homeschooled students, while it is very established that income plays a major role in academic performance (Source).
  4. Gender didn’t make any difference as opposed to in public schools (Source).

THAT’S AMAZING!!!
Let’s think about this for a second. Homeschoolers avoid the one place that is supposed to prepare us for standardized tests and they actually perform BETTER than public school students. Furthermore, the parents’ education, the families’ socioeconomic status, the students’ gender and having certified teacher parents had no effect on the academic success of the student.

The study also has two major drawbacks:

  1. Public school students took standardized tests on a mandatory basis, while homeschoolers volunteered. It may have been that homeschoolers scored so high relative to public school students because only parents who were pretty sure that their child would perform well actually signed them up.
  2. The study was conducted by the HSLDA, an institute that advocates homeschooling. This is not necessarily a bad thing, but their report is not as transparent as a published research paper and it may be that some significant aspects of the study were left out. In any case, we can’t check.

Study 2

The study (Source) had 4 advantages over the first one:

  1. The study was conducted by an independent researcher group from Canada.
  2. Both public and homeschoolers were recruited on a voluntary basis.
  3. Students from public schools were matched with homeschoolers based on (1) similar family income and (2) similar parental education, to ensure that the differences between those groups are not due to these factors.
  4. The researchers differentiated between structured homeschoolers who are more inclined to replicate school at home and unschoolers who don’t impose any structure over their children.

One major disadvantage is that the study only had 37 public and 37 homeschooling participants, which means that there is a large probability that the findings of the study are simply due to chance. Keep that in mind when considering the following results.

Results:

  1. Structured homeschooling students were at least one grade level ahead of public school students in 5 out of 7 test areas (word identification, phonic decoding, science, social science, humanities), almost half a year ahead in math, and slightly, but not significantly advanced in reading comprehension.
  2. Unstructured homeschoolers performed significantly worse than structured homeschoolers. In 5 of 7 areas, the differences were substantial, ranging from 1.32 grade levels for the math test to 4.2 grade levels for the word identification test.
  3. Unstructured homeschoolers perform worst than public school kids, but the difference is statistically not significant.

Screen Shot 2015-04-05 at 17.15.45

The authors argue that homeschooled students may benefit from multiple aspects. They usually have smaller classes, often even one-on-one classes. Therefore, the instructions they receive may be more tailored to them. They may also be spending more time in general on academics. With regards to the relatively bad performance of unstructured homeschoolers (unschoolers) one could argue that students need some kind of structure or guidance to learn effectively. Alternatively, it may also be that unschoolers never took standardized tests before and therefore lack test-testing abilities.

 

What About Non-Academic Effects?

Sadly, I was only able to find one survey study by the HSLDA that asked homeschoolers many years later about their day-to-day lives (Source). Here are the most striking results:

Homeschoolers are happier with their lives than public school student

Screen Shot 2015-04-05 at 18.12.04

 

Homeschoolers are happier with their jobs than public school students

Screen Shot 2015-04-05 at 18.12.25

 

Homeschoolers participate more in protests or boycotts and they vote more often than public school students

Screen Shot 2015-04-05 at 18.07.27

 

The large majority is very satisfied with having been homeschooled and would homeschool their own children as well.

Screen Shot 2015-04-15 at 13.33.30

 

Shouldn’t all of us Homeschool their children?!

The evidence that speaks FOR homeschooling is very appealing, no doubt. However, none of the studies had a level of scientific rigour and quality that could have made me into a blind homeschooling believer. I would feel very unsafe with making such an important decision based on these 2 studies. There is still a mountain of work to be done in order to establish homeschooling in the education landscape as a real alternative to public schools. Important questions might be: how much structure contributes to the advancement of the student and and what point does it become counterproductive? What factors are absolutely necessary for homeschooling to work? How can the quality of their education be measured in order to ensure equal access to jobs and higher education?

Although unschoolers performed bad relative to structured homeschoolers and public school students, it doesn’t mean that unschooling is generally a bad thing. Unschooling is a movement that resulted out of a growing dissatisfaction with the rigours and grade-focused school system. The philosophy that every child has an inborn curiosity is most certainly true and it is hardly disputable that many lose their curiosity due to the setup of the school. Unschooling is simply the extreme end of a ‘structure spectrum’. It will be up to researchers to figure out how much structure is beneficial and how much is counterproductive for the advancement of the student. Many schools are already experimenting with less structure by giving students more agency over what they learn and how they learn it (Source).

 

Conclusion

Before schools existed, students learned at home and within their community. The concept of schooling emerged in order to make information more accessible. Nowadays, information is ubiquitous and available in palatable forms such as online courses and study software. Students are performing better academically when staying outside of the one institution (school) that was specifically designed to advance them in this regard. Critics of homeschooling often argue that homeschoolers would not be socialized enough, but that doesn’t seem to be the case. I would never have expected to find research studies that speak so clearly in favour of homeschooling. Given these results, I am very surprised that researchers are not taking this domain more serious by running more studies. There are hurdles, primarily the lack of data, but these can surely be overcome.

I think that we trust our school system too much and ourselves too little, which is why the homeschooling movement will most likely stay small for now. That doesn’t mean that we should ignore it. On the contrary, we should try to learn from it. If we could find out what makes homeschooling so great we could bring these aspects into the classroom. We already have a great education infrastructure with buildings, teachers and financial resources. We should try to manage all of our resources in a better way. That’s what we can do with the help of the homeschooling community. I am very curious to see how the homeschooling movement develops further. Maybe my kids won’t go to school, who knows!


Please subscribe if you’d like to stay updated
Leave a comment.

Testing – A Powerful Learning Method

Tests have a bad reputation and it’s time to change that. Their effectiveness as a learning tool is tremendous, but students don’t use tests as much as they should. We’ll look at recent scientific studies that compare testing to other learning methods, the neuroscience of memories and forgetting and how testing could be implemented effectively in schools and universities.

 

child with learning difficultiesIntroduction

During my years at school and university I often found myself the day before an exam, trying to frantically cram all the study material into my brain. After having written the exam, the learning process went into reverse mode and all of the study material was wiped clean from my memory. This type of learning is also commonly referred to as bulimic learning – devour and regurgitate.

My teachers always used to say “we don’t learn for school, but for life”. As a school kid that was hard to believe since I was restarting my brain drive right after exam period. Many people – including myself – like to mock the phrase by reversing its meaning: “we don’t learn for life, but for school”. This version of the phrase is actually the original one by Lucius Seneca, a Roman philosopher. Seneca was actually criticizing schools for not teaching life-relevant material.

What I’d like to argue today is two-fold. First, todays’ bulimic learning is a terrible way for students to learn because it is highly inefficient and it shifts the students’ focus away from the process of learning and towards examination. Second, recent scientific evidence points out that testing can be used as a highly effective learning tool, which means that we might have to change the way we use testing in the classroom and for ourselves.

 

How Students Learn

A 2009 study looked at the most prominent learning methods used by students (see table below) (Source). Rereading notes or textbook is by far the most popular learning method with 83.6% of students applying it. Doing practice problems (42.9%) and flashcards (40.1%) were also high up, but nowhere near rereading notes. But how do these methods compare in efficiency? If students were to test out methods in an experimental setting, would they be able to retain more information with one method versus another? And if one method is better than another, how large is the difference?

Screen Shot 2015-03-12 at 23.08.01

When practicing for an exam we traditionally think in two stages: studying and testing. During stage one (studying) students expose themselves to the material (reading a book, watching a lecture) so that new memories can be formed. During stage two (testing) we reproduce what we have learned into a measurable form. This is what tests have been designed to do. However, research findings found that a significant portion of the learning process actually happens during testing:

 

The Testing Effect

In an experiment from 2005, university students were asked to alternate between studying 40 English-Swahili word-pairs and testing their knowledge (Source). Take a look at the table below. The students went through 4 study and 4 testing periods (the columns from 1 to 8). They started with a study period (S) followed by a testing period (T) and so on. There were 4 different conditions (look at the rows). The first group (ST) continued studying and testing ALL 40 word-pairs, irrespective of whether word-pairs had been recalled successfully in the last testing period (T). This is like having a deck of flashcards and continue studying them all, irrespective of how well you feel acquainted with the card. You can see in the ST row that the number of studied and tested words is always 40. All word-pairs stay in the loop.

test

The second group (SNT) removed those word-pairs from their study periods if they had been recalled correctly during the previous testing period. In other words, this is like leaving out flashcards that you feel comfortable with from your studying, but still testing yourself on them. ST and SNT are the same with respect to the testing period – both continue testing themselves on all flashcards.
The two remaining groups (STN and SNTN) take successfully recalled word-pairs out of the test. The SNTN group also takes successfully recalled word-pairs out of the studying period. The nice thing about this design is that we can trace back with very high certainty the relative contribution of both studying and testing to the actual learning.

Screen Shot 2015-03-01 at 16.11.45To the results: The students were tested 1 week after the experiment to see how many of the word-pairs they still remembered. What do you think was more predictive of successful recall: Keeping words in the studying phase or keeping them in the testing phase?

The bar chart shows that two groups that kept all the word-pairs in the testing period (ST and SNT) recalled ~80%, while the ones that had dropped them from the testing period only recalled ~30%. Whether word-pairs remained in the study period, however, had practically no effect on their performance. The first two groups (ST and SNT) don’t differ from each other although ST continued studying all word-pairs and SNT dropped them. The same goes for STN and SNTN. Again: Whether word-pairs remained in the study period had virtually no effect on performance.

 

What does this mean? First of all, most of the learning occurred during testing and not during studying! Testing is traditionally used as a measuring tool of how much a student has learned, but if most of the learning occurs during testing then we have to change the way we think of testing. Second, remember that 84% of students preferred studying by rereading notes? This form of studying is pretty much what the students in this experiment were doing in the study period – exposing themselves to the information. The second most popular study method – flashcards – is used by about 43% of students, which is impressive, but still very distant from the whopping 84% of students that reread notes. Flashcards are more similar to what students were doing in the testing period – trying to recall information.

The difference of these two methods – exposing yourself to the information (rereading notes) and trying to recall information (flashcards) – is what we are going to look at in more detail now by taking a crash course in the neuroscience of learning. We will be able to categorize all study methods into two groups: passive review and active recall.

 

Memories: A Battle Between Learning and Forgetting

In very very simple terms: Your brain is made up of cells called neurons that form a giant network of connections amongst each other through which they can send electric signals. These connections and the signals that run through them basically make up everything you are. Every single thought and memory you have is a specific activity pattern in the neural mesh. A memory is simply a specific network of neurons and when these send electric signals to each other they give rise to that memory they stand for.

footprints-in-snow11A memory can be forgotten if it is not activated enough. Think of forgetting in terms of a snow-covered parking lot. Walking on the snow leaves traces – your memory. Forgetting is like fresh snow that makes your traces slowly fade. The only way to keep the memory from fading is to activate it frequently – to re-walk the path.

There are two ways to re-walk the path: passive review and active recall. During passive review previously formed memories are activated through the mere exposure to the material. For example, seeing a picture of your dog makes you remember your dog. Reading your shopping list makes you remember the items you wanted to buy. Rereading notes – the most popular study method – is passive review because you expose yourself to previously encoded information. Active recall on the other hand does not involve exposure, but the student has to actively remember the information. For example, remembering your new mobile number or what you ate last evening for dinner are active recalls. Flashcards involve active recall. “Who invented dynamite?” or “what is the capital of Japan?” Because the answer is written on the back of the card no exposure to the information can occur and it has to be recalled with effort.

Forgetting can be prevented if the memory is used frequently. Even Aristotle knew this when he wrote “exercise in repeatedly recalling a thing strengthens the memory” (Source). This is why the muscle is used as an analogy to describe how the brain works – both deteriorate if not used.

In summary, neurons form connections with each other through which signals can be sent. A memory is a network of neurons that are connected to each other. Recalling a memory means sending a signal through that network. Forgetting happens when a memory is not being activated for a period of time. As a consequence, the network deteriorates or becomes inaccessible. Finally, a memory can be activated through passive review (exposure, no effort) or active recall (no exposure, effort). Rereading notes is a form of passive review while flashcards are a form of active recall.

 

Metacognitive failure

The question that remains to be answered is why the large majority of students like to use rereading notes – a relatively ineffective study method. Are they able to judge the (in)effectiveness of their methods? A similar study as the one from before compared testing to elaborative concept mapping (building a mind map from the most important of a text that the students read). Although 84% of students that underwent testing performed better than the ones doing concept mapping, 75% of students believed that concept mapping was equal to or better than testing (Source).

The ability to observe ones’ own thinking (‘thinking about thinking’) is referred to as metacognition an extremely important ability that allows us to do all kinds of things: question our approaches, find alternative solutions, adopt more effective ways of thinking. In our case, students were unable to successfully deploy metacognition to notice the ineffectiveness of rereading notes relative to other methods. One plausible explanation is that students get a wrong sense of mastery during rereading study notes. The more you read your notes the more familiar they will become. If I were to take a multiple-choice exam without the A, B, C and D answer options I would not be able to answer as many questions as compared to when they are presented. I can recognize and pick out the one that seems familiar to me and probably be right as well. This feeling of familiarity gives the illusion of mastery. This concept is closely linked to passive review. The simple exposure of the study material activates the memory, but that doesn’t mean that we will be able to activate the memory if not being exposed to it.

Testing on the other hand does not lead to an illusion of mastery – rather to the opposite. You either know what is written on the back of the flashcard or you don’t. Students coming from rereading notes to testing may feel very much discouraged by the untainted reality of their state of knowledge. Students want to feel prepared for the test and the illusion of mastery offers just that. It feels good after all!

One final point may be that students simply prefer reading notes because it requires less preparation. Writing flashcards takes a lot of time and why should students do it if they already spent time preparing their study notes? This is of course possible.

 

What do we do with all this?

We first looked at the most popular study methods used in school and asked how effective these are in helping students learn effectively. A study found strong evidence that questions our traditional understanding that testing only serves as a measuring tool of what we have learned during studying. Instead, much of the actual learning happens during testing. But testing is just a form of active recall, which is the effortful process of remembering something versus exposing oneself to it (passive review). Finally, we speculated that students are unable to successfully compare the effectiveness of study methods for learning because of the illusion of mastery. Rereading study notes is simply the exposure to the material as a means to activating a memory, but recalling the information without exposure in the future is not guaranteed.

What do we do with this?

  1. Get students to use testing
    Students must learn about the effectiveness of testing as a study method. They are interested in getting good grades and if there is a better way of achieving that they would probably change their study behavior even if that means a tougher reality. The long-term outcome will anyway be better! I would do it! The authors of the discussed study invented a study method that is fairly simple to use and independent of the teacher. It’s called the 3R method (Read-Recite-Review). Students read a text, recite everything they remember and reread the passage to confirm. This simple method has been found to outperform note-taking and rereading both for simple and complex texts (Source).
  2. Introduce technology
    Here is a problem: when testing is used to grade students, then they will study. Studying takes time, which means you cannot have too many tests. If tests are not graded, then students will not study and the test cannot have the desired learning effect. Learning apps offer a way out of this dilemma. Such apps confront their users with lots of tests, which are important to show off learning progress. In the digital world, tests can be perceived as a fun activity – like a challenge or even a game. There is certainly some incentive for students to learn, but the tests won’t have a real impact on their lives. Therefore, they can use the way we want them to use tests: as a learning method.
  3. Testing for feedback
    Frequent testing is a form of formative evaluation (Previous post: Feedback) that can help students make adjustments to their performance early on and avoid nasty surprises such as a failed final exam. Frequent testing can have a nice side-effect: when the final exam is just around the corner, students will probably still remember far more because of the tests spread throughout the year. Therefore, bulimic learning will not be as extreme as before.

Feedback

Should students be praised for for good work? Is questioning an effective teaching method? And what is formative evaluation? All of these examples are forms of feedback. Giving feedback is probably the most important activity a teacher can perform, the effects of which vary widely – from helpful to counter-productive. This post will take a look at the scientific evidence on how different kinds of feedback –  (1) rewards/praise, (2) formative evaluation and (3) questioning – can affect students’ achievement as well as the flipped classroom approach as a way of giving feedback a more central role in school.

This is the first post in a series of posts that will cover the different chapters of Prof. Hattie’s book ‘Visible Learning’. Hattie’s research summarises about 50,000 studies involving 240 million students worldwide, which is probably by far the greatest scientific contribution to the field of learning sciences. Therefore, we will tackle his findings and their implications in the coming posts.

 

Meta-Analysis and Effect Sizes

As boring as it may be, we’ve got to lay some ground work in order to be able to fully appreciate Hattie’s findings! First off: It’s a meta-analysis, which is a way of summarizing multiple papers that looked into the same thing and making one large paper out of it. If the initial findings of the single studies still exist after being merged it strengthens the findings significantly. It tells you that the findings are robust across multiple research groups that used different participants. Hattie found 800 meta-analyses that covered 50,000 single studies and decided to summarise these into a ‘meta-meta-analysis’. His findings encompass about 80 million students world wide, which makes his research very robust and credible. You see now why his research is so important?!

Hattie wanted to answer a single question – what is the effect of ‘X’ on students’ academic achievement. All of the findings will refer to the same outcome – students’ academic achievement. But the variable (X) could be swapped with anything that we might think could have an effect on students’ academic achievement (e.g. homework, kindergarten or feedback for that matter!). Check out the graph below. It shows all ‘X’s’  grouped into effect size ranges (x-axis). The effect size (d) is an expression for how big our variable is on students’ academic achievement. The bigger the value the more it contributes.Screen Shot 2015-01-10 at 16.22.52

One thing that Hattie noticed is that nearly every intervention seems to have a positive effect on students! The average effect size is d =  0.4 and hardly anything we do to kids has a negative effect on their achievement. In fact, 95-97% of the things we do have a positive effect (source). Hence, if we want to know what differentiates good from great, we need to look at effect sizes above d = 0.4! Keep that in mind.

Theses effect sizes can also be translated back into more tangible forms.  For example, an effect size of d = 1.0 can mean “…advancing children’s achievement by two to three years, improving the rate of learning by 50%,… When implementing a new program, an effect size of 1.0 would mean that, on average, students receiving that treatment would exceed 84% of students not receiving that treatment” (Source). That last bit in bold helps me understand effect sizes a bit better. It’s called the common language effect (CLE) and it basically turns ‘d = 1.0’ into ‘on average, 84 out of 100 students benefit from the treatment.’

 

Back to Feedback

I spent way too much time trying to understand what feedback actually is and apparently, John Hattie also had problems with it, as this quote shows: “When I was presenting these early results in Hong Kong, a questioner asked what was meant by feedback, and I have struggled to understand the concept of feedback ever since” (Source). Feedback is always something that follows instructions (teaching). If a student is taught a new method in math and given a set of practice examples, feedback would try to maximize the learning in whatever way is more suitable. Here is a more elaborate definition of feedback: “Feedback is information with which a learner can confirm, add to, overwrite, tune, or restructure information in memory, whether that information is domain knowledge, meta-cognitive knowledge, beliefs about self and tasks, or cognitive tactics and strategies” (Source).

Feedback is among the 10 largest effects that John Hattie found in his research. The effect size is d = 0.73 (based on 1,287 studies and 67,931 students). That means that 52 out of 100 students would show higher performance when given feedback vs. no feedback. That might not sound that great when you think of the other 48% of students for whom feedback apparently didn’t result in higher achievement, but d = 0.73 only refers to the average effect of feedback found across 1,287 studies – which all tested different kinds of feedback. Some lie far below d = 0.73 and some way above. I have picked the three most interesting types of feedback for us to look at: rewards/praise, formative evaluation and questioning.

 

Feedback Type 1 – Rewards and Praise

2Who hasn’t been told at some point by their teacher that they’ll get a smiley stamp on their test if they do well in school? But what effects do such rewards have on kids’ learning behaviour? A 1999 meta-analysis looked at how tangible rewards affect students’ intrinsic motivation. Whether it’s about asking students to engage in a task (d = -0.4), asking students to finish a task (d = -0.36) and asking students to reach a certain performance level (d = -0.28) – every type of tangible reward reduced intrinsic motivation (Deci et al., 1999). That is at first sight rather surprising because you’d think that students would work harder to reach the goal. However, rewards completely change the way we learn in that our intrinsic motivation is shifted away from the act of learning and toward the reward. Rewards hijack students’ intrinsic motivation and learning becomes a means to an end. That doesn’t mean that rewards should never be given. Unexpected rewards can be a way of conveying that a task was done well and can motivate students to continue with their work. If unexpected rewards become too frequent, they become expected, making the reward the center of attention again (Deci et al., 1999).

1We often praise students in the hopes of motivating them to continue making an effort. A 1987 study from Israel placed students into 4 groups that got different types of feedback – comments, praise, grades and no feedback (control). Students could use the feedback to improve their performance. The results showed that the students who got the comments performed the best, but the students who were praised rated their performance significantly higher than all the other groups (Butler, 1987). We can draw two conclusions from this. (1) Praise is less effective in helping students perform than comments. (2) it gives the impression of performing well. That is actually quite a dangerous realization. However, if praise is given in form of positive feedback, thus combining feedback and praise, it can have a positive impact on students indeed (Deci et al., 1999).

But praise has even more pitfalls. It works similar to rewards in that it is a positive feeling we get when others think highly of us. The problem with this is that students will want others to continue thinking highly of them. As a result, they often develop a high fear of failure and consequently stick to manageable assignments rather than pushing themselves with harder challenges.

 

Feedback Type 2 – Formative Evaluation

With an effect size of d = 0.9, formative evaluation just blows away most of the other effect sizes that Hattie found throughout his studies. Formative evaluation allows a teacher to keep track of the state of knowledge of her students in order to adjust her teaching style on the go. Formative evaluation stands in contrast to summative evaluation, which measures students’ performance at the end (e.g. final exam). The advantage of formative evaluation is that changes can be made early on to reach a desired goal more safely. A language teacher may ask her students to select the best thesis statement from a range of options. If all students select the right one she moves on. In contrast,”…if most of the students have chosen incorrectly, she might choose to review the work on thesis statements with the class. But if some have answered correctly while others have not, then she might initiate a class discussion. Moreover, because she knows which students chose which option, she can use this information to guide the discussion more effectively” (Source).

When the student’s needs are assessed attentively the teacher is able to give feedback to its fullest potential. Formative evaluation works best if the teacher listens carefully to the needs of the student. “It is the attention to the purposes of innovations, the willingness to seek negative evidence (i.e., seeking evidence on where students are not doing well) to improve the teaching innovation, the keenness to see the effects on all students, and the openness to new experiences that make the difference” (Source).

 

Feedback Type 3 – Questioning

Screen Shot 2015-01-24 at 20.28.37Teachers spend about 35-50% of teaching time posing questions (van Lier, 1998). That makes questioning the most dominant teaching method, which is reason enough to talk about it. Hattie pooled together 7 meta-analyses and found an overall effect size of d = 0.46. That is slightly above the average of d = 0.4 that Hattie determined as the yardstick against which all effects should be compared to. The whole point of posing questions is to improve students’ understanding of things by getting them to think. Unlike instructions during which information is consumed rather passively, students are almost certainly thinking actively when a question has been asked.

Questions can be divided into low and high-level questions. 82% of all questions are of the low-level type and they typically ask students to recall facts or procedures (Cotton, 1989). Teachers still think that “… their role is to impart knowledge and information about a subject, and student learning is the acquisition of this information through processes of repetition, memorization, and recall: hence the need for much questioning to check that they have recalled this information” (Source). That is all well! The process of repetition is essential for memorization. But higher-level questions are also necessary for reaching deeper understanding. Socratic questioning is systematic and disciplined. For example, asking “why do you think that might be true?” would force students to clarify their own argument, while “what is the counter-argument?” would open them up to the possibility of other opinions. Consequently, the student may realize that their own perspective of things is not the truth, but rather only one way of looking at something.

 

Maximising Feedback – The Flipped Classroom Approach

Since feedback is so important for the performance of students, shouldn’t we try to optimize schools so that the power of feedback can develop its full potential? One way of doing it would be to ask students to learn the materials from online videos (e.g. Khan Academy) at home and do practice examples under the teacher’s supervision in school. With all of these wonderful free online resources, wouldn’t it make more sense if the teacher focused on giving feedback? This approach – now termed the flipped classroom approach – has been popularized lately. A recent article showcases the Clintondale High School in Michigan that was able to reduce the failure rate from 35% to 10%. College enrolment increased from 63% to 80% in only 2 years (Source). Those are quite impressive findings, but it remains to be seen whether the approach results in higher student achievement under scientific conditions. There is surprisingly very little work looking into this (Source), but is of high relevance, given the Clintondale High School success.

 

What has been found so far shows that “…student perceptions of the flipped classroom are somewhat mixed, but are generally positive overall. Students tend to prefer in-person lectures to video lectures, but prefer interactive classroom activities over lectures” (Source). I had talked about Massive Open Online Courses (MOOCs) in a previous post (here), which showed that also university students show much worse performance when in-person lectures are replaced with video lectures. Students complained about the lack of access to the teachers and reported feeling left alone with the tasks. So you see, the discussion regarding flipped classrooms is hardly over, but has rather just begun.

Summary

  1. John Hattie has done some marvellous work on what helps students reach higher levels of achievement in the classroom.
  2. Feedback (d = 0.73) is one of the largest effect sizes Hattie found and it is one of the most important activities a teacher can perform. There are some types of feedback that work and others that are counter-productive.
  3. Rewards and praise actually have negative effects on student achievement and reduce their intrinsic motivation.
  4. Formative evaluation (d = 0.9) is by far the greatest thing a teacher can do, but it only works well if the teacher is willing to listen carefully to what the student needs.
  5. Questioning (d = 0.46) is the most used teaching method with a moderate effect size. Both low and high-level questions are necessary to ensure learning of facts as well as deep understanding.
  6. The Flipped Classroom Approach is a powerful way of unleashing the power of feedback to a much larger degree than in traditional teaching approaches, but it has not been properly validated by scientific studies yet.

Feedback is powerful and I think it will become even more important as access to instructions is becoming more easy. A student may be able to listen to great lectures online and download wonderful education apps, but having a teacher that can help improve the learning process through feedback is essential. If done right, teachers can make students enjoy the learning process, maybe even for a lifetime.

Reaching Universal Primary Education Worldwide

What’s the big picture here? How many kids around the world are still out of school? What is the world community doing to change that? Is there progress and could we speed things up a little – let’s say, with the internet?

The Difference between Improving and Spreading Education

People are generally trying to do one of two things in education – improve it or spread it. Under improvement fall things like considering scientific findings regarding brain development and kids and adjusting the learning material accordingly. It also means to try out different concepts (e.g. Waldorf, Montessori, all-day schools or boarding schools – see previous post: Grades, What They Are and What They Measure) as well as introducing more technology into the classroom. These are the topics that are most commonly dealt with in developed countries where education is accessible to everyone. We talk about creating equal opportunities, because that is the next step in the development of our education systems that we have to figure out. However, many countries around the globe still struggle with providing universal access to education.

Spreading education is a completely different domain. You might be struggling with convincing a teacher to come twice a week to an isolated village to teach math, while improvers try to lobby for more equal opportunities for kids with different backgrounds. Today’s industrial nations once struggled with spreading education as well. When the Prussian king Friedrich Wilhelm I announced in 1717 the first mandatory school system there was no education infrastructure present. There were no school buildings and no teachers, but in 1740 Prussia had already constructed 1480 schools (source). That is a rate of 1 new school every 5-6 days over 23 years! Developments of course continued, but what you can see from this example is that it takes time to build an education infrastructure (see previous post: The Origins of Education).

Things can surely move much faster nowadays than in the 18th century. While the industrial revolution took more than a century to complete in Europe it is now being achieved at a fraction of that time in todays’ developing countries. I guess that pioneers always have it tougher because they cannot copy from others on how to do things in the best way and what pitfalls to rather avoid. Progress is all about trial and error. But when it comes to spreading education we can rely on prior experience. We know what is required to build education systems. There are countless NGOs, funds and volunteers who try to bring the number of kids without an education down. This mostly involves being on site, training or sending teachers, providing learning material and financial support for families. Some of these programs are really good and they achieve great things, but they seem a bit like a drop in the ocean when looking at education on a global scale.

 

Scalability – An Introduction

Screen Shot 2014-11-28 at 13.15.17I want to take a little detour to talk about the concept of scalability. In the business world, scalability means to expand rapidly without dramatically increasing one’s costs [or effort] (Source). The graph shows how a scalable business can grow exponentially while the extra effort hardly required hardly increases. Take for example the McDonalds franchise system. If you have the necessary financial resources you can open your own McDonalds. All you have to do is stick to the McDonalds guidelines (e.g. how long the meat needs to be on the grill or what is generally on the menu) and pay branding/licensing fees . You would benefit from the brand and McDonalds spreads their brand and earns more money without moving a finger. There are many such examples, but the large majority of the most recent scalable businesses can be found on the internet. Any teenager who knows how to build a website can offer a service and cater customers across the globe. Take Mark Zuckerberg’s Facebook, which he created single-handedly and which was used by 864 million people daily in September 2014 (source). Since the content is generated by the users, Facebook just has to make sure that their servers don’t crash to keep their service online.
Before taking the concept of scalability into education let’s take a look at how we, as a species, are working together to spread education across the globe.

 

Universal Primary Education

Screen Shot 2014-11-27 at 22.29.46The United Nations Millennium Development Goals (UN MDG) that were initiated by the UN Secretary Kofi Annan in 2000 set 8 ambitious goals that were targeted and had to be met by 2015. Large sums of money were annually allocated to low and middle income countries to help reach those goals. The second goal is about education. It reads that “…by 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling achieving universal primary education until 2015” (Source). The UN defines primary education to be 6 years of basic schooling. The graph on the right is taken from the United Nations’ MDG Report 2014 (Source). You can see massive improvements between 1990 and 2012. Only Sub-Saharan Africa has not yet been able to bring primary education to at least 90% of its children, but major improvements have been observed there as well over the last decade. The gap is closing, but not at a fast enough rate. In 2012, there were still 58 million kids without primary education worldwide. That is 1 in 10 children out of school (Source). We are not moving at a fast enough pace to achieve this goal until 2015, which is why people have been looking for alternatives.

 

Scalability and Education

Scalability does not seem to be working in education, at least for now. The most prominent example of scaling education comes from higher education. In 2011, David Stavens and Mike Sokolsky – two roboticists from Stanford University – offered that students from around the globe could join their lecture on artificial intelligence for free on the web. They had initially expected no more than 2,000 students, but eventually 200,000 people had signed up (source). Since then, many such ‘Massive Open Online Courses’ (MOOCs) have emerged, the most famous websites that offer online courses are the MIT/Harvard non-profit company http://www.edx.org as well as the Stanford for-profit spin-offs Coursera and Udacity. Many have believed and still believe that companies such as Udacity are “…poised to potentially revolutionise education” (source). MOOCs deliver high quality lectures for free to anyone in the world with an internet connection. BUT “So far, most MOOCs have had dropout rates exceeding 90 percent” (Source). Udacity MOOCs have also been introduced at the San Jose State University to cut costs, but the student passing rate was alarmingly low: Elementary statistics (50.5%), College algebra (25.4%), Elementary Math (23.8%) (Source).

Why are online courses failing as a substitute for conventional courses? The documentary Ivory Tower argues that students were unhappy with the lack of feedback and lack of general interaction with the professors as indicated in the courses’ comment sections (Source). The importance of human interaction in learning has probably been underestimated quite a bit even in the realm of higher education where students are usually expected to work independently. Companies like Udacity may one day find a format that works, but it will surely be even more challenging to get young school kids to participate in online lectures. Education remains to a large degree non-scalable (linear).

 

Teachers are Essential

MOOCs require students to work very independently, which is a skill that kids from primary school do often not have. They heavily rely on the guidance of their teachers and parents. Just how important the teacher is for the success of a child is impressively demonstrated by John Hattie, Professor of Education and Director of the Melbourne Education Research Institute at the University of Melbourne, Australia. Over the course of about 15 years he has aggregated the results of more than 50,000 studies on what works best in education (Source). The result: “…the best way to get higher achievement is to improve the level of interaction between pupils and their teachers” (Source). The student-teacher relationship has an effect size of d=0.72 on students’ success, which is quite impressive (the effect size measures the difference between the control group and the experimental group, with a d-value of 0.72 signifying a medium to large effect).

 

What Hinders the Spread of Education

The UN MDG Report 2014 mentions 6 factors that put certain groups of kids at a higher risk of not receiving primary education. I have listed them in the table below.

Screen Shot 2014-11-28 at 15.48.47

What this table shows quite nicely is how multi-faceted and intertwined education is with society. Less primary education attendance is linked to politics (conflict-affected areas and rural areas), the economy (poverty) as well as culture (gender and disability inequality). Universal primary education can be achieved more easily if certain basic societal conditions are met. Wars need to stop, average incomes have to rise and the role of women and disabled within society has to change. It is obvious that the billions of dollars that the UN has made available to low and middle income countries to achieve universal primary education by 2015 cannot solve the problem of poverty and wars. These factors that clearly affect school attendance have to be dealt with by economics departments and politicians. However, the available money can be spent to battle some of the other factors. For example, building schools in rural areas would get more rural kids into school and offering special education programs for disabled students is also feasible.

Screen Shot 2014-11-28 at 16.24.57The goal of reaching universal primary education has gotten into our reach and UN member state efforts have most likely contributed to it. There appears to be a clear correlation between UN aid and progress being made (Source). Money seems to help solve the education problem, but is enough money available? And are governments actually spending enough money to reach the MDG? It turns out, finding out how governments spend their money is not so easy. The NGO Government Spending Watch has – just 32 months before the MDG deadline – released a report on whether governments are spending sufficient money to reach each of these goals. 20% of total spending is estimated to be sufficient to reach the goal, but only 12 out of 51 countries that have not reached universal primary education yet are spending that amount.

The graph below shows a handful of selected countries and their spending on education. EFA stands for Education for All, which is a new standard that includes pre-school care and fighting adult illiteracy. The same report also states that “Education spending has risen slightly, but most countries have been reducing it as % of budget or GDP” (Source). In other words, the piece of the GDP pie that is allocated to education is becoming smaller. That does not necessarily mean that spending on education is decreasing because pies can grow in absolute size! But still, targets are not being met by the large majority of countries.Screen Shot 2014-11-29 at 13.37.36

 

There appears to be insufficient money available to spend on education. The recent world economic crisis has caused an estimated $140 billion in revenue loss in poor countries. 40% of their extra spending was actually funded with borrowing money (Source). This drives countries into debt. How can countries be expected to keep investing in education if revenues are going down? Other sectors such as food and water security are obviously more important than education. This is probably where the money will and should go first. As a consequence, the international community needs to increase its total aid to help develop education systems. The OECD reports a $61.2 billion aid gap (for all MDG combined) that would have to be closed in order to reach targets (Source). Another final issue is that we do not know whether the UN aid is being spent on what it was initially meant for (corruption).

 

In Conclusion

Whether a kid can go to school very much depends on global factors such as the recent world economic crisis. But much of it is also luck. Being poor, a girl, living in a rural and/or conflict-affected area – these are all factors that reduce the chance of attending school. It is interesting to see how the spread of education goes hand in hand with other things we value: eradication of poverty and war and achieving equality. Due to the interconnectedness of these various topics we can work on one of these issues and solve others indirectly.

The MDG of achieving universal primary education by 2015 is a noble goal, but whether we reach similar goals in the future depends to a large extent on how much we are willing to help each other in terms of finances as well as how that money is ultimately being spent. Money does help and we need more of it because education is linear, not scalable. As long as there is no brilliant solution to change that we will have to raise our efforts.

Why Grades Should Measure Individual Progress Instead of Comparing Students to Each Other

Todays’ content:

  1. Why Do Schools Have Grades? What they measure and why they are so important to us.
  2. When Grades Compare: Why is it so important to compare students using grades?
  3. Effects of Comparison on Your Mindset: The negative impact grades can have on self-confidence and perception of own abilities.
  4. External Factors vs. Talent: How much of performance is more linked to effort rather than talent?
  5. Two Solutions: Self-Comparison and Uneven Support: Stopping comparisons and helping some more than others.
  6. Further Reading

 

Why Do Schools Have Grades?

I thought quite some time about what grades are designed to do, but the answer is surprisingly ambiguous. Obviously, grades are supposed to measure something, but what exactly they measure is unclear.

  • Is it memorization skills? Sure.
  • Is it effort? I hope so.
  • It it the ability to follow instructions? Maybe.
  • Is it how much a teacher likes you? Hopefully not.

A short stream of consciousness:
I remember my teachers pushing me to work hard and take something out of class (to learn something). Is learning the same as memorization or something more deeper? If so, how do we measure it? And if grades measure hard work, shouldn’t a talented student be graded worst for the same effort?

You can now see how difficult it is to come up with a definition for what grades are measuring. Turns out we don’t have to know what grades measure in order to talk about them. We can talk about their effects on students and what their role in a larger context is.

Memes-and-gradesRemember the feeling of dread when you got a bad grade or the immense joy when you aced your exam? The fact that we can experience such intense emotions when it comes to grades shows how important they are to us personally. When I was a kid I never thought about why I had to get good grades or what they mean. I simply wanted them because everyone else seemed to value them so much. My friends wanted them and my parents and teachers wanted me to have them as well. It felt too good to be praised for good grades – and it felt too bad to be scolded for bad ones. What motivated me to get good grades wasn’t what they would allow me to do one day, but rather the praise and good feelings that came with them. At a more advanced age I learned about university applications and job opportunities. The general rule was and still is that good grades open doors – opportunities.

 

When Grades Compare

class-average-was-76-percent-got-a-80I must admit that grades motivated me because they showed that I could be better than others. And you know how it goes: If one is at the top, another has to be at the bottom. Many of my classmates felt discouraged by bad grades and started falling behind. But I’d argue that this isn’t because of grades, but rather how they are used. Grades are used as a tool for comparison. They are usually not presented on an individual level but as a distribution of all grades in a class. The degree of comparison varies. Some countries list the students’ names along side the grades. There are some countries (e.g. Finland) and school types (e.g. Waldorf) that refuse to compare grades for most of the school time, but these are rather the minority. Competition does not stop in school, but extends to national and international test such as the SAT, GRE or PISA. What has happened, in essence, is that students’ performance has been relativized to other students. This creates a student hierarchy in which the top will feel great and the bottom sometimes completely devastated and these feelings are not entirely based on internal factors such as effort, but the external und uncontrollable performance of other students as well.

This would not be all too bad if grades didn’t have such tremendous influence over our future lives. Two examples.
In Korea, a students’ score on the final high school examination pretty much determines which university they can attend. In Germany and other countries, a student can only be admitted to certain programs such as medicine or psychology if their grade point average lies above a certain threshold that is defined over the student with the lowest GPA that got accepted the previous year. School ranking not only opens doors for some, but also shuts them for others. The ranking is a product of two things: limited positions and the wish to select applicants more fairly and easily for those positions.

From an economic perspective selection starts if there is a mismatch between demand and supply. If 500 students apply to 30 spots, 470 students will be disappointed. It is not that they were bad applicants, but that others were ‘better’. Universities want to recruit students with the best grades because it predicts quite well the likelihood that they will successfully pull through their studies (Source). There are of course other criteria that the admissions office looks at, but when spots are limited the most fair way of deciding who should get them are grades.
The take home message: competition occurs when there is a higher demand than supply.

I see two problems with comparison. First, giving students access to limited spots in academia and on the job market based on grades is crazy if we don’t know what they measure. What does better mean? Second, comparison can have detrimental effects on students, which I want to zone in on now.

 

Effects Of Comparison On Your Mindset

grades_01An example from my school life: When I moved from Washington D.C. (USA) to Beijing (China) I was put into a much tougher school. I had to work quite hard to keep up with classes and still I got pretty bad grades. I may have thought things like “I am a loser” or “I am stupid”. When saying such things I was essentially defining my abilities and self-worth over how good I was relative to others. I may have studied all night long, shown tremendous effort and still got a worst grade than my friends. The tables turned when I moved to Germany and switched to a less demanding school. I was suddenly among the best! Keep in mind that my performance didn’t change, but only the environment to which I was comparing myself. I may have thought things like “I am smart” and “studying was totally worth it”. It is commonly known that university freshmen – often having been among the best at their school – now face a completely different point of comparison: the best students of other schools. The same vicious self-doubt cycle starts and can be extended to job life, family life and others. I personally switched from “I am a loser” to “I am smart” because I was fortunate to experience a different environment. Therefore, comparison is essentially nonsense and the effects can be very detrimental to a students’ beliefs in themselves.

I recently read Carol Dweck’s book ‘Mindset’, which I’d like to bring in at this point. She is a psychologist at Stanford University and has spent decades researching how ones mindset can have dramatic influences on success in life. I’ll let her describe it in her own words:

“In a fixed mindset students believe their basic abilities, their intelligence, their talents, are just fixed traits. They have a certain amount and that’s that, and then their goal becomes to look smart all the time and never look dumb. In a growth mindset students understand that their talents and abilities can be developed through effort, good teaching and persistence. They don’t necessarily think everyone’s the same or anyone can be Einstein, but they believe everyone can get smarter if they work at it.” (Source)

Comparison can lead to a loss in self-confidence and a fixed mindset. I have experienced it first hand and I know that many other students suffer from it as well. I think it is one of the main reasons why some students don’t enjoy school at all and stop care to learn and be curious. This not only has detrimental effects on students, but also on society. I think we don’t want people that seek elite study programs and well paying jobs because it makes them feel good, but rather individuals that learn to improve themselves and act out of conviction.

 

External Factors vs. Talent

I certainly thought of myself as a loser when comparing myself to my friends who got better grades. And since effort didn’t change much of my situation I was led to believe that some people were simply more gifted than others. Research is starting to show that abilities are much less set in stone and more malleable than previously assumed. Students are not aware of disadvantages they might have experienced from early on, but which greatly influence their performance in school. A simple example is dyslexia (difficulty learning to read), which obviously undermines student performance in school.

Another external factor is money. “According to figures, 64 per cent of students from independent schools went on to these universities [Oxford, Cambridge, Imperial College London, University College London and other members of the Russell Group] in 2010/11, compared with 24 per cent from state schools – a 40 percentage point gap.” (Source)

Wealthy parents can send their kids to private (i.e. independent) schools and these schools may have a good reputation, which increases chances of students to get accepted at good universities. But it might also just be that private school prepare their students better! Getting a worse score on the SAT when coming from a public school and going against mostly such well prepared and well funded elite students is hardly surprising.

vocabularyOther factors have to do with child rearing. Let’s suppose two first graders begin with school. One of them – Kid A – has parents with a professional background and the other one – Kid B – has parents on welfare. Studies have shown that the number of words an infant hears in its early years predicts its academic success as well as IQ. A study from 1995 has shown that professional parents talk more to their infants than parents on welfare, which means that Kid A who has professional parents will most likely be better in school (Source).

There are countless studies that show how external factors can have an effect on our abilities. Effort leads to improvement, no doubt about that. But all these external factors are not being considered when comparing kids to each other. Students have different backgrounds and therefore also different starting points.

Talent may very well be something that exists, but its effect is much smaller than we usually think. Geniuses have to work very hard for their achievements sometimes much harder than others. Dweck writes “Michael Jordan wasn’t a natural, either. He was the hardest-working athlete, perhaps in the history of sport” (Source). Beethoven centered his whole life around music, which was often the only source of joy in his life. The famous journalist Malcom Gladwell stresses in his book ‘Outliers’ how extraordinary performance has more due to do with hard work rather than talent (Source). Even Kenyan marathon runners are by some not thought to be superb performers due to some inborn abilities, but rather due to their cultural training in enduring incredible amounts of pain (Source).

 

Two Solutions

I want to propose two ways of improving the situation in schools. First, grades should not be used to compare students’ abilities, but rather individual progress. Second, we should recognize and combat disadvantages that students have relative to others.

 

Self-Comparison

We know now that grades – although thought of as being a fair way of measuring performance – actually aren’t fair at all.

confidentWe want students to advance, but we shouldn’t do it by creating a performance hierarchy that compares apples with pears. This creates irrational fear for apples to perform like pears and vice versa. Consequently, students’ self-confidence will suffer and they begin believing in a fixed mindset – the belief that abilities are inborn rather than trainable. Instead of comparing students to each other, we should compare them to themselves so that their motivation for learning about being better than their current selves. Emphasizing personal progress in schools maintains self-confidence and helps develop a growth mindset – the belief that abilities are trainable rather than inborn.

5510922The ‘Evangelische Schule Berlin Zentrum’ led by the headmaster Margret Rasfeld has shown that this can work. At this school, students don’t take an exam together, but each one does it on their own – whenever they feel ready. This flexibility adjusts to the learning pace of students rather than forcing them to work under time pressure. Furthermore, grades are not used for comparison, but rather to measure individual progress (Source). The students even outperformed other students from a comparable school in Hamburg. Although having started with lower performances the students outperformed the Hamburg kids in reading, grammar and natural science as well as equaled out in English and math (Source).

The school is actively trying to build a close community among teachers and students and remove sources of fear. Mrs. Rasfeld and her students want to create a school that is enjoyable for all.

 

Uneven Support

Screen Shot 2014-11-15 at 18.14.56If students perform differently due to their personal advantages (biological or environmental) but are treated equally in school, they suffer a disadvantage. A logical consequence would be to invest more time and money into more needy students so that everyone performs about equally. Extra tutoring classes for students with dyslexia or learning disabilities would be one example. Amanda Ripley describes in her book ‘The Smartest Kids in the World: And How They Got That Way” how the Finish education system does exactly that. Here is an excerpt from the book:

“Until students reached age sixteen, though, Finnish schools followed a strict ethic of equity. Teachers could not, as a rule, hold kids back or promote them when they weren’t ready. That left only one option: All kids had to learn. To make this possible, Finland’s education system funneled money toward kids who needed help. As soon as young kids showed signs of slipping, teachers descended upon them like a pit crew before they fell further behind. About a third of kids got special help during their first nine years of school. Only 2 percent repeated a grade in Finnish primary school (compared to 11 percent in the United States, which was above average for the developed world).” (Source).

This example shows very nicely how abilities can be developed with the right support (fostering growth mindset), but more importantly it shows it is possible to build structures that create more equal opportunities than we currently have.

 

Concluding Remarks

Although designed to be fair, grades are actually not as fair as we thought or still think. When considering that students can be vastly different from each other, comparison makes little sense. Yet, our education system has been built around these numbers although we don’t even know for sure what they measure. As a side-effect, comparison creates fear of failure, lack of self-confidence and a belief that abilities are more fixed than they are. The fixation on grades leads to the loss of curiosity for the subject matter.

The solution is to use grades, not to compare students to each other, but students to themselves. Teachers initially set out to teach their students and they will be able to do it much better if students can learn and improve themselves without the fear of failure. Individuals with  disadvantages should receive additional support so that we can come closer to the ideal of true equal opportunities.

 

Further Reading:

  1. The Smartest Kids in the World: And How They Got That Way – by Amanda Ripley
  2. How Children Succeed: Grit, Curiosity, and the Hidden Power of Character – by Paul Tough
  3. Outliers – by Malcom Gladwell

  4. Learning in Finland – by David Rothenberg