Judging Learning Effectiveness During Learning

When students study, they may monitor their progress during a study session by periodically forming judgments on how well they are remembering the material. Such judgments guide how much further study is deemed necessary. Researchers have studied this matter in the case of paired associate learning (where you learn lists of word pairs like dogs-cats, newspaper-book, etc.).  In particular, researchers looked for correlations between later memory recall either immediately after learning or after a short delay in which judgments about learning are based on a covert attempt to recall. Results indicate that making judgments about how well something will be remembered can be just as efficient as taking an actual test.

In delayed judgments, the student typically makes an initial covert recall effort and then, based on that, judges how well the material was learned. Future recall tends to correlate with predictions on recall on a future test. That is not so surprising, other than the fact that other studies have shown students over-estimate what they have learned and under-estimate how much additional student would be beneficial.

A retrieval attempt directly reveals evidence of how well memory has formed. The act of retrieval itself may enhance learning. Successful retrieval could constitute an additional reinforcing learning opportunity. Indeed, other studies have shown that testing may lead to better final recall than a comparable amount of study. When an item is retrieved in covert recall and leads to a high judgment of learning, the item gets a long-term memory boost.

In the present study, the researchers directly compared final recall and delayed judgments of learning of paired association of lists of 40 words in Swedish (the native language of the subjects) and Swahili under differing testing conditions. One hundred twenty-one Swedish college students were divided into experimental groups: 1) repeated study and testing (study-test, “ST group”), 2) repeated study and termination of testing after the first successful recall test (study-test, dropout,“STd” group), and 3) repeated study and judgments of learning (study-judgment of learning “STjol” group).

Testing involved presenting the first word of a pair to serve as a cue to probe for recall of the associated word. All groups received four initial learning episodes with 5 seconds per item, after which they had 8 seconds to respond to test on the item (ST, STd) or render a judgment on their prediction of cued recall for that item a week later (STjol). All groups were compared for their performance on the same test a week later.

The ST group went through four successive study-test sessions, in each of which they studied all 40 word pairs on a computer screen. Immediately at the end of the list, the students took a 30-second distractor test (math quizzes). Then their recall was explicitly tested by presenting each Swahili word, whereupon they had 8 seconds to provide the Swedish equivalent. Students experienced four such study and test sessions. A similar procedure was used for the STd group, except that on any given test, each correctly recalled item was dropped from subsequent tests; thus the number of word pairs dropped from 40 on the first test to the number of pairs missed on the previous test.

The STjol group experienced a similar process including the 30 second distractor task, except that the test trials were replaced by jol trials. That is, instead of being required to provide the Swedish word that matched the Swahili probe word, the subjects were given 8 second to render a judgment for each word pair by answering this question: “How certain are you that you will recall the Swedish word in a week when we test you again? “ Students used a rating scale of 20% sure, 40%, 60%, 80%, 100%.

During the learning phase, the two ST groups increased their scores at about the same rate from the first session to the fourth. Thus, dropping a pair from testing once it was recalled correctly did not seem a disadvantage to learning. Authors assume the jol groups would have increased scores similarly, but of course they were not explicitly tested during the learning phase. Their prediction scores did, however, increase over the four sessions at a similar rate as recall did in the ST groups.

The key issue was elucidated on the memory test a week later. The ST group had better recall than the STd group, thus revealing that dropping items during study had long-term consequences. This is reminiscent of studies by others on flash cards that showed that best long-term recall was produced by re-testing with all cards in the deck, including those that were

answered correctly in a previous self-test.

The jol group performed better on the final test than the STd group but results were about the same as those for the ST group. Thus, making a delayed judgment during the learning phase about how well one has learned was just as effective for recall a week later as actually being tested during the learning phase.

What do we make of that? It seems that to make a prediction for ability to recall word pairs, a person has to first make a covert recall effort. If you covertly recall a word readily, you would like give a judgment rate of 100%, whereas if you struggled with covert recall, you might judge future recall at only 40%, for example. To make such judgments, the learner has to monitor the learning in real time. Such monitoring in order to render a judgment entails covert self-testing, which these results suggest is just as effective long term as explicit testing.

Such results confirm what we know about memory being promoted by self-testing, whether explicit or covert. As a practical matter for study strategies where it is inconvenient to take actual tests during study sessions, it prudent to conduct covert self-testing wherein a student asks questions like “how well have I remembered this item?” If the answer is “not well,” more study is called for.  Confident judgments will tend to be confirmed when taking a real test later. In other words, making judgments about learning effectiveness during a learning phase helps a student to monitor progress and know how much time to devote to study. In addition, making such estimates seems, in itself, to promote learning because covert self-testing is required.

Sources:

Jönsson, F. U., Hedner, M., and Olsson, M. J. (2012). The testing effect as a function of explicit testing of instructions and judgments of learning. Experimental Psychology. 59 (5): 251-257.

Roediger, H. L., and Karpicke, J. D. 2006. Test enhanced learning. Taking memory tests improves long-term retentio