Article Information

Svea Nicolaisen^1*, Christa Thöne-Reineke¹, Lisa Buchwald², Harm Kuper², Mechthild Wiegard¹

¹Institute of Animal Welfare, Animal Behavior and Laboratory Animal Science, School of Veterinary Medicine, Freie Universität Berlin, 14163 Berlin, Germany

²Department of Education and Psychology, Freie Universität Berlin, 14195 Berlin, Germany

^*Corresponding Author: Svea Nicolaisen, Institute of Animal Welfare, Animal Behavior and Laboratory Animal Science, School of Veterinary Medicine, Freie Universität Berlin, 14163 Berlin, Germany

Received: 23 August 2023; Accepted: 04 September 2023; Published: 18 September 2023

Citation: Svea Nicolaisen, Christa Thöne-Reineke, Lisa Buchwald, Harm Kuper, Mechthild Wiegard. Analysis of the Knowledge Level of Slaughterhouse Employees and Evaluation of Online Training to Improve Animal Welfare. Journal of Food Science and Nutrition Research. 6 (2023): 127-138.

Abstract

Keywords

Animal welfare, Knowledge test, E-learning, Slaughter, Training, Evaluation

Article Details

Introduction

Animal welfare is an important topic in the current public debate [1]. In Western society, the view of the social environment and the perception of animals and their use by humans has changed [2]. According to Dirscherl [3], animal ethical issues are increasingly included in social discourse. The inclusion of animal welfare as a state target in the German Basic Law in 2002 clearly expresses society's changing values and mindsets [4]. Statements critical of society regarding modern livestock farming and the associated production of food of animal origin have been increasingly voiced in recent years [5]. The debate focuses on animal welfare concerns and considerations of more animal-friendly husbandry and meat production systems [5]. In a representative survey conducted in Germany, a large proportion of respondents rejected methods that cause animal suffering during transport and slaughter [1]. In particular, meat production is not only about aspects of animal husbandry, but also about animal transport and the handling of live animals in the slaughterhouse [6]. The German Veterinary Association (Bundestierärztekammer e. V.) also confirms that, regardless of the size of the slaughterhouse, there are deficits in animal welfare between the process steps of unloading up to bleeding [7]. Their attitude towards livestock can influence slaughterhouse employees’ behavior, which can impact the welfare of the animals [8]. The Animal Health and Welfare Committee of the European Food Safety Authority identified and characterized 40 risks to cattle welfare during slaughter [6]. In total, 39 of those 40 risks were caused by humans and are mainly associated with lack of skills or fatigue [6]. Slaughterhouse employees who have direct contact with animals must be trained to ensure animal welfare during slaughter [1,9]. National and European laws and regulations provide the general requirements for the protection of animals during slaughter and transport. Transport of animals must be carried out by approved transportation companies and by livestock drivers holding a certificate of competence [10]. Reg. (EC) No. 1099/2009 on the protection of animals at the time of killing regulates the handling of animals during stunning and slaughter. Only persons with a certificate of competence can stun and slaughter animals [11]. The certificate of competence is acquired after a training course, or after acknowledgment of and apprenticeship or study [11]. In Germany, the provisions of Regulation No. 1099/2009 are implemented and supplemented by the Animal Welfare Slaughter Ordinance [12]. This ordinance requires that for the certificate of competence, both theoretical and practical examinations must be passed [12]. The certificate of competence is issued by an official institution, and animal welfare officers at slaughterhouses can contribute to personnel training by providing them with the background information necessary to obtain the certificate [13]. Animal welfare officers are responsible for verifying that personnel have the necessary knowledge to carry out their duties and for ensuring that these competencies are maintained [13]. The certificate of competence is valid indefinitely, unless it is withdrawn after violation of Regulation (EC) No. 1099/2009 [12].

The aims of the study were to analyze the level of knowledge of slaughterhouse staff before and after participation in an e-learning program and to test and evaluate the conception and implementation of the animal welfare training. This study was conducted as part of the joint research project eSchulTS2 (development of target group-specific learning modules to improve animal welfare during the transport and slaughter of cattle and pigs and is intended to support improvement of online training materials.

2. Materials and Methods

2.1 Online Platform

The online platform tet.folio [14], located on a server of the Free University of Berlin, was used to create the e-learning program. The project team eSchulTS2 were the only authors who had access to the administration and stored data of the e-learning platform. No server, platform or content maintenance or updating was scheduled during the study period.

2.2 Design of the e-learning program and the knowledge test

The e-learning program consisted of two pilot modules, and the languages German or Romanian could be selected in advance. The first pilot module, animal behavior, consisted of four sections with three training videos and a final quiz. The main topics in this module were to understand the sight and hearing of cattle and to recognize their different states of mind (Table 1). The second pilot module, cattle handling, contained five sections, five training videos, and a final quiz. In this module, the main topics were animal-friendly moving of cattle with and without driving aids (Table 1). For each module, the same knowledge test (Table 2) was placed before (pre-test) and after (post-test) the learning. In order to assess the respondents' level of knowledge, the questions were the same before and after each pilot module. The questions were developed within the project team and adapted to the content and wording of the modules. In order not to overburden the participants and to prevent them from dropping out of the modules, there were only eight questions per pilot module (four before and four after the module). Each question had three different possible answers, but only one answer was correct. Test questions were single-choice questions; the pre- and post-test were each scored 4 points in each module. The knowledge test had accessibility-friendly options, whereby participants clicked on loudspeaker symbols to listen to the questions and possible answers.

Table 1: Topics and objectives of the pilot modules.

Table 2: Design of the knowledge test.

2.3 Design of the evaluation

The evaluation was included in the tet.folio program and had to be filled in by the participants at the end of each module. For each module, the evaluation contained 13 statements and 2 open questions (Table 3). The answers to the evaluation statements used a five-point Likert scale: 1 (I fully agree), 2 (I agree) representing an agreement and 3 (neither nor) representing neither agree nor disagree, 4 (do not agree) and 5 (do not agree at all) representing a disagreement. To keep the evaluation simple and understandable, the Likert scale was presented in the form of five different smileys that could be selected.

Table 3: Design of the evaluation.

2.4 Setting

The knowledge test took place from December 2022 to the end of January 2023. In prior consultation with the cooperating slaughterhouse companies an internet-capable computer with loudspeaker or headset was provided for the participants. The participants were employees in various German cattle slaughterhouses. The animal welfare officers of the slaughterhouses were sent the link to the pilot modules, a list of user names and passwords, and an Excel spreadsheet to be filled in by them with further information about the participants. The animal welfare officer collected the following demographic information from participants after they completed both knowledge tests: age, gender, position in slaughterhouse, years in slaughterhouse, years of formal education, country of origin, and ability to read and write. Each participant was seated alone at a computer and provided with their own username and password in order they could log in and to enable retrospective linking of participant with collected data. After the knowledge tests and evaluations were completed, the animal welfare officers sent the anonymized MS Excel (Microsoft®) spreadsheet to the authors. In addition, the cooperating slaughterhouse company was not able to see how their employees performed in the knowledge test. The participants were not provided with their knowledge test results. After the end of the response period, the results of the knowledge tests and evaluations were exported from tet.folio to MS Excel, and the statistical program SPSS (IBM®) was used to carry out a statistical evaluation in the form of descriptive statistics and a t-test (α = 0.05) to compare the pre- and post-tests.

3. Results

3.1 Demographic data

A total of 25 participants viewed both pilot modules, completed the knowledge tests, and evaluated the modules. The demographic characteristics of the study participants are shown in table 3. Most (22/25) participants chose German as their language for e-learning, while 3 participants chose Romanian. The group had an age average of 42.9 years, the youngest participant was 28 and the oldest 65 years old (Table 3). Of the 25 participants, 7 were female, 17 were male and one unknown. Most (86.4%) of the German-speaking participants had a 3-year apprenticeship or a higher degree, while 2 of the 3 Romanian participants had no apprenticeship or higher degree. Most participants had a position in slaughterhouse management (32.0%), followed by butchers (28.0%) and veterinarians (20.0%). Participants’ countries of origin were Germany, 64.0%, Romania, 12.0%, and other countries, 5.0% (Ukraine, Argentina, Iraq, and Russia). The German-speaking participants had an average of 6.5 years of experience in the slaughter industry and the Romanian-speaking participants 9.7 years. One participant reported they were unable to read and write.

Table 3: Demographics of slaughterhouse employees by language selected.

3.2 Knowledge test

The participants needed an average of 35 minutes to complete both modules, including the knowledge tests (Table 4). The German-speaking participants needed an average of 6 minutes less time than the Romanian-speaking participants. German-speaking participants had a higher score (3.7) on the pre-test of the animal behaviour module than Romanian-speaking participants, but Romanian-speaking participants scored better on the post-test (4.0) (Table 5). Romanian-speaking participants performed better in the pre- and post-test (4.0, 4.0) of the cattle handling module than the German-speaking participants (3.7, 3.9). Scores for both language groups significantly improved from the pre-test to the post-test (P < 0.05). The bar chart shows that participants with no apprenticeship scored lower on the pre-test than participants with apprenticeship or a higher degree (Figure 1). In pre-tests, more correct answers were selected in cattle handling than in animal behaviour (Figure 2). In the pre-test, the first question (Q1 Pre AB) in animal behaviour and the fourth question (Q4 Pre CH) in cattle handling were the most frequently incorrectly selected answers, both with 12% (Figure 2). At post-test, the second question (Q2 Post AB) in animal behaviour and questions three (Q3 Post CH) and four (Q4 Post CH) in cattle handling were answered 100% correctly (Figure 2).

Table 4: Time required for modules and knowledge tests

Table 5: Pre-test score, post-test score, and difference in test scores by language group. Significance is denoted by superscripts * at P < 0.05.

Figure 1: Relationship between number of participants and the level of education in the pre-test.

Figure 2: Pre- and post-test comparison of the number of correct and incorrect answers from both modules.

3.3 Evaluation

In the first part of the animal behavior module evaluation, participants were asked to rate the curriculum. Most, 87.0% (n = 20), fully agreed that the course was clearly structured in terms of content. The remaining 13.0% (n = 3) agreed. The majority, 82.6% (n = 19), of participants fully agreed, and 17.4% (n = 4) agreed that the training was taught in a way that could be understood. Among participants, 87.0% (n = 20) fully agreed and 13.0% (n =3) agreed that they could understand the aim of the training (Figure 3). The second part of the animal behavior module evaluation focused on usability. Almost all participants, 91.3% (n = 20), fully agreed that the photos and videos were easy to view, and 8.7% (n = 2) agreed. Drawings and animations were easy to recognize, as 87.0% (n = 20) fully agreed and 13.0% (n =3) agreed. Of the participants, 82.6% fully agreed, 13.0% (n = 3) agreed, and 4.3% (n = 1) neither agreed nor disagreed that the narrator spoke in an understandable manner. Most, 91.3% (n = 21), fully agreed with the statement that they were able to orient themselves well in the module, and two participants (8.7%) agreed (Figure 3). The third part of the animal behavior module evaluation focused on comprehension. Of the participants, 82.6% (n = 19) fully agreed and 17.4% (n = 4) agreed that they understood all the words in the training. The majority (91.3% (n = 21) fully agreed and 8.7% (n =2) agreed) confirmed they understood the content of the training and that the scope of the training was appropriate for solving the tasks (Figure 3). The fourth part of the animal behavior module evaluation dealt with transfer of knowledge from training to practical work. Participants felt the content of the training related thematically to their own work; 87.0% (n = 20) fully agreed, 8.7% (n = 2) agreed and one participant neither agreed nor disagreed. For the statement 'I learned something new for my job during training', 56.5% (n = 13) fully agreed, 13.0% (n = 3) agreed, 17.4% (n = 4) neither agreed nor disagreed, but 4.3% (n = 1) disagreed, and 8.7% (n = 2) did not agree at all. When asked to rate the statement, 'what I learned in training I can apply to my work in the future', 65.2% (n = 15) fully agreed, 26.1% (n = 6) agreed, 4.3% (n = 1) neither agreed nor disagreed, and one participant did not agree at all (Figure 3).

Figure 3: Results of the evaluation of the animal behavior module in the areas of curriculum, usability, comprehension, and transfer (n = 23).

In the first part of the of the cattle handling module evaluation, participants were asked to rate the curriculum (Figure 4). Most, 79.2% (n = 19), fully agreed that the course was clearly structured in terms of content as 12.5% (n = 3) agreed and 8.3% (n = 2) neither agreed nor disagreed. Most 79.2% (n = 19) participants fully agreed, 8.3% (n = 2) agreed, and 12.5% (n = 3) neither agreed nor disagreed that the training was communicated in an understandable manner. Among participants, 79.2% (n = 19) fully agreed, 16.7% (n = 4) agreed, and 4.2% (n = 1) neither agreed nor disagreed that they were able to understand the aim of the training (Figure 4). The second part of the cattle handling module evaluation focused on module usability. Almost all participants, 79.2% (n = 19), fully agreed and 16.7% (n = 4) agreed, although 4.2% (n = 1) disagreed that the photos and videos were easy to view. That the drawings and animations were easy to recognize, 79.2% (n = 19) fully agreed and 20.8% (n = 5) agreed. Of the participants, 83.4% (n = 20) fully agreed and 16.7% (n = 4) agreed that the narrator spoke understandably. To the statement that they were able to orient themselves well in the module, 83.3% (n = 20) fully agreed and four participants (16.7%) agreed (Figure 4). The third part of the cattle handling module evaluation focused on participants’ understanding of the module. Of the participants, 91.7% (n = 22) fully agreed and 8.3% (n = 2) agreed that they understood all of the words in the training. The majority, 83.3% (n = 20), fully agreed and 16.7% (n = 4) agreed that they understood the content of the training. Altogether, 87.3% (n = 20) of participants fully agreed and 12.5% (n = 3) agreed that the scope of the training was appropriate for solving the tasks (Figure 4). The fourth part of the cattle handling module evaluation focused on knowledge transfer from module training to practice. Most participants, 75.0% (n = 18) fully agreed that the training related thematically to their work, 12.5% (n = 3) agreed, and 12.5% (n = 3) neither agreed nor disagreed. Over half the participants, 62.5% (n = 15), fully agreed that they had learned something new for their jobs during training, 4.2% (n = 1) agreed, and 16.7% (n = 4) neither agreed nor disagreed, but 4.2% (n = 1) disagreed or did not agree at all (12.5%; n = 3). When asked about the statement, ‘what I learned in training I will be able to apply to my work in the future,’ over three-quarters of participants either fully agreed, 70.8% (n = 17), or agreed, 12.5% (n = 3), while 4.2% (n = 1) neither agreed nor disagreed, but 4.2% (n = 1) disagreed, and 8.3% (n = 2) did not agree at all (Figure 4). A total of 10 comments were written in reply to the open questions (Table 6).

Figure 4: Results of the evaluation of the cattle handling module in the areas of curriculum, usability, comprehension, and transfer (n = 24).

Table 6: Positive comments (+) and suggestions for improvement (-) for the modules written by the participants. Comments were translated and summarized by the authors from the German originals.

4. Discussion

Ultimately, the aim of this e-learning training course for slaughterhouse employees was to demonstrate whether employees who participated in the online training were able to improve their knowledge of cattle handling and animal behavior, judged by pre-test and post-test scores. The demographic data on the selected language in the e-leaning training course (German or Romanian) in relation to the reported demographic information (country of origin) was consistent. The demographic data collected also confirmed previous findings; most slaughterhouse workers are from Germany, followed by a large number of workers from Romania [15]. In terms of the time required for the modules, the Romanian-speaking participants took longer than the German-speaking group. The Romanian translations kindly provided by employees of the slaughterhouse involved in the project, and it is possible the translations were difficult, which could have resulted in the Romanian-speaking participants taking longer for the modules. Marchitan [16] stated that Romanian is a Romance language that has preserved the historical meaning for many lexical items, which leads to confusion for translators who do not have a complete command of the language. It is possible that the group of those who took longer to complete the modules than others had reading deficits, even though accessibility was high, as loudspeaker symbols allowed participants to listen to all texts. Due to work commitments, it is also possible that some employees could take less time for the training than others. One limitation of this study was that the selection of participants was carried out by animal welfare officers. It is, therefore, not clear if these participants simply had time for the training, or if they were especially chosen in order to perform as well as possible in the tests. The animal welfare officers stated that all participants could read and write, but it can be assumed that illiterate people also work at slaughterhouses, and it would have been interesting to find out how these staff would have performed. In Germany, about 6.2 million adults are considered functionally illiterate, and 47.4% of them do not have German as their language of origin [17]. A limitation of the study is the small number of only 25 participants. However, it should be taken into account that the number of people working at cattle slaughterhouses in Germany is smaller than the in pig slaughterhouses [18]. Accordingly, there are probably fewer available employees who can participate in training cattle welfare compared with in pig welfare. Another limitation of the study was that the post-tests were only administered once per module, immediately after module completion. While these measurements could indicate a short-term memory gain, it is impossible to know how much knowledge was retained by course participants in the long term- for this, additional post-tests are needed after several weeks or months. In addition, further exploration of the impact of the training on Romanian-speaking participants was limited in this study due to the small sample size (n = 3). Both language groups (German and Romanian) improved between pre-test and post-test scores. In the animal behavior module, the Romanian-speaking participants showed the greatest differences between the pre-test and post-test scores, even though their pre-test scores were the lowest of the two language groups. In the pre-test, the levels of knowledge in both language groups were high, possibly due to the participants' years (mean 6.9 years) of experience in slaughterhouse work. These results show that the participants already had good knowledge of animal behavior and cattle handling before the online training. This could be due to the fact that the slaughterhouses, in addition to the certificate of competence, also conducts further training for the employees on the topic of animal welfare. A previous study conducted in German-speaking countries showed that slaughter companies train their employees primarily once per year [15]. In the pre-test, the employees with post-school education performed better than the employees without training. It should be noted that it is not easy to compare training from abroad with the German standards for education. However, immigrants can have lower skills than natives, including German reading skills, and lower levels of education [19-21]. Questions Q1 Pre AB and Q4 Pre CH were the most frequently answered incorrectly. In previous courses/training, these questions could have been provided with misleading answers or been covered inappropriately. Questions Q2 Post AB, Q3 Post CH, and Q4 Post CH were answered 100% correctly in the post-test. In contrast to the other sections, there were two animated videos in the cattle handling module that discussed the learning content of questions Q3 Post CH and Q4 Post CH. A previous study showed that video-based e-learning with animation can lead to better learning outcomes [22]. It is possible that this type of knowledge transfer was particularly effective for our participants. In the evaluation, there was great agreement among the participants concerning the animal behavior module in the areas of curriculum design and comprehension of the module. In the usability of the animal behavior module, there was one rating of 'neither nor' for the statement on the way the narrator spoke. Since most of the participants agreed, it is possible that this participant had technical problems with the speakers or headphones on their computer. For the animal behavior module, the greatest number of negative evaluations were given in the area of knowledge transfer. Three participants disagreed with the statement that the training had taught them something new for their work. The reasons for this could be that the participants were well trained or that this training was not comprehensive enough. It is also possible that some participants were overconfident in stating their information competencies compared to their actual competencies, so displaying behavior known as the Dunning-Kruger effect [23]. One participant disagreed with the statement that they can use the information learned for their work in the future. This may be because this participant does not work in the live cattle sector of the slaughterhouse, or the training did not provide sufficient information for their work. For the cattle handling module, some participants neither agreed nor disagreed with the curriculum design in terms of the structure of the learning content, communication, and objectives of the training. The reason for this could have been the learning objectives of the cattle handling module, which deals, among other things, with the controversial topic of electric prodding. However, the majority of participants agreed with the design of the curriculum. The transfer aspects of the cattle handling module, as for the animal behavior module, were not evaluated agreeably in terms of what was newly learned and how it could be applied to future work. Again, this could be because the participants were very well educated, or the training module had no new information to offer. In the open questions on the animal behavior module, there were comments that real videos of aggressive cattle were missing and that the statements on the topic of vision should be made more concrete. These comments are useful and should be taken into account in further revision of the training. In the cattle handling module, one participant commented, 'it is not clearly regulated by law how often the electric prod can be used', which is untrue, as Regulation No. 1099/2009 states that electric prodding of only a maximum of one second duration can be applied to cattle, and it may not be repeated if the cattle do not react. One participant criticized the presentation of the separation of cattle while using the electric prod, which we regard as a good suggestion that should be taken into account in the module revision. For the comment, 'the statement about the driving aids is not well chosen, the driving paddle belonged in with it', it was unclear what exactly statement should be improved and included in a revision. The modules were generally given positive approval ratings by the participants, which indicates the modules provided further information and knowledge. However, there is room for module improvement in some areas, as some participants expressed suggestions for improvement.

5. Conclusion

This study shows that employees at slaughterhouses were able to improve their measured knowledge of cattle handling and animal welfare through the online training provided in this study. The two modules studied should be improved in accordance with the results of this study. Given the largely positive view participants have of the training they undertook, other e-learning materials with different learning objectives around animal welfare at slaughter should be made available to employees. The development and implementation of culturally, intellectually, and linguistically appropriate training for slaughterhouse employees must be future priorities for slaughterhouse companies.

6. Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

7. Funding

This study was conducted in the framework of the joint research project eSchulTS2 (Development of target group-specific e-learning modules to improve animal welfare during transport and slaughter of cattle and pigs) which is funded by the German Ministry of Food and Agriculture (grant no. 2817806A18). The publication of this article was funded by the Freie Universität Berlin.

8. Acknowledgments

We would like to thank the members of the entire eSchulTS2 team for their excellent cooperation.

References

1. German Ethics Council. Animal welfare - the responsible handling of livestock (2020).
2. Gottwald FT. Creatures like us. On the responsibility of humans for farm animals. Munich (2004).
3. Dirscherl C. Meat consumption and animal husbandry in the current socio-ethical debate. Berichte über Landwirtschaft 91 (2013): 3.
4. Deimel I, Franz A, Frentrup M, et al. Perspectives for a European Animal Welfare Label (2010).
5. European Food Safety Authority. Animal Welfare at slaughter (2020).
6. Tiedemann U. It is undisputed that there are deficits in animal welfare. Fleischwirtschaft 6 (2021): 8-9.
7. Hultgren J, Wiberg S, Berg C, et al. Cattle behaviours and stockperson actions related to impaired animal welfare at Swedish slaughter plants. Applied Animal Behaviour Science 152 (2014): 23-37.
8. Iulietto MF, Sechi P, Gaudenzi CM, et al. Noise assessment in slaughterhouses by means of a smartphone app. Italian Jorunal of Food Science 7 (2018): 79-82.
9. European Commission. Council Regulation (EC) No 1/2005 of 22 December 2004 on the Protection of Animals during Transport and Related Operations and Amending Directives 64/432/EEC and 93/119/EC and Regulation (EC) No 1255/97 (2005).
10. European Community. Council Regulation (EC) No 1099/2009 of 24 September 2009 on the Protection of Animals at the Time of Killing. (2009).
11. Regulation on the protection of animals in connection with slaughter or killing and on the implementation of Council Regulation (EC) No 1099/2009 (Tierschut -Schlachtverordnung-TierSchlV) (2012).
12. European Commission, Directorate-General for Health and Consumers. The animal welfare officer in the European Union, European Commission (2015).
13. Haase S. The Technology Enhanced Textbook. (2016).
14. Isbrandt R, Meemken D, Langkabel N. Animal welfare training at cattle and pig slaughterhouses - results of an online survey from German-speaking countries. Berliner und Münchener Tierärztliche Wochenschrift 135 (2022): 1-16.
15. Marchitan M. A translator´s false friends: dealing with English-Romanian language cognates. Articol (2021): 1-7.
16. Grotlüschen A, Buddeberg K, Dutz G, Heilmann L, Stammer C. LEO 2018 - Living with low literacy. Hamburg, Germany: Press brochure (2019).
17. Federal Office for Agriculture and Food. Meat Market and Supply Situation Report, (2023).
18. Stanat P, Rauch D, Segeritz M. Students with a migration background. In: Klieme E, Artelt C, Hartig J, Jude N, PISA 2009: Taking stock after a decade. Münster: Waxmann (2010), 200-230.
19. Autorengruppe Bildungsberichterstattung. Education in Germany. Bielefeld: Bertelsmann Verlag (2012).
20. Maaz K, Dumont H. Educational acquisition by social origin, migration background and gender. In: Köller O, Hasselhorn M, Hesse F, Maaz K, Schrader J. The Education System in Germany. Current status and potential. Bad Heilbrunn: Verlag Julius Klinkhardt (2019) : 299-332.
21. Bilal EL-Ariss B, Essam Zaneldin E. Waleed Ahmed, W. Using Videos in Blended E-Learning for a Structural Steel Design Course. Education Science 11 (2021): 290.
22. Mahmood K. Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Kruger effect. Communications in Information Literacy 10 (2016): 198-213.