Ex) Article Title, Author, Keywords
New Phys.: Sae Mulli 2022; 72: 311-318
Published online April 29, 2022 https://doi.org/10.3938/NPSM.72.311
Copyright © New Physics: Sae Mulli.
Sentot Kusairi1, .. Sutopo1, Ahmad Suryadi2*
1Department of Physics, Faculty of Mathematics and Natural Sciences, Universitas Negeri Malang, Malang 65145, Indonesia
2Department of Physics Education, Faculty of Education and Teacher Training, Universitas Islam Negeri Syarif Hidayatullah Jakarta, Jakarta 15412, Indonesia
Correspondence to:*E-mail: firstname.lastname@example.org
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Isomorphic multiple-choice items (IMCI) are a potential tool to diagnose students' misconceptions. This survey research aimed to identify the characteristics of students' responses on IMCI and diagnose students' misconceptions. The relationship between students' responses to diagnostic results and their success in solving open-ended items was also investigated. The IMCI consisted of nine multiple-choice items that were categorized into three IMCI groups according to the force and motion concepts covered in the questions. To evaluate the accuracy of IMCI diagnosis, an open-ended problem was also developed. Referring to their responses to the IMCI, students were classified into three groups: those who understand the concept, those who hold misconceptions, and those who are inconsistent. It was found that students who understand the concept well also demonstrated a high ability to solve the open-ended problem as shown by the average score of those students, which was the highest among the groups. The isomorphic scores were correlated sig¬nificantly with open-ended scores, while conventional scores did not show any significant correlation.
Keywords: Isomorphic, Multiple-choice, Misconception, Force, Motion
Misconceptions have been recognized as a barrier in physics teaching and learning. Previous research has shown that misconceptions could inhibit students from learning new concepts[1,2]. Identification of students' misconception patterns must be conducted by teachers. By identifying students' misconceptions before the learning process, the teacher can observe students' learning development during the learning process[4,5]. Thus, identifying students' misconceptions before the learning process is essential to design proper learning[6,7].
Misconception identification in science learning has been conducted in various contexts and educational levels including elementary school, high school, university, and teachers[11,12]. The two-tier multiple-choice test has been widely implemented. The disadvantage of the two-tier test is its tendency to justify students' common mistakes to be misconceptions. To address this drawback, the three- and four-tier instruments have been developed. However, the three-tier test is still considered to confuse children choosing between the main answer (the first tier) and the reason (the second tier). Furthermore, scoring the four-tier instrument is still challenging due to the rich variations of possible answers. Therefore, the effort to develop a diagnostic test that is able to uncover students' misconceptions quickly with an easy scoring process is essential. However, a quick and appropriate procedure to identify misconceptions is still a challenge in physics learning.
One of the alternatives that can diagnose the misconception quickly and accurately is isomorphic questions. An accurate diagnosis is important as a reference in providing appropriate feedback to the students. Providing quick feedback is essential to understanding concepts correctly. Previous studies have introduced isomorphic items in identifying misconceptions[16-18]. The consistency of students' answers from one context to the others can be determined using isomorphic questions. Bollen
This work reports using and validating isomorphic multiple-choice question (IMCI) usage to identify students' misconceptions. There are three aims of this research including (1) to identify the characteristic of students' responses to IMCI, (2) to diagnose students' misconceptions based on IMCI in physics students and natural science students, and (3) to identify a correlation between students' categorization by IMCI and their skills in solving the open-ended problem.
Survey research was conducted with 221 students in the Faculty of Mathematics and Natural Science in Universitas Negeri Malang, Indonesia. The participants are 103 physics students and 118 natural science students (chemistry students and a natural science student-teacher). All of the participants involved in this survey have taken an introductory physics course, and most were in their 2nd year of study.
The research instrument used in this study is nine multiple-choice with a Cronbach's Alpha coefficient of 0.79 and one open-ended question item. The multiple-choice items are classified into three groups. Each group consists of three parallel questions in different contexts: an IMCI. IMCI tests the ability to understand physics concepts through the nuance of context problems. Each multiple-choice item has three options from the most frequent misconceptions and one true option. IMCI identifies the consistency of misconception and the scientific conception from students. Each option is marked as category 1, 2, 3, or 4. Category 1 to category 3 are for the options with misconceptions; category 4 is for the true answer. The categorization is meant to identify the consistency of students' answers. An open-ended question is used to validate diagnostic justification conducted by IMCI analysis.
Each group of IMCI questioned a particular physics concept, but each multiple-choice item had a different context. The instrument was developed in four stages: 1) reviewing the literature of misconception research, 2) developing essay questions, 3) testing students to get a multiple-choice distractor, and 4) developing isomorphic items. The questions that have been developed were reviewed internally to check the validity, and the instrument was tested. The concept of each isomorphic item group, the items, and the items' parameter is shown in Table 1.
To evaluate the accuracy of diagnosis with IMCI, an open-ended problem was also developed. Students were tasked with solving an open-ended problem involving interactive friction forces. To answer correctly, students are required to have the ability to understand all concepts covered in the IMCI instrument. The open-ended question used in this survey is shown as follows.
Andi and Budi are having a match of tug of war using a strong rope but smooth enough not to injure their hands. The rules of match are as follows. (1) Both have to stay upright straight during the match (should not be bent because of the pull of the other), and (2) both should not swing their feet even when they cannot hold the pull. Hence, the only movement they can make when they lose is getting closer to the other side. The data of both of the players are as follows.
|The players' weight and their shoe characteristics||Andi||Budi|
|Weights (in Newton)||600||400|
|The coefficient of static friction between the shoes and the oor||0.5||0.8|
|The coefficient of kinetic friction between the shoes and the oor||0.4||0.7|
a) The winner of the match!
b) The acceleration of the loser when being pulled!
The analysis of students' responses to IMCI was conducted by examining the consistency of the answers. If the answers were consistent in two or more particular categories, then the student was categorized in one of the categories, whether the answer was correct (scientific thinking) or incorrect (misconception). Meanwhile, if the answers in all three items were different, the students' response was justified as ``inconsistent". Furthermore, the scoring of essay questions used a rubric. Based on this result, the percentage of students classified in several categories was calculated, and the relationship between students' categories and their open-ended score was also calculated.
A two-way ANOVA was conducted to investigate the differences in open-ended scores in different category groups. Correlation analysis was also carried out to investigate the relationship between ``conventional scores" and ``isomorphic scores" on open-ended results. Conventional scoring depended on the number of correct answers. For example, when students correct two items, the score is two. In comparison, isomorphic scoring relies on correct answer consistency. Score 2 was coded for three consistent answers in a row, 1 for two consistent answers in a row, and 0 for 1 or fewer correct answers.
With 221 students answering IMCI and open-ended questions, the number of students' responses for three groups of IMCI was 663 responses. Student responses possibly will be consistent toward certain categories for all three questions, consistent toward certain categories for two questions, or inconsistent by answering all the questions with different categories. Students' consistency toward certain categories for all three questions appeared in 51 responses (7.7%), students' consistency toward certain categories for two questions appeared in 469 responses (70.7%), and students' provided 143 inconsistent responses (21.65%). The percentages of students' response consistency on each IMCI group are shown in Table 2.
The data show that the percentage of the consistency of 3 answers is below 10%. These results are consistent with the study results using RFCI. Low presentations happened in all three IMCI groups. Therefore, it can be decided that misconception justification based on the consistency for all three questions could not be implemented. So, in this following report, the consistency of both two and three questions is used to justify students' misconceptions.
The students' responses to each IMCI group are explained in the following sections. The first group of IMCI is related to the concept of action-reaction pair of forces. One example of the multiple-choice items in this group is as follows.
A block is tied under a roof using a string, as shown in the picture. Between the pair of the forces shown, the pair of action-reaction forces is...
The force of the string on the block (
Block's gravitational force (
Block's gravitational force (
The force of the string on the roof (
Students' responses on the first IMCI group related to the action-reaction pair are shown in Fig. 1.
Based on Fig. 1, it can be seen that students' responses are relatively different between physics and natural science students. In the 4th category (the scientifically correct answer), the percentage of physics students was more than natural science students. For physics students, answers were dominated by inconsistency, while natural science students considered equilibrium forces was action-reaction pairs. It indicates that there were still difficult to answer the question. The number of all students who answered consistently and correctly was relatively low (18.6%).
Figure 2 shows the diagnostic result of the second IMCI group. In this group, students were instructed to analyze the magnitude of forces in the case of object interaction.
Based on this data, it can be seen that 12.7% all students answered consistently correctly. Most of the misconceptions experienced by the students were caused by the assumption that in the interaction of forces, the heavier object exerts a larger force. On the contrary, experts developed the concept that both objects exert equal-magnitude forces in forces interaction. It is interesting since most of the previous researchers also found similar results. The percentage of physics student groups mastered the concept at a higher rate than natural science students. This is in line with the fact that physics students learn more intensively about this concept than natural science students. This is also supported by data that the inconsistent percentage of natural science students is higher than physics students. The third group of IMCI is related to Newton's 2nd Law. One of the items is as follow.
A cupboard made of wood is placed on a horizontal rough floor. A worker tries to move the cupboard horizontally by using a string but the cupboard is not moving. In this phenomenon…
The worker's pulling force is smaller than the cupboard's weight (2nd category)
The worker's pulling force is smaller than the frictional force between the cupboard and the floor (3rd category)
The worker's pulling force is equal to the frictional force between the cupboard and the floor (4th category)
The worker's weight is smaller than the cupboard's weight (1st category)
The diagnostic result of this multiple-choice item group is shown in Fig. 3.
Figure 3 shows that the answers of the 3rd category dominate students' responses. Students are suspected of misconception in the case of an object at rest; the pulling or pushing force is smaller than the friction force between the object and the floor that causes the object to stay at rest. It indicates that most of the students were still having naïve thoughts from their daily life even if they had studied fundamental physics. Students who are also diagnosed suffer a misconception that there is a relationship between the pulling force and the object's weight. Even if conceptually, weight and the pulling force are not exerted on the same line. The number of students who answered consistently correctly is relatively low, 8.6%. There is a tendency for different groups to have the same diagnostic result. It indicates a misconception similarity experienced by different groups as in other reports.
The open-ended question was provided to measure students' understanding of the diagram of the forces' interaction. In addition, students also need to determine which objects are moving and not moving depending on static and kinetic friction. A minor percentage of students can solve open-ended questions correctly. Student responses to open-ended questions can be seen in Table 3.
Some student answers have succeeded in determining the magnitude of static and kinetic friction and using it to determine the winner. One example of student answers shows in Fig. 4.
Most students can determine the winner is. However, they failed in deciding how much acceleration is present when the first shift occurs. Students cannot analyze the role of maximum static friction when objects are static and kinetic friction when objects begin to move (Fig. 5). Some students try to use the free-body diagram (Fig. 5) but do not choose correctly between static and kinetic friction forces.
Figure 6 shows the correlation between students' responses and the average problem-solving score. It can be seen that students who understand the concepts have higher open-ended scores compared to students with misconceptions. Students who were inconsistent also tended to have the lowest score. It shows that using multiple-choice items could be a good predictor for solving problems involving several concepts.
Two-way ANOVA was conducted to analyze differences in scores in groups of categories. The statistics for each group of the isomorphic item are shown in Table 4.
All isomorphic groups have a significance value below 0.05. This study shows that the category groups in each isomorphic question cause a difference in the open-ended score. In other words, different categories will cause different open-ended scores.
In comparison, this study also performed conventional scoring and isomorphic scoring. Descriptive statistics of the two scoring techniques can be seen in Table 5.
Against these scores, the correlation test was done between conventional and isomorphic scores on open-ended scores. Correlation results can be seen in Table 6.
Data analysis showed that the isomorphic scores correlated significantly with open-ended scores, while conventional scores did not show a significant correlation. It can be concluded that an isomorphic score can be a better predictor for students' abilities in implementing Newton Law concepts.
By utilizing IMCI, this study has proven that students' physics misconceptions can be easily identified. Justification for students' misconception by utilizing IMCI needs two consistent answers by considering the low percentage of 3 consistent answers. In contrast to diagnostics of students' misconception using three-tier items, the use of IMCI does not require students to identify the level of student confidence in answering questions; IMCI can identify the students' thinking that is not consistent. The difficulty in getting students' consistent responses in answering three items is related to the difficulty in developing IMCI items. It is not easy to develop some parallel questions. Commonly, different contexts or representations are used to make IMCI with a different item. Meanwhile, previous researchers found that representations also influences students' skills with problem-solving[22,26].
This work also indicated a positive relationship between conceptual understanding and students' ability to answer an open-ended item. It was found that students who were justified in understanding the concept also had a higher success rate in solving an open-ended problem. Meanwhile, justified inconsistent students tended to fail in solving an open-ended problem. The diagnostic results with IMCI showed that groups of physics and natural science students tended to have similar forms of misconception. Previous research on misconceptions also reported similar identification results among groups. The misconceptions that occurred in physics, biology, chemistry, and environmental science students toward the topic of energy conservation were similar. Research results regarding students' misconceptions and their failure in problem-solving are consistent with previous studies, such as those on mechanics, electromagnetism[13,29], thermodynamics, and modern physics.
This study showed that the percentage of students from natural science whose understanding of concepts is inconsistent tends to be higher than that of physics students. Indeed, this is a positive indication of the use of IMCI because the content tested on this instrument is physics. In other words, IMCI measures a person's physics understanding. Despite all participants studying the topics being tested, physics students have a better understanding of physics concepts. One of the factors that cause students to choose physics is having good physics competence[31,32]. This study does not intend to generalize these claims in a broader sense. This is a preliminary study, and the use of IMCI with content from other disciplines such as the general concept of chemistry, biology, or mathematics should be considered. The study can be conducted by examining whether or not IMCI is consistent in recording participants' inconsistency by major.
IMCI also has several challenges. First, developing IMCI is not easy. Each item multiple-choice item must include misconception distractors that students often encounter. Developing alternative options like that is not easy. Second, options on the IMCI limit the students' thinking as there is a possibility that the options do not include concepts from the students' knowledge base. Third, each IMCI requires three multiple-choice items; this can cause the number of items in the test to be high and extend the time needed for the test.
Notwithstanding these limitations, the use of IMCI has several advantages. The process of identifying misconceptions can be done quickly, considering multiple-choice questions can be analyzed easily using computer assistance. IMCI could categorize students based on their misconception types and identify students lacking information in answering problems. Unlike diagnostics students' misconceptions using three-tier items, IMCI does not produce many combinations, making it easy to provide feedback to students. IMCI also does not require students to provide student confidence in answering questions. Thus, the use of IMCI in diagnosing students' misconceptions can be applied in physics teaching and learning. Misconception identification using IMCI begins with developing multiple-choice items with one correct answer and other options, including the most frequent misconceptions for students. Two other isomorphic items must be developed with another context from one multiple-choice item.
This research found that misconception identification using IMCI could be conducted by identifying two consistent answers or more. Misconception identification could categorize students into several groups based on their conceptual understanding, misconception, and inconsistency in understanding. There is a correlation between the justification conducted using IMCI and the tendency to solve open-ended problems that involve several concepts.