pISSN 0374-4914 eISSN 2289-0041

## Research Paper

New Phys.: Sae Mulli 2022; 72: 311-318

Published online April 29, 2022 https://doi.org/10.3938/NPSM.72.311

## Utilizing Isomorphic Multiple-Choice Items to Diagnose Students' Misconceptions in Force and Motion

1Department of Physics, Faculty of Mathematics and Natural Sciences, Universitas Negeri Malang, Malang 65145, Indonesia
2Department of Physics Education, Faculty of Education and Teacher Training, Universitas Islam Negeri Syarif Hidayatullah Jakarta, Jakarta 15412, Indonesia

Received: January 12, 2022; Revised: March 2, 2022; Accepted: March 4, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License(http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

### Abstract

Isomorphic multiple-choice items (IMCI) are a potential tool to diagnose students' misconceptions. This survey research aimed to identify the characteristics of students' responses on IMCI and diagnose students' misconceptions. The relationship between students' responses to diagnostic results and their success in solving open-ended items was also investigated. The IMCI consisted of nine multiple-choice items that were categorized into three IMCI groups according to the force and motion concepts covered in the questions. To evaluate the accuracy of IMCI diagnosis, an open-ended problem was also developed. Referring to their responses to the IMCI, students were classified into three groups: those who understand the concept, those who hold misconceptions, and those who are inconsistent. It was found that students who understand the concept well also demonstrated a high ability to solve the open-ended problem as shown by the average score of those students, which was the highest among the groups. The isomorphic scores were correlated sig¬nificantly with open-ended scores, while conventional scores did not show any significant correlation.

Keywords: Isomorphic, Multiple-choice, Misconception, Force, Motion

### I. INTRODUCTION

Misconceptions have been recognized as a barrier in physics teaching and learning. Previous research has shown that misconceptions could inhibit students from learning new concepts[1,2]. Identification of students' misconception patterns must be conducted by teachers[3]. By identifying students' misconceptions before the learning process, the teacher can observe students' learning development during the learning process[4,5]. Thus, identifying students' misconceptions before the learning process is essential to design proper learning[6,7].

Misconception identification in science learning has been conducted in various contexts and educational levels including elementary school[8], high school[9], university[10], and teachers[11,12]. The two-tier multiple-choice test has been widely implemented. The disadvantage of the two-tier test is its tendency to justify students' common mistakes to be misconceptions. To address this drawback, the three-[13] and four-tier instruments[14] have been developed. However, the three-tier test is still considered to confuse children choosing between the main answer (the first tier) and the reason (the second tier). Furthermore, scoring the four-tier instrument is still challenging due to the rich variations of possible answers[15]. Therefore, the effort to develop a diagnostic test that is able to uncover students' misconceptions quickly with an easy scoring process is essential. However, a quick and appropriate procedure to identify misconceptions is still a challenge in physics learning.

One of the alternatives that can diagnose the misconception quickly and accurately is isomorphic questions. An accurate diagnosis is important as a reference in providing appropriate feedback to the students. Providing quick feedback is essential to understanding concepts correctly[3]. Previous studies have introduced isomorphic items in identifying misconceptions[16-18]. The consistency of students' answers from one context to the others can be determined using isomorphic questions[19]. Bollen et al. [20] conducted a study to reveal students' answers consistently in graphical questions; they found that students' answers depend on the context of the problem. If students understand the physics concepts well, their answers will stay consistent even when the context is different. In other words, the isomorphic question can evaluate students' representation [21] so it can be used to uncover students' misconceptions. Using a website can also assist with isomorphic items, so diagnosing misconceptions could be faster [16]. However, the report regarding the validity of isomorphic questions is rarely conducted.

This work reports using and validating isomorphic multiple-choice question (IMCI) usage to identify students' misconceptions. There are three aims of this research including (1) to identify the characteristic of students' responses to IMCI, (2) to diagnose students' misconceptions based on IMCI in physics students and natural science students, and (3) to identify a correlation between students' categorization by IMCI and their skills in solving the open-ended problem.

### II. METHOD

Survey research was conducted with 221 students in the Faculty of Mathematics and Natural Science in Universitas Negeri Malang, Indonesia. The participants are 103 physics students and 118 natural science students (chemistry students and a natural science student-teacher). All of the participants involved in this survey have taken an introductory physics course, and most were in their 2nd year of study.

The research instrument used in this study is nine multiple-choice with a Cronbach's Alpha coefficient of 0.79 and one open-ended question item. The multiple-choice items are classified into three groups. Each group consists of three parallel questions in different contexts: an IMCI. IMCI tests the ability to understand physics concepts through the nuance of context problems. Each multiple-choice item has three options from the most frequent misconceptions and one true option. IMCI identifies the consistency of misconception and the scientific conception from students. Each option is marked as category 1, 2, 3, or 4. Category 1 to category 3 are for the options with misconceptions; category 4 is for the true answer. The categorization is meant to identify the consistency of students' answers. An open-ended question is used to validate diagnostic justification conducted by IMCI analysis.

Each group of IMCI questioned a particular physics concept, but each multiple-choice item had a different context. The instrument was developed in four stages: 1) reviewing the literature of misconception research, 2) developing essay questions, 3) testing students to get a multiple-choice distractor, and 4) developing isomorphic items. The questions that have been developed were reviewed internally to check the validity, and the instrument was tested. The concept of each isomorphic item group, the items, and the items' parameter is shown in Table 1.

The concepts and item parameters (Cronbach' Alpha).

GroupConceptItems NumberDifficulty IndexDiscrim-ination IndexPoint-Biserial
1Direction of action- reaction force10.190.530,60
40.260.630.60
90.550.740.45
2Magnitude of action-reaction force20.210.530.64
50.160.560.73
70.150.370.52
3Force of object on rest30.120.320.42
60.110.320.47
80.170.410.46

To evaluate the accuracy of diagnosis with IMCI, an open-ended problem was also developed. Students were tasked with solving an open-ended problem involving interactive friction forces. To answer correctly, students are required to have the ability to understand all concepts covered in the IMCI instrument. The open-ended question used in this survey is shown as follows.

Andi and Budi are having a match of tug of war using a strong rope but smooth enough not to injure their hands. The rules of match are as follows. (1) Both have to stay upright straight during the match (should not be bent because of the pull of the other), and (2) both should not swing their feet even when they cannot hold the pull. Hence, the only movement they can make when they lose is getting closer to the other side. The data of both of the players are as follows.

The players' weight and their shoe characteristics Andi Budi
Weights (in Newton) 600 400
The coefficient of static friction between the shoes and the oor 0.5 0.8
The coefficient of kinetic friction between the shoes and the oor 0.4 0.7

Determine.

• a) The winner of the match!

• b) The acceleration of the loser when being pulled!

The analysis of students' responses to IMCI was conducted by examining the consistency of the answers. If the answers were consistent in two or more particular categories, then the student was categorized in one of the categories, whether the answer was correct (scientific thinking) or incorrect (misconception). Meanwhile, if the answers in all three items were different, the students' response was justified as inconsistent". Furthermore, the scoring of essay questions used a rubric. Based on this result, the percentage of students classified in several categories was calculated, and the relationship between students' categories and their open-ended score was also calculated.

A two-way ANOVA was conducted to investigate the differences in open-ended scores in different category groups. Correlation analysis was also carried out to investigate the relationship between conventional scores" and isomorphic scores" on open-ended results. Conventional scoring depended on the number of correct answers. For example, when students correct two items, the score is two. In comparison, isomorphic scoring relies on correct answer consistency. Score 2 was coded for three consistent answers in a row, 1 for two consistent answers in a row, and 0 for 1 or fewer correct answers.

### III. RESULTS

With 221 students answering IMCI and open-ended questions, the number of students' responses for three groups of IMCI was 663 responses. Student responses possibly will be consistent toward certain categories for all three questions, consistent toward certain categories for two questions, or inconsistent by answering all the questions with different categories. Students' consistency toward certain categories for all three questions appeared in 51 responses (7.7%), students' consistency toward certain categories for two questions appeared in 469 responses (70.7%), and students' provided 143 inconsistent responses (21.65%). The percentages of students' response consistency on each IMCI group are shown in Table 2.

The Percentage of Consistency in IMCI Response.

ConceptsConsistent for all three questions (%)Consistent for two questions (%)Inconsistent (%)
Action-reaction pair of forces8.165.226.7
The magnitude of action-reaction pair of forces6.377.416.3
Application of Newton's Laws on Object at Rest8.669.721.7

The data show that the percentage of the consistency of 3 answers is below 10%. These results are consistent with the study results using RFCI[21]. Low presentations happened in all three IMCI groups. Therefore, it can be decided that misconception justification based on the consistency for all three questions could not be implemented. So, in this following report, the consistency of both two and three questions is used to justify students' misconceptions.

The students' responses to each IMCI group are explained in the following sections. The first group of IMCI is related to the concept of action-reaction pair of forces. One example of the multiple-choice items in this group is as follows.

A block is tied under a roof using a string, as shown in the picture. Between the pair of the forces shown, the pair of action-reaction forces is...

• The force of the string on the block (T1) and the force of the string on the roof (T2) (1st category)

• Block's gravitational force (w) and the force of the string on the block (T1) (2nd category)

• Block's gravitational force (w) and the force of the roof on the string (F) (3rd category)

• The force of the string on the roof (T2) and the force of the roof on the string (F) (4th category)*

• Students' responses on the first IMCI group related to the action-reaction pair are shown in Fig. 1.

Figure 1. (Color online) Students' Responses on Action-Reaction Pair of Forces.
Notes:
1st category: misconception that every force with the same magnitude and opposite in the directions on different objects are pairs of action-reaction
2nd category: misconception that equilibrium forces on an object are pairs of action-reaction
3rd category: misconception that forces with the same magnitude and opposite in the directions placed at the very end of a system are a pair of action-reaction
4th category: the student understands the concept of Newton's 3rd Law about action-reaction pair of forces

Based on Fig. 1, it can be seen that students' responses are relatively different between physics and natural science students. In the 4th category (the scientifically correct answer), the percentage of physics students was more than natural science students. For physics students, answers were dominated by inconsistency, while natural science students considered equilibrium forces was action-reaction pairs. It indicates that there were still difficult to answer the question. The number of all students who answered consistently and correctly was relatively low (18.6%).

Figure 2 shows the diagnostic result of the second IMCI group. In this group, students were instructed to analyze the magnitude of forces in the case of object interaction.

Figure 2. (Color online) Students' Responses on the Concept of Action-Reaction Forces Magnitude
Notes:
1st category: the misconception that the force is exerted by the heavier object
2nd category: the misconception that the larger force is exerted by the lighter object
3rd category: the misconception that the larger force is exerted by the heavier object
4th category: student understands that on the interaction of the forces, the magnitude of action-reaction forces are equal

Based on this data, it can be seen that 12.7% all students answered consistently correctly. Most of the misconceptions experienced by the students were caused by the assumption that in the interaction of forces, the heavier object exerts a larger force. On the contrary, experts developed the concept that both objects exert equal-magnitude forces in forces interaction. It is interesting since most of the previous researchers also found similar results. The percentage of physics student groups mastered the concept at a higher rate than natural science students. This is in line with the fact that physics students learn more intensively about this concept than natural science students. This is also supported by data that the inconsistent percentage of natural science students is higher than physics students. The third group of IMCI is related to Newton's 2nd Law. One of the items is as follow.

A cupboard made of wood is placed on a horizontal rough floor. A worker tries to move the cupboard horizontally by using a string but the cupboard is not moving. In this phenomenon…

• The worker's pulling force is smaller than the cupboard's weight (2nd category)

• The worker's pulling force is smaller than the frictional force between the cupboard and the floor (3rd category)

• The worker's pulling force is equal to the frictional force between the cupboard and the floor (4th category)

• The worker's weight is smaller than the cupboard's weight (1st category)

• The diagnostic result of this multiple-choice item group is shown in Fig. 3.

Figure 3. (Color online) Students' Responses on Newton's 2nd Law of Motion.
Notes:
1st category: students have misconception that weight influences force interaction
2nd category: students have misconception that the pulling force is smaller than the pulled object weight
3rd category: students have misconception that the pulling force is smaller than the frictional force on an object at rest
4th category: students understand the concept of Newton's law on object at rest

Figure 3 shows that the answers of the 3rd category dominate students' responses. Students are suspected of misconception in the case of an object at rest; the pulling or pushing force is smaller than the friction force between the object and the floor that causes the object to stay at rest. It indicates that most of the students were still having naïve thoughts from their daily life even if they had studied fundamental physics. Students who are also diagnosed suffer a misconception that there is a relationship between the pulling force and the object's weight. Even if conceptually, weight and the pulling force are not exerted on the same line. The number of students who answered consistently correctly is relatively low, 8.6%. There is a tendency for different groups to have the same diagnostic result. It indicates a misconception similarity experienced by different groups as in other reports.

The open-ended question was provided to measure students' understanding of the diagram of the forces' interaction. In addition, students also need to determine which objects are moving and not moving depending on static and kinetic friction. A minor percentage of students can solve open-ended questions correctly. Student responses to open-ended questions can be seen in Table 3.

Students Response to Open-Ended Question.

DescriptionPhysics Students (%)Natural Science Students (%)
Students can determine who wins based on the magnitude of the maximum static friction and determine the acceleration correctly5.80
Students can determine who wins based on the magnitude of the maximum static friction but cannot determine the acceleration correctly27.912.7
Students can determine the winner correctly but without a clear reason6.735.6
Students cannot determine who won correctly59.651.7

Some student answers have succeeded in determining the magnitude of static and kinetic friction and using it to determine the winner. One example of student answers shows in Fig. 4.

Figure 4. (Color online) The example of student answer correctly based on friction force problem.

Most students can determine the winner is. However, they failed in deciding how much acceleration is present when the first shift occurs. Students cannot analyze the role of maximum static friction when objects are static and kinetic friction when objects begin to move (Fig. 5). Some students try to use the free-body diagram (Fig. 5) but do not choose correctly between static and kinetic friction forces.

Figure 5. (Color online) Examples of student answers that failure to identify acceleration.

Figure 6 shows the correlation between students' responses and the average problem-solving score. It can be seen that students who understand the concepts have higher open-ended scores compared to students with misconceptions. Students who were inconsistent also tended to have the lowest score. It shows that using multiple-choice items could be a good predictor for solving problems involving several concepts.

Figure 6. (Color online) Average Scores of The Open-Ended Item for Each Category.

Two-way ANOVA was conducted to analyze differences in scores in groups of categories. The statistics for each group of the isomorphic item are shown in Table 4.

Two-way ANOVA.

SourceType III Sum of SquaresdfMean SquareFSig.Noncent. ParameterObserved Powerb
Corrected Model47377.387a67707.1252.822.000189.0511.000
Intercept18695.075118695.07574.599.00074.5991.000
ISO_013291.2104822.8023.283.01313.133.828
ISO_025664.61441416.1545.651.00022.604.977
ISO_035523.99041380.9985.511.00022.043.974
ISO_01 * ISO_025708.39714407.7431.627.07822.778.866
ISO_01 * ISO_032703.38216168.961.674.81610.787.443
ISO_02 * ISO_034754.03313365.6951.459.13918.970.795
ISO_01 * ISO_02 * `ISO_033787.81612315.6511.260.24815.115.692
Error38342.749153250.606
Total130050.000221
Corrected Total85720.136220

All isomorphic groups have a significance value below 0.05. This study shows that the category groups in each isomorphic question cause a difference in the open-ended score. In other words, different categories will cause different open-ended scores.

In comparison, this study also performed conventional scoring and isomorphic scoring. Descriptive statistics of the two scoring techniques can be seen in Table 5.

Descriptive statistics of Conventional, IMCI, and Problem-Solving Score.

StatisticsConventionalIMCIOpen-ended
mean16.447.1614.16
Min000
Max10010090
SD17.2615.2119.69

Against these scores, the correlation test was done between conventional and isomorphic scores on open-ended scores. Correlation results can be seen in Table 6.

Correlation between traditional and open-ended score and between isomorphic and open-ended.

Spearman's rhoOpen-ended
ConventionalCorrelation Coefficient.077
Sig.257
N221
IsomorphicCorrelation Coefficient0.350
Sig.000
N221

Data analysis showed that the isomorphic scores correlated significantly with open-ended scores, while conventional scores did not show a significant correlation. It can be concluded that an isomorphic score can be a better predictor for students' abilities in implementing Newton Law concepts.

### IV. DISCUSSION

By utilizing IMCI, this study has proven that students' physics misconceptions can be easily identified. Justification for students' misconception by utilizing IMCI needs two consistent answers by considering the low percentage of 3 consistent answers. In contrast to diagnostics of students' misconception using three-tier items, the use of IMCI does not require students to identify the level of student confidence in answering questions; IMCI can identify the students' thinking that is not consistent. The difficulty in getting students' consistent responses in answering three items is related to the difficulty in developing IMCI items. It is not easy to develop some parallel questions. Commonly, different contexts or representations are used to make IMCI with a different item. Meanwhile, previous researchers found that representations also influences students' skills with problem-solving[22,26].

This work also indicated a positive relationship between conceptual understanding and students' ability to answer an open-ended item. It was found that students who were justified in understanding the concept also had a higher success rate in solving an open-ended problem. Meanwhile, justified inconsistent students tended to fail in solving an open-ended problem. The diagnostic results with IMCI showed that groups of physics and natural science students tended to have similar forms of misconception. Previous research on misconceptions also reported similar identification results among groups. The misconceptions that occurred in physics, biology, chemistry, and environmental science students toward the topic of energy conservation were similar[27]. Research results regarding students' misconceptions and their failure in problem-solving are consistent with previous studies, such as those on mechanics[28], electromagnetism[13,29], thermodynamics[30], and modern physics[9].

This study showed that the percentage of students from natural science whose understanding of concepts is inconsistent tends to be higher than that of physics students. Indeed, this is a positive indication of the use of IMCI because the content tested on this instrument is physics. In other words, IMCI measures a person's physics understanding. Despite all participants studying the topics being tested, physics students have a better understanding of physics concepts. One of the factors that cause students to choose physics is having good physics competence[31,32]. This study does not intend to generalize these claims in a broader sense. This is a preliminary study, and the use of IMCI with content from other disciplines such as the general concept of chemistry, biology, or mathematics should be considered. The study can be conducted by examining whether or not IMCI is consistent in recording participants' inconsistency by major.

IMCI also has several challenges. First, developing IMCI is not easy. Each item multiple-choice item must include misconception distractors that students often encounter. Developing alternative options like that is not easy. Second, options on the IMCI limit the students' thinking as there is a possibility that the options do not include concepts from the students' knowledge base. Third, each IMCI requires three multiple-choice items; this can cause the number of items in the test to be high and extend the time needed for the test.

Notwithstanding these limitations, the use of IMCI has several advantages. The process of identifying misconceptions can be done quickly, considering multiple-choice questions can be analyzed easily using computer assistance. IMCI could categorize students based on their misconception types and identify students lacking information in answering problems. Unlike diagnostics students' misconceptions using three-tier items, IMCI does not produce many combinations, making it easy to provide feedback to students. IMCI also does not require students to provide student confidence in answering questions. Thus, the use of IMCI in diagnosing students' misconceptions can be applied in physics teaching and learning. Misconception identification using IMCI begins with developing multiple-choice items with one correct answer and other options, including the most frequent misconceptions for students. Two other isomorphic items must be developed with another context from one multiple-choice item.

### V. CONCLUSION

This research found that misconception identification using IMCI could be conducted by identifying two consistent answers or more. Misconception identification could categorize students into several groups based on their conceptual understanding, misconception, and inconsistency in understanding. There is a correlation between the justification conducted using IMCI and the tendency to solve open-ended problems that involve several concepts.

### References

1. J. L. Docktor and J. P. Mestre, Phys. Rev. ST Phys. Educ. Res. 10, 020119 (2014).
2. M. O. Bekkink et al, BMC Med. Educ. 16, 221 (2016).
3. S. An and Z. Wu, Int. J. Sci. Math. Educ. 10, 717 (2012).
4. P. Eaton, K. Vavruska and S. Willoughby, Phys. Rev. Phys. Educ. Res. 15, 010123 (2019).
5. D. Kaltakci-Gurel, A. Eryilmaz and L. C. McDermott, Res. Sci. Technol. Educ. 35, 238 (2017).
6. D.-C. Yang and I. A. J. Sianturi, Int. J. Sci. Math. Educ. 17, 1519 (2018).
7. N. Erceg et al, Phys. Rev. Phys. Educ. Res. 12, 020139 (2016).
8. T. Zhang, A. Chen and C. Ennis, Sport Educ. Soc. 24, 25 (2019).
9. E. Taslidere, Res. Sci. Technol. Educ. 34, 164 (2016).
10. Y. J. Jafer, Int. J. Sci. Math. Educ. 18, 657 (2020).
11. J. N. Burgoon, M. L. Heddle and E. Duran, J. Sci. Teach. Educ. 21, 859 (2010).
12. M. J. Hermida, M. S. Segretin, A. Soni García and S. J. Lipina, Educ. Res. 58, 457 (2016).
13. H. Pesman and A. Erylmaz, J. Educ. Res. 103, 208 (2010).
14. D. Kaltakci-Gurel, A. Eryilmaz and L. C. McDermott, Eurasia J. Math. Sci. Tech. Ed 11, 989 (2015).
15. H. O. Arslan, C. Cigdemoglu and C. Moseley, Int. J. Sci. Edu. 34, 1667 (2012).
16. S. Kusairi, A. Hidayat and N. Hidayat, Chem.: Bulg. J. Sci. Educ. 26, 526 (2017).
17. S.-Y. Lin and C. Singh, Phys. Rev. ST Phys. Educ. Res. 7, 020104 (2011).
18. S.-Y. Lin and C. Singh, Phys. Rev. ST Phys. Educ. Res. 9, 020114 (2013).
19. C. Singh, Phys. Rev. ST Phys. Educ. Res. 4, 010105 (2008).
20. L. Bollen et al, Phys. Rev. Phys. Educ. Res. 12, 010108 (2016).
21. P. Nieminen, A. Savinainen and J. Viiri, Phys. Rev. ST Phys. Educ. Res. 6, 020109 (2010).
22. P. Barniol and G. Zavala, Phys. Rev. ST Phys. Educ. Res. 10, 020115 (2014).
23. W. M. Christensen and J. R. Thompson, in Proceed-ings of the 13 Th Annual Conference on Research in Undergraduate Mathematics Education (2010).
24. A. F. Heckler and T. M. Scaife, Phys. Rev. ST Phys. Educ. Res. 11, 010101 (2015).
25. P. Klein, A. Müller and J. Kuhn, Phys. Rev. Phys. Educ. Res. 13, 010132 (2017).
26. D.-H. Nguyen and N. S. Rebello, Phys. Rev. ST Phys. Educ. Res. 7, 010112 (2011).
27. M. Park and X. Liu, Res. Sci. Educ. 51, 43 (2021).
28. A. Desstya et al, Int. J. Instr. 12, 201 (2019).
29. J. Leppavirta, Int. J. Sci. Math. Educ. 10, 1099 (2012).
30. R. Leinonen, M. A. Asikainen and P. E. Hirvonen, Res. Sci. Educ. 42, 1165 (2012).
31. T. Gill and J. F. Bell, Int. J. Sci. Educ. 35, 753 (2013).
32. Z. Hazari, G. Sonnert, P. M. Sadler and M.-C. Shanahan, J. Res. Sci. Teach. 47, 978 (2010).