arXiv subjects

N. G. Holmes

Publications and source records attributed to N. G. Holmes.

At least 37 records · Page 2Linked to original sources

arXiv2020

Online Administration of Research-Based Assessments

The number and use of research-based assessments (RBAs) has grown significantly over the last several decades. Data from RBAs can be compared against national datasets to provide instructors with empirical evidence on the efficacy of their teaching practices. Many physics instructors, however, opt not to use RBAs due to barriers such as having to use class time to administer them. In this article we examine how these barriers can be mitigated through online administrations of RBAs, particularly through the use of free online RBA platforms that automate administering, scoring, and analyzing RBAs (e.g., the Learning About STEM Student Outcomes [LASSO], Colorado Learning Attitudes About Science Survey for Experimental Physics [E-CLASS], Physics Lab Inventory of Critical thinking [PLIC], and PhysPort DataExplorer platforms). We also explore the research into common concerns of administering RBAs online and conclude with a practical how-to guide for instructors.

physics.ed-ph↗

arXiv2020

Preliminary evidence for available roles in mixed-gender and all-women lab groups

Group work during lab instruction can be a source of inequity between male and female students. In this preliminary study, we explored the activities male and female students take on during a lab session at a university in Denmark. Different from many studies, the class was majority-female, so three of the seven groups were all female and the rest were mixed-gender. We found that students in mixed-gender groups divide tasks in similar ways to mixed-gender groups at North American institutions, with men handling the equipment and women handling the computer more often. We also found that women in single-gender groups took on each of the available roles with approximately equal frequency, but women in single-gender groups spent more time on the equipment than students in mixed-gender groups. We interpret the results through poststructual gender theory and the notion of `doing physics' and `doing gender' in physics labs.

physics.ed-ph↗

arXiv2020

What do equitable physics lab groups look like in light of inchargeness?

In physics labs, students experience a wide range of equitable and inequitable interactions. We developed a methodology to characterize different lab groups in terms of their bid exchanges and inchargeness. An equitable group is one in which every student's bids are heard and acknowledged. Our analysis of equitable and inequitable groups raises questions about how inchargeness and gender interact to affect the functionality of a lab group.

physics.ed-ph↗

arXiv2020

Student reasoning about sources of experimental measurement uncertainty in quantum versus classical mechanics

Measurement uncertainty and experimental error are important concepts taught in undergraduate physics laboratories. Although student ideas about error and uncertainty in introductory classical mechanics lab experiments have been studied extensively, there is relatively limited research on student thinking about experimental measurement uncertainty in quantum mechanics. In this work, we used semi-structured interviews to study advanced physics students' interpretations of fictitious data distributions from two common undergraduate laboratory experiments in quantum mechanics and one in classical mechanics. To analyze these interpretations, we developed a coding scheme that classifies student responses based on what factors they believe create uncertainty and differentiates between different types of uncertainty (e.g. imprecision, inaccuracy). We found that participants in our study expressed a variety of ideas about measurement uncertainty that varied with the context (classical/quantum) and the type of uncertainty.

physics.ed-ph↗

arXiv2020

Sense of agency, gender, and students' perception in open-ended physics labs

Instructional physics labs are critical junctures for many STEM majors to develop an understanding of experimentation in the sciences. Students can acquire useful experimental skills and grow their identities as scientists. However, many traditionally-instructed labs do not necessarily involve authentic physics experimentation features in their curricula. Recent research calls for a reformation in undergraduate labs to incorporate more student agency and choice in the learning processes. In our institution, we have adopted open-ended lab teaching in the introductory physics courses. By using reformed curricula that provide higher student agency, we analyzed approximately 100 students in the introductory-level lab courses to examine their views towards the open-ended physics labs. Between the start and the end of the semester, we found a statistically significant shift in students' perceptions about the agency afforded in lab activities. We also examined students' responses to "Which lab unit was your favorite and why?". The analysis showed that majority of the students preferred Project Lab, which had the highest student agency and coding analysis showed that "freedom" was the most frequent response for students' reason for picking Project Lab. Finally, we also examined student views across gender and found no significant gender effect on students' sense of agency.

physics.ed-ph↗

arXiv2020

Student evaluation of more or better experimental data in classical and quantum mechanics

Prior research has shown that physics students often think about experimental procedures and data analysis very differently from experts. One key framework for analyzing student thinking has found that student thinking is more point-like, putting emphasis on the results of a single experimental trial, whereas set-like thinking relies on the results of many trials. Recent work, however, has found that students rarely fall into one of these two extremes, which may be a limitation of how student thinking is evaluated. Measurements of student thinking have focused on probing students' procedural knowledge by asking them, for example, what steps they might take next in an experiment. Two common refrains are to collect more data, or to improve the experiment and collect better data. In both of these cases, the underlying reasons behind student responses could be based in point-like or set-like thinking. In this study we use individual student interviews to investigate how advanced physics students believe the collection of more and better data will affect the results of a classical and a quantum mechanical experiment. The results inform future frameworks and assessments for characterizing students thinking between the extremes of point and set reasoning in both classical and quantum regimes.

physics.ed-ph↗

arXiv2020

Group roles in unstructured labs show inequitable gender divide

Instructional labs are being transformed to better reflect authentic scientific practice, often by removing aspects of pedagogical structure to support student agency and decision-making. We explored how these changes impact men's and women's participation in group work associated with labs through clustering methods on the quantified behavior of students. We compared the group roles students take on in two different types of instructional settings; (1) highly structured traditional labs, and (2) less structured inquiry-based labs. Students working in groups in the inquiry-based (less structured) labs assumed different roles within their groups, however men and women systematically took on different roles and men behaved differently when in single- versus mixed-gender groups. We found no such systematic differences in role division among male and female students in the traditional (highly-structured) labs. Students in the inquiry-based labs were not overtly assigned these roles, indicating that the inequitable division of roles was not a result of explicit assignment. Our results highlight the importance of structuring equitable group dynamics in educational settings, as a gendered division of roles can emerge without active intervention. As the culture in physics evolves to remove systematic gender biases in the field, instructors in educational settings must not only remove explicitly biased aspects of curricula but also take active steps to ensure that potentially discriminatory aspects are not inadvertently reinforced.

physics.ed-ph↗

arXiv2020

Ready Student One: Exploring the predictors of student learning in virtual reality

Immersive virtual reality (VR) has enormous potential for education, but classroom resources are limited. Thus, it is important to identify whether and when VR provides sufficient advantages over other modes of learning to justify its deployment. In a between-subjects experiment, we compared three methods of teaching Moon phases (a hands-on activity, VR, and a desktop simulation) and measured student improvement on existing learning and attitudinal measures. While a substantial majority of students preferred the VR experience, we found no significant differences in learning between conditions. However, we found differences between conditions based on gender, which was highly correlated with experience with video games. These differences may indicate certain groups have an advantage in the VR setting.

physics.ed-ph↗

arXiv2019

Seeing the real world: Comparing learning from verification labs and traditional or enhanced lecture demonstrations

Instructors in introductory physics courses often use labs and demonstrations to reinforce that the physics equations introduced in lectures and textbooks describe what actually happens in the real world. The surface features and instructional learning goals of such labs and demonstrations may seem quite similar (predict-observe-explain). Nonetheless, physics education research has found very different impacts of these instructional methods on students' physics content knowledge that depend critically on several instructional characteristics. In this paper, we disentangle the research by comparing details of the learning methods in verification labs, traditional lecture demonstrations, and enhanced lecture demonstrations. We discuss possible mechanisms for the measurably different learning outcomes by dissecting the activities according to the role of prediction, cognitive load, and engagement. We use this characterization to motivate rethinking the goals of labs towards experimentation skills and beliefs, rather than for verifying physics content.

physics.ed-ph↗

arXiv2019

Participating in the Physics Lab: Does Gender Matter?

It is a well-studied notion that women are under-represented in the physical sciences, with a "leaky pipeline" metaphor describing how the number of women decreases at higher levels in academia[1,2]. It is unclear, however, where the major leaks exist and what factors are responsible for this[2]. Our focus here is on women in physics with an emphasis on practical laboratory work.

physics.ed-ph↗

arXiv2019

Quantifying critical thinking: Development and validation of the Physics Lab Inventory of Critical thinking (PLIC)

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the development and validation of the Physics Lab Inventory of Critical thinking (PLIC). We define critical thinking as the ability to use data and evidence to decide what to trust and what to do. The PLIC is a 10-question, closed-response assessment that probes student critical thinking skills in the context of physics experimentation. Using interviews and data from 5584 students at 29 institutions, we demonstrate, through qualitative and quantitative means, the validity and reliability of the instrument at measuring student critical thinking skills. This establishes a valuable new assessment instrument for instructional labs.

physics.ed-ph↗

arXiv2018

Virtual Reality as a Teaching Tool for Moon Phases and Beyond

A ball on a stick is a common and simple activity for teaching the phases of the Moon. This activity, like many others in physics and astronomy, gives students a perspective they otherwise could only imagine. For Moon phases, a third person view and control over time allows students to rapidly build a mental model that connects all the moving parts. Computer simulations of many traditional physics and astronomy activities provide new features, controls, or vantage points to enhance learning beyond a hands-on activity. Virtual reality provides the capabilities of computer simulations and embodied cognition experiences through a hands-on activity making it a natural step to improve learning. We recreated the traditional ball-and-stick moon phases activity in virtual reality and compared participant learning using this simulation with using traditional methods. We found a strong participant preference for VR relative to the traditional methods. However, we observed no difference across conditions in average levels of performance on a pre/post knowledge test.

physics.ed-ph↗

arXiv2018

Assessment of critical thinking in physics labs

Despite the significant amount of time undergraduate students spend in introductory physics labs, there is little consensus on instructional goals and accepted diagnostic assessments for these labs. In response to these issues, we have developed the Physics Lab Inventory of Critical thinking (PLIC) to assess students' proficiency with critical thinking in a physics lab context. Specifically, the PLIC aims to evaluate students' skills in making sense of data, variability, models, and experimental methods and to assess the effectiveness of lab courses at developing these skills. Here, we discuss the statistical and expert validation of this instrument using 2681 valid student responses collected from 12 institutions during the 2017-2018 academic year. As a part of our validation, we address the impact of lab design on student performance on the instrument.

physics.ed-ph↗

arXiv2018

Operationalizing the AAPT Learning Goals for the Lab

Calls for reform to instructional labs means many instructors and departments are facing the daunting task of identifying goals for their introductory lab courses. Fortunately, the American Association of Physics Teachers (AAPT) released a set of recommendations for learning goals for the lab to support lab redevelopment [1]. Here we outline the process we have undergone to identify a set of learning goals for the labs that operationalize those provided by the AAPT. We also provide two examples of newly developed lab activities that aim to meet those goals to demonstrate this operationalization. We aim to provide departments and instructors with a few ideas of a procedure that they can follow or a set of goals that they can use to align lab instruction with the AAPT learning goals.

physics.ed-ph↗

arXiv2018

Who does what now? How physics lab instruction impacts student behaviors

While laboratory instruction is a cornerstone of physics education, the impact of student behaviours in labs on retention, persistence in the field, and the formation of students' physics identity remains an open question. In this study, we performed in-lab observations of student actions over two semesters in two pedagogically different sections of the same introductory physics course. We used a cluster analysis to identify different categories of student behaviour and analyzed how they correlate with lab structure and gender. We find that, in lab structures which fostered collaborative group work and promoted decision making, there was a task division along gender lines with respect to laptop and equipment usage (and found no such divide among students in guided verification labs).

physics.ed-ph↗

arXiv2018

Surprise! Shifting students away from model-verifying frames in physics labs

Framing affects how students interpret, approach, and accomplish tasks. Little is known, however, about how students frame tasks in physics labs. During the first lab of a sequence designed to teach students about modeling and critical thinking with data, students test a simple model of a pendulum that breaks down with improved measurements. Using in-lab video and follow-up interviews, we identified students' frequent use of a model-verifying frame that substantially interferes with the instructional goals. We present a case study analysis of two students who approach the lab with a model-verifying frame, engage in problematic behaviors including questionable research practices, but later shift their frames to accommodate goals aligned with instructional intention. As instructors transition their instructional labs to open-inquiry experiences, an activity that directly challenges the model-verifying frame may be productive for shifting students away from this problematic frame to one that supports their engagement in authentic experimentation.

physics.ed-ph↗

arXiv2018

Confirming what we know: Understanding questionable research practices in intro physics labs

Many institutions are changing the focus of their introductory physics labs from verifying physics content towards teaching students about the skills and nature of science. As instruction shifts, so too will the ways students approach and behave in the labs. In this study, we evaluated students' lab notes from an early activity in an experimentation-focused lab course. We found that about 30% of student groups (out of 107 groups at three institutions) recorded questionable research practices in their lab notes, such as subjective interpretations of results or manipulating equipment and data. The large majority of these practices were associated with confirmatory goals, which we suspect stem from students' prior exposure to verification labs. We propose ways for experimentation-focused labs to better engage students in the responsible conduct of research and authentic scientific practice.

physics.ed-ph↗

arXiv2018

Interview validation of the Physics Lab Inventory of Critical thinking (PLIC)

Although an important goal of introductory physics labs is to train students in scientific reasoning and critical thinking, currently there are no standard tests in physics designed to assess such skills. We are in the process of developing and validating the Physics Lab Inventory of Critical thinking (PLIC), an assessment to probe students' critical thinking abilities in physics lab courses. The instrument asks students to critique a set of experimental methods and data and use them to evaluate a particular physical model (the period of a mass on a spring). Currently, we are validating the closed-response survey through interviews with students and present the results of 12 such interviews here. We describe a trend that has emerged from these interviews, with students' reasoning falling into three main patterns of behavior: selecting all options, cuing on keywords, and critically analyzing. We have found ways to shift students to the last and more desirable behavior. We discuss ways in which these findings are likely relevant to the design of other concept inventories.

physics.ed-ph↗