Searcharxiv⌕ Search

arXiv subjects

Carl Wieman

Publications and source records attributed to Carl Wieman.

14 recordsLinked to original sources
arXiv2025

Directed Self Guided Learning of Blended Math-Science Sensemaking for Historically Marginalized STEM Learners

Blended math science sensemaking (MSS) is reflected in a student ability to integrate math and science knowledge to develop mathematical descriptions of observations. While an important component of scientific thinking, there is little research on teaching MSS. Students from backgrounds historically marginalized in STEM often lack prior learning opportunities needed to succeed in STEM. Supporting them in developing MSS could help them build quantitative understanding and improve STEM performance. Fostering MSS calls for supporting transferable, lifelong skills. We designed and tested a self-guided learning model grounded in simulations and a cognitive framework to achieve transferable MSS. We tested the activities grounded in the framework with 27 students from backgrounds historically marginalized in STEM attending minority serving 2- and 4-year US colleges. We evaluated long and short-term MSS transfer by asking student to complete transfer tasks immediately following and at least 1 week post activity. Most students started at the lowest and progressed to the highest MSS level and demonstrated considerable short and long-term transfer after completing just one activity, suggesting the effectiveness of this approach for fostering transferable MSS.

physics.ed-ph↗
arXiv2024

Scaffold or Crutch? Examining College Students' Use and Views of Generative AI Tools for STEM Education

Developing problem-solving competency is central to Science, Technology, Engineering, and Mathematics (STEM) education, yet translating this priority into effective approaches to problem-solving instruction and assessment remain a significant challenge. The recent proliferation of generative artificial intelligence (genAI) tools like ChatGPT in higher education introduces new considerations about how these tools can help or hinder students' development of STEM problem-solving competency. Our research examines these considerations by studying how and why college students use genAI tools in their STEM coursework, focusing on their problem-solving support. We surveyed 40 STEM college students from diverse U.S. institutions and 28 STEM faculty to understand instructor perspectives on effective genAI tool use and guidance in STEM courses. Our findings reveal high adoption rates and diverse applications of genAI tools among STEM students. The most common use cases include finding explanations, exploring related topics, summarizing readings, and helping with problem-set questions. The primary motivation for using genAI tools was to save time. Moreover, over half of student participants reported simply inputting problems for AI to generate solutions, potentially bypassing their own problem-solving processes. These findings indicate that despite high adoption rates, students' current approaches to utilizing genAI tools often fall short in enhancing their own STEM problem-solving competencies. The study also explored students' and STEM instructors' perceptions of the benefits and risks associated with using genAI tools in STEM education. Our findings provide insights into how to guide students on appropriate genAI use in STEM courses and how to design genAI-based tools to foster students' problem-solving competency.

physics.ed-ph↗
arXiv2023

Examining the Potential and Pitfalls of ChatGPT in Science and Engineering Problem-Solving

The study explores the capabilities of OpenAI's ChatGPT in solving different types of physics problems. ChatGPT (with GPT-4) was queried to solve a total of 40 problems from a college-level engineering physics course. These problems ranged from well-specified problems, where all data required for solving the problem was provided, to under-specified, real-world problems where not all necessary data were given. Our findings show that ChatGPT could successfully solve 62.5% of the well-specified problems, but its accuracy drops to 8.3% for under-specified problems. Analysis of the model's incorrect solutions revealed three distinct failure modes: 1) failure to construct accurate models of the physical world, 2) failure to make reasonable assumptions about missing data, and 3) calculation errors. The study offers implications for how to leverage LLM-augmented instructional materials to enhance STEM education. The insights also contribute to the broader discourse on AI's strengths and limitations, serving both educators aiming to leverage the technology and researchers investigating human-AI collaboration frameworks for problem-solving and decision-making.

cs.AI↗
arXiv2023

Investigating Blended Math-Science Sensemaking with a diverse undergraduate population

Blended mathematical sensemaking in science (MSS) involves deep conceptual understanding of quantitative relationships describing scientific phenomena. Previously we developed the cognitive framework describing proficiency in MSS across STEM disciplines and validated it with undergraduate students from dominant backgrounds (White, middle class) using assessment built around PhET simulations. In this study we investigate whether the framework can characterize engagement in MSS among undergraduate students from diverse backgrounds and identify potential differences between the two populations. The framework is effective in characterizing engagement in MSS by undergraduate students from diverse backgrounds and largely functions as a learning progression. We have also uncovered a distinct pattern of engagement in MSS reflected in students successfully developing the mathematical relationship describing their observations without engaging in a type of MSS focused on quantitative pattern identification. Unlike students from dominant backgrounds, diverse students leverage lower level MSS to develop the formula for the phenomenon. Further, diverse students use PhET simulations to make sense of the phenomenon more and are more successful in using the simulations to find the correct mathematical relationship compared to students from dominant backgrounds. Math preparation has a stronger effect at level 1 compared to levels 2 and 3 of the framework. The framework can guide the development of instructional and assessment strategies to support students from diverse backgrounds in building MSS skills. Further, PhET simulations provide a suitable and effective learning environment for supporting engagement in and learning of MSS skills, and their capabilities should be leveraged for designing learning experiences in the future.

physics.ed-ph↗
arXiv2023

Introducing an Instructional Model for Teaching Blended Math-Science Sensemaking in Undergraduate STEM Courses Using Computer Simulations

The ability to express scientific concepts in mathematical terms and integrate scientific and mathematical reasoning about a phenomenon is a foundational cognitive process involved in scientific thinking. This process called blended math-science sensemaking (blended MSS) is a desired skill for all STEM students, but few students are learning it, and there is little research on how to teach it. In this work we introduce the development and testing of a novel instructional method for teaching blended (MSS) that is suitable for use in STEM courses in undergraduate and K-12 educational settings. This study builds on our past work on developing and validating a framework for characterizing in detail the cognitive levels involved in such sensemaking. This work uses the unique power of interactive simulations for assessing and developing blended MSS. We designed instructional activities to help students use blended MSS in the contexts of heat capacity and Coulombs law. The Heat Capacity activity was piloted in a freshmen Chemistry course and the Coulombs law activity was piloted in a freshmen Physics course. The results indicate that for students who came in with no knowledge of the relevant equation the activity supported the development of both the equation, and their understanding of the mathematical relationships of the equation. These results indicate that the teaching approach helps students engage in blended MSS at higher levels of cognitive complexity.

physics.ed-ph↗
arXiv2022

Cognitive Framework for Blended Mathematical Sensemaking in Science

Blended mathematical sensemaking in science (blended MSS) involves deep conceptual understanding of quantitative relationships describing phenomena in science and has been studies in various disciplines. However, no unified characterization of blended MSS exists. We developed a theoretical cognitive model fro blended MSS grounded in prior work. The model contains three broad levels representing increasingly sophisticated ways of engaging in blended MSS: 1) deriving qualitative relationships among relevant variables describing a phenomenon (qualitative level, 2) deriving mathematical relationships among these variables (quantitative level), 3) explaining how the mathematical operations used in the formula relate to the phenomenon (conceptual levels). Each level contains three sub-levels. We used PhET simulations to design dynamic assessment scenarios in various disciplines to test the model. We used these assessments to interview undergraduate students with a wide range of math skills. Interview analysis provided validity evidence for the cognitive model. It also revealed that students tend to perform at the same level across different disciplinary contexts, suggesting that blended MSS is a distinct cognitive construct, independent of specific disciplinary context. This paper presents a first-ever published validated cognitive model for blended MSS which can guide instruction, curriculum, and assessment development.

physics.ed-ph↗
arXiv2022

Absence of a COVID-induced academic drop in high-school physics learning

At the start of the COVID 19 pandemic, the majority of secondary instruction in the United States transitioned to an online environment. In many parts of the country, online schooling continued for upwards of two years. Many experts have hypothesized an "academic slide" (a reduction in student learning) following this period of online instruction. We investigated the change in student preparation for introductory college physics in incoming Stanford university students between the Fall term of 2019 to the Fall term of 2021. We did this by looking at the performance on a validated physics diagnostic exam that all Stanford students intending to take a physics course took before enrolling in an introductory physics course. We found no statistically or educationally significant change in scores. Despite many anecdotal faculty reports, at least for this population, the level of student preparation in physics and related math appears to be unchanged.

physics.ed-ph↗
arXiv2021

An equitable and effective approach to introductory mechanics

Introductory mechanics ("physics 1") is a critical gateway course for students desiring to pursue a STEM career. A major challenge with this course is that there is a large spread in the students' incoming physics preparation, and this level of preparation is strongly predictive of a students' performance. The level of incoming preparation is also largely determined by a student's educational privilege, and so this course can amplify inequities in K-12 education and provide a barrier to a STEM career for students from marginalized groups. Here, we present a novel introductory course design to address such equity challenges in physics 1. We designed the course based on the concept of deliberate practice to give students targeted, scaffolded, and repeated opportunities to engage in research-identified practices and decisions required for effective problem-solving. We used real-world problems, as they carry less resemblance to physics high school problems, and so even the students with the best high school physics instruction have little experience or skill in solving them. The students learned the physics content knowledge they needed in future courses, particularly in engineering, and their problem-solving skills improved substantially. Furthermore, the success in the course was not correlated with incoming physics preparation, in stark contrast to the outcomes from traditional physics 1 courses. These findings suggest that we made physics 1 more equitable by employing a deliberate practice approach in the context of real-world problem-solving.

physics.ed-ph↗
arXiv2020

A Detailed Characterization of the Expert Problem-Solving Process in Science and Engineering; Guidance for Teaching and Assessment

A primary goal of science and engineering (S & E) education is to produce good problem solvers, but how to best teach and measure the quality of problem-solving remains unclear. The process is complex, multifaceted, and not fully characterized. Here we present a theoretical framework of the S & E problem-solving process as a set of specific interlinked decisions. This theory is empirically grounded and describes the entire process. To develop this theory, we interviewed 52 successful scientists and engineers (experts) spanning different disciplines, including biology and medicine. They described how they solved a typical but important problem in their work, and we analyzed the interviews in terms of decisions made. Surprisingly, we found that across all experts and fields, the solution process was framed around making a set of just twenty-nine specific decisions. We also found that the process of making those discipline-general decisions (selecting between alternative actions) relied heavily on domain-specific predictive models that embodied the relevant disciplinary knowledge. This set of decisions provides a guide for the detailed measurement and teaching of S & E problem-solving. This decision framework also provides a more specific, complete, and empirically based theory describing the practices of science.

physics.ed-ph↗
arXiv2019

The impact of incoming preparation and demographics on performance in Physics I: a multi-institution comparison

We have studied the impact of incoming preparation and demographic variables on student performance on the final exam in physics 1, the standard introductory, calculus-based mechanics course This was done at three different institutions using multivariable regression analysis to determine the extent to which exam scores can be predicted by a variety of variables that are available to most faculty and departments. We have found that the results are surprisingly consistent across the institutions, with the only two variables that have predictive power being math SAT/ACT scores and concept inventory pre-scores. The importance of both variables is comparable and fairly similar across the institutions. They explain 20 - 30 percent of the variation in students' performance on the final exam. Most notably, the demographic variables (gender, under-represented minority, first generation to attend college) are not significant. In all cases, although there appear to be gaps in exam performance if one considers only the demographic variable, once these two proxies of incoming preparation are included in the model, there is no longer a demographic gap. There is only a preparation gap that applies equally across the entire student population. This work shows that to properly understand differences in student performance across a diverse population, and hence to design more effective instruction, it is important to do statistical analyses that take multiple variables into account. It also illustrates the importance of having measures that are sensitive to both subject specific and more general preparation. The results suggest that better matching of the course design and teaching to the incoming student preparation will likely be the most effective way to eliminate observed performance gaps across demographic groups while also improving the success of all students.

physics.ed-ph↗
arXiv2018

Interview validation of the Physics Lab Inventory of Critical thinking (PLIC)

Although an important goal of introductory physics labs is to train students in scientific reasoning and critical thinking, currently there are no standard tests in physics designed to assess such skills. We are in the process of developing and validating the Physics Lab Inventory of Critical thinking (PLIC), an assessment to probe students' critical thinking abilities in physics lab courses. The instrument asks students to critique a set of experimental methods and data and use them to evaluate a particular physical model (the period of a mass on a spring). Currently, we are validating the closed-response survey through interviews with students and present the results of 12 such interviews here. We describe a trend that has emerged from these interviews, with students' reasoning falling into three main patterns of behavior: selecting all options, cuing on keywords, and critically analyzing. We have found ways to shift students to the last and more desirable behavior. We discuss ways in which these findings are likely relevant to the design of other concept inventories.

physics.ed-ph↗
arXiv2015

Measuring the impact of an instructional laboratory on the learning of introductory physics

We have analyzed the impact of taking an associated lab course on the scores on final exam questions in two large introductory physics courses. Approximately a third of the students who completed each course also took an accompanying instructional lab course. The lab courses were fairly conventional, although they focused on supporting the mastery of a subset of the introductory physics topics covered in the associated course. Performance between students who did and did not take the lab course was compared using final exam questions from the associated courses that related to concepts from the lab courses. The population of students who took the lab in each case was somewhat different from those who did not enroll in the lab course in terms of background and major. Those differences were taken into account by normalizing their performance on the lab-related questions with scores on the exam questions that did not involve material covered in the lab. When normalized in this way, the average score on lab-related questions of the students who took the lab, in both courses, was within 1% of the score of students who did not, with an uncertainty of 2%. This result raises questions as to the effectiveness of labs at supporting mastery of physics content.

physics.ed-ph↗
arXiv2013

Use of research-based instructional strategies: how to avoid faculty quitting

We have examined the teaching practices of faculty members who adopted research-based instructional strategies as part of the Carl Wieman Science Education Initiative (CWSEI) at the University of British Columbia. Of the 70 that adopted such strategies with the support of the CWSEI program, only one subsequently stopped using these strategies. This is a tiny fraction of the 33% stopping rate for physics faculty in general [Henderson, Dancy, and Niewiadomska-Bugaj, PRST-PER, 8, 020104 (2012)]. Nearly all of these UBC faculty members who had an opportunity to subsequently use RBIS in other courses (without CWSEI support) did so. We offer possible explanations for the difference in quitting rates. The direct support of the faculty member by a trained science education specialist in the discipline during the initial implementation of the new strategies may be the most important factor.

physics.ed-ph↗