arXiv subjects

Scott T. Grafton

Publications and source records attributed to Scott T. Grafton.

At least 19 recordsLinked to original sources

arXiv2026

Actionable Guidance Outperforms Map and Compass Cues in Demanding Immersive VR Wayfinding

Navigation aids are central to immersive virtual reality (VR) experiences that involve physical locomotion. Their effectiveness depends not only on how much spatial information they provide, but also on how directly that information supports movement decisions. We compared three common guidance techniques for immersive VR wayfinding: a directional arrow, a minimap, and a compass. In a controlled room-scale VR study with 42 participants completing 1008 trials, participants navigated to target landmarks in a time-pressured maze with reduced visibility and forced route replanning. Across behavioral and eye-tracking measures, arrow guidance produced the strongest navigation performance, minimap guidance yielded intermediate performance, and compass cues performed worst, suggesting that during immersive locomotion users benefit from guidance that can be interpreted rapidly while moving. These results suggest that in demanding immersive locomotion tasks, interfaces that translate spatial information directly into actionable movement cues can outperform richer but more interpretive spatial representations. Our findings highlight the importance of designing XR navigation interfaces that minimize the cognitive translation between spatial information and movement decisions.

cs.HC↗

arXiv2022

M3L: Language-based Video Editing via Multi-Modal Multi-Level Transformers

Video editing tools are widely used nowadays for digital design. Although the demand for these tools is high, the prior knowledge required makes it difficult for novices to get started. Systems that could follow natural language instructions to perform automatic editing would significantly improve accessibility. This paper introduces the language-based video editing (LBVE) task, which allows the model to edit, guided by text instruction, a source video into a target video. LBVE contains two features: 1) the scenario of the source video is preserved instead of generating a completely different video; 2) the semantic is presented differently in the target video, and all changes are controlled by the given instruction. We propose a Multi-Modal Multi-Level Transformer (M$^3$L) to carry out LBVE. M$^3$L dynamically learns the correspondence between video perception and language semantic at different levels, which benefits both the video understanding and video frame synthesis. We build three new datasets for evaluation, including two diagnostic and one from natural videos with human-labeled text. Extensive experimental results show that M$^3$L is effective for video editing and that LBVE can lead to a new field toward vision-and-language research.

cs.CV↗

arXiv2020

StressNet: Detecting Stress in Thermal Videos

Precise measurement of physiological signals is critical for the effective monitoring of human vital signs. Recent developments in computer vision have demonstrated that signals such as pulse rate and respiration rate can be extracted from digital video of humans, increasing the possibility of contact-less monitoring. This paper presents a novel approach to obtaining physiological signals and classifying stress states from thermal video. The proposed network--"StressNet"--features a hybrid emission representation model that models the direct emission and absorption of heat by the skin and underlying blood vessels. This results in an information-rich feature representation of the face, which is used by spatio-temporal network for reconstructing the ISTI ( Initial Systolic Time Interval: a measure of change in cardiac sympathetic activity that is considered to be a quantitative index of stress in humans ). The reconstructed ISTI signal is fed into a stress-detection model to detect and classify the individual's stress state ( i.e. stress or no stress ). A detailed evaluation demonstrates that StressNet achieves estimated the ISTI signal with 95% accuracy and detect stress with average precision of 0.842. The source code is available on Github.

cs.CV↗

arXiv2020

Crystallinity characterization of white matter in the human brain

White matter microstructure underpins cognition and function in the human brain through the facilitation of neuronal communication, and the non-invasive characterization of this structure remains a research frontier in the neuroscience community. Efforts to assess white matter microstructure, however, are hampered by the sheer amount of information needed for characterization. Current techniques within neuroimaging deal with this problem by representing white matter features with single scalars that are often not easy to interpret. Here, we address these issues by introducing tools from materials science for the characterization of white matter microstructure. We investigate structure on a mesoscopic scale by analyzing its homogeneity and determining which regions of the brain are structurally homogeneous, or "crystalline" in the context of materials science. We find that crystallinity provides novel information and varies across the brain along interpretable lines of anatomical difference, with highest homogeneity in regions adjacent to the corpus callosum. Furthermore, crystallinity is highly reliable, and yet also varies between individuals, making it a potentially useful biomarker to examine individual differences in white matter along various dimensions. We also parcellate white matter into "crystal grains," or contiguous sets of voxels of high structural similarity, and find overlap with other white matter parcellations. Finally, we characterize the shapes of local white matter signatures through another tool from materials science - bond-orientational order parameters - and locate fiber crossings and fascicles with this new technique. Our results provide new means of assessing white matter microstructure on multiple length scales, and open new avenues of future inquiry. We hope that this work fosters future productive dialogue between the fields of soft matter and neuroscience.

cond-mat.soft↗

arXiv2019

Estimating localized complexity of white-matter wiring with GANs

In-vivo examination of the physical connectivity of axonal projections through the white matter of the human brain is made possible by diffusion weighted magnetic resonance imaging (dMRI) Analysis of dMRI commonly considers derived scalar metrics such as fractional anisotrophy as proxies for "white matter integrity," and differences of such measures have been observed as significantly correlating with various neurological diagnosis and clinical measures such as executive function, presence of multiple sclerosis, and genetic similarity. The analysis of such voxel measures is confounded in areas of more complicated fiber wiring due to crossing, kissing, and dispersing fibers. Recently, Volz et al. introduced a simple probabilistic measure of the count of distinct fiber populations within a voxel, which was shown to reduce variance in group comparisons. We propose a complementary measure that considers the complexity of a voxel in context of its local region, with an aim to quantify the localized wiring complexity of every part of white matter. This allows, for example, identification of particularly ambiguous regions of the brain for tractographic approaches of modeling global wiring connectivity. Our method builds on recent advances in image inpainting, in which the task is to plausibly fill in a missing region of an image. Our proposed method builds on a Bayesian estimate of heteroscedastic aleatoric uncertainty of a region of white matter by inpainting it from its context. We define the localized wiring complexity of white matter as how accurately and confidently a well-trained model can predict the missing patch. In our results, we observe low aleatoric uncertainty along major neuronal pathways which increases at junctions and towards cortex boundaries. This directly quantifies the difficulty of lesion inpainting of dMRI images at all parts of white matter.

eess.IV↗

arXiv2018

Spatial Coherence of Oriented White Matter Microstructure: Applications to White Matter Regions Associated with Genetic Similarity

We present a method to discover differences between populations with respect to the spatial coherence of their oriented white matter microstructure in arbitrarily shaped white matter regions. This method is applied to diffusion MRI scans of a subset of the Human Connectome Project dataset: 57 pairs of monozygotic and 52 pairs of dizygotic twins. After controlling for morphological similarity between twins, we identify 3.7% of all white matter as being associated with genetic similarity (35.1k voxels, $p < 10^{-4}$, false discovery rate 1.5%), 75% of which spatially clusters into twenty-two contiguous white matter regions. Furthermore, we show that the orientation similarity within these regions generalizes to a subset of 47 pairs of non-twin siblings, and show that these siblings are on average as similar as dizygotic twins. The regions are located in deep white matter including the superior longitudinal fasciculus, the optic radiations, the middle cerebellar peduncle, the corticospinal tract, and within the anterior temporal lobe, as well as the cerebellum, brain stem, and amygdalae. These results extend previous work using undirected fractional anisotrophy for measuring putative heritable influences in white matter. Our multidirectional extension better accounts for crossing fiber connections within voxels. This bottom up approach has at its basis a novel measurement of coherence within neighboring voxel dyads between subjects, and avoids some of the fundamental ambiguities encountered with tractographic approaches to white matter analysis that estimate global connectivity.

stat.AP↗

arXiv2017

Analytic tractography: A closed-form solution for estimating local white matter connectivity with diffusion MRI

White matter structures composed of myelinated axons in the living human brain are primarily studied by diffusion-weighted MRI (dMRI). These long-range projections are typically characterized in a two-step process: dMRI is used to estimate the orientation of axons within each voxel, then these local orientations are linked together to estimate the spatial extent of putative white matter bundles. Tractography, the process of tracing bundles across voxels, either requires computationally expensive (probabilistic) simulations to model uncertainty in fiber orientation or ignores it completely (deterministic). Probabilistic simulation necessarily generates a finite number of trajectories, introducing "simulation error" to trajectory estimates. Here we introduce a method to analytically (via a closed-form solution) take an orientation distribution function (ODF) from each voxel and calculate the probabilities that a trajectory projects from a voxel into each directly adjacent voxel. We validate our method by demonstrating that probabilistic simulations converge to our analytically computed probabilities at the voxel level as the number of simulated seeds increases. We show that our method accurately calculates the ground-truth transition probabilities from a phantom dataset. As a demonstration, we incoroporate our analytic method for voxel transition probabilities into the Voxel Graph framework, creating a quantitative framework for assessing white matter structure that we call "analytic tractography". The long-range connectivity problem is reduced to finding paths in a graph whose adjacency structure reflects voxel-to-voxel analytic transition probabilities. We demonstrate this approach performs comparably to many current probabilistic and deterministic approaches at a fraction of the computational cost. Open source software software is provided.

q-bio.NC↗

arXiv2017

Brain State Flexibility Accompanies Motor-Skill Acquisition

Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging -- and to assess their dynamics during learning -- remain limited. Here, we describe an approach based on a novel application of graph theory in which points in time are represented by network nodes, and similarities in brain states between two different time points are represented as network edges. We use a graph-based clustering technique to identify clusters of time points representing canonical brain states, and to assess the manner in which the brain moves from one state to another as learning progresses. We observe the presence of two primary states characterized by either high activation in sensorimotor cortex or high activation in a frontal-subcortical system. Flexible switching among these primary states and other less common states becomes more frequent as learning progresses, and is inversely correlated with individual differences in learning rate. These results are consistent with the notion that the development of automaticity is associated with a greater freedom to use cognitive resources for other processes. Taken together, our work offers new insights into the constrained, low dimensional nature of brain dynamics characteristic of early learning, which give way to less constrained, high-dimensional dynamics in later learning.

q-bio.NC↗

arXiv2017

Riemannian-geometry-based modeling and clustering of network-wide non-stationary time series: The brain-network case

This paper advocates Riemannian multi-manifold modeling in the context of network-wide non-stationary time-series analysis. Time-series data, collected sequentially over time and across a network, yield features which are viewed as points in or close to a union of multiple submanifolds of a Riemannian manifold, and distinguishing disparate time series amounts to clustering multiple Riemannian submanifolds. To support the claim that exploiting the latent Riemannian geometry behind many statistical features of time series is beneficial to learning from network data, this paper focuses on brain networks and puts forth two feature-generation schemes for network-wide dynamic time series. The first is motivated by Granger-causality arguments and uses an auto-regressive moving average model to map low-rank linear vector subspaces, spanned by column vectors of appropriately defined observability matrices, to points into the Grassmann manifold. The second utilizes (non-linear) dependencies among network nodes by introducing kernel-based partial correlations to generate points in the manifold of positive-definite matrices. Capitilizing on recently developed research on clustering Riemannian submanifolds, an algorithm is provided for distinguishing time series based on their geometrical properties, revealed within Riemannian feature spaces. Extensive numerical tests demonstrate that the proposed framework outperforms classical and state-of-the-art techniques in clustering brain-network states/structures hidden beneath synthetic fMRI time series and brain-activity signals generated from real brain-network structural connectivity matrices.

cs.LG↗

arXiv2017

Optimal Trajectories of Brain State Transitions

The complexity of neural dynamics stems in part from the complexity of the underlying anatomy. Yet how the organization of white matter architecture constrains how the brain transitions from one cognitive state to another remains unknown. Here we address this question from a computational perspective by defining a brain state as a pattern of activity across brain regions. Drawing on recent advances in network control theory, we model the underlying mechanisms of brain state transitions as elicited by the collective control of region sets. Specifically, we examine how the brain moves from a specified initial state (characterized by high activity in the default mode) to a specified target state (characterized by high activity in primary sensorimotor cortex) in finite time. Across all state transitions, we observe that the supramarginal gyrus and the inferior parietal lobule consistently acted as efficient, low energy control hubs, consistent with their strong anatomical connections to key input areas of sensorimotor cortex. Importantly, both these and other regions in the fronto-parietal, cingulo-opercular, and attention systems are poised to affect a broad array of state transitions that cannot easily be classified by traditional notions of control common in the engineering literature. This theoretical versatility comes with a vulnerability to injury. In patients with mild traumatic brain injury, we observe a loss of specificity in putative control processes, suggesting greater susceptibility to damage-induced noise in neurophysiological activity. These results offer fundamentally new insights into the mechanisms driving brain state transitions in healthy cognition and their alteration following injury.

q-bio.NC↗

arXiv2016

Emerging Frontiers of Neuroengineering: A Network Science of Brain Connectivity

Neuroengineering is faced with unique challenges in repairing or replacing complex neural systems that are composed of many interacting parts. These interactions form intricate patterns over large spatiotemporal scales, and produce emergent behaviors that are difficult to predict from individual elements. Network science provides a particularly appropriate framework in which to study and intervene in such systems, by treating neural elements (cells, volumes) as nodes in a graph and neural interactions (synapses, white matter tracts) as edges in that graph. Here, we review the emerging discipline of network neuroscience, which uses and develops tools from graph theory to better understand and manipulate neural systems, from micro- to macroscales. We present examples of how human brain imaging data is being modeled with network analysis and underscore potential pitfalls. We then highlight current computational and theoretical frontiers, and emphasize their utility in informing diagnosis and monitoring, brain-machine interfaces, and brain stimulation. A flexible and rapidly evolving enterprise, network neuroscience provides a set of powerful approaches and fundamental insights critical to the neuroengineer's toolkit.

q-bio.NC↗

arXiv2016

Explicitly Linking Regional Activation and Function Connectivity: Community Structure of Weighted Networks with Continuous Annotation

A major challenge in neuroimaging is understanding the mapping of neurophysiological dynamics onto cognitive functions. Traditionally, these maps have been constructed by examining changes in the activity magnitude of regions related to task performance. Recently, network neuroscience has produced methods to map connectivity patterns among many regions to certain cognitive functions by drawing on tools from network science and graph theory. However, these two different views are rarely addressed simultaneously, largely because few tools exist that account for patterns between nodes while simultaneously considering activation of nodes. We address this gap by solving the problem of community detection on weighted networks with continuous (non-integer) annotations by deriving a generative probabilistic model. This model generates communities whose members connect densely to nodes within their own community, and whose members share similar annotation values. We demonstrate the utility of the model in the context of neuroimaging data gathered during a motor learning paradigm, where edges are task-based functional connectivity and annotations to each node are beta weights from a general linear model that encoded a linear decrease in blood-oxygen-level-dependent signal with practice. Interestingly, we observe that individuals who learn at a faster rate exhibit the greatest dissimilarity between functional connectivity and activation magnitudes, suggesting that activation and functional connectivity are distinct dimensions of neurophysiology that track behavioral change. More generally, the tool that we develop offers an explicit, mathematically principled link between functional activation and functional connectivity, and can readily be applied to a other similar problems in which one set of imaging data offers network data, and a second offers a regional attribute.

q-bio.NC↗

arXiv2016

Structural Pathways Supporting Swift Acquisition of New Visuo-Motor Skills

Human skill learning requires fine-scale coordination of distributed networks of brain regions that are directly linked to one another by white matter tracts to allow for effective information transmission. Yet how individual differences in these anatomical pathways may impact individual differences in learning remains far from understood. Here, we test the hypothesis that individual differences in the organization of structural networks supporting task performance predict individual differences in the rate at which humans learn a visuo-motor skill. Over the course of 6 weeks, twenty-two healthy adult subjects practiced a discrete sequence production task, where they learned a sequence of finger movements based on discrete visual cues. We collected structural imaging data during four MRI scanning sessions spaced approximately two weeks apart, and using deterministic tractography, structural networks were generated for each participant to identify streamlines that connect cortical and sub-cortical brain regions. We observed that increased white matter connectivity linking early visual (but not motor) regions was associated with a faster learning rate. Moreover, we observed that the strength of multi-edge paths between motor and visual modules was also correlated with learning rate, supporting the role of polysynaptic connections in successful skill acquisition. Our results demonstrate that the combination of diffusion imaging and tractography-based connectivity can be used to predict future individual differences in learning capacity, particularly when combined with methods from network science and graph theory.

q-bio.NC↗

arXiv2016

The Energy Landscape of Neurophysiological Activity Implicit in Brain Network Structure

A critical mystery in neuroscience lies in determining how anatomical structure impacts the complex functional dynamics of human thought. How does large-scale brain circuitry constrain states of neuronal activity and transitions between those states? We address these questions using a maximum entropy model of brain dynamics informed by white matter tractography. We demonstrate that the most probable brain states -- characterized by minimal energy -- display common activation profiles across brain areas: local spatially-contiguous sets of brain regions reminiscent of cognitive systems are co-activated frequently. The predicted activation rate of these systems is highly correlated with the observed activation rate measured in a separate resting state fMRI data set, validating the utility of the maximum entropy model in describing neurophysiologial dynamics. This approach also offers a formal notion of the energy of activity within a system, and the energy of activity shared between systems. We observe that within- and between-system energies cleanly separate cognitive systems into distinct categories, optimized for differential contributions to integrated v.s. segregated function. These results support the notion that energetic and structural constraints circumscribe brain dynamics, offering novel insights into the roles that cognitive systems play in driving whole-brain activation patterns.

q-bio.NC↗

arXiv2016

Individual Differences in Dynamic Functional Brain Connectivity Across the Human Lifespan

Individual differences in brain functional networks may be related to complex personal identifiers, including health, age, and ability. Understanding and quantifying these differences is a necessary first step towards developing predictive methods derived from network topology. Here, we present a method to quantify individual differences in brain functional dynamics by applying hypergraph analysis, a method from dynamic network theory. Using a summary metric derived from the hypergraph formalism---hypergraph cardinality---we investigate individual variations in two separate and complementary data sets. The first data set ("multi-task") consists of 77 individuals engaging in four consecutive cognitive tasks. We observed that hypergraph cardinality exhibits variation across individuals while remaining consistent within individuals between tasks; moreover, one of the memory tasks evinced a marginally significant correspondence between hypergraph cardinality and age. This finding motivated a similar analysis of the second data set ("age-memory"), in which 95 individuals of varying ages performed a memory task with a similar structure to the multi-task memory task. With the increased age range in the age-memory data set, the correlation between hypergraph cardinality and age correspondence becomes significant. We discuss these results in the context of the well-known finding linking age with network structure, and suggest that age-related changes in brain function can be better understood by taking an integrative approach that incorporates information about the dynamics of functional interactions.

q-bio.NC↗

arXiv2016

Graph Frequency Analysis of Brain Signals

This paper presents methods to analyze functional brain networks and signals from graph spectral perspectives. The notion of frequency and filters traditionally defined for signals supported on regular domains such as discrete time and image grids has been recently generalized to irregular graph domains, and defines brain graph frequencies associated with different levels of spatial smoothness across the brain regions. Brain network frequency also enables the decomposition of brain signals into pieces corresponding to smooth or rapid variations. We relate graph frequency with principal component analysis when the networks of interest denote functional connectivity. The methods are utilized to analyze brain networks and signals as subjects master a simple motor skill. We observe that brain signals corresponding to different graph frequencies exhibit different levels of adaptability throughout learning. Further, we notice a strong association between graph spectral properties of brain networks and the level of exposure to tasks performed, and recognize the most contributing and important frequency signatures at different task familiarity.

q-bio.NC↗

arXiv2016

Accurately Predicting Functional Connectivity from Diffusion Imaging

Understanding the relationship between the dynamics of neural processes and the anatomical substrate of the brain is a central question in neuroscience. On the one hand, modern neuroimaging technologies, such as diffusion tensor imaging, can be used to construct structural graphs representing the architecture of white matter streamlines linking cortical and subcortical structures. On the other hand, temporal patterns of neural activity can be used to construct functional graphs representing temporal correlations between brain regions. Although some studies provide evidence that whole-brain functional connectivity is shaped by the underlying anatomy, the observed relationship between function and structure is weak, and the rules by which anatomy constrains brain dynamics remain elusive. In this article, we introduce a methodology to predict with high accuracy the functional connectivity of a subject at rest from his or her structural graph. Using our methodology, we are able to systematically unveil the role of structural paths in the formation of functional correlations. Furthermore, in our empirical evaluations, we observe that the eigen-modes of the predicted functional connectivity are aligned with activity patterns associated with different cognitive systems. Our work offers the potential to infer properties of brain dynamics in clinical or developmental populations with low tolerance for functional neuroimaging.

q-bio.NC↗

arXiv2016

Stimulation-based control of dynamic brain networks

The ability to modulate brain states using targeted stimulation is increasingly being employed to treat neurological disorders and to enhance human performance. Despite the growing interest in brain stimulation as a form of neuromodulation, much remains unknown about the network-level impact of these focal perturbations. To study the system wide impact of regional stimulation, we employ a data-driven computational model of nonlinear brain dynamics to systematically explore the effects of targeted stimulation. Validating predictions from network control theory, we uncover the relationship between regional controllability and the focal versus global impact of stimulation, and we relate these findings to differences in the underlying network architecture. Finally, by mapping brain regions to cognitive systems, we observe that the default mode system imparts large global change despite being highly constrained by structural connectivity. This work forms an important step towards the development of personalized stimulation protocols for medical treatment or performance enhancement.

q-bio.NC↗