arXiv subjects

Yujiang Wang

Publications and source records attributed to Yujiang Wang.

At least 91 records · Page 5Linked to original sources

arXiv2021

Face Mask Extraction in Video Sequence

Inspired by the recent development of deep network-based methods in semantic image segmentation, we introduce an end-to-end trainable model for face mask extraction in video sequence. Comparing to landmark-based sparse face shape representation, our method can produce the segmentation masks of individual facial components, which can better reflect their detailed shape variations. By integrating Convolutional LSTM (ConvLSTM) algorithm with Fully Convolutional Networks (FCN), our new ConvLSTM-FCN model works on a per-sequence basis and takes advantage of the temporal correlation in video clips. In addition, we also propose a novel loss function, called Segmentation Loss, to directly optimise the Intersection over Union (IoU) performances. In practice, to further increase segmentation accuracy, one primary model and two additional models were trained to focus on the face, eyes, and mouth regions, respectively. Our experiment shows the proposed method has achieved a 16.99% relative improvement (from 54.50% to 63.76% mean IoU) over the baseline FCN model on the 300 Videos in the Wild (300VW) dataset.

cs.CV↗

arXiv2021

Dynamic Face Video Segmentation via Reinforcement Learning

For real-time semantic video segmentation, most recent works utilised a dynamic framework with a key scheduler to make online key/non-key decisions. Some works used a fixed key scheduling policy, while others proposed adaptive key scheduling methods based on heuristic strategies, both of which may lead to suboptimal global performance. To overcome this limitation, we model the online key decision process in dynamic video segmentation as a deep reinforcement learning problem and learn an efficient and effective scheduling policy from expert information about decision history and from the process of maximising global return. Moreover, we study the application of dynamic video segmentation on face videos, a field that has not been investigated before. By evaluating on the 300VW dataset, we show that the performance of our reinforcement key scheduler outperforms that of various baselines in terms of both effective key selections and running speed. Further results on the Cityscapes dataset demonstrate that our proposed method can also generalise to other scenarios. To the best of our knowledge, this is the first work to use reinforcement learning for online key-frame decision in dynamic video segmentation, and also the first work on its application on face videos.

cs.CV↗

arXiv2020

Focal to bilateral tonic-clonic seizures are associated with widespread network abnormality in temporal lobe epilepsy

Objective: To identify if whole-brain structural network alterations in patients with temporal lobe epilepsy (TLE) and focal to bilateral tonic-clonic seizures (FBTCS) differ from alterations in patients without FBTCS. Methods: We dichotomized a cohort of 83 drug-resistant patients with TLE into those with and without FBTCS and compared each group to 29 healthy controls. For each subject, we used diffusion MRI to construct whole-brain structural networks. First, we measured the extent of alterations by performing FBTCS-negative (FBTCS-) versus control and FBTCS-positive (FBTCS+) versus control comparisons, thereby delineating altered sub-networks of the whole-brain structural network. Second, by standardising networks of each patient using control networks, we measured the subject-specific abnormality at every brain region in the network, thereby quantifying the spatial localisation and the amount of abnormality in every patient. Results: Both FBTCS+ and FBTCS- patient groups had altered sub-networks with reduced fractional anisotropy (FA) and increased mean diffusivity (MD) compared to controls. The altered subnetwork in FBTCS+ patients was more widespread than in FBTCS- patients (441 connections altered at t>3, p<0.001 in FBTCS+ compared to 21 connections altered at t>3, p=0.01 in FBTCS-). Significantly greater abnormalities-aggregated over the entire brain network as well as assessed at the resolution of individual brain areas-were present in FBTCS+ patients (p<0.001, d=0.82). In contrast, the fewer abnormalities present in FBTCS- patients were mainly localised to the temporal and frontal areas. Significance: The whole-brain structural network is altered to a greater and more widespread extent in patients with TLE and FBTCS. We suggest that these abnormal networks may serve as an underlying structural basis or consequence of the greater seizure spread observed in FBTCS.

q-bio.NC↗

arXiv2020

Multivariate white matter alterations are associated with epilepsy duration

Previous studies investigating associations between white matter alterations and duration of temporal lobe epilepsy (TLE) have shown differing results, and were typically limited to univariate analyses of tracts in isolation. In this study we apply a multivariate measure (the Mahalanobis distance), to capture the distinct ways white matter may differ in individual patients, and relate this to epilepsy duration. Diffusion MRI, from a cohort of 94 subjects (28 healthy controls, 33 left-TLE and 33 right-TLE), was used to assess associations between tract fractional anisotropy (FA) and epilepsy duration. Using ten white matter tracts, we analysed associations using traditional univariate analyses (z-scores) and a complementary multivariate approach (Mahalanobis distance), incorporating multiple white matter tracts into a single unified analysis. In patients with right-TLE, FA was not significantly associated with epilepsy duration for any tract studied in isolation. In patients with left-TLE, the FA of two limbic tracts (ipsilateral fornix, contralateral cingulum gyrus) was significantly negatively associated with epilepsy duration (Bonferonni corrected p<0.05). Using a multivariate approach we found significant ipsilateral positive associations with duration in both left, and right-TLE cohorts (left-TLE: Spearman's rho=0.487, right-TLE: Spearman's rho=0.422). Extrapolating our multivariate results to duration equals zero (i.e. at onset) we found no significant difference between patients and controls. Associations using the multivariate approach were more robust than univariate methods. The multivariate distance measure provides non-overlapping and more robust results than traditional univariate analyses. Future studies should consider adopting both frameworks into their analysis in order to ascertain a more complete understanding of epilepsy progression, regardless of laterality.

q-bio.NC↗

arXiv2020

Reliability and comparability of human brain structural covariance networks

Structural covariance analysis is a widely used structural MRI analysis method which characterises the co-relations of morphology between brain regions over a group of subjects. To our knowledge, little has been investigated in terms of the comparability of results between different data sets or the reliability of results over the same subjects in different rescan sessions, image resolutions, or FreeSurfer versions. In terms of comparability, our results show substantial differences in the structural covariance matrix between data sets of age- and sex-matched healthy human adults. These differences persist after site correction, they are exacerbated by low sample sizes, and they are most pronounced when using average cortical thickness as a morphological measure. Down-stream graph theoretic analyses further show statistically significant differences. In terms of reliability, substantial differences were also found when comparing repeated scan sessions of the same subjects, and image resolutions and FreeSurfer versions of the same image. We could further estimate the relative measurement error and showed that it is largest when using thickness. With simulated data, we argue that cortical thickness is least reliable because of larger relative measurement errors. Practically, we make the following recommendations (1) pooling subjects across sites into one group should be avoided, particularly if sites differ in image resolutions, demographics, or preprocessing; (2) surface area and volume should be preferred as morphological measures over cortical thickness; (3) a large number of subjects should be used to estimate structural covariance; (4) measurement error should be assessed where repeated measurements are available; (5) if combining sites is critical, univariate site-correction is insufficient, but error covariance should be explicitly measured and modelled.

q-bio.NC↗

arXiv2020

Band power modulation through intracranial EEG stimulation and its cross-session consistency

Background: Direct electrical stimulation of the brain through intracranial electrodes is currently used to probe the epileptic brain as part of pre-surgical evaluation, and it is also being considered for therapeutic treatments through neuromodulation. It is still unknown, however, how consistent intracranial direct electrical stimulation responses are across sessions, to allow effective neuromodulation design. Objective: To investigate the cross-session consistency of the electrophysiological effect of electrical stimulation delivered through intracranial EEG. Methods: We analysed data from 79 epilepsy patients implanted with intracranial EEG who underwent brain stimulation as part of a memory experiment. We quantified the effect of stimulation in terms of band power modulation and compared this effect from session to session. As a reference, we applied the same measures during baseline periods. Results: In most sessions, the effect of stimulation on band power could not be distinguished from baseline fluctuations of band power. Stimulation effect was also not consistent across sessions; only a third of the session pairs had a higher consistency than the baseline standards. Cross-session consistency is mainly associated with the strength of positive stimulation effects, and it also tends to be higher when the baseline conditions are more similar between sessions. Conclusion: These findings can inform our practices for designing neuromodulation with greater efficacy when using direct electrical brain stimulation as a therapeutic treatment.

q-bio.NC↗

arXiv2020

Independent components of human brain morphology

Quantification of brain morphology has become an important cornerstone in understanding brain structure. Measures of cortical morphology such as thickness and surface area are frequently used to compare groups of subjects or characterise longitudinal changes. However, such measures are often treated as independent from each other. A recently described scaling law, derived from a statistical physics model of cortical folding, demonstrates that there is a tight covariance between three commonly used cortical morphology measures: cortical thickness, total surface area, and exposed surface area. We show that assuming the independence of cortical morphology measures can hide features and potentially lead to misinterpretations. Using the scaling law, we account for the covariance between cortical morphology measures and derive novel independent measures of cortical morphology. By applying these new measures, we show that new information can be gained; in our example we show that distinct morphological alterations underlie healthy ageing compared to temporal lobe epilepsy, even on the coarse level of a whole hemisphere. We thus provide a conceptual framework for characterising cortical morphology in a statistically valid and interpretable manner, based on theoretical reasoning about the shape of the cortex.

q-bio.NC↗

arXiv2020

Predicting the Impact of Electric Field Stimulation in a Detailed Computational Model of Cortical Tissue

Neurostimulation using weak electric fields has generated excitement in recent years due to its potential as a medical intervention. However, study of this stimulation modality has been hampered by inconsistent results and large variability within and between studies. In order to begin addressing this variability we need to properly characterise the impact of the current on the underlying neuron populations. To develop and test a computational model capable of capturing the impact of electric field stimulation on networks of neurons. We construct a cortical tissue model with distinct layers and explicit neuron morphologies. We then apply a model of electrical stimulation and carry out multiple test case simulations. The cortical slice model is compared to experimental literature and shown to capture the main features of the electrophysiological response to stimulation. Namely, the model showed 1) a similar level of depolarisation in individual pyramidal neurons, 2) acceleration of intrinsic oscillations, and 3) retention of the spatial profile of oscillations in different layers. We then apply alternative electric fields to demonstrate how the model can capture differences in neuronal responses to the electric field. We demonstrate that the tissue response is dependent on layer depth, the angle of the apical dendrite relative to the field, and stimulation strength. We present publicly available computational modelling software that predicts the neuron network population response to electric field stimulation.

q-bio.NC↗

arXiv2019

Shape Constrained Network for Eye Segmentation in the Wild

Semantic segmentation of eyes has long been a vital pre-processing step in many biometric applications. Majority of the works focus only on high resolution eye images, while little has been done to segment the eyes from low quality images in the wild. However, this is a particularly interesting and meaningful topic, as eyes play a crucial role in conveying the emotional state and mental well-being of a person. In this work, we take two steps toward solving this problem: (1) We collect and annotate a challenging eye segmentation dataset containing 8882 eye patches from 4461 facial images of different resolutions, illumination conditions and head poses; (2) We develop a novel eye segmentation method, Shape Constrained Network (SCN), that incorporates shape prior into the segmentation network training procedure. Specifically, we learn the shape prior from our dataset using VAE-GAN, and leverage the pre-trained encoder and discriminator to regularise the training of SegNet. To improve the accuracy and quality of predicted masks, we replace the loss of SegNet with three new losses: Intersection-over-Union (IoU) loss, shape discriminator loss and shape embedding loss. Extensive experiments shows that our method outperforms state-of-the-art segmentation and landmark detection methods in terms of mean IoU (mIoU) accuracy and the quality of segmentation masks. The eye segmentation database is available at https://www.dropbox.com/s/yvveouvxsvti08x/Eye_Segmentation_Database.zip?dl=0.

cs.CV↗

arXiv2019

Interictal intracranial EEG for predicting surgical success: the importance of space and time

Predicting post-operative seizure freedom using functional correlation networks derived from interictal intracranial EEG has shown some success. However, there are important challenges to consider. 1: electrodes physically closer to each other naturally tend to be more correlated causing a spatial bias. 2: implantation location and number of electrodes differ between patients, making cross-subject comparisons difficult. 3: functional correlation networks can vary over time but are currently assumed as static. In this study we address these three substantial challenges using intracranial EEG data from 55 patients with intractable focal epilepsy. Patients additionally underwent preoperative MR imaging, intra-operative CT, and post-operative MRI allowing accurate localisation of electrodes and delineation of removed tissue. We show that normalising for spatial proximity between nearby electrodes improves prediction of post-surgery seizure outcomes. Moreover, patients with more extensive electrode coverage were more likely to have their outcome predicted correctly (ROC-AUC >0.9, p<<0.05), but not necessarily more likely to have a better outcome. Finally, our predictions are robust regardless of the time segment. Future studies should account for the spatial proximity of electrodes in functional network construction to improve prediction of post-surgical seizure outcomes. Greater coverage of both removed and spared tissue allows for predictions with higher accuracy.

q-bio.NC↗

arXiv2019

Carbogen inhalation during Non-Convulsive Status Epilepticus: A quantitative analysis of EEG recordings

Objective: To quantify the effect of inhaled 5% carbon-dioxide/95% oxygen on EEG recordings from patients in non-convulsive status epilepticus (NCSE). Methods: Five children of mixed aetiology in NCSE were given high flow of inhaled carbogen (5% carbon dioxide/95% oxygen) using a face mask for maximum 120s. EEG was recorded concurrently in all patients. The effects of inhaled carbogen on patient EEG recordings were investigated using band-power, functional connectivity and graph theory measures. Carbogen effect was quantified by measuring effect size (Cohen's d) between "before", "during" and "after" carbogen delivery states. Results: Carbogen's apparent effect on EEG band-power and network metrics across all patients for "before-during" and "before-after" inhalation comparisons was inconsistent across the five patients. Conclusion: The changes in different measures suggest a potentially non-homogeneous effect of carbogen on the patients' EEG. Different aetiology and duration of the inhalation may underlie these non-homogeneous effects. Tuning the carbogen parameters (such as ratio between CO2 and O2, duration of inhalation) on a personalised basis may improve seizure suppression in future.

q-bio.NC↗

arXiv2019

End-to-End Visual Speech Recognition for Small-Scale Datasets

Visual speech recognition models traditionally consist of two stages, feature extraction and classification. Several deep learning approaches have been recently presented aiming to replace the feature extraction stage by automatically extracting features from mouth images. However, research on joint learning of features and classification remains limited. In addition, most of the existing methods require large amounts of data in order to achieve state-of-the-art performance, otherwise they under-perform. In this work, we present an end-to-end visual speech recognition system based on fully-connected layers and Long-Short Memory (LSTM) networks which is suitable for small-scale datasets. The model consists of two streams which extract features directly from the mouth and difference images, respectively. The temporal dynamics in each stream are modelled by a Bidirectional LSTM (BLSTM) and the fusion of the two streams takes place via another BLSTM. An absolute improvement of 0.6%, 3.4%, 3.9%, 11.4% over the state-of-the-art is reported on the OuluVS2, CUAVE, AVLetters and AVLetters2 databases, respectively.

cs.CV↗

arXiv2019

Personalised network modelling in epilepsy

Epilepsy is a disorder characterised by spontaneous, recurrent seizures. Both local and network abnormalities have been associated with epilepsy, and the exact processes generating seizures are thought to be heterogeneous and patient-specific. Due to the heterogeneity, treatments such as surgery and medication are not always effective in achieving full seizure control and choosing the best treatment for the individual patient can be challenging. Predictive models constrained by the patient's own data therefore offer the potential to assist in clinical decision making. In this chapter, we describe how personalised patient-derived networks from structural or functional connectivity can be incorporated into predictive models. We focus specifically on dynamical systems models which are composed of differential equations capable of simulating brain activity over time. Here we review recent studies which have used these models, constrained by patient data, to make personalised patient-specific predictions about seizure features (such as propagation patterns) or treatment outcomes (such as the success of surgical resection). Finally, we suggest future research directions for patient-specific network models in epilepsy, including their application to integrate information from multiple modalities, to predict long-term disease evolution, and to account for within-subject variability for treatment.

q-bio.NC↗

arXiv2018

Fractal and Multifractal Properties of Electrographic Recordings of Human Brain Activity: Toward Its Use as a Signal Feature for Machine Learning in Clinical Applications

The brain is a system operating on multiple time scales, and characterisation of dynamics across time scales remains a challenge. One framework to study such dynamics is that of fractal geometry. However, currently there exists no established method for the study of brain dynamics using fractal geometry, due to the many challenges in the conceptual and technical understanding of the methods. We aim to highlight some of the practical challenges of applying fractal geometry to brain dynamics and propose solutions to enable its wider use in neuroscience. Using intracranially recorded EEG and simulated data, we compared monofractal and multifractal methods with regards to their sensitivity to signal variance. We found that both correlate closely with signal variance, thus not offering new information about the signal. However, after applying an epoch-wise standardisation procedure to the signal, we found that multifractal measures could offer non-redundant information compared to signal variance, power and other established EEG signal measures. We also compared different multifractal estimation methods and found that the Chhabra-Jensen algorithm performed best. Finally, we investigated the impact of sampling frequency and epoch length on multifractal properties. Using epileptic seizures as an example event in the EEG, we show that there may be an optimal time scale for detecting temporal changes in multifractal properties around seizures. The practical issues we highlighted and our suggested solutions should help in developing a robust method for the application of fractal geometry in EEG signals. Our analyses and observations also aid the theoretical understanding of the multifractal properties of the brain and might provide grounds for new discoveries in the study of brain signals. These could be crucial for understanding of neurological function and for the developments of new treatments.

q-bio.NC↗

arXiv2018

Computer modelling of connectivity change suggests epileptogenesis mechanisms in idiopathic generalised epilepsy

Patients with idiopathic generalised epilepsy (IGE) typically have normal conventional magnetic resonance imaging (MRI), hence MRI based diagnosis is challenging. Anatomical abnormalities underlying brain dysfunctions in IGE are unclear and their relation to the pathomechanisms of epileptogenesis is poorly understood. In this study, we applied connectometry, an advanced quantitative neuroimaging technique for investigating localised changes in white-matter tissue. Analysing white matter structures of 32 subjects we incorporated our findings in a computational model of seizure dynamics to suggest a plausible mechanism of epileptogenesis. Patients with IGE have significant bilateral alterations in major white-matter fascicles. In the cingulum, fornix, and superior longitudinal fasciculus, tract integrity is compromised, whereas in specific parts of tracts between thalamus and the precentral gyrus, tract integrity is enhanced in patients. Combining these alterations in a logistic regression model, we computed the decision boundary that discriminated patients and controls. The computational model, informed with the findings on the tract abnormalities, specifically highlighted the importance of enhanced cortico-reticular connections along with impaired cortico-cortical connections in inducing pathological seizure-like dynamics. We emphasise taking directionality of brain connectivity into consideration towards understanding the pathological mechanisms; this is possible by combining neuroimaging and computational modelling. Our imaging evidence of structural alterations suggest the loss of cortico-cortical and enhancement of cortico-thalamic fibre integrity in IGE. We further suggest that impaired connectivity from cortical regions to the thalamic reticular nucleus offers a therapeutic target for selectively modifying the brain circuit for reversing the mechanisms leading to epileptogenesis.

q-bio.NC↗

arXiv2018

Universality in human cortical folding across lobes of individual brains

Background: We have previously demonstrated that cortical folding across mammalian species follows a universal scaling law that can be derived from a simple theoretical model. The same scaling law has also been shown to hold across brains of our own species, irrespective of age or sex. These results, however, only relate measures of complete cortical hemispheres. There are known systematic variations in morphology between different brain regions, and region-specific changes with age. It is therefore of interest to extend our analyses to different cortical regions, and analyze the scaling law within an individual brain. Methods: To directly compare the morphology of sub-divisions of the cortical surface in a size-independent manner, we base our method on a topological invariant of closed surfaces. We reconstruct variables of a complete hemisphere from each lobe of the brain so that it has the same gyrification index, average thickness and average Gaussian curvature. Results: We show that different lobes are morphologically diverse but obey the same scaling law that was observed across human subjects and across mammalian species. This is also the case for subjects with Alzheimer's disease. The age-dependent offset changes at similar rates for all lobes in healthy subjects, but differs most dramatically in the temporal lobe in Alzheimer's disease. Significance: Our results further support the idea that while morphological parameters can vary locally across the cortical surface/across subjects of the same species/across species, the processes that drive cortical gyrification are universal.

q-bio.NC↗

arXiv2018

The impact of epilepsy surgery on the structural connectome and its relation to outcome

Temporal lobe surgical resection brings seizure remission in up to 80% of patients, with long-term complete seizure freedom in 41%. However, it is unclear how surgery impacts on the structural white matter network, and how the network changes relate to seizure outcome. We used white matter fibre tractography on preoperative diffusion MRI to generate a structural white matter network, and postoperative T1-weighted MRI to retrospectively infer the impact of surgical resection on this network. We then applied graph theory and machine learning to investigate the properties of change between the preoperative and predicted postoperative networks. Temporal lobe surgery had a modest impact on global network efficiency, despite the disruption caused. This was due to alternative shortest paths in the network leading to widespread increases in betweenness centrality post-surgery. Measurements of network change could retrospectively predict seizure outcomes with 79% accuracy and 65% specificity, which is twice as high as the empirical distribution. Fifteen connections which changed due to surgery were identified as useful for prediction of outcome, eight of which connected to the ipsilateral temporal pole. Our results suggest that the use of network change metrics may have clinical value for predicting seizure outcome. This approach could be used to prospectively predict outcomes given a suggested resection mask using preoperative data only.

q-bio.NC↗

arXiv2018

Learning Local Metrics and Influential Regions for Classification

The performance of distance-based classifiers heavily depends on the underlying distance metric, so it is valuable to learn a suitable metric from the data. To address the problem of multimodality, it is desirable to learn local metrics. In this short paper, we define a new intuitive distance with local metrics and influential regions, and subsequently propose a novel local metric learning method for distance-based classification. Our key intuition is to partition the metric space into influential regions and a background region, and then regulate the effectiveness of each local metric to be within the related influential regions. We learn local metrics and influential regions to reduce the empirical hinge loss, and regularize the parameters on the basis of a resultant learning bound. Encouraging experimental results are obtained from various public and popular data sets.

cs.LG↗