arXiv subjects

Lijuan Liu

Publications and source records attributed to Lijuan Liu.

At least 37 records · Page 2Linked to original sources

arXiv2022

Towards Better Semantic Understanding of Mobile Interfaces

Improving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users. To stimulate research in this direction, we release a human-annotated dataset with approximately 500k unique annotations aimed at increasing the understanding of the functionality of UI elements. This dataset augments images and view hierarchies from RICO, a large dataset of mobile UIs, with annotations for icons based on their shapes and semantics, and associations between different elements and their corresponding text labels, resulting in a significant increase in the number of UI elements and the categories assigned to them. We also release models using image-only and multimodal inputs; we experiment with various architectures and study the benefits of using multimodal inputs on the new dataset. Our models demonstrate strong performance on an evaluation set of unseen apps, indicating their generalizability to newer screens. These models, combined with the new dataset, can enable innovative functionalities like referring to UI elements by their labels, improved coverage and better semantics for icons etc., which would go a long way in making UIs more usable for everyone.

cs.HC↗

arXiv2022

On the nature of photospheric horizontal magnetic field increase in major solar flares

Rapid increase of horizontal magnetic field ($B_h$) around the flaring polarity inversion line is the most prominent photospheric field change during flares. It is considered to be caused by the contraction of flare loops, the details behind which is still not fully understood. Here we investigate the $B_h$-increase in 35 major flares using HMI high-cadence vector magnetograms. We find that $B_h$-increase is always accompanied by the increase of field inclination. It usually initiates near the flare ribbons, showing step-like change in between the ribbons. In particular, its evolution in early flare phase shows close spatio-temporal correlation to flare ribbons. We further find that $B_h$-increase tends to have similar intensity in confined and eruptive flares, but larger spatial-extent in eruptive flares in a statistical sense. Its intensity and timescale have inverse and positive correlations to the initial ribbon separations, respectively. The results altogether are well consistent with a recent proposed scenario which suggests that the reconnection-driven contraction of flare loops enhances photospheric $B_h$ according to the ideal induction equation, providing statistical evidence to the reconnection-driven origin for $B_h$-increase for the first time.

astro-ph.SR↗

arXiv2022

DETR++: Taming Your Multi-Scale Detection Transformer

Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12]. With the sweeping reform of Transformers [27] in natural language processing, Carion et al. [2] introduce the Transformer-based detection method, i.e., DETR. However, due to the quadratic complexity in the self-attention mechanism in the Transformer, DETR is never able to incorporate multi-scale features as performed in existing CNN-based detectors, leading to inferior results in small object detection. To mitigate this issue and further improve performance of DETR, in this work, we investigate different methods to incorporate multi-scale features and find that a Bi-directional Feature Pyramid (BiFPN) works best with DETR in further raising the detection precision. With this discovery, we propose DETR++, a new architecture that improves detection results by 1.9% AP on MS COCO 2017, 11.5% AP on RICO icon detection, and 9.1% AP on RICO layout extraction over existing baselines.

cs.CV↗

arXiv2022

Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering

In this work we develop a generalizable and efficient Neural Radiance Field (NeRF) pipeline for high-fidelity free-viewpoint human body synthesis under settings with sparse camera views. Though existing NeRF-based methods can synthesize rather realistic details for human body, they tend to produce poor results when the input has self-occlusion, especially for unseen humans under sparse views. Moreover, these methods often require a large number of sampling points for rendering, which leads to low efficiency and limits their real-world applicability. To address these challenges, we propose a Geometry-guided Progressive NeRF (GP-NeRF). In particular, to better tackle self-occlusion, we devise a geometry-guided multi-view feature integration approach that utilizes the estimated geometry prior to integrate the incomplete information from input views and construct a complete geometry volume for the target human body. Meanwhile, for achieving higher rendering efficiency, we introduce a progressive rendering pipeline through geometry guidance, which leverages the geometric feature volume and the predicted density values to progressively reduce the number of sampling points and speed up the rendering process. Experiments on the ZJU-MoCap and THUman datasets show that our method outperforms the state-of-the-arts significantly across multiple generalization settings, while the time cost is reduced > 70% via applying our efficient progressive rendering pipeline.

cs.CV↗

arXiv2022

The Rotation of Magnetic Flux Rope Formed during Solar Eruption

The eruptions of solar filaments often show rotational motion about their rising direction, but it remains elusive what mechanism governs such rotation and how the rotation is related to the initial morphology of the pre-eruptive filament (and co-spatial sigmoid), filament chirality, and magnetic helicity. The conventional view regarding the rotation as a result of a magnetic flux rope (MFR) under-going the ideal kink instability still has confusion in explaining these relationships. Here we proposed an alternative explanation for the rotation during eruptions, by analyzing a magnetohydrodynamic simulation in which magnetic reconnection initiates an eruption from a sheared arcade configuration and an MFR is formed during eruption through the reconnection. The simulation reproduces a reverse S-shaped MFR with dextral chirality, and the axis of this MFR rotates counterclockwise while rising, which compares favorably with a typical filament eruption observed from dual viewing angles. By calculating the twist and writhe numbers of the modeled MFR during its eruption, we found that accompanied with the rotation, the nonlocal writhe of the MFR's axis decreases while the twist of its surrounding field lines increases, and this is distinct from the kink instability, which converts magnetic twist into writhe of the MFR axis.

astro-ph.SR↗

arXiv2021

Resolving Two Distinct Thermal X-ray Components in A compound Solar Flare

X-ray emission provides the most direct diagnostics of the energy-release process in solar flares. Occasionally, a superhot X-ray source is found to be above hot flare loops of ~10 MK temperature. While the origin of the superhot plasma is still elusive, it has conjured up an intriguing image of in-situ plasma heating near the reconnection site high above the flare loops, in contrast to the conventional picture of chromospheric evaporation. Here we investigate an extremely long-duration solar flare, in which EUV images show two distinct flare loop systems that appear successively along a Gamma-shaped polarity inversion line (PIL). When both flare loop systems are present, the HXR spectrum is found to be well fitted by combining a hot component (Te ~12 MK) and a superhot component (Te ~30 MK). Associated with a fast CME, the superhot X-ray source is located at top of the flare arcade that appears earlier, straddling and extending along the long "arm" of the Gamma-shaped PIL. Associated with a slow CME, the hot X-ray source is located at the top of the flare arcade that appears later and sits astride the short "arm" of the Gamma-shaped PIL. Aided by observations from a different viewing angle, we are able to verify that the superhot X-ray source is above the hot one in projection, but the two sources belong to different flare loop systems. Thus, this case study provides a stereoscopic observation explaining the co-existence of superhot and hot X-ray emitting plasmas in solar flares.

astro-ph.SR↗

arXiv2021

PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling

We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in onlin emessaging. PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation. Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context. In addition, for both tasks, we provide baseline models using the state-of-the-art models and report their benchmark performances. The best image retrieval model achieves 10.4% recall@1 (out of 1000 candidates) and the best photo intent prediction model achieves 58.1% F1 score, indicating that the dataset presents interesting yet challenging real-world problems. We are releasing PhotoChat to facilitate future research work among the community.

cs.IR↗

arXiv2021

In-game Residential Home Planning via Visual Context-aware Global Relation Learning

In this paper, we propose an effective global relation learning algorithm to recommend an appropriate location of a building unit for in-game customization of residential home complex. Given a construction layout, we propose a visual context-aware graph generation network that learns the implicit global relations among the scene components and infers the location of a new building unit. The proposed network takes as input the scene graph and the corresponding top-view depth image. It provides the location recommendations for a newly-added building units by learning an auto-regressive edge distribution conditioned on existing scenes. We also introduce a global graph-image matching loss to enhance the awareness of essential geometry semantics of the site. Qualitative and quantitative experiments demonstrate that the recommended location well reflects the implicit spatial rules of components in the residential estates, and it is instructive and practical to locate the building units in the 3D scene of the complex construction.

cs.CV↗

arXiv2021

The configuration and failed eruption of a complex magnetic flux rope above a δ sunspot region

Aims. We investigate the configuration of a complex flux rope above a δ sunspot region in NOAA AR 11515, and its eruptive expansion during a confined M5.3-class flare. Methods. We study the formation of the δ sunspot using continuum intensity images and photospheric vector magnetograms provided by SDO/HMI. We use EUV and UV images provided by SDO/AIA, and hard X-ray emission recorded by RHESSI to investigate the eruptive details. The coronal magnetic field is extrapolated with a non-linear force free field (NLFFF) method, based on which the flux rope is identified by calculating the twist number Tw and squashing factor Q. We search the null point via a modified Powell hybrid method. Results. The collision between two emerging spot groups form the δ sunspot. A bald patch (BP) forms at the collision location, above which a complex flux rope is identified. The flux rope has multiple layers, with one compact end and one bifurcated end, having Tw decreasing from the core to the boundary. A null point is located above the flux rope. The eruptive process consists of precursor flaring at a 'v'-shaped coronal structure, rise of the filament, and flaring below the filament, corresponding well with the NLFFF topological structures, including the null point and the flux rope with BP and hyperbolic flux tube (HFT). Two sets of post-flare loops and three flare ribbons support the bifurcation configuration of the flux rope. Conclusions. The precursor reconnection, which occurs at the null point, weakens the overlying confinement to allow the flux rope to rise, fitting the breakout model. The main phase reconnection, which may occur at the BP or HFT, facilitates the flux rope rising. The results suggest that the δ spot configuration presents an environment prone to the formation of complex magnetic configurations which will work together to produce activities.

astro-ph.SR↗

arXiv2021

ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces

As mobile devices are becoming ubiquitous, regularly interacting with a variety of user interfaces (UIs) is a common aspect of daily life for many people. To improve the accessibility of these devices and to enable their usage in a variety of settings, building models that can assist users and accomplish tasks through the UI is vitally important. However, there are several challenges to achieve this. First, UI components of similar appearance can have different functionalities, making understanding their function more important than just analyzing their appearance. Second, domain-specific features like Document Object Model (DOM) in web pages and View Hierarchy (VH) in mobile applications provide important signals about the semantics of UI elements, but these features are not in a natural language format. Third, owing to a large diversity in UIs and absence of standard DOM or VH representations, building a UI understanding model with high coverage requires large amounts of training data. Inspired by the success of pre-training based approaches in NLP for tackling a variety of problems in a data-efficient way, we introduce a new pre-trained UI representation model called ActionBert. Our methodology is designed to leverage visual, linguistic and domain-specific features in user interaction traces to pre-train generic feature representations of UIs and their components. Our key intuition is that user actions, e.g., a sequence of clicks on different UI components, reveals important information about their functionality. We evaluate the proposed model on a wide variety of downstream tasks, ranging from icon classification to UI component retrieval based on its natural language description. Experiments show that the proposed ActionBert model outperforms multi-modal baselines across all downstream tasks by up to 15.5%.

cs.CL↗

arXiv2021

The Source Locations of Major Flares and CMEs in the Emerging Active Regions

Major flares and coronal mass ejections (CMEs) tend to originate from the compact polarity inversion lines (PILs) in the solar active regions (ARs). Recently, a scenario named as "collisional shearing" is proposed by \citet{Chintzoglou_2019} to explain the phenomenon, which suggests that the collision between different emerging bipoles is able to form the compact PIL, driving the shearing and flux cancellation that are responsible to the subsequent large activities. In this work, through tracking the evolution of 19 emerging ARs from their birth until they produce the first major flares or CMEs, we investigated the source PILs of the activities, i.e., the active PILs, to explore the generality of "collisional shearing". We find that none of the active PILs is the self PIL (sPIL) of a single bipole. We further find that 11 eruptions originate from the collisional PILs (cPILs) formed due to the collision between different bipoles, 6 from the conjoined systems of sPIL and cPIL, and 2 from the conjoined systems of sPIL and ePIL (external PIL between the AR and the nearby preexisting polarities). Collision accompanied by shearing and flux cancellation is found developing at all PILs prior to the eruptions, with $84\%$ (16/19) cases having collisional length longer than 18~Mm. Moreover, we find that the magnitude of the flares is positively correlated with the collisional length of the active PILs, indicating that the intenser activities tend to originate from the PILs with severer collision. The results suggest that the "collisional shearing", i.e., bipole-bipole interaction during the flux emergence is a common process in driving the major activities in emerging ARs.

astro-ph.SR↗

arXiv2020

Magnetic Flux of Active Regions Determining the Eruptive Character of Large Solar Flares

We establish the largest eruptive/confined flare database to date and analyze 322 flares of \emph{GOES} class M1.0 and larger that occurred during 2010$-$2019, i.e., almost spanning the entire solar cycle 24. We find that the total unsigned magnetic flux ($Φ$$_{AR}$) of active regions (ARs) is a key parameter in governing the eruptive character of large flares, with the proportion of eruptive flares exhibiting a strong anti-correlation with $Φ$$_{AR}$. This means that an AR containing a large magnetic flux has a lower probability for the large flares it produces to be associated with a coronal mass ejection (CME). This finding is supported by the high positive correlation we obtained between the critical decay index height and $Φ$$_{AR}$, implying that ARs with a larger $Φ$$_{AR}$ have a stronger magnetic confinement. Moreover, the confined flares originating from ARs larger than 1.0$\times$$10^{23}$ Mx have several characteristics in common: stable filament, slipping magnetic reconnection and strongly sheared post-flare loops. Our findings reveal new relations between the magnetic flux of ARs and the occurrence of CMEs in association with large flares. These relations obtained here provide quantitative criteria for forecasting CMEs and adverse space weather, and have also important implications for "superflares" on solar-type stars and stellar CMEs. The link of database is https://doi.org/10.12149/101030.

astro-ph.SR↗

arXiv2020

Cause and Kinematics of a Jet-Like CME

In this article, we present the multi-viewpoint and multi-wavelength analysis of an atypical solar jet based on the data from Solar Dynamics Observatory, SOlar, and Heliospheric Observatory, and Solar TErrestrial RElations Observatory. It is usually believed that the coronal mass ejections (CMEs) are developed from the large scale solar eruptions in the lower atmosphere. However, the kinematical and spatial evolution of the jet on 2013 April 28 guide us that the jet was clearly associated with a narrow CME having a width of approx 25 degrees with a speed of 450 km/s. To better understand the link between the jet and the CME, we did the coronal potential field extrapolation from the line of sight magnetogram of the AR. The extrapolations present that the jet eruption follows exactly the same path of the open magnetic field lines from the source region which provides the route for the jet material to escape from the solar surface towards the outer corona.

astro-ph.SR↗

arXiv2020

Electric Currents through J-shaped and Non-J-shaped Flare Ribbons

Recently solar flares exhibiting a double J-shaped ribbons in the lower solar atmosphere have been paid increasing attention in the context of extending the two-dimensional standard flare model to three dimensions, as motivated by the spatial correlation between photospheric current channels and flare ribbons. Here we study the electric currents through the photospheric area swept by flare ribbons (termed synthesized ribbon area or SRA), with a sample of 71 two-ribbon flares, of which 36 are J-shaped. Electric currents flowing through one ribbon are highly correlated with those through the other, therefore belonging to the same current system. The non-neutrality factor of this current system is independent of the flare magnitude, implying that both direct and return currents participate in flares. J-shaped flares are distinct from non-J-shaped flares in the following aspects: 1) Electric current densities within J-shaped SRA are significantly smaller than those within non-J-shaped SRA, but J-shaped SRA and its associated magnetic flux are also significantly larger. 2) Electric currents through SRA are positively correlated with the flare magnitude, but J-shaped flares show stronger correlation than non-J-shaped flares. 3) The majority (75\%) of J-shaped flares are eruptive, while the majority (86\%) of non-J-shaped flares are confined; accordingly, hosting active regions of J-shaped flares are more likely to be sigmoidal than non-J-shaped flares. Thus, J-shaped flares constitute a distinct subset of two-ribbon flares, probably the representative of eruptive ones. Further, we found that combining SRA and its associated magnetic flux has the potential to differentiate eruptive from confined flares.

astro-ph.SR↗

arXiv2020

Complex-valued valuations on Lp spaces

All continuous translation invariant complex-valued valuations on Lebesgue measurable functions are completely classified. And all continuous rotation invariant complex-valued valuations on spherical Lebesgue measurable functions are also completely classified.

math.MG↗

arXiv2020

Quality Control of Neuron Reconstruction Based on Deep Learning

Neuron reconstruction is essential to generate exquisite neuron connectivity map for understanding brain function. Despite the significant amount of effect that has been made on automatic reconstruction methods, manual tracing by well-trained human annotators is still necessary. To ensure the quality of reconstructed neurons and provide guidance for annotators to improve their efficiency, we propose a deep learning based quality control method for neuron reconstruction in this paper. By formulating the quality control problem into a binary classification task regarding each single point, the proposed approach overcomes the technical difficulties resulting from the large image size and complex neuron morphology. Not only it provides the evaluation of reconstruction quality, but also can locate exactly where the wrong tracing begins. This work presents one of the first comprehensive studies for whole-brain scale quality control of neuron reconstructions. Experiments on five-fold cross validation with a large dataset demonstrate that the proposed approach can detect 74.7% errors with only 1.4% false alerts.

cs.CV↗

arXiv2020

The Relationship between Chirality, Sense of Rotation, and Hemispheric Preference of Solar Eruptive Filaments

The orientation, chirality, and dynamics of solar eruptive filaments is a key to understanding the magnetic field of coronal mass ejections (CMEs) and therefore to predicting the geoeffectiveness of CMEs arriving at Earth. However, confusion and contention remain over the relationship between the filament chirality, magnetic helicity, and sense of rotation during eruption. To resolve the ambiguity in observations, in this paper, we used stereoscopic observations to determine the rotation direction of filament apex and the method proposed by Chen et al. (2014) to determine the filament chirality. Our sample of 12 eruptive active-region filaments establishes a strong one-to-one relationship, i.e., during the eruption, sinistral/dextral filaments (located in the southern/northern hemisphere) rotate clockwise/counterclockwise when viewed from above, and corroborates a weak hemispheric preference, i.e., a filament and related sigmoid both exhibit a forward (reverse) S shape in the southern (northern) hemisphere, which suggests that the sigmoidal filament is associated with a low-lying magnetic flux rope with its axis dipped in the middle. As a result of rotation, the projected S shape of a filament is anticipated to be reversed during eruption.

astro-ph.SR↗

arXiv2019

Formation of a Magnetic Flux Rope in the Early Emergence Phase of NOAA Active Region 12673

In this work, we investigate the formation of a magnetic flux rope (MFR) above the central polarity inversion line (PIL) of NOAA Active Region 12673 during its early emergence phase. Through analyzing the photospheric vector magnetic field, extreme ultraviolet (EUV) and ultraviolet (UV) images, extrapolated three-dimensional (3D) non-linear force-free fields (NLFFFs), as well as the photospheric motions, we find that with the successive emergence of different bipoles in the central region, the conjugate polarities separate, resulting in collision between the non-conjugated opposite polarities. Nearly-potential loops appear above the PIL at first, then get sheared and merge at the collision locations as evidenced by the appearance of a continuous EUV sigmoid on 2017 September 4, which also indicates the formation of an MFR. The 3D NLFFFs further reveal the gradual buildup of the MFR, accompanied by the appearance of two elongated bald patches (BPs) at the collision locations and a very low-lying hyperbolic flux tube configuration between the BPs. The final MFR has relatively steady axial flux and average twist number of around $2.1\times 10^{20}$~Mx and -1.5, respective. Shearing motions are found developing near the BPs when the collision occurs, with flux cancellation and UV brightenings being observed simultaneously, indicating the development of a process named as "collisional shearing" (firstly identified by Chintzoglou et al. 2019). The results clearly show that the MFR is formed by "collisional shearing", i.e., through shearing and flux cancellation driven by the collision between non-conjugated opposite polarities during their emergence.

astro-ph.SR↗