arXiv subjects

Jiong Wang

Publications and source records attributed to Jiong Wang.

33 records · Page 2Linked to original sources

arXiv2024

QACP: An Annotated Question Answering Dataset for Assisting Chinese Python Programming Learners

In online learning platforms, particularly in rapidly growing computer programming courses, addressing the thousands of students' learning queries requires considerable human cost. The creation of intelligent assistant large language models (LLMs) tailored for programming education necessitates distinct data support. However, in real application scenarios, the data resources for training such LLMs are relatively scarce. Therefore, to address the data scarcity in intelligent educational systems for programming, this paper proposes a new Chinese question-and-answer dataset for Python learners. To ensure the authenticity and reliability of the sources of the questions, we collected questions from actual student questions and categorized them according to various dimensions such as the type of questions and the type of learners. This annotation principle is designed to enhance the effectiveness and quality of online programming education, providing a solid data foundation for developing the programming teaching assists (TA). Furthermore, we conducted comprehensive evaluations of various LLMs proficient in processing and generating Chinese content, highlighting the potential limitations of general LLMs as intelligent teaching assistants in computer programming courses.

cs.CL↗

arXiv2023

LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark

Large language models have emerged as a promising approach towards achieving general-purpose AI agents. The thriving open-source LLM community has greatly accelerated the development of agents that support human-machine dialogue interaction through natural language processing. However, human interaction with the world extends beyond only text as a modality, and other modalities such as vision are also crucial. Recent works on multi-modal large language models, such as GPT-4V and Bard, have demonstrated their effectiveness in handling visual modalities. However, the transparency of these works is limited and insufficient to support academic research. To the best of our knowledge, we present one of the very first open-source endeavors in the field, LAMM, encompassing a Language-Assisted Multi-Modal instruction tuning dataset, framework, and benchmark. Our aim is to establish LAMM as a growing ecosystem for training and evaluating MLLMs, with a specific focus on facilitating AI agents capable of bridging the gap between ideas and execution, thereby enabling seamless human-AI interaction. Our main contribution is three-fold: 1) We present a comprehensive dataset and benchmark, which cover a wide range of vision tasks for 2D and 3D vision. Extensive experiments validate the effectiveness of our dataset and benchmark. 2) We outline the detailed methodology of constructing multi-modal instruction tuning datasets and benchmarks for MLLMs, enabling rapid scaling and extension of MLLM research to diverse domains, tasks, and modalities. 3) We provide a primary but potential MLLM training framework optimized for modality extension. We also provide baseline models, comprehensive experimental observations, and analysis to accelerate future research. Our baseline model is trained within 24 A100 GPU hours, framework supports training with V100 and RTX3090 is available thanks to the open-source society.

cs.CV↗

arXiv2023

Model Compression for DNN-based Speaker Verification Using Weight Quantization

DNN-based speaker verification (SV) models demonstrate significant performance at relatively high computation costs. Model compression can be applied to reduce the model size for lower resource consumption. The present study exploits weight quantization to compress two widely-used SV models, namely ECAPA-TDNN and ResNet. Experimental results on VoxCeleb show that weight quantization is effective for compressing SV models. The model size can be reduced multiple times without noticeable degradation in performance. Compression of ResNet shows more robust results than ECAPA-TDNN with lower-bitwidth quantization. Analysis of the layer weights suggests that the smooth weight distribution of ResNet may be related to its better robustness. The generalization ability of the quantized model is validated via a language-mismatched SV task. Furthermore, analysis by information probing reveals that the quantized models can retain most of the speaker-relevant knowledge learned by the original models.

eess.AS↗

arXiv2023

Set-Based Face Recognition Beyond Disentanglement: Burstiness Suppression With Variance Vocabulary

Set-based face recognition (SFR) aims to recognize the face sets in the unconstrained scenario, where the appearance of same identity may change dramatically with extreme variances (e.g., illumination, pose, expression). We argue that the two crucial issues in SFR, the face quality and burstiness, are both identity-irrelevant and variance-relevant. The quality and burstiness assessment are interfered with by the entanglement of identity, and the face recognition is interfered with by the entanglement of variance. Thus we propose to separate the identity features with the variance features in a light-weighted set-based disentanglement framework. Beyond disentanglement, the variance features are fully utilized to indicate face quality and burstiness in a set, rather than being discarded after training. To suppress face burstiness in the sets, we propose a vocabulary-based burst suppression (VBS) method which quantizes faces with a reference vocabulary. With interword and intra-word normalization operations on the assignment scores, the face burtisness degrees are appropriately estimated. The extensive illustrations and experiments demonstrate the effect of the disentanglement framework with VBS, which gets new state-of-the-art on the SFR benchmarks. The code will be released at https://github.com/Liubinggunzu/set_burstiness.

cs.CV↗

arXiv2023

Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration

Weakly Supervised Object Localization (WSOL), which aims to localize objects by only using image-level labels, has attracted much attention because of its low annotation cost in real applications. Recent studies leverage the advantage of self-attention in visual Transformer for long-range dependency to re-active semantic regions, aiming to avoid partial activation in traditional class activation mapping (CAM). However, the long-range modeling in Transformer neglects the inherent spatial coherence of the object, and it usually diffuses the semantic-aware regions far from the object boundary, making localization results significantly larger or far smaller. To address such an issue, we introduce a simple yet effective Spatial Calibration Module (SCM) for accurate WSOL, incorporating semantic similarities of patch tokens and their spatial relationships into a unified diffusion model. Specifically, we introduce a learnable parameter to dynamically adjust the semantic correlations and spatial context intensities for effective information propagation. In practice, SCM is designed as an external module of Transformer, and can be removed during inference to reduce the computation cost. The object-sensitive localization ability is implicitly embedded into the Transformer encoder through optimization in the training phase. It enables the generated attention maps to capture the sharper object boundaries and filter the object-irrelevant background area. Extensive experimental results demonstrate the effectiveness of the proposed method, which significantly outperforms its counterpart TS-CAM on both CUB-200 and ImageNet-1K benchmarks. The code is available at https://github.com/164140757/SCM.

cs.CV↗

arXiv2022

Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering

Multi-modal video question answering aims to predict correct answer and localize the temporal boundary relevant to the question. The temporal annotations of questions improve QA performance and interpretability of recent works, but they are usually empirical and costly. To avoid the temporal annotations, we devise a weakly supervised question grounding (WSQG) setting, where only QA annotations are used and the relevant temporal boundaries are generated according to the temporal attention scores. To substitute the temporal annotations, we transform the correspondence between frames and subtitles to Frame-Subtitle (FS) self-supervision, which helps to optimize the temporal attention scores and hence improve the video-language understanding in VideoQA model. The extensive experiments on TVQA and TVQA+ datasets demonstrate that the proposed WSQG strategy gets comparable performance on question grounding, and the FS self-supervision helps improve the question answering and grounding performance on both QA-supervision only and full-supervision settings.

cs.CV↗

arXiv2022

A general theoretical scheme for shape-programming of incompressible hyperelastic shells through differential growth

In this paper, we study the problem of shape-programming of incompressible hyperelastic shells through differential growth. The aim of the current work is to determine the growth tensor (or growth functions) that can produce the deformation of a shell to the desired shape. First, a consistent finite-strain shell theory is introduced. The shell equation system is established from the 3D governing system through a series expansion and truncation approach. Based on the shell theory, the problem of shape-programming is studied under the stress-free assumption. For a special case in which the parametric coordinate curves generate a net of curvature lines on the target surface, the sufficient condition to ensure the vanishing of the stress components is analyzed, from which the explicit expression of the growth tensor can be derived. In the general case, we conduct the variable changes and derive the total growth tensor by considering a two-step deformation of the shell. With these obtained results, a general theoretical scheme for shape-programming of thin hyperelastic shells through differential growth is proposed. To demonstrate the feasibility and efficiency of the proposed scheme, several nature-inspired examples are studied. The derived growth tensors in these examples have also been implemented in the numerical simulations to verify their correctness and accuracy. The simulation results show that the target shapes of the shell samples can be recovered completely. The scheme for shape-programming proposed in the current work is helpful in designing and manufacturing intelligent soft devices.

math.NA↗

arXiv2022

On a general multi-layered hyperelastic plate theory of growth

In this paper, we propose a multi-layered hyperelastic plate theory of growth within the framework of nonlinear elasticity. First, the 3D governing system for a general multi-layered hyperelastic plate is established, which incorporates the growth effect, and the material and geometrical parameters of the different layers. Then, a series expansion-truncation approach is adopted to eliminate the thickness variables in the 3D governing system. An elaborate calculation scheme is applied to derive the iteration relations of the coefficient functions in the series expansions. Through some further manipulations, a 2D vector plate equation system with the associated boundary conditions is established, which only contains the unknowns in the bottom layer of the plate. To show the efficiency of the current plate theory, three typical examples regarding the growth-induced deformations and instabilities of multi-layered plate samples are studied. Some analytical and numerical solutions to the plate equation are obtained, which can provide accurate predictions on the growth behaviors of the plate samples. Furthermore, the problem of `shape-programming' of multi-layered hyperelastic plates through differential growth is studied. The explicit formulas of shape-programming for some typical multi-layered plates are derived, which involve the fundamental quantities of the 3D target shapes. By using these formulas, the shape evolutions of the plates during the growing processes can be controlled accurately. The results obtained in the current work are helpful for the design of intelligent soft devices with multi-layered plate structures.

cond-mat.mtrl-sci↗

arXiv2022

VLAD-VSA: Cross-Domain Face Presentation Attack Detection with Vocabulary Separation and Adaptation

For face presentation attack detection (PAD), most of the spoofing cues are subtle, local image patterns (e.g., local image distortion, 3D mask edge and cut photo edges). The representations of existing PAD works with simple global pooling method, however, lose the local feature discriminability. In this paper, the VLAD aggregation method is adopted to quantize local features with visual vocabulary locally partitioning the feature space, and hence preserve the local discriminability. We further propose the vocabulary separation and adaptation method to modify VLAD for cross-domain PADtask. The proposed vocabulary separation method divides vocabulary into domain-shared and domain-specific visual words to cope with the diversity of live and attack faces under the cross-domain scenario. The proposed vocabulary adaptation method imitates the maximization step of the k-means algorithm in the end-to-end training, which guarantees the visual words be close to the center of assigned local features and thus brings robust similarity measurement. We give illustrations and extensive experiments to demonstrate the effectiveness of VLAD with the proposed vocabulary separation and adaptation method on standard cross-domain PAD benchmarks. The codes are available at https://github.com/Liubinggunzu/VLAD-VSA.

cs.CV↗

arXiv2022

Theoretical scheme on shape-programming of thin hyperelastic plates through differential growth

In this paper, a theoretical scheme is proposed for shape-programming of thin hyperelastic plates through differential growth. First, starting from the 3D governing system of a hyperelastic (neo-Hookean) plate, a consistent finite-strain plate equation system is formulated through a series-expansion and truncation approach. Based on the plate equation system, the problem of shape-programming is studied under the stress-free assumption. By equating the stress components in the plate equations to be zero, the explicit relations between growth functions and geometrical quantities of the target shape of the plate are derived. Then, a theoretical scheme of shape-programming is proposed, which can be used to identify the growth fields corresponding to arbitrary 3D shapes of the plate. To demonstrate the efficiency of the scheme, some typical examples are studied. The predicted growth functions in these examples are adopted in the numerical simulations, from which the target shapes of the plate can be recovered completely. The scheme of shape-programming proposed in the current work is applicable for manufacture of intelligent soft devices.

cs.CE↗

arXiv2020

Learning a Reinforced Agent for Flexible Exposure Bracketing Selection

Automatically selecting exposure bracketing (images exposed differently) is important to obtain a high dynamic range image by using multi-exposure fusion. Unlike previous methods that have many restrictions such as requiring camera response function, sensor noise model, and a stream of preview images with different exposures (not accessible in some scenarios e.g. some mobile applications), we propose a novel deep neural network to automatically select exposure bracketing, named EBSNet, which is sufficiently flexible without having the above restrictions. EBSNet is formulated as a reinforced agent that is trained by maximizing rewards provided by a multi-exposure fusion network (MEFNet). By utilizing the illumination and semantic information extracted from just a single auto-exposure preview image, EBSNet can select an optimal exposure bracketing for multi-exposure fusion. EBSNet and MEFNet can be jointly trained to produce favorable results against recent state-of-the-art approaches. To facilitate future research, we provide a new benchmark dataset for multi-exposure selection and fusion.

cs.CV↗

arXiv2018

Attention-based Pyramid Aggregation Network for Visual Place Recognition

Visual place recognition is challenging in the urban environment and is usually viewed as a large scale image retrieval task. The intrinsic challenges in place recognition exist that the confusing objects such as cars and trees frequently occur in the complex urban scene, and buildings with repetitive structures may cause over-counting and the burstiness problem degrading the image representations. To address these problems, we present an Attention-based Pyramid Aggregation Network (APANet), which is trained in an end-to-end manner for place recognition. One main component of APANet, the spatial pyramid pooling, can effectively encode the multi-size buildings containing geo-information. The other one, the attention block, is adopted as a region evaluator for suppressing the confusing regional features while highlighting the discriminative ones. When testing, we further propose a simple yet effective PCA power whitening strategy, which significantly improves the widely used PCA whitening by reasonably limiting the impact of over-counting. Experimental evaluations demonstrate that the proposed APANet outperforms the state-of-the-art methods on two place recognition benchmarks, and generalizes well on standard image retrieval datasets.

cs.CV↗

arXiv2017

Exact solution for growth-induced large bending deformation of a hyperelastic plate

In this paper, the growth-induced bending deformation of a thin hyperelastic plate is studied. For a plane-strain problem, the governing PDE system is formulated, which is composed of the mechanical equilibrium equations, the constraint equation and the boundary conditions. By adopting a gradient growth field with the growth value changes linearly along the thickness direction, the exact solution to the governing PDE system can be derived. With the obtained solution, some important features of the bending deformation in the plate can be found and the effects of the different growth parameters can be revealed. This exact solution can serve as a benchmark one for testing the correctness of numerical schemes and approximate plate models in growth theory.

physics.class-ph↗

arXiv2009

Instabilities Induced by Phase Fronts Coalescence During the Phase Transitions in a Thin Sma Layer: Mechanism and Analytical Descriptions

Systematic experiments on stress-induced phase transitions in thin SMA structures in literature have revealed two interesting instability phenomena: the coalescence of two martensite-austenite fronts leads to a sudden stress drop and that of two austenite-martensite fronts leads to a sudden stress jump. In order to get an insight into these two phenomena, in this work we carry out an analytical study on the stress-induced phase transitions in a thin SMA layer (a simple structure in which the two coalescence processes can happen). We derive a quasi-2D model with a non-convex effective strain energy function while taking into account the rate-independent dissipation effect. By using a coupled series-asymptotic expansion method, we manage to express the total energy dissipation in terms of the leading-order term of the axial strain. The equilibrium equations are obtained by maximizing the total energy dissipation, which are then solved analytically under suitable boundary conditions. The analytical results reveal that the mechanism for such instabilities is the presence of "limited points", which cause the switch of nontrivial solution modes to trivial solution modes. Descriptions for the whole coalescence processes of two fronts are also provided based on the analytical solutions, which also capture the morphology varies of the specimen. It is also revealed the key role played by the thickness-length ratio on these instabilities: the zero limit of which can lead to the smooth switch of nontrivial modes to trivial modes with no stress drop or stress jump.

physics.class-ph↗

arXiv2008

On Constructing the Asymptotic Solutions for Phase Transitions in a Slender Cylinder Composed of a Compressible Hyperelastic Material with Clamped End Conditions

In this paper, we study phase transitions in a slender circular cylinder composed of a compressible hyperelastic material with a non-convex strain energy function. We aim to construct the asymptotic solutions based on an axisymmetrical three-dimensional setting and use the results to describe the key features (in particular, instability phenomena) observed in the experiments by others. The difficult problem of the solution bifurcations of the governing nonlinear partial differential equations (PDE's) is solved through a novel approach. By using a methodology involving coupled series-asymptotic expansions, we derive the normal form equation of the original complicated system of nonlinear PDE's. By writing the normal form equation into a first-order dynamical system and with a phase-plane analysis, we manage to deduce the global bifurcation properties and to solve the boundary-value problem analytically. The asymptotic solutions (including post-bifurcation solutions) in terms of integrals are obtained. The engineering stress-strain curve plotted from the asymptotic solutions can capture the key features of the curve measured in a few experiments (e.g., the stress drop, the stress plateau, and the small stress valley). It appears that the asymptotic solutions obtained shed certain light on the instability phenomena associated with phase transitions in a cylinder, in particular the role played by the radius-length ratio. Also, an important feature of this work is that we consider the clamped end conditions, which are more practical but rarely used in literature for phase transition problems.

physics.class-ph↗