arXiv subjects

Jiawan Zhang

Publications and source records attributed to Jiawan Zhang.

At least 19 recordsLinked to original sources

arXiv2026

Figma2Code: Automating Multimodal Design to Code in the Wild

Front-end development constitutes a substantial portion of software engineering, yet converting design mockups into production-ready User Interface (UI) code remains tedious and costly. While recent work has explored automating this process with Multimodal Large Language Models (MLLMs), existing approaches typically rely solely on design images. As a result, they must infer complex UI details from images alone, often leading to degraded results. In real-world development workflows, however, design mockups are usually delivered as Figma files, a widely used tool for front-end design, that embed rich multimodal information (e.g., metadata and assets) essential for generating high-quality UI. To bridge this gap, we introduce Figma2Code, a new task that advances design-to-code into a multimodal setting and aims to automate design-to-code in the wild. Specifically, we collect paired design images and their corresponding metadata files from the Figma community. We then apply a series of processing operations, including rule-based filtering, human- and MLLM-based annotation and screening, and metadata refinement. This process yields 3,055 samples, from which designers curate a balanced dataset of 213 high-quality cases. Using this dataset, we benchmark ten state-of-the-art open-source and proprietary MLLMs. Our results show that while proprietary models achieve superior visual fidelity, they remain limited in layout responsiveness and code maintainability. Further experiments across modalities and ablation studies corroborate this limitation, partly due to models' tendency to directly map primitive visual attributes from Figma metadata.

cs.SE↗

arXiv2026

Quantifying LLM Biases Across Instruction Boundary in Mixed Question Forms

Large Language Models (LLMs) annotated datasets are widely used nowadays, however, large-scale annotations often show biases in low-quality datasets. For example, Multiple-Choice Questions (MCQs) datasets with one single correct option is common, however, there may be questions attributed to none or multiple correct options; whereas true-or-false questions are supposed to be labeled with either True or False, but similarly the text can include unsolvable elements, which should be further labeled as Unknown. There are problems when low-quality datasets with mixed question forms can not be identified. We refer to these exceptional label forms as Sparse Labels, and LLMs' ability to distinguish datasets with Sparse Labels mixture is important. Since users may not know situations of datasets, their instructions can be biased. To study how different instruction settings affect LLMs' identifications of Sparse Labels mixture, we introduce the concept of Instruction Boundary, which systematically evaluates different instruction settings that lead to biases. We propose BiasDetector, a diagnostic benchmark to systematically evaluate LLMs on datasets with mixed question forms under Instruction Boundary settings. Experiments show that users' instructions induce large biases on our benchmark, highlighting the need not only for LLM developers to recognize risks of LLM biased annotation resulting in Sparse Labels mixture, but also problems arising from users' instructions to identify them. Code, datasets and detailed implementations are available at https://github.com/ZpLing/Instruction-Boundary.

cs.CL↗

arXiv2026

Face Normal Estimation from Rags to Riches

Although recent approaches to face normal estimation have achieved promising results, their effectiveness heavily depends on large-scale paired data for training. This paper concentrates on relieving this requirement via developing a coarse-to-fine normal estimator. Concretely, our method first trains a neat model from a small dataset to produce coarse face normals that perform as guidance (called exemplars) for the following refinement. A self-attention mechanism is employed to capture long-range dependencies, thus remedying severe local artifacts left in estimated coarse facial normals. Then, a refinement network is customized for the sake of mapping input face images together with corresponding exemplars to fine-grained high-quality facial normals. Such a logical function split can significantly cut the requirement of massive paired data and computational resource. Extensive experiments and ablation studies are conducted to demonstrate the efficacy of our design and reveal its superiority over state-of-the-art methods in terms of both training expense as well as estimation quality. Our code and models are open-sourced at: https://github.com/AutoHDR/FNR2R.git.

cs.CV↗

arXiv2025

Adaptive Frequency Domain Alignment Network for Medical image segmentation

High-quality annotated data plays a crucial role in achieving accurate segmentation. However, such data for medical image segmentation are often scarce due to the time-consuming and labor-intensive nature of manual annotation. To address this challenge, we propose the Adaptive Frequency Domain Alignment Network (AFDAN)--a novel domain adaptation framework designed to align features in the frequency domain and alleviate data scarcity. AFDAN integrates three core components to enable robust cross-domain knowledge transfer: an Adversarial Domain Learning Module that transfers features from the source to the target domain; a Source-Target Frequency Fusion Module that blends frequency representations across domains; and a Spatial-Frequency Integration Module that combines both frequency and spatial features to further enhance segmentation accuracy across domains. Extensive experiments demonstrate the effectiveness of AFDAN: it achieves an Intersection over Union (IoU) of 90.9% for vitiligo segmentation in the newly constructed VITILIGO2025 dataset and a competitive IoU of 82.6% on the retinal vessel segmentation benchmark DRIVE, surpassing existing state-of-the-art approaches.

cs.CV↗

arXiv2025

PixelatedScatter: Arbitrary-level Visual Abstraction for Large-scale Multiclass Scatterplots

Overdraw is inevitable in large-scale scatterplots. Current scatterplot abstraction methods lose features in medium-to-low density regions. We propose a visual abstraction method designed to provide better feature preservation across arbitrary abstraction levels for large-scale scatterplots, particularly in medium-to-low density regions. The method consists of three closely interconnected steps: first, we partition the scatterplot into iso-density regions and equalize visual density; then, we allocate pixels for different classes within each region; finally, we reconstruct the data distribution based on pixels. User studies, quantitative and qualitative evaluations demonstrate that, compared to previous methods, our approach better preserves features and exhibits a special advantage when handling ultra-high dynamic range data distributions.

cs.MM↗

arXiv2023

Face Inverse Rendering via Hierarchical Decoupling

Previous face inverse rendering methods often require synthetic data with ground truth and/or professional equipment like a lighting stage. However, a model trained on synthetic data or using pre-defined lighting priors is typically unable to generalize well for real-world situations, due to the gap between synthetic data/lighting priors and real data. Furthermore, for common users, the professional equipment and skill make the task expensive and complex. In this paper, we propose a deep learning framework to disentangle face images in the wild into their corresponding albedo, normal, and lighting components. Specifically, a decomposition network is built with a hierarchical subdivision strategy, which takes image pairs captured from arbitrary viewpoints as input. In this way, our approach can greatly mitigate the pressure from data preparation, and significantly broaden the applicability of face inverse rendering. Extensive experiments are conducted to demonstrate the efficacy of our design, and show its superior performance in face relighting over other state-of-the-art alternatives. {Our code is available at \url{https://github.com/AutoHDR/HD-Net.git}}

cs.CV↗

arXiv2022

SemanticAxis: Exploring Multi-attribute Data by Semantics Construction and Ranking Analysis

Mining the distribution of features and sorting items by combined attributes are two common tasks in exploring and understanding multi-attribute (or multivariate) data. Up to now, few have pointed out the possibility of merging these two tasks into a united exploration context and the potential benefits of doing so. In this paper, we present SemanticAxis, a technique that achieves this goal by enabling analysts to build a semantic vector in two-dimensional space interactively. Essentially, the semantic vector is a linear combination of the original attributes. It can be used to represent and explain abstract concepts implied in local (outliers, clusters) or global (general pattern) features of reduced space, as well as serving as a ranking metric for its defined concepts. In order to validate the significance of combining the above two tasks in multi-attribute data analysis, we design and implement a visual analysis system, in which several interactive components cooperate with SemanticAxis seamlessly and expand its capacity to handle complex scenarios. We prove the effectiveness of our system and the SemanticAxis technique via two practical cases.

cs.DB↗

arXiv2022

Dual Space Coupling Model Guided Overlap-Free Scatterplot

The overdraw problem of scatterplots seriously interferes with the visual tasks. Existing methods, such as data sampling, node dispersion, subspace mapping, and visual abstraction, cannot guarantee the correspondence and consistency between the data points that reflect the intrinsic original data distribution and the corresponding visual units that reveal the presented data distribution, thus failing to obtain an overlap-free scatterplot with unbiased and lossless data distribution. A dual space coupling model is proposed in this paper to represent the complex bilateral relationship between data space and visual space theoretically and analytically. Under the guidance of the model, an overlap-free scatterplot method is developed through integration of the following: a geometry-based data transformation algorithm, namely DistributionTranscriptor; an efficient spatial mutual exclusion guided view transformation algorithm, namely PolarPacking; an overlap-free oriented visual encoding configuration model and a radius adjustment tool, namely $f_{r_{draw}}$. Our method can ensure complete and accurate information transfer between the two spaces, maintaining consistency between the newly created scatterplot and the original data distribution on global and local features. Quantitative evaluation proves our remarkable progress on computational efficiency compared with the state-of-the-art methods. Three applications involving pattern enhancement, interaction improvement, and overdraw mitigation of trajectory visualization demonstrate the broad prospects of our method.

cs.GR↗

arXiv2022

Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion

Uncalibrated photometric stereo is proposed to estimate the detailed surface normal from images under varying and unknown lightings. Recently, deep learning brings powerful data priors to this underdetermined problem. This paper presents a new method for deep uncalibrated photometric stereo, which efficiently utilizes the inter-image representation to guide the normal estimation. Previous methods use optimization-based neural inverse rendering or a single size-independent pooling layer to deal with multiple inputs, which are inefficient for utilizing information among input images. Given multi-images under different lighting, we consider the intra-image and inter-image variations highly correlated. Motivated by the correlated variations, we designed an inter-intra image feature fusion module to introduce the inter-image representation into the per-image feature extraction. The extra representation is used to guide the per-image feature extraction and eliminate the ambiguity in normal estimation. We demonstrate the effect of our design on a wide range of samples, especially on dark materials. Our method produces significantly better results than the state-of-the-art methods on both synthetic and real data.

cs.CV↗

arXiv2021

Towards Visual Explainable Active Learning for Zero-Shot Classification

Zero-shot classification is a promising paradigm to solve an applicable problem when the training classes and test classes are disjoint. Achieving this usually needs experts to externalize their domain knowledge by manually specifying a class-attribute matrix to define which classes have which attributes. Designing a suitable class-attribute matrix is the key to the subsequent procedure, but this design process is tedious and trial-and-error with no guidance. This paper proposes a visual explainable active learning approach with its design and implementation called semantic navigator to solve the above problems. This approach promotes human-AI teaming with four actions (ask, explain, recommend, respond) in each interaction loop. The machine asks contrastive questions to guide humans in the thinking process of attributes. A novel visualization called semantic map explains the current status of the machine. Therefore analysts can better understand why the machine misclassifies objects. Moreover, the machine recommends the labels of classes for each attribute to ease the labeling burden. Finally, humans can steer the model by modifying the labels interactively, and the machine adjusts its recommendations. The visual explainable active learning approach improves humans' efficiency of building zero-shot classification models interactively, compared with the method without guidance. We justify our results with user studies using the standard benchmarks for zero-shot classification.

cs.HC↗

arXiv2021

Regshock: Interactive Visual Analytics of Systemic Risk in Financial Networks

Financial regulatory agencies are struggling to manage the systemic risks attributed to negative economic shocks. Preventive interventions are prominent to eliminate the risks and help to build a more resilient financial system. Although tremendous efforts have been made to measure multi-risk severity levels, understand the contagion behaviors and other risk management problems, there still lacks a theoretical framework revealing what and how regulatory intervention measurements can mitigate systemic risk. Here we demonstrate regshock, a practical visual analytical approach to support the exploration and evaluation of financial regulation measurements. We propose risk-island, an unprecedented risk-centered visualization algorithm to help uncover the risk patterns while preserving the topology of financial networks. We further propose regshock, a novel visual exploration and assessment approach based on the simulation-intervention-evaluation analysis loop, to provide a heuristic surgical intervention capability for systemic risk mitigation. We evaluate our approach through extensive case studies and expert reviews. To our knowledge, this is the first practical systemic method for the financial network intervention and risk mitigation problem; our validated approach potentially improves the risk management and control capabilities of financial experts.

cs.HC↗

arXiv2020

iConViz: Interactive Visual Exploration of the Default Contagion Risk of Networked-Guarantee Loans

Groups of enterprises can serve as guarantees for one another and form complex networks when obtaining loans from commercial banks. During economic slowdowns, corporate default may spread like a virus and lead to large-scale defaults or even systemic financial crises. To help financial regulatory authorities and banks manage the risk associated with networked loans, we identified the default contagion risk, a pivotal issue in developing preventive measures, and established iConVis, an interactive visual analysis tool that facilitates the closed-loop analysis process. A novel financial metric, the contagion effect, was formulated to quantify the infectious consequences of guarantee chains in this type of network. Based on this metric, we designed and implement a series of novel and coordinated views that address the analysis of financial problems. Experts evaluated the system using real-world financial data. The proposed approach grants practitioners the ability to avoid previous ad hoc analysis methodologies and extend coverage of the conventional Capital Accord to the banking industry.

q-fin.RM↗

arXiv2020

regvis.net -- A Visual Bibliography of Regulatory Visualization

Information visualization and visual analytics technology has attracted significant attention from the financial regulation community. In this research, we present regvis.net, a visual survey of regulatory visualization that allows researchers from both the computing and financial communities to review their literature of interest. We have collected and manually tagged more than 80 regulation visualization related publications. To the best of our knowledge, this is the first publication set tailored for regulatory visualization. We have provided a webpage (http://regvis.net) for interactive searches and filtering. Each publication is represented by a thumbnail of the representative system interface or key visualization chart, and users can conduct multi-condition screening explorations and fixed text searches.

cs.CY↗

arXiv2020

$E^3$: Visual Exploration of Spatiotemporal Energy Demand

Understanding demand-side energy behaviour is critical for making efficiency responses for energy demand management. We worked closely with energy experts and identified the key elements of the energy demand problem including temporal and spatial demand and shifts in spatiotemporal demand. To our knowledge, no previous research has investigated the shifts in spatiotemporal demand. To fill this research gap, we propose a unified visual analytics approach to support exploratory demand analysis; we developed E3, a highly interactive tool that support users in making and verifying hypotheses through human-client-server interactions. A novel potential flow based approach was formalized to model shifts in energy demand and integrated into a server-side engine. Experts then evaluated and affirmed the usefulness of this approach through case studies of real-world electricity data. In the future, we will improve the modelling algorithm, enhance visualisation, and expand the process to support more forms of energy data.

cs.HC↗

arXiv2020

Visual analytics for networked-guarantee loans risk management

Groups of enterprises guarantee each other and form complex guarantee networks when they try to obtain loans from banks. Such secured loan can enhance the solvency and promote the rapid growth in the economic upturn period. However, potential systemic risk may happen within the risk binding community. Especially, during the economic down period, the crisis may spread in the guarantee network like a domino. Monitoring the financial status, preventing or reducing systematic risk when crisis happens is highly concerned by the regulatory commission and banks. We propose visual analytics approach for loan guarantee network risk management, and consolidate the five analysis tasks with financial experts: i) visual analytics for enterprises default risk, whereby a hybrid representation is devised to predict the default risk and developed an interface to visualize key indicators; ii) visual analytics for high default groups, whereby a community detection based interactive approach is presented; iii) visual analytics for high defaults pattern, whereby a motif detection based interactive approach is described, and we adopt a Shneiderman Mantra strategy to reduce the computation complexity. iv) visual analytics for evolving guarantee network, whereby animation is used to help understanding the guarantee dynamic; v) visual analytics approach and interface for default diffusion path. The temporal diffusion path analysis can be useful for the government and bank to monitor the default spread status. It also provides insight for taking precautionary measures to prevent and dissolve systemic financial risk. We implement the system with case studies on a real-world guarantee network. Two financial experts are consulted with endorsement on the developed tool. To the best of our knowledge, this is the first visual analytics tool to explore the guarantee network risks in a systematic manner.

cs.SI↗

arXiv2019

Dunhuang Grottoes Painting Dataset and Benchmark

This document introduces the background and the usage of the Dunhuang Grottoes Dataset and the benchmark. The documentation first starts with the background of the Dunhuang Grotto, which is widely recognised as an priceless heritage. Given that digital method is the modern trend for heritage protection and restoration. Follow the trend, we release the first public dataset for Dunhuang Grotto Painting restoration. The rest of the documentation details the painting data generation. To enable a data driven fashion, this dataset provided a large number of training and testing example which is sufficient for a deep learning approach. The detailed usage of the dataset as well as the benchmark is described.

cs.CV↗

arXiv2019

OVSNet : Towards One-Pass Real-Time Video Object Segmentation

Video object segmentation aims at accurately segmenting the target object regions across consecutive frames. It is technically challenging for coping with complicated factors (e.g., shape deformations, occlusion and out of the lens). Recent approaches have largely solved them by using backforth re-identification and bi-directional mask propagation. However, their methods are extremely slow and only support offline inference, which in principle cannot be applied in real time. Motivated by this observation, we propose a efficient detection-based paradigm for video object segmentation. We propose an unified One-Pass Video Segmentation framework (OVS-Net) for modeling spatial-temporal representation in a unified pipeline, which seamlessly integrates object detection, object segmentation, and object re-identification. The proposed framework lends itself to one-pass inference that effectively and efficiently performs video object segmentation. Moreover, we propose a maskguided attention module for modeling the multi-scale object boundary and multi-level feature fusion. Experiments on the challenging DAVIS 2017 demonstrate the effectiveness of the proposed framework with comparable performance to the state-of-the-art, and the great efficiency about 11.5 FPS towards pioneering real-time work to our knowledge, more than 5 times faster than other state-of-the-art methods.

cs.CV↗

arXiv2019

Kindling the Darkness: A Practical Low-light Image Enhancer

Images captured under low-light conditions often suffer from (partially) poor visibility. Besides unsatisfactory lightings, multiple types of degradations, such as noise and color distortion due to the limited quality of cameras, hide in the dark. In other words, solely turning up the brightness of dark regions will inevitably amplify hidden artifacts. This work builds a simple yet effective network for \textbf{Kin}dling the \textbf{D}arkness (denoted as KinD), which, inspired by Retinex theory, decomposes images into two components. One component (illumination) is responsible for light adjustment, while the other (reflectance) for degradation removal. In such a way, the original space is decoupled into two smaller subspaces, expecting to be better regularized/learned. It is worth to note that our network is trained with paired images shot under different exposure conditions, instead of using any ground-truth reflectance and illumination information. Extensive experiments are conducted to demonstrate the efficacy of our design and its superiority over state-of-the-art alternatives. Our KinD is robust against severe visual defects, and user-friendly to arbitrarily adjust light levels. In addition, our model spends less than 50ms to process an image in VGA resolution on a 2080Ti GPU. All the above merits make our KinD attractive for practical use.

cs.CV↗