arXiv subjects

Jun Jia

Publications and source records attributed to Jun Jia.

54 records · Page 3Linked to original sources

arXiv2024

AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted code and models. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for efficient video quality assessment of user-generated content.

cs.CV↗

arXiv2024

NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results

This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i.e., Kuaishou/Kwai Platform. The KVQ database is divided into three parts, including 2926 videos for training, 420 videos for validation, and 854 videos for testing. The purpose is to build new benchmarks and advance the development of S-UGC VQA. The competition had 200 participants and 13 teams submitted valid solutions for the final testing phase. The proposed solutions achieved state-of-the-art performances for S-UGC VQA. The project can be found at https://github.com/lixinustc/KVQChallenge-CVPR-NTIRE2024.

eess.IV↗

arXiv2024

Text2QR: Harmonizing Aesthetic Customization and Scanning Robustness for Text-Guided QR Code Generation

In the digital era, QR codes serve as a linchpin connecting virtual and physical realms. Their pervasive integration across various applications highlights the demand for aesthetically pleasing codes without compromised scannability. However, prevailing methods grapple with the intrinsic challenge of balancing customization and scannability. Notably, stable-diffusion models have ushered in an epoch of high-quality, customizable content generation. This paper introduces Text2QR, a pioneering approach leveraging these advancements to address a fundamental challenge: concurrently achieving user-defined aesthetics and scanning robustness. To ensure stable generation of aesthetic QR codes, we introduce the QR Aesthetic Blueprint (QAB) module, generating a blueprint image exerting control over the entire generation process. Subsequently, the Scannability Enhancing Latent Refinement (SELR) process refines the output iteratively in the latent space, enhancing scanning robustness. This approach harnesses the potent generation capabilities of stable-diffusion models, navigating the trade-off between image aesthetics and QR code scannability. Our experiments demonstrate the seamless fusion of visual appeal with the practical utility of aesthetic QR codes, markedly outperforming prior methods. Codes are available at \url{https://github.com/mulns/Text2QR}

cs.CV↗

arXiv2024

Exploring the Naturalness of AI-Generated Images

The proliferation of Artificial Intelligence-Generated Images (AGIs) has greatly expanded the Image Naturalness Assessment (INA) problem. Different from early definitions that mainly focus on tone-mapped images with limited distortions (e.g., exposure, contrast, and color reproduction), INA on AI-generated images is especially challenging as it has more diverse contents and could be affected by factors from multiple perspectives, including low-level technical distortions and high-level rationality distortions. In this paper, we take the first step to benchmark and assess the visual naturalness of AI-generated images. First, we construct the AI-Generated Image Naturalness (AGIN) database by conducting a large-scale subjective study to collect human opinions on the overall naturalness as well as perceptions from technical and rationality perspectives. AGIN verifies that naturalness is universally and disparately affected by technical and rationality distortions. Second, we propose the Joint Objective Image Naturalness evaluaTor (JOINT), to automatically predict the naturalness of AGIs that aligns human ratings. Specifically, JOINT imitates human reasoning in naturalness evaluation by jointly learning both technical and rationality features. We demonstrate that JOINT significantly outperforms baselines for providing more subjectively consistent results on naturalness assessment.

cs.CV↗

arXiv2023

Perceptual Quality Assessment for Video Frame Interpolation

The quality of frames is significant for both research and application of video frame interpolation (VFI). In recent VFI studies, the methods of full-reference image quality assessment have generally been used to evaluate the quality of VFI frames. However, high frame rate reference videos, necessities for the full-reference methods, are difficult to obtain in most applications of VFI. To evaluate the quality of VFI frames without reference videos, a no-reference perceptual quality assessment method is proposed in this paper. This method is more compatible with VFI application and the evaluation scores from it are consistent with human subjective opinions. A new quality assessment dataset for VFI was constructed through subjective experiments firstly, to assess the opinion scores of interpolated frames. The dataset was created from triplets of frames extracted from high-quality videos using 9 state-of-the-art VFI algorithms. The proposed method evaluates the perceptual coherence of frames incorporating the original pair of VFI inputs. Specifically, the method applies a triplet network architecture, including three parallel feature pipelines, to extract the deep perceptual features of the interpolated frame as well as the original pair of frames. Coherence similarities of the two-way parallel features are jointly calculated and optimized as a perceptual metric. In the experiments, both full-reference and no-reference quality assessment methods were tested on the new quality dataset. The results show that the proposed method achieves the best performance among all compared quality assessment methods on the dataset.

eess.IV↗

arXiv2023

BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality Assessment

Banding, also known as staircase-like contours, frequently occurs in flat areas of images/videos processed by the compression or quantization algorithms. As undesirable artifacts, banding destroys the original image structure, thus degrading users' quality of experience (QoE). In this paper, we systematically investigate the banding image quality assessment (IQA) problem, aiming to detect the image banding artifacts and evaluate their perceptual visual quality. Considering that the existing image banding databases only contain limited content sources and banding generation methods, and lack perceptual quality labels (i.e. mean opinion scores), we first build the largest banding IQA database so far, named Banding Artifact Noticeable Database (BAND-2k), which consists of 2,000 banding images generated by 15 compression and quantization schemes. A total of 23 workers participated in the subjective IQA experiment, yielding over 214,000 patch-level banding class labels and 44,371 reliable image-level quality ratings. Subsequently, we develop an effective no-reference (NR) banding evaluator for banding detection and quality assessment by leveraging frequency characteristics of banding artifacts. A dual convolutional neural network is employed to concurrently learn the feature representation from the high-frequency and low-frequency maps, thereby enhancing the ability to discern banding artifacts. The quality score of a banding image is generated by pooling the banding detection maps masked by the spatial frequency filters. Experiments demonstrate that our banding evaluator achieves a remarkably high accuracy in banding detection and also exhibits high SRCC and PLCC results with the perceptual quality labels. These findings unveil the strong correlations between the intensity of banding artifacts and the perceptual visual quality, thus validating the necessity of banding quality assessment.

cs.CV↗

arXiv2023

Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images

Computer graphics images (CGIs) are artificially generated by means of computer programs and are widely perceived under various scenarios, such as games, streaming media, etc. In practice, the quality of CGIs consistently suffers from poor rendering during production, inevitable compression artifacts during the transmission of multimedia applications, and low aesthetic quality resulting from poor composition and design. However, few works have been dedicated to dealing with the challenge of computer graphics image quality assessment (CGIQA). Most image quality assessment (IQA) metrics are developed for natural scene images (NSIs) and validated on databases consisting of NSIs with synthetic distortions, which are not suitable for in-the-wild CGIs. To bridge the gap between evaluating the quality of NSIs and CGIs, we construct a large-scale in-the-wild CGIQA database consisting of 6,000 CGIs (CGIQA-6k) and carry out the subjective experiment in a well-controlled laboratory environment to obtain the accurate perceptual ratings of the CGIs. Then, we propose an effective deep learning-based no-reference (NR) IQA model by utilizing both distortion and aesthetic quality representation. Experimental results show that the proposed method outperforms all other state-of-the-art NR IQA methods on the constructed CGIQA-6k database and other CGIQA-related databases. The database is released at https://github.com/zzc-1998/CGIQA6K.

cs.CV↗

arXiv2023

StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability

Video shakiness is an unpleasant distortion of User Generated Content (UGC) videos, which is usually caused by the unstable hold of cameras. In recent years, many video stabilization algorithms have been proposed, yet no specific and accurate metric enables comprehensively evaluating the stability of videos. Indeed, most existing quality assessment models evaluate video quality as a whole without specifically taking the subjective experience of video stability into consideration. Therefore, these models cannot measure the video stability explicitly and precisely when severe shakes are present. In addition, there is no large-scale video database in public that includes various degrees of shaky videos with the corresponding subjective scores available, which hinders the development of Video Quality Assessment for Stability (VQA-S). To this end, we build a new database named StableDB that contains 1,952 diversely-shaky UGC videos, where each video has a Mean Opinion Score (MOS) on the degree of video stability rated by 34 subjects. Moreover, we elaborately design a novel VQA-S model named StableVQA, which consists of three feature extractors to acquire the optical flow, semantic, and blur features respectively, and a regression layer to predict the final stability score. Extensive experiments demonstrate that the StableVQA achieves a higher correlation with subjective opinions than the existing VQA-S models and generic VQA models. The database and codes are available at https://github.com/QMME/StableVQA.

cs.CV↗

arXiv2023

Acoustic frequency atomic spin oscillator in the quantum regime

We experimentally demonstrate quantum behavior of a macroscopic atomic spin oscillator in the acoustic frequency range. Quantum back-action of the spin measurement, ponderomotive squeezing of light, and oscillator spring softening are observed at spin oscillation frequencies down to 6 kHz. Quantum noise sources characteristic of spin oscillators operating in the near-DC frequency range are identified and means for their mitigation are presented. These results constitute an important step towards quantum noise reduction and entanglement-enhanced sensing in the acoustic range using a negative-mass reference frame. In particular, the results are relevant for broadband noise reduction in gravitational wave detectors.

quant-ph↗

arXiv2023

RAWIW: RAW Image Watermarking Robust to ISP Pipeline

Invisible image watermarking is essential for image copyright protection. Compared to RGB images, RAW format images use a higher dynamic range to capture the radiometric characteristics of the camera sensor, providing greater flexibility in post-processing and retouching. Similar to the master recording in the music industry, RAW images are considered the original format for distribution and image production, thus requiring copyright protection. Existing watermarking methods typically target RGB images, leaving a gap for RAW images. To address this issue, we propose the first deep learning-based RAW Image Watermarking (RAWIW) framework for copyright protection. Unlike RGB image watermarking, our method achieves cross-domain copyright protection. We directly embed copyright information into RAW images, which can be later extracted from the corresponding RGB images generated by different post-processing methods. To achieve end-to-end training of the framework, we integrate a neural network that simulates the ISP pipeline to handle the RAW-to-RGB conversion process. To further validate the generalization of our framework to traditional ISP pipelines and its robustness to transmission distortion, we adopt a distortion network. This network simulates various types of noises introduced during the traditional ISP pipeline and transmission. Furthermore, we employ a three-stage training strategy to strike a balance between robustness and concealment of watermarking. Our extensive experiments demonstrate that RAWIW successfully achieves cross-domain copyright protection for RAW images while maintaining their visual quality and robustness to ISP pipeline distortions.

eess.IV↗

arXiv2022

SU(2)-in-SU(1,1) Nested Interferometer for Highly Sensitive, Loss-Tolerant Quantum Metrology

We present experimental and theoretical results on a new interferometer topology that nests a SU(2) interferometer, e.g., a Mach-Zehnder or Michelson interferometer, inside a SU(1,1) interferometer, i.e., a Mach-Zehnder interferometer with parametric amplifiers in place of beam splitters. This SU(2)-in-SU(1,1) nested interferometer (SISNI) simultaneously achieves high signal-to-noise ratio (SNR), sensitivity beyond the standard quantum limit (SQL) and tolerance to photon losses external to the interferometer, e.g., in detectors. We implement a SISNI using parametric amplification by four-wave mixing (FWM) in Rb vapor and a laser-fed Mach-Zehnder SU(2) interferometer. We observe path-length sensitivity with SNR 2.2 dB beyond the SQL at power levels (and thus SNR) 2 orders of magnitude beyond those of previous loss-tolerant interferometers. We find experimentally the optimal FWM gains and find agreement with a minimal quantum noise model for the FWM process. The results suggest ways to boost the in-practice sensitivity of high-power interferometers, e.g., gravitational wave interferometers, and may enable high-sensitivity, quantum-enhanced interferometry at wavelengths for which efficient detectors are not available.

quant-ph↗

arXiv2021

Deep Natural Language Processing for LinkedIn Search

Many search systems work with large amounts of natural language data, e.g., search queries, user profiles, and documents. Building a successful search system requires a thorough understanding of textual data semantics, where deep learning based natural language processing techniques (deep NLP) can be of great help. In this paper, we introduce a comprehensive study for applying deep NLP techniques to five representative tasks in search systems: query intent prediction (classification), query tagging (sequential tagging), document ranking (ranking), query auto completion (language modeling), and query suggestion (sequence to sequence). We also introduce BERT pre-training as a sixth task that can be applied to many of the other tasks. Through the model design and experiments of the six tasks, readers can find answers to four important questions: (1). When is deep NLP helpful/not helpful in search systems? (2). How to address latency challenges? (3). How to ensure model robustness? This work builds on existing efforts of LinkedIn search, and is tested at scale on LinkedIn's commercial search engines. We believe our experiences can provide useful insights for the industry and research communities.

cs.IR↗

arXiv2021

Deep Natural Language Processing for LinkedIn Search Systems

Many search systems work with large amounts of natural language data, e.g., search queries, user profiles and documents, where deep learning based natural language processing techniques (deep NLP) can be of great help. In this paper, we introduce a comprehensive study of applying deep NLP techniques to five representative tasks in search engines. Through the model design and experiments of the five tasks, readers can find answers to three important questions: (1) When is deep NLP helpful/not helpful in search systems? (2) How to address latency challenges? (3) How to ensure model robustness? This work builds on existing efforts of LinkedIn search, and is tested at scale on a commercial search engine. We believe our experiences can provide useful insights for the industry and research communities.

cs.CL↗

arXiv2019

Robust Invisible Hyperlinks in Physical Photographs Based on 3D Rendering Attacks

In the era of multimedia and Internet, people are eager to obtain information from offline to online. Quick Response (QR) codes and digital watermarks help us access information quickly. However, QR codes look ugly and invisible watermarks can be easily broken in physical photographs. Therefore, this paper proposes a novel method to embed hyperlinks into natural images, making the hyperlinks invisible for human eyes but detectable for mobile devices. Our method is an end-to-end neural network with an encoder to hide information and a decoder to recover information. From original images to physical photographs, camera imaging process will introduce a series of distortion such as noise, blur, and light. To train a robust decoder against the physical distortion from the real world, a distortion network based on 3D rendering is inserted between the encoder and the decoder to simulate the camera imaging process. Besides, in order to maintain the visual attraction of the image with hyperlinks, we propose a loss function based on just noticeable difference (JND) to supervise the training of encoder. Experimental results show that our approach outperforms the previous method in both simulated and real situations.

eess.IV↗

arXiv2019

Floquet control of global $PT$ symmetry in 2D arrays of quadrimer waveguides

Manipulating the global $PT$ symmetry of a non-Hermitian composite system is a rather significative and challenging task. Here, we investigate Floquet control of global $PT$ symmetry in 2D arrays of quadrimer waveguides with transverse periodic structure along $x$-axis and longitudinal periodic modulation along $z$-axis. For unmodulated case with inhomogeneous inter- and intra- quadrimer coupling strength $κ_1\neqκ$, in addition to conventional global $PT$-symmetric phase and $PT$-symmetry-breaking phase, we find that there is exotic phase where global $PT$ symmetry is broken under open boundary condition, whereas it still is unbroken under periodical boundary condition. The boundary of phase is analytically given as $κ_1\geqκ+\sqrt{2}$ and $1\leqγ\leq2$, where there exists a pair of zero-energy edge states with purely imaginary energy eigenvalues localized at the left boundary, whereas other $4N-2$ eigenvalues are real. Especially, the domain of the exotic phase can be manipulated narrow and even disappeared by tuning modulation parameter. More interestingly, whether or not the array has initial global $PT$ symmetry, periodic modulation not only can restore the broken global $PT$ symmetry, but also can control it by tuning modulation amplitude. Therefore, the global property of transverse periodic structure of such a 2D array can be manipulated by only tuning modulation amplitude of longitudinal periodic modulation.

physics.optics↗

arXiv2019

A novel two-mode squeezed light based on double-pump phase-matching

A novel two-mode non-degenerate squeezed light is generated based on a four-wave mixing (4WM) process driven by two pump fields crossing at a small angle. By exchanging the roles of the pump beams and the probe and conjugate beams, we have demonstrated the frequency-degenerate two-mode squeezed light with separated spatial patterns. Different from a 4WM process driven by one pump field, the refractive index of the corresponding probe field $n_{p}$ can be converted to a value that is greater than $1$ or less than $1$ by an angle adjustment. In the new region with $n_{p}<1$, the bandwidth of the gain is relatively large due to the slow change in the refractive index with the two-photon detuning. As the bandwidth is important for the practical application of a quantum memory, the wide-bandwidth intensity-squeezed light fields provide new prospects for quantum memories.

quant-ph↗

arXiv2017

Absolute sensitivity of phase measurement in an SU(1,1) type interferometer

Absolute sensitivity is measured for the phase measurement in an SU(1,1) type interferometer and the results are compared to that of a Mach-Zehnder interferometer operated under the condition of the same intra-interferometer intensity. The interferometer is phase locked to a point with the largest quantum noise cancellation, and a simulated phase modulation is added in one arm of SU(1,1) interferometer. Both the signal and noise level are estimated at the same frequency range, and we obtain 3dB improvement in sensitivity for the SU(1,1) interferometer over the Mach-Zehnder interferometer. Our results demonstrate a direct phase estimation, and may pave the way for practical applications of nonlinear interferometer.

quant-ph↗

arXiv2015

MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation

MADNESS (multiresolution adaptive numerical environment for scientific simulation) is a high-level software environment for solving integral and differential equations in many dimensions that uses adaptive and fast harmonic analysis methods with guaranteed precision based on multiresolution analysis and separated representations. Underpinning the numerical capabilities is a powerful petascale parallel programming environment that aims to increase both programmer productivity and code scalability. This paper describes the features and capabilities of MADNESS and briefly discusses some current applications in chemistry and several areas of physics.

cs.MS↗