arXiv subjects

Hanyu Wang

Publications and source records attributed to Hanyu Wang.

72 records · Page 4Linked to original sources

arXiv2023

Multimodality-guided Image Style Transfer using Cross-modal GAN Inversion

Image Style Transfer (IST) is an interdisciplinary topic of computer vision and art that continuously attracts researchers' interests. Different from traditional Image-guided Image Style Transfer (IIST) methods that require a style reference image as input to define the desired style, recent works start to tackle the problem in a text-guided manner, i.e., Text-guided Image Style Transfer (TIST). Compared to IIST, such approaches provide more flexibility with text-specified styles, which are useful in scenarios where the style is hard to define with reference images. Unfortunately, many TIST approaches produce undesirable artifacts in the transferred images. To address this issue, we present a novel method to achieve much improved style transfer based on text guidance. Meanwhile, to offer more flexibility than IIST and TIST, our method allows style inputs from multiple sources and modalities, enabling MultiModality-guided Image Style Transfer (MMIST). Specifically, we realize MMIST with a novel cross-modal GAN inversion method, which generates style representations consistent with specified styles. Such style representations facilitate style transfer and in principle generalize any IIST methods to MMIST. Large-scale experiments and user studies demonstrate that our method achieves state-of-the-art performance on TIST task. Furthermore, comprehensive qualitative results confirm the effectiveness of our method on MMIST task and cross-modal style interpolation.

cs.CV↗

arXiv2023

AnySyn: A Cost-Generic Logic Synthesis Framework with Customizable Cost Functions

Modern technology-independent logic synthesis has been developed to optimize for the size and depth of AND-Inverter Graphs (AIGs) as a proxy of CMOS circuit area and delay. However, for non-CMOS-based emerging technologies, AIG size and depth may not be good cost estimations. Dedicated algorithms optimizing for more complex cost functions have been proven effective for their specific target applications yet require time and experts in both logic synthesis and the targeted technology to develop. In this work, we propose AnySyn, a cost-generic optimization framework for agile experimentation and prototyping of various customized cost functions before investing in developing specialized algorithms. Experimental results show that AnySyn outperforms non-specialized size and depth optimization algorithms by 14% and 19% on average and achieves comparable results to specialized algorithms within acceptable CPU time.

cs.AR↗

arXiv2023

PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning

Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the long-term execution. An alternative method is to train a DRL policy that outputs the navigation path directly. However, two roadblocks arise for training a DRL policy that outputs paths: (1) The action space for potential paths often involves higher dimensions comparing to low-level commands, which increases the difficulties of training; (2) It takes multiple time steps to track a path instead of a single time step, which requires the path to predicate the interactions of the robot w.r.t. the dynamic environment in multiple time steps. This, in turn, amplifies the challenges associated with training. In response to these challenges, we propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. Specifically, we employ specific action space discretization techniques and tailored state space representation methods to address the associated challenges. In our experiments, PathRL achieves better success rates and reduces angular rotation variability compared to other DRL navigation methods, facilitating stable and smooth robot movement. We demonstrate the competitive edge of PathRL in both real-world scenarios and multiple challenging simulation environments.

cs.RO↗

arXiv2023

Chop & Learn: Recognizing and Generating Object-State Compositions

Recognizing and generating object-state compositions has been a challenging task, especially when generalizing to unseen compositions. In this paper, we study the task of cutting objects in different styles and the resulting object state changes. We propose a new benchmark suite Chop & Learn, to accommodate the needs of learning objects and different cut styles using multiple viewpoints. We also propose a new task of Compositional Image Generation, which can transfer learned cut styles to different objects, by generating novel object-state images. Moreover, we also use the videos for Compositional Action Recognition, and show valuable uses of this dataset for multiple video tasks. Project website: https://chopnlearn.github.io.

cs.CV↗

arXiv2023

Unveiling Real Triple Degeneracies in Crystals: Exploring Link and Compound Structures

With their non-Abelian topological charges, real multi-bandgap systems challenge the conventional topological phase classifications. As the minimal sector of multi-bandgap systems, real triple degeneracies (RTPs), which serve as real 'Weyl points', lay the foundation for the research on real topological phases. However, experimental demonstration of physical systems with global band configurations consisting of multiple RTPs in crystals has not been reported. Here we present experimental evidence of RTPs in photonic meta-crystals, characterizing them using the Euler number, and establishing their connection with both Abelian and non-Abelian charges. By considering RTPs as the basic elements, we further propose the concept of a topological compound, akin to a chemical compound, where we find that certain phases are not topologically allowed. The topological classification of RTPs in crystals demonstrated in our work plays a similar role as the 'no-go' theorem in Weyl systems.

cond-mat.mes-hall↗

arXiv2023

Towards Scalable Neural Representation for Diverse Videos

Implicit neural representations (INR) have gained increasing attention in representing 3D scenes and images, and have been recently applied to encode videos (e.g., NeRV, E-NeRV). While achieving promising results, existing INR-based methods are limited to encoding a handful of short videos (e.g., seven 5-second videos in the UVG dataset) with redundant visual content, leading to a model design that fits individual video frames independently and is not efficiently scalable to a large number of diverse videos. This paper focuses on developing neural representations for a more practical setup -- encoding long and/or a large number of videos with diverse visual content. We first show that instead of dividing videos into small subsets and encoding them with separate models, encoding long and diverse videos jointly with a unified model achieves better compression results. Based on this observation, we propose D-NeRV, a novel neural representation framework designed to encode diverse videos by (i) decoupling clip-specific visual content from motion information, (ii) introducing temporal reasoning into the implicit neural network, and (iii) employing the task-oriented flow as intermediate output to reduce spatial redundancies. Our new model largely surpasses NeRV and traditional video compression techniques on UCF101 and UVG datasets on the video compression task. Moreover, when used as an efficient data-loader, D-NeRV achieves 3%-10% higher accuracy than NeRV on action recognition tasks on the UCF101 dataset under the same compression ratios.

cs.CV↗

arXiv2022

NIRVANA: Neural Implicit Representations of Videos with Adaptive Networks and Autoregressive Patch-wise Modeling

Implicit Neural Representations (INR) have recently shown to be powerful tool for high-quality video compression. However, existing works are limiting as they do not explicitly exploit the temporal redundancy in videos, leading to a long encoding time. Additionally, these methods have fixed architectures which do not scale to longer videos or higher resolutions. To address these issues, we propose NIRVANA, which treats videos as groups of frames and fits separate networks to each group performing patch-wise prediction. This design shares computation within each group, in the spatial and temporal dimensions, resulting in reduced encoding time of the video. The video representation is modeled autoregressively, with networks fit on a current group initialized using weights from the previous group's model. To further enhance efficiency, we perform quantization of the network parameters during training, requiring no post-hoc pruning or quantization. When compared with previous works on the benchmark UVG dataset, NIRVANA improves encoding quality from 37.36 to 37.70 (in terms of PSNR) and the encoding speed by 12X, while maintaining the same compression rate. In contrast to prior video INR works which struggle with larger resolution and longer videos, we show that our algorithm is highly flexible and scales naturally due to its patch-wise and autoregressive designs. Moreover, our method achieves variable bitrate compression by adapting to videos with varying inter-frame motion. NIRVANA achieves 6X decoding speed and scales well with more GPUs, making it practical for various deployment scenarios.

cs.CV↗

arXiv2022

Deep learning forward and reverse primer design to detect SARS-CoV-2 emerging variants

Surges that have been observed at different periods in the number of COVID-19 cases are associated with the emergence of multiple SARS-CoV-2 (Severe Acute Respiratory Virus) variants. The design of methods to support laboratory detection are crucial in the monitoring of these variants. Hence, in this paper, we develop a semi-automated method to design both forward and reverse primer sets to detect SARS-CoV-2 variants. To proceed, we train deep Convolution Neural Networks (CNNs) to classify labelled SARS-CoV-2 variants and identify partial genomic features needed for the forward and reverse Polymerase Chain Reaction (PCR) primer design. Our proposed approach supplements existing ones while promoting the emerging concept of neural network assisted primer design for PCR. Our CNN model was trained using a database of SARS-CoV-2 full-length genomes from GISAID and tested on a separate dataset from NCBI, with 98\% accuracy for the classification of variants. This result is based on the development of three different methods of feature extraction, and the selected primer sequences for each SARS-CoV-2 variant detection (except Omicron) were present in more than 95 \% of sequences in an independent set of 5000 same variant sequences, and below 5 \% in other independent datasets with 5000 sequences of each variant. In total, we obtain 22 forward and reverse primer pairs with flexible length sizes (18-25 base pairs) with an expected amplicon length ranging between 42 and 3322 nucleotides. Besides the feature appearance, in-silico primer checks confirmed that the identified primer pairs are suitable for accurate SARS-CoV-2 variant detection by means of PCR tests.

q-bio.GN↗

arXiv2022

Neural Space-filling Curves

We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression. Code and additional results will be made available at https://hywang66.github.io/publication/neuralsfc.

cs.CV↗

arXiv2021

NeRV: Neural Representations for Videos

We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by 25x to 70x, the decoding speed by 38x to 132x, while achieving better video quality. With such a representation, we can treat videos as neural networks, simplifying several video-related tasks. For example, conventional video compression methods are restricted by a long and complex pipeline, specifically designed for the task. In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H.264, HEVC \etc). Besides compression, we demonstrate the generalization of NeRV for video denoising. The source code and pre-trained model can be found at https://github.com/haochen-rye/NeRV.git.

cs.CV↗

arXiv2021

Reversible modulation of metal-insulator transition in VO2 via chemically-induced oxygen migration

Metal-insulator transitions (MIT),an intriguing correlated phenomenon induced by the subtle competition of the electrons' repulsive Coulomb interaction and kinetic energy, is of great potential use for electronic applications due to the dramatic change in resistivity. Here, we demonstrate a reversible control of MIT in VO2 films via oxygen stoichiometry engineering. By facilely depositing and dissolving a water-soluble yet oxygen-active Sr3Al2O6 capping layer atop the VO2 at room temperature, oxygen ions can reversibly migrate between VO2 and Sr3Al2O6, resulting in a gradual suppression and a complete recovery of MIT in VO2. The migration of the oxygen ions is evidenced in a combination of transport measurement, structural characterization and first-principles calculations. This approach of chemically-induced oxygen migration using a water-dissolvable adjacent layer could be useful for advanced electronic and iontronic devices and studying oxygen stoichiometry effects on the MIT.

cond-mat.mtrl-sci↗

arXiv2021

Motion-Aware Robotic 3D Ultrasound

Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object's motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a step-wise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning.

cs.RO↗

arXiv2021

Enhanced metal-insulator transition in freestanding VO2 down to 5 nm thickness

Ultrathin freestanding membranes with a pronounced metal-insulator transition (MIT) provides huge potential in future flexible electronic applications as well as a unique aspect of the study of lattice-electron interplay. However, the reduction of the thickness to an ultrathin region (a few nm) is typically detrimental to the MIT in epitaxial films, and even catastrophic for their freestanding form. Here, we report an enhanced MIT in VO2-based freestanding membranes, with a lateral size up to millimetres and VO2 thickness down to 5 nm. The VO2-membranes were detached by dissolving a Sr3Al2O6 sacrificial layer between the VO2 thin film and c-Al2O3(0001) substrate, allowing a transfer onto arbitrary surfaces. Furthermore, the MIT in the VO2-membrane was greatly enhanced by inserting an intermediate Al2O3 buffer layer. In comparison to the best available ultrathin VO2-membranes, the enhancement of MIT is over 400% at 5 nm VO2 thickness and more than one order of magnitude for VO2 above 10 nm. Our study widens the spectrum of functionality in ultrathin and large-scale membranes, and enables the potential integration of MIT into flexible electronics and photonics.

cond-mat.mtrl-sci↗

arXiv2021

Gate-tunable chiral phonons in low-buckled group-IVA monolayers

We investigate the electric response of chiral phonons on the low-buckled group-IVA monolayers by performing first-principles calculations. The vertical electric field breaks the degeneracy of phonon modes at high-symmetry $\pm K$ points of the phonon Brillouin zone, and the size of the phononic gap is proportional to the strength of the electric field. The gapped phonon modes at $\pm K$ possess chiralities with considerable phonon circular polarizations and discrete phonon pseudoangular momenta. The chiralities of phonons are robust against the variation of the field strength, but reversed by changing the field direction. Electric control of chiral phonons adds a new dimension to the study of chiral phonons, which has potential use in the design of phononic and valley devices.

cond-mat.mtrl-sci↗

arXiv2020

Logic Synthesis Meets Machine Learning: Trading Exactness for Generalization

Logic synthesis is a fundamental step in hardware design whose goal is to find structural representations of Boolean functions while minimizing delay and area. If the function is completely-specified, the implementation accurately represents the function. If the function is incompletely-specified, the implementation has to be true only on the care set. While most of the algorithms in logic synthesis rely on SAT and Boolean methods to exactly implement the care set, we investigate learning in logic synthesis, attempting to trade exactness for generalization. This work is directly related to machine learning where the care set is the training set and the implementation is expected to generalize on a validation set. We present learning incompletely-specified functions based on the results of a competition conducted at IWLS 2020. The goal of the competition was to implement 100 functions given by a set of care minterms for training, while testing the implementation using a set of validation minterms sampled from the same function. We make this benchmark suite available and offer a detailed comparative analysis of the different approaches to learning

cs.LG↗

arXiv2019

Highly degenerate photonic flat bands arising from complete graph configurations

Inspired by complete graph theory, we demonstrate that a metallic claw "meta-atom" structure can carry a high number of nearly degenerate resonant modes. A photonic meta-crystal composing of a lattice of such meta-atoms exhibits a large number of flat bands that are squeezed into a narrow frequency window, and these flat bands can be designed to locate in a wide complete 3D bandgap. The degeneracy dimension (Nf) of the flat bands is determined by the number of branches (Nb) of the metallic claw with Nf=Nb-3, which is geometrically related to the complete graph theory. Different from those flat bands emerging from special lattice arrangements (e.g., Kagome lattice), the isolated flat bands here are insensitive to lattice perturbations. The proposed mechanism offers a new platform for realizing various dispersion-less phenomena and a new paradigm to realize high density of states and spectra compressing.

physics.class-ph↗

arXiv2015

Study of Memristor-based Oscillatory Neural Networks using PPV modeling

Memristor-based oscillator is becoming promising thanks to its inherent NDR (Negative Differential Region) property and compact circuit structure. This paves the way to the large scale oscillatory neural network (ONN) and the realization of pattern recognition based on its global synchronization. However, the simulation of large scale ONN encounters the problem of long simulation time because of the large number of oscillators. Here we propose a highly efficient method to abstract the phase sensitivity characteristic of the memristor-based oscillator, i.e., its PPV (Perturbation Projection Vector), which allows reducing considerably the complexity of ONN simulation, and speeding up the simulation more than 2000 times. Our study also reveals the impact of the circuit parameters on the pattern recognition accuracy and the robustness against the frequency mismatch.

cs.ET↗

arXiv2015

PPV modelling of memristor-based oscillator

In this letter, we propose for the first time a method of abstracting the PPV (Perturbation Projection Vector) characteristic of the up-to-date memristor-based oscillators. Inspired from biological oscillators and its characteristic named PRC (Phase Response Curve), we build a bridge between PRC and PPV. This relationship is verified rigorously using the transistor level simulation of Colpitts and ring oscillators, i.e., comparing the PPV converted from PRC and the PPV obtained from accurate PSS+PXF simulation. Then we apply this method to the PPV calculation of the memristor-based oscillator. By keeping the phase dynamics of the oscillator and dropping the details of voltage/current amplitude, the PPV modelling is highly efficient to describe the phase dynamics due to the oscillator coupling, and will be very suitable for the fast simulation of large scale oscillatory neural networks.

cs.ET↗