arXiv subjects

Zheng Yan

Publications and source records attributed to Zheng Yan.

At least 163 records · Page 9Linked to original sources

arXiv2020

A Reinforcement Learning Method For Power Suppliers' Strategic Bidding with Insufficient Information

Power suppliers can exercise market power to gain higher profit. However, this becomes difficult when external information is extremely rare. To get a promising performance in an extremely incomplete information market environment, a novel model-free reinforcement learning algorithm based on the Learning Automata (LA) is proposed in this paper. Besides, this paper analyses the rationality and convergence of the algorithm in case studies based on the Cournot market model.

eess.SY↗

arXiv2020

6G White paper: Research challenges for Trust, Security and Privacy

The roles of trust, security and privacy are somewhat interconnected, but different facets of next generation networks. The challenges in creating a trustworthy 6G are multidisciplinary spanning technology, regulation, techno-economics, politics and ethics. This white paper addresses their fundamental research challenges in three key areas. Trust: Under the current "open internet" regulation, the telco cloud can be used for trust services only equally for all users. 6G network must support embedded trust for increased level of information security in 6G. Trust modeling, trust policies and trust mechanisms need to be defined. 6G interlinks physical and digital worlds making safety dependent on information security. Therefore, we need trustworthy 6G. Security: In 6G era, the dependence of the economy and societies on IT and the networks will deepen. The role of IT and the networks in national security keeps rising - a continuation of what we see in 5G. The development towards cloud and edge native infrastructures is expected to continue in 6G networks, and we need holistic 6G network security architecture planning. Security automation opens new questions: machine learning can be used to make safer systems, but also more dangerous attacks. Physical layer security techniques can also represent efficient solutions for securing less investigated network segments as first line of defense. Privacy: There is currently no way to unambiguously determine when linked, deidentified datasets cross the threshold to become personally identifiable. Courts in different parts of the world are making decisions about whether privacy is being infringed, while companies are seeking new ways to exploit private data to create new business revenues. As solution alternatives, we may consider blockchain, distributed ledger technologies and differential privacy approaches.

cs.CR↗

arXiv2020

DOB-Net: Actively Rejecting Unknown Excessive Time-Varying Disturbances

This paper presents an observer-integrated Reinforcement Learning (RL) approach, called Disturbance OBserver Network (DOB-Net), for robots operating in environments where disturbances are unknown and time-varying, and may frequently exceed robot control capabilities. The DOB-Net integrates a disturbance dynamics observer network and a controller network. Originated from conventional DOB mechanisms, the observer is built and enhanced via Recurrent Neural Networks (RNNs), encoding estimation of past values and prediction of future values of unknown disturbances in RNN hidden state. Such encoding allows the controller generate optimal control signals to actively reject disturbances, under the constraints of robot control capabilities. The observer and the controller are jointly learned within policy optimization by advantage actor critic. Numerical simulations on position regulation tasks have demonstrated that the proposed DOB-Net significantly outperforms a conventional feedback controller and classical RL algorithms.

cs.RO↗

arXiv2020

Effective p-wave Fermi-Fermi Interaction Induced by Bosonic Superfluids

We study the two-dimensional Bose-Fermi mixture on square lattice at finite temperature by using the determinant quantum Monte Carlo method within the weakly interacting regime. Here we consider the attractive Bose-Hubbard model and free spinless fermions. In the absence of bosonfermion interactions, we obtain the boundary of the collapsed state of the attractive bosons. In the presence of boson-fermion interactions, an effective p-wave interaction between fermions will be induced as far as the bosons are in a superfluid state. Moreover, we find the emergence of the composite fermion pairs at low temperatures.

cond-mat.str-el↗

arXiv2019

Enjoy the Untrusted Cloud: A Secure, Scalable and Efficient SQL-like Query Framework for Outsourcing Data

While the security of the cloud remains a concern, a common practice is to encrypt data before outsourcing them for utilization. One key challenging issue is how to efficiently perform queries over the ciphertext. Conventional crypto-based solutions, e.g. partially/fully homomorphic encryption and searchable encryption, suffer from low performance, poor expressiveness and weak compatibility. An alternative method that utilizes hardware-assisted trusted execution environment, i.e., Intel SGX, has emerged recently. On one hand, such work lacks of supporting scalable access control over multiple data users. On the other hand, existing solutions are subjected to the key revocation problem and knowledge extractor vulnerability. In this work, we leverage the newly hardware-assisted methodology and propose a secure, scalable and efficient SQL-like query framework named QShield. Building upon Intel SGX, QShield can guarantee the confidentiality and integrity of sensitive data when being processed on an untrusted cloud platform. Moreover, we present a novel lightweight secret sharing method to enable multi-user access control in QShield, while tackling the key revocation problem. Furthermore, with an additional trust proof mechanism, QShield guarantees the correctness of queries and significantly alleviates the possibility to build a knowledge extractor. We implemented a prototype for QShield and show that QShield incurs minimum performance cost.

cs.CR↗

arXiv2019

Universal critical behavior in the ferromagnetic superconductor Eu(Fe$_{0.75}$Ru$_{0.25}$)$_{2}$As$_{2}$

The study of universal critical behavior is a crucial issue in a continuous phase transition, which groups various critical phenomena into universality classes for revealing microscopic electronic behaviors. The understanding of the nature of magnetism in Eu-based ferromagnetic superconductors is largely impeded by the infeasibility of performing inelastic neutron scattering measurements to deduce the microscopic magnetic behaviors and the effects on the superconductivity, due to the significant neutron absorption effect of natural $^{152}$Eu and unavailability of large single crystals. However, by systematically combining the neutron diffraction experiment, the first-principles calculations, and the quantum Monte Carlo simulations, we have obtained a perfectly consistent universal critical exponent value of $β=0.385(13)$ experimentally and theoretically for Eu(Fe$_{0.75}$Ru$_{0.25}$)$_{2}$As$_{2}$, from which the magnetism in the Eu-based ferromagnetic superconductors is identified as the universal class of a three-dimensional anisotropic quantum Heisenberg model with long-range magnetic exchange coupling. This study not only clarifies the nature of microscopic magnetic behaviors in the Eu-based ferromagnetic superconductors, but also opens a new avenue of systemic methodology for studying the universal critical behaviors associated with magnetic phase transitions in the area of magnetism and the spin fluctuations effects on the unconventional superconductivity.

cond-mat.str-el↗

arXiv2019

Fast and Exact Nearest Neighbor Search in Hamming Space on Full-Text Search Engines

A growing interest has been witnessed recently from both academia and industry in building nearest neighbor search (NNS) solutions on top of full-text search engines. Compared with other NNS systems, such solutions are capable of effectively reducing main memory consumption, coherently supporting multi-model search and being immediately ready for production deployment. In this paper, we continue the journey to explore specifically how to empower full-text search engines with fast and exact NNS in Hamming space (i.e., the set of binary codes). By revisiting three techniques (bit operation, subs-code filtering and data preprocessing with permutation) in information retrieval literature, we develop a novel engineering solution for full-text search engines to efficiently accomplish this special but important NNS task. In the experiment, we show that our proposed approach enables full-text search engines to achieve significant speed-ups over its state-of-the-art term match approach for NNS within binary codes.

cs.IR↗

arXiv2019

An Empirical Comparison of FAISS and FENSHSES for Nearest Neighbor Search in Hamming Space

In this paper, we compare the performances of FAISS and FENSHSES on nearest neighbor search in Hamming space--a fundamental task with ubiquitous applications in nowadays eCommerce. Comprehensive evaluations are made in terms of indexing speed, search latency and RAM consumption. This comparison is conducted towards a better understanding on trade-offs between nearest neighbor search systems implemented in main memory and the ones implemented in secondary memory, which is largely unaddressed in literature.

cs.LG↗

arXiv2019

Dual channels of helicity cascade in turbulent flows

Helicity, as one of only two inviscid invariants in three-dimensional turbulence, plays an important role in the generation and evolution of turbulence. From the traditional viewpoint, there exists only one channel of helicity cascade similar to that of kinetic energy cascade. Through theoretical analysis, we find that there are two channels in helicity cascade process. The first channel mainly originates from vortex twisting process, and the second channel mainly originates from vortex stretching process. By analysing the data of direct numerical simulations of typical turbulent flows, we find that these two channels behave differently. The ensemble averages of helicity flux in different channels are equal in homogeneous and isotropic turbulence, while they are different in other type of turbulent flows. The second channel is more intermittent and acts more like a scalar, especially on small scales. Besides, we find a novel mechanism of hindered even inverse energy cascade, which could be attributed to the second-channel helicity flux with large amplitude.

physics.flu-dyn↗

arXiv2019

Towards Practical Visual Search Engine within Elasticsearch

In this paper, we describe our end-to-end content-based image retrieval system built upon Elasticsearch, a well-known and popular textual search engine. As far as we know, this is the first time such a system has been implemented in eCommerce, and our efforts have turned out to be highly worthwhile. We end up with a novel and exciting visual search solution that is extremely easy to be deployed, distributed, scaled and monitored in a cost-friendly manner. Moreover, our platform is intrinsically flexible in supporting multimodal searches, where visual and textual information can be jointly leveraged in retrieval. The core idea is to encode image feature vectors into a collection of string tokens in a way such that closer vectors will share more string tokens in common. By doing that, we can utilize Elasticsearch to efficiently retrieve similar images based on similarities within encoded sting tokens. As part of the development, we propose a novel vector to string encoding method, which is shown to substantially outperform the previous ones in terms of both precision and latency. First-hand experiences in implementing this Elasticsearch-based platform are extensively addressed, which should be valuable to practitioners also interested in building visual search engine on top of Elasticsearch.

cs.CV↗

arXiv2019

Sweeping cluster algorithm for quantum spin systems with strong geometric restrictions

Quantum spin systems with strong geometric restrictions give rise to rich quantum phases such as valence bond solids and spin liquid states. However, the geometric restrictions often hamper the application of sophisticated numerical approaches. Based on the stochastic series expansion method, we develop an efficient and exact quantum Monte Carlo "sweeping cluster" algorithm which automatically satisfies the geometrical restrictions. Here we use the quantum dimer model as a benchmark to demonstrate the reliability and power of this algorithm. Comparing to existing numerical methods, we can obtain higher accuracy results for a wider parameter region and much more substantial system sizes.

cond-mat.stat-mech↗

arXiv2019

Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting

Weather forecasting is usually solved through numerical weather prediction (NWP), which can sometimes lead to unsatisfactory performance due to inappropriate setting of the initial states. In this paper, we design a data-driven method augmented by an effective information fusion mechanism to learn from historical data that incorporates prior knowledge from NWP. We cast the weather forecasting problem as an end-to-end deep learning problem and solve it by proposing a novel negative log-likelihood error (NLE) loss function. A notable advantage of our proposed method is that it simultaneously implements single-value forecasting and uncertainty quantification, which we refer to as deep uncertainty quantification (DUQ). Efficient deep ensemble strategies are also explored to further improve performance. This new approach was evaluated on a public dataset collected from weather stations in Beijing, China. Experimental results demonstrate that the proposed NLE loss significantly improves generalization compared to mean squared error (MSE) loss and mean absolute error (MAE) loss. Compared with NWP, this approach significantly improves accuracy by 47.76%, which is a state-of-the-art result on this benchmark dataset. The preliminary version of the proposed method won 2nd place in an online competition for daily weather forecasting.

cs.LG↗

arXiv2019

Revisiting Skip-Gram Negative Sampling Model with Rectification

We revisit skip-gram negative sampling (SGNS), one of the most popular neural-network based approaches to learning distributed word representation. We first point out the ambiguity issue undermining the SGNS model, in the sense that the word vectors can be entirely distorted without changing the objective value. To resolve the issue, we investigate the intrinsic structures in solution that a good word embedding model should deliver. Motivated by this, we rectify the SGNS model with quadratic regularization, and show that this simple modification suffices to structure the solution in the desired manner. A theoretical justification is presented, which provides novel insights into quadratic regularization . Preliminary experiments are also conducted on Google's analytical reasoning task to support the modified SGNS model.

cs.CL↗

arXiv2018

A Multi-step Piecewise Linear Approximation Based Solution for Load Pick-up Problem in Electrical Distribution System

The load pick-up (LPP) problem searches the optimal configuration of the electrical distribution system (EDS), aiming to minimize the power loss or provide maximum power to the load ends. The piecewise linearization (PWL) approximation method can be used to tackle the nonlinearity and nonconvexity in network power flow (PF) constraints, and transform the LPP model into a mixed-integer linear programming model (LPP-MILP model).However, for the PWL approximation based PF constraints, big linear approximation errors will affect the accuracy and feasibility of the LPP-MILP model's solving results. And the long modeling and solving time of the direct solution procedure of the LPP-MILP model may affect the applicability of the LPP optimization scheme. This paper proposes a multi-step PWL approximation based solution for the LPP problem in the EDS. In the proposed multi-step solution procedure, the variable upper bounds in the PWL approximation functions are dynamically renewed to reduce the approximation errors effectively. And the multi-step solution procedure can significantly decrease the modeling and solving time of the LPP-MILP model, which ensure the applicability of the LPP optimization scheme. For the two main application schemes for the LPP problem (i.e. network optimization reconfiguration and service restoration), the effectiveness of the proposed method is demonstrated via case studies using a real 13-bus EDS and a real 1066-bus EDS.

math.OC↗

arXiv2018

Self-optimal Piecewise Linearization Based Network Power Flow Constraints in Electrical Distribution System Optimization

As a representative mathematical expression of power flow (PF) constraints in electrical distribution system (EDS), the piecewise linearization (PWL) based PF constraints have been widely used in different EDS optimization scenarios. However, the linearized approximation errors originating from the currently-used PWL approximation function can be very large and thus affect the applicability of the PWL based PF constraints. This letter analyzes the approximation error self-optimal (ESO) condition of the PWL approximation function, refines the PWL function formulas, and proposes the self-optimal (SO)-PWL based PF constraints in EDS optimization which can ensure the minimum approximation errors. Numerical results demonstrate the effectiveness of the proposed method.

math.OC↗

arXiv2018

Distributionally Robust Co-Optimization of Power Dispatch and Do-Not-Exceed Limits

To address the challenge of the renewable energy uncertainty, the ISO New England (ISO-NE) has proposed to apply do-not-exceed (DNE) limits, which represent the maximum nodal injection of renewable energy the grid can accommodate. Unfortunately, it appears challenging to compute DNE limits that simultaneously maintain the system flexibility and incorporate a large portion of the available renewable energy at the minimum cost. In addition, it is often challenging to accurately estimate the joint probability distribution of the renewable energy. In this paper, we propose a two-stage distributionally robust optimization model that co-optimizes the power dispatch and the DNE limits, by adopting an affinely adjustable power re-dispatch and an adjustable joint chance constraint that measures the renewable utilization. Notably, this model admits a second-order conic reformulation that can be efficiently solved by the commercial solvers (e.g., MOSEK). We conduct case studies based on modified IEEE test instances to demonstrate the effectiveness of the proposed approach and analyze the trade-off among the system flexibility, the renewable utilization, and the dispatch cost.

math.OC↗

arXiv2018

End-to-End Neural Ranking for eCommerce Product Search: an application of task models and textual embeddings

We consider the problem of retrieving and ranking items in an eCommerce catalog, often called SKUs, in order of relevance to a user-issued query. The input data for the ranking are the texts of the queries and textual fields of the SKUs indexed in the catalog. We review the ways in which this problem both resembles and differs from the problems of IR in the context of web search. The differences between the product-search problem and the IR problem of web search necessitate a different approach in terms of both models and datasets. We first review the recent state-of-the-art models for web search IR, distinguishing between two distinct types of model which we call the distributed type and the local-interaction type. The different types of relevance models developed for IR have complementary advantages and disadvantages when applied to eCommerce product search. Further, we explain why the conventional methods for dataset construction employed in the IR literature fail to produce data which suffices for training or evaluation of models for eCommerce product search. We explain how our own approach, applying task modeling techniques to the click-through logs of an eCommerce site, enables the construction of a large-scale dataset for training and robust benchmarking of relevance models. Our experiments consist of applying several of the models from the IR literature to our own dataset. Empirically, we have established that, when applied to our dataset, certain models of local-interaction type reduce ranking errors by one-third compared to the baseline tf-idf. Applied to our dataset, the distributed models fail to outperform the baseline. As a basis for a deployed system, the distributed models have several advantages, computationally, over the local-interaction models. This motivates an ongoing program of work, which we outline at the conclusion of the paper.

cs.IR↗

arXiv2017

Resolving ECRH deposition broadening due to edge turbulence in DIII-D by heat deposition measurement

Interaction between microwave power, used for local heating and mode control, and density fluctuations can produce a broadening of the injected beam, as confirmed in experiment and simulation. Increased power deposition width could impact suppression of tearing mode structures on ITER. This work discusses the experimental portion of an effort to understand scattering of injected microwaves by turbulence on the DIII-D tokamak. The corresponding theoretical modeling work can be found in M.B. Thomas et. al.: Submitted to Nuclear Fusion (2017)[Author Note - this paper to be published in same journal]. In a set of perturbative heat transport experiments, tokamak edge millimeter-scale fluctuation levels and microwave heat deposition are measured simultaneously. Beam broadening is separated from heat transport through fitting of modulated fluxes. Electron temperature measurements from a 500 kHz, 48-channel ECE radiometer are Fourier analyzed and used to calculate a deposition-dependent flux. Consistency of this flux with a transport model is evaluated. A diffusive and convective transport solution is linearized and compared with energy conservation-derived fluxes. Comparison between these two forms of heat flux is used to evaluate the quality of ECRF deposition profiles, and fitting finds a significant broadening of 1D equilibrium ray tracing calculations from the benchmarked TORAY-GA ray tracing code is needed. The physical basis, cross-validation, and application of the heat flux method is presented. The method is applied to a range of DIII-D discharges and finds a broadening factor of the deposition profile width which scales linearly with edge density fluctuation level. These experimental results are found to be consistent with the full-wave beam broadening measured by the 3D full wave simulations in the same discharges.

physics.plasm-ph↗