arXiv subjects

Ziru Chen

Publications and source records attributed to Ziru Chen.

32 records · Page 2Linked to original sources

arXiv2024

eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data

With tremendous efforts on developing effective e-commerce models, conventional e-commerce models show limited success in generalist e-commerce modeling, and suffer from unsatisfactory performance on new users and new products - a typical out-of-domain generalization challenge. Meanwhile, large language models (LLMs) demonstrate outstanding performance in generalist modeling and out-of-domain generalizability in many fields. Toward fully unleashing their power for e-commerce, in this paper, we construct ECInstruct, the first open-sourced, large-scale, and high-quality benchmark instruction dataset for e-commerce. Leveraging ECInstruct, we develop eCeLLM, a series of e-commerce LLMs, by instruction-tuning general-purpose LLMs. Our comprehensive experiments and evaluation demonstrate that eCeLLM models substantially outperform baseline models, including the most advanced GPT-4, and the state-of-the-art task-specific models in in-domain evaluation. Moreover, eCeLLM exhibits excellent generalizability to out-of-domain settings, including unseen products and unseen instructions, highlighting its superiority as a generalist e-commerce model. Both the ECInstruct dataset and the eCeLLM models show great potential in empowering versatile and effective LLMs for e-commerce. ECInstruct and eCeLLM models are publicly accessible through https://ninglab.github.io/eCeLLM.

cs.CL↗

arXiv2024

When is Tree Search Useful for LLM Planning? It Depends on the Discriminator

In this paper, we examine how large language models (LLMs) solve multi-step problems under a language agent framework with three components: a generator, a discriminator, and a planning method. We investigate the practical utility of two advanced planning methods, iterative correction and tree search. We present a comprehensive analysis of how discrimination accuracy affects the overall performance of agents when using these two methods or a simpler method, re-ranking. Experiments on two tasks, text-to-SQL parsing and mathematical reasoning, show that: (1) advanced planning methods demand discriminators with at least 90% accuracy to achieve significant improvements over re-ranking; (2) current LLMs' discrimination abilities have not met the needs of advanced planning methods to achieve such improvements; (3) with LLM-based discriminators, advanced planning methods may not adequately balance accuracy and efficiency. For example, compared to the other two methods, tree search is at least 10--20 times slower but leads to negligible performance gains, which hinders its real-world applications. Code and data are available at https://github.com/OSU-NLP-Group/llm-planning-eval.

cs.CL↗

arXiv2023

Error Detection for Text-to-SQL Semantic Parsing

Despite remarkable progress in text-to-SQL semantic parsing in recent years, the performance of existing parsers is still far from perfect. Specifically, modern text-to-SQL parsers based on deep learning are often over-confident, thus casting doubt on their trustworthiness when deployed for real use. In this paper, we propose a parser-independent error detection model for text-to-SQL semantic parsing. Using a language model of code as its bedrock, we enhance our error detection model with graph neural networks that learn structural features of both natural language questions and SQL queries. We train our model on realistic parsing errors collected from a cross-domain setting, which leads to stronger generalization ability. Experiments with three strong text-to-SQL parsers featuring different decoding mechanisms show that our approach outperforms parser-dependent uncertainty metrics. Our model could also effectively improve the performance and usability of text-to-SQL semantic parsers regardless of their architectures. (Our implementation is available at https://github.com/OSU-NLP-Group/Text2SQL-Error-Detection)

cs.CL↗

arXiv2023

Exploring Chain-of-Thought Style Prompting for Text-to-SQL

In-context learning with large language models (LLMs) has recently caught increasing attention due to its superior few-shot performance on various tasks. However, its performance on text-to-SQL parsing still has much room for improvement. In this paper, we hypothesize that a crucial aspect of LLMs to improve for text-to-SQL parsing is their multi-step reasoning ability. Thus, we systematically study how to enhance LLMs' reasoning ability through chain of thought (CoT) style prompting, including the original chain-of-thought prompting (Wei et al., 2022b) and least-to-most prompting (Zhou et al., 2023). Our experiments demonstrate that iterative prompting as in Zhou et al. (2023) may be unnecessary for text-to-SQL parsing, and using detailed reasoning steps tends to have more error propagation issues. Based on these findings, we propose a new CoT-style prompting method for text-to-SQL parsing. It brings 5.2 and 6.5 point absolute gains on the Spider development set and the Spider Realistic set, respectively, compared to the standard prompting method without reasoning steps; 2.4 and 1.5 point absolute gains, compared to the least-to-most prompting method.

cs.CL↗

arXiv2023

Automatic Evaluation of Attribution by Large Language Models

A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. However, evaluating the attribution, i.e., verifying whether the generated statement is fully supported by the cited reference, remains an open problem. Although human evaluation is common practice, it is costly and time-consuming. In this paper, we investigate the automatic evaluation of attribution given by LLMs. We begin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from related tasks such as question answering, fact-checking, natural language inference, and summarization. We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing. Our results on this curated test set and simulated examples from existing benchmarks highlight both promising signals and challenges. We hope our problem formulation, testbeds, and findings will help lay the foundation for future studies on this important problem.

cs.CL↗

arXiv2023

Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue System

We introduce TacoBot, a user-centered task-oriented digital assistant designed to guide users through complex real-world tasks with multiple steps. Covering a wide range of cooking and how-to tasks, we aim to deliver a collaborative and engaging dialogue experience. Equipped with language understanding, dialogue management, and response generation components supported by a robust search engine, TacoBot ensures efficient task assistance. To enhance the dialogue experience, we explore a series of data augmentation strategies using LLMs to train advanced neural models continuously. TacoBot builds upon our successful participation in the inaugural Alexa Prize TaskBot Challenge, where our team secured third place among ten competing teams. We offer TacoBot as an open-source framework that serves as a practical example for deploying task-oriented dialogue systems.

cs.CL↗

arXiv2023

Text-to-SQL Error Correction with Language Models of Code

Despite recent progress in text-to-SQL parsing, current semantic parsers are still not accurate enough for practical use. In this paper, we investigate how to build automatic text-to-SQL error correction models. Noticing that token-level edits are out of context and sometimes ambiguous, we propose building clause-level edit models instead. Besides, while most language models of code are not specifically pre-trained for SQL, they know common data structures and their operations in programming languages such as Python. Thus, we propose a novel representation for SQL queries and their edits that adheres more closely to the pre-training corpora of language models of code. Our error correction model improves the exact set match accuracy of different parsers by 2.4-6.5 and obtains up to 4.3 point absolute improvement over two strong baselines. Our code and data are available at https://github.com/OSU-NLP-Group/Auto-SQL-Correction.

cs.CL↗

arXiv2022

Bootstrapping a User-Centered Task-Oriented Dialogue System

We present TacoBot, a task-oriented dialogue system built for the inaugural Alexa Prize TaskBot Challenge, which assists users in completing multi-step cooking and home improvement tasks. TacoBot is designed with a user-centered principle and aspires to deliver a collaborative and accessible dialogue experience. Towards that end, it is equipped with accurate language understanding, flexible dialogue management, and engaging response generation. Furthermore, TacoBot is backed by a strong search engine and an automated end-to-end test suite. In bootstrapping the development of TacoBot, we explore a series of data augmentation strategies to train advanced neural language processing models and continuously improve the dialogue experience with collected real conversations. At the end of the semifinals, TacoBot achieved an average rating of 3.55/5.0.

cs.CL↗

arXiv2022

A Deep Reinforcement Learning based Approach for NOMA-based Random Access Network with Truncated Channel Inversion Power Control

As a main use case of 5G and Beyond wireless network, the ever-increasing machine type communications (MTC) devices pose critical challenges over MTC network in recent years. It is imperative to support massive MTC devices with limited resources. To this end, Non-orthogonal multiple access (NOMA) based random access network has been deemed as a prospective candidate for MTC network. In this paper, we propose a deep reinforcement learning (RL) based approach for NOMA-based random access network with truncated channel inversion power control. Specifically, each MTC device randomly selects a pre-defined power level with a certain probability for data transmission. Devices are using channel inversion power control yet subject to the upper bound of the transmission power. Due to the stochastic feature of the channel fading and the limited transmission power, devices with different achievable power levels have been categorized as different types of devices. In order to achieve high throughput with considering the fairness between all devices, two objective functions are formulated. One is to maximize the minimum long-term expected throughput of all MTC devices, the other is to maximize the geometric mean of the long-term expected throughput for all MTC devices. A Policy based deep reinforcement learning approach is further applied to tune the transmission probabilities of each device to solve the formulated optimization problems. Extensive simulations are conducted to show the merits of our proposed approach.

cs.NI↗

arXiv2020

Optimizing Non-Orthogonal Multiple Access in Random Access Networks

Non-orthogonal multiple access (NOMA) has been considered as a promising solution for improving the spectrum efficiency of next-generation wireless networks. In this paper, the performance of a p-persistent slotted ALOHA system in support of NOMA transmissions is investigated. Specifically, wireless users can choose to use high or low power for data transmissions with certain probabilities. To achieve the maximum network throughput, an analytical framework is developed to analyze the successful transmission probability of NOMA and long term average throughput of users involved in the non-orthogonal transmissions. The feasible region of the maximum number of concurrent users using high and low power to ensure successful NOMA transmissions are quantified. Based on the analysis, an algorithm is proposed to find the optimal transmission probabilities for users to choose high and low power to achieve the maximum system throughput. In addition, the impact of power settings on the network performance is further investigated. Simulations are conducted to validate the analysis.

cs.NI↗

arXiv2019

Analysis of RF Energy Harvesting in Uplink-NOMA IoT-based Network

Internet of Things (IoT) systems in general consist of a lot of devices with massive connectivity. Those devices are usually constrained with limited energy supply and can only operate at low power and low rate. In this paper, we investigate a cellular-based IoT system combined with energy harvesting and NOMA. We consider all base stations (BS) and IoT devices follow the Poisson Point Process (PPP) distribution in a given area. The unit time slot is divided into two phases, energy harvesting phase in downlink (DL) and data transmission phase in uplink (uplink). That is, IoT devices will first harvest energy from all BS transmissions and then use the harvested energy to do the NOMA information transmission. We define an energy harvesting circle within which all IoT devices can harvest enough energy for NOMA transmission. The design objective is to maximize the total throughput in uplink within the circle by varying the duration T of energy harvesting phase. In our work, we also consider the inter-cell interference in the throughput calculation. The analysis of Probability Mass Function (PMF) for IoT devices in the energy harvesting circle is also compared with simulation results. It is shown that the BS density needs to be carefully set so that the IoT devices in the energy harvesting circle receive relatively smaller interference and energy circles overlap only with a small probability. Our simulations show that there exists an optimal T to achieve the maximum throughput. When the BSs are densely deployed consequently the total throughput will decrease because of the interference.

eess.SP↗

arXiv2018

A Stochastic Geometry Analysis of Energy Harvesting in Large Scale Wireless Networks

In this paper, the theoretical sustainable capacity of wireless networks with radio frequency (RF) energy harvesting is analytically studied. Specifically, we consider a large scale wireless network where base stations (BSs) and low power wireless devices are deployed by homogeneous Poisson point process (PPP) with different spatial densities. Wireless devices exploit the downlink transmissions from the BSs for either information delivery or energy harvesting. Generally, a BS schedules downlink transmission to wireless devices. The scheduled device receives the data information while other devices harvest energy from the downlink signals. The data information can be successfully received by the scheduled device only if the device has sufficient energy for data processing, i.e., the harvested energy is larger than a threshold. Given the densities of BSs and users, we apply stochastic geometry to analyze the expected number of users per cell and the successful information delivery probability of a wireless device, based on which the total network throughput can be derived. It is shown that the maximum network throughput per cell can be achieved under the optimal density of BSs. Extensive simulations validate the analysis.

cs.NI↗

arXiv2018

A Hybrid Approach for Efficient Wireless Information and Power Transfer in Green C-RAN

In this paper, we consider a green cloud radio access network (C-RAN) with simultaneous wireless and power transfer ability. In order to reduce the energy consumed for updating the chanel state information (CSI), energy users are divided into two different groups, including the free charge group and the MIMO group. Then a semi-definite programming problem is formulated under the constraints of energy and information transmission requirements. To minimize the total energy consumption, two algorithms are developed to authorize the energy users into two group divisions in single time slot. Then the algorithms are extended to long term scenarios consisting training and long term stages, the CSI of free charge energy users are not required during the long term stage. Simulation and numerical results are presented to demonstrate the efficiency of the proposed algorithms in significantly reducing the energy consumption of C-RAN systems.

cs.NI↗

arXiv2017

Energy-Throughput Tradeoff in Sustainable Cloud-RAN with Energy Harvesting

In this paper, we investigate joint beamforming for energy-throughput tradeoff in a sustainable cloud radio access network system, where multiple base stations (BSs) powered by independent renewable energy sources will collaboratively transmit wireless information and energy to the data receiver and the energy receiver simultaneously. In order to obtain the optimal joint beamforming design over a finite time horizon, we formulate an optimization problem to maximize the throughput of the data receiver while guaranteeing sufficient RF charged energy of the energy receiver. Although such problem is non-convex, it can be relaxed into a convex form and upper bounded by the optimal value of the relaxed problem. We further prove tightness of the upper bound by showing the optimal solution to the relaxed problem is rank one. Motivated by the optimal solution, an efficient online algorithm is also proposed for practical implementation. Finally, extensive simulations are performed to verify the superiority of the proposed joint beamforming strategy to other beamforming designs.

cs.IT↗