arXiv subjects

Shaojie Tang

Publications and source records attributed to Shaojie Tang.

At least 91 records · Page 5Linked to original sources

arXiv2020

Cascade Submodular Maximization: Question Selection and Sequencing in Online Personality Quiz

Personality quiz is a powerful tool that enables costumer segmentation by actively asking them questions, and marketers are using it as an effective method of generating leads and increasing e-commerce sales. In this paper, we study the problem of how to select and sequence a group of quiz questions so as to optimize the quality of customer segmentation. We assume that the customer will sequentially scan the list of questions. After reading a question, the customer makes two, possibly correlated, random decisions: 1) she first decides whether to answer this question or not, and then 2) decides whether to continue reading the next question or not. We further assume that the utility of questions that have been answered can be captured by a monotone and submodular function. In general, our problem falls into the category of non-adaptive active learning based customer profiling. Note that under the our model, the probability of a question being answered depends on the location of that question, as well as the set of other questions placed ahead of that question, this makes our problem fundamentally different from existing studies on submodular optimization. We develop a series of question selection and sequencing strategies with provable performance bound. Although we focus on the application of quiz design in this paper, our results apply to a broad range of applications, including assortment optimization with position bias effect.

cs.LG↗

arXiv2020

Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability

Federated learning is a new distributed machine learning framework, where a bunch of heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated learning in mobile environments: intermittent client availability, where the set of eligible clients may change during the training process. Such intermittent client availability would seriously deteriorate the performance of the classical Federated Averaging algorithm (FedAvg for short). Thus, we propose a simple distributed non-convex optimization algorithm, called Federated Latest Averaging (FedLaAvg for short), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg attains the convergence rate of $O(E^{1/2}/(N^{1/4} T^{1/2}))$, achieving a sublinear speedup with respect to the total number of clients. We implement FedLaAvg along with several baselines and evaluate them over the benchmarking MNIST and Sentiment140 datasets. The evaluation results demonstrate that FedLaAvg achieves more stable training than FedAvg in both convex and non-convex settings and indeed reaches a sublinear speedup.

stat.ML↗

arXiv2020

Beyond Pointwise Submodularity: Non-Monotone Adaptive Submodular Maximization in Linear Time

In this paper, we study the non-monotone adaptive submodular maximization problem subject to a cardinality constraint. We first revisit the adaptive random greedy algorithm proposed in \citep{gotovos2015non}, where they show that this algorithm achieves a $1/e$ approximation ratio if the objective function is adaptive submodular and pointwise submodular. It is not clear whether the same guarantee holds under adaptive submodularity (without resorting to pointwise submodularity) or not. Our first contribution is to show that the adaptive random greedy algorithm achieves a $1/e$ approximation ratio under adaptive submodularity. One limitation of the adaptive random greedy algorithm is that it requires $O(n\times k)$ value oracle queries, where $n$ is the size of the ground set and $k$ is the cardinality constraint. Our second contribution is to develop the first linear-time algorithm for the non-monotone adaptive submodular maximization problem. Our algorithm achieves a $1/e-ε$ approximation ratio (this bound is improved to $1-1/e-ε$ for monotone case), using only $O(nε^{-2}\log ε^{-1})$ value oracle queries. Notably, $O(nε^{-2}\log ε^{-1})$ is independent of the cardinality constraint. For the monotone case, we propose a faster algorithm that achieves a $1-1/e-ε$ approximation ratio in expectation with $O(n \log \frac{1}ε)$ value oracle queries. We also generalize our study by considering a partition matroid constraint, and develop a linear-time algorithm for monotone and fully adaptive submodular functions.

cs.LG↗

arXiv2020

SelectScale: Mining More Patterns from Images via Selective and Soft Dropout

Convolutional neural networks (CNNs) have achieved remarkable success in image recognition. Although the internal patterns of the input images are effectively learned by the CNNs, these patterns only constitute a small proportion of useful patterns contained in the input images. This can be attributed to the fact that the CNNs will stop learning if the learned patterns are enough to make a correct classification. Network regularization methods like dropout and SpatialDropout can ease this problem. During training, they randomly drop the features. These dropout methods, in essence, change the patterns learned by the networks, and in turn, forces the networks to learn other patterns to make the correct classification. However, the above methods have an important drawback. Randomly dropping features is generally inefficient and can introduce unnecessary noise. To tackle this problem, we propose SelectScale. Instead of randomly dropping units, SelectScale selects the important features in networks and adjusts them during training. Using SelectScale, we improve the performance of CNNs on CIFAR and ImageNet.

cs.CV↗

arXiv2020

Assortment Optimization with Repeated Exposures and Product-dependent Patience Cost

In this paper, we study the assortment optimization problem faced by many online retailers such as Amazon. We develop a \emph{cascade multinomial logit model}, based on the classic multinomial logit model, to capture the consumers' purchasing behavior across multiple stages. Different from existing studies, our model allows for repeated exposures of a product, i.e., the same product can be displayed multiple times across different stages. In addition, each consumer has a \emph{patience budget} that is sampled from a known distribution and each product is associated with a \emph{patience cost}, which captures the cognitive efforts spent on browsing that product. Given an assortment of products, a consumer sequentially browses them stage by stage. After browsing all products in one stage, if the utility of a product exceeds the utility of the outside option, the consumer proceeds to purchase the product and leave the platform. Otherwise, if the patience cost of all products browsed up to that point is no larger than her patience budget, she continues to view the next stage. We propose an approximation solution to this problem.

cs.LG↗

arXiv2020

Linear-Time Algorithms for Adaptive Submodular Maximization

In this paper, we develop fast algorithms for two stochastic submodular maximization problems. We start with the well-studied adaptive submodular maximization problem subject to a cardinality constraint. We develop the first linear-time algorithm which achieves a $(1-1/e-ε)$ approximation ratio. Notably, the time complexity of our algorithm is $O(n\log\frac{1}ε)$ (number of function evaluations) which is independent of the cardinality constraint, where $n$ is the size of the ground set. Then we introduce the concept of fully adaptive submodularity, and develop a linear-time algorithm for maximizing a fully adaptive submoudular function subject to a partition matroid constraint. We show that our algorithm achieves a $\frac{1-1/e-ε}{4-2/e-2ε}$ approximation ratio using only $O(n\log\frac{1}ε)$ number of function evaluations.

cs.LG↗

arXiv2020

Semantic Segmentation Using Deep Learning to Extract Total Extraocular Muscles and Optic Nerve from Orbital Computed Tomography Images

Objectives: Precise segmentation of total extraocular muscles (EOM) and optic nerve (ON) is essential to assess anatomical development and progression of thyroid-associated ophthalmopathy (TAO). We aim to develop a semantic segmentation method based on deep learning to extract the total EOM and ON from orbital CT images in patients with suspected TAO. Materials and Methods: A total of 7,879 images obtained from 97 subjects who underwent orbit CT scans due to suspected TAO were enrolled in this study. Eighty-eight patients were randomly selected into the training/validation dataset, and the rest were put into the test dataset. Contours of the total EOM and ON in all the patients were manually delineated by experienced radiologists as the ground truth. A three-dimensional (3D) end-to-end fully convolutional neural network called semantic V-net (SV-net) was developed for our segmentation task. Intersection over Union (IoU) was measured to evaluate the accuracy of the segmentation results, and Pearson correlation analysis was used to evaluate the volumes measured from our segmentation results against those from the ground truth. Results: Our model in the test dataset achieved an overall IoU of 0.8207; the IoU was 0.7599 for the superior rectus muscle, 0.8183 for the lateral rectus muscle, 0.8481 for the medial rectus muscle, 0.8436 for the inferior rectus muscle and 0.8337 for the optic nerve. The volumes measured from our segmentation results agreed well with those from the ground truth (all R>0.98, P<0.0001). Conclusion: The qualitative and quantitative evaluations demonstrate excellent performance of our method in automatically extracting the total EOM and ON and measuring their volumes in orbital CT images. There is a great promise for clinical application to assess these anatomical structures for the diagnosis and prognosis of TAO.

physics.med-ph↗

arXiv2020

A Novel Method for ECG Signal Classification via One-Dimensional Convolutional Neural Network

This paper presents an end-to-end ECG signal classification method based on a novel segmentation strategy via 1D Convolutional Neural Networks (CNN) to aid the classification of ECG signals. The ECG segmentation strategy named R-R-R strategy (i.e., retaining ECG data between the R peaks just before and after the current R peak) for segmenting the original ECG data into segments in order to train and test the 1D CNN models. The novel strategy mimics physicians in scanning ECG to a greater extent, and maximizes the inherent information of ECG segments. The performance of the classification models for 5-class and 6-class are verified with ECG signals from 48 records of the MIT-BIH arrhythmia database. As the heartbeat types are divided into 5 classes (i.e., normal beat, left bundle branch block beat, right bundle branch block beat, ventricular ectopic beat, and paced beat) in the MIT-BIH, the best classification accuracy, the area under the curve (AUC), the sensitivity and the F1-score reach 99.24%, 0.9994, 0.99 and 0.99, respectively. As the heartbeat types are divided into 6 classes (i.e., normal beat, left bundle branch block beat, right bundle branch block beat, ventricular ectopic beat, paced beat and other beats) in the MIT-BIH, the beat classification accuracy, the AUC, the sensitivity, and the F1-score reach 97.02%, 0.9966, 0.97, and 0.97, respectively. Meanwhile, according to the recommended practice from the Association for Advancement of Medical Instrumentation (AAMI), the heartbeat types are divided into 5 classes (i.e., normal beat, supraventricular ectopic beats, ventricular ectopic beats, fusion beats, and unclassifiable beats), the beat classification accuracy, the sensitivity, and the F1-score reach 97.45%, 0.97, and 0.97, respectively. The experimental results show that the proposed method achieves better performance than the state-of-the-art methods.

eess.SP↗

arXiv2020

Distributed Optimization over Block-Cyclic Data

We consider practical data characteristics underlying federated learning, where unbalanced and non-i.i.d. data from clients have a block-cyclic structure: each cycle contains several blocks, and each client's training data follow block-specific and non-i.i.d. distributions. Such a data structure would introduce client and block biases during the collaborative training: the single global model would be biased towards the client or block specific data. To overcome the biases, we propose two new distributed optimization algorithms called multi-model parallel SGD (MM-PSGD) and multi-chain parallel SGD (MC-PSGD) with a convergence rate of $O(1/\sqrt{NT})$, achieving a linear speedup with respect to the total number of clients. In particular, MM-PSGD adopts the block-mixed training strategy, while MC-PSGD further adds the block-separate training strategy. Both algorithms create a specific predictor for each block by averaging and comparing the historical global models generated in this block from different cycles. We extensively evaluate our algorithms over the CIFAR-10 dataset. Evaluation results demonstrate that our algorithms significantly outperform the conventional federated averaging algorithm in terms of test accuracy, and also preserve robustness for the variance of critical parameters.

cs.LG↗

arXiv2019

A new PIR-based method for real-time tracking

Pyroelectric infrared (PIR) sensors are considered to be promising devices for device-free localization due to its advantages of low cost, less intrusive, and the immunity from multi-path fading. However, most of the existing PIR-based localization systems only utilize the binary information of PIR sensors and therefore require a large number of PIR sensors and a careful deployment. A few works directly map the raw data of PIR sensors to one's location using machine learning approaches. However, these approaches require to collect abundant training data and suffer from environmental change. In this paper, we propose a PIR-based device-free localization approach based on the raw data of PIR sensors. The key of this approach is to extract a new type of location information called as the azimuth change. The extraction of the azimuth change relies on the physical properties of PIR sensors. Therefore, no abundant training data are needed and the system is robust to environmental change. Through experiments, we demonstrated that a device-free localization system incorporating the information of azimuth change outperforms the state-of-the-art approaches in terms of higher location accuracy. In addition, the information of the azimuth change can be easily integrated with other PIR-based localization systems to improve their localization accuracy.

eess.SP↗

arXiv2019

Product Sequencing and Pricing under Cascade Browse Model

In this paper, we study the joint product sequencing and pricing problem faced by many online retailers such as Amazon. We assume that consumers' purchase behavior can be explained by a ``consider-then-choose'' model: they first form a consideration set by screening a subset of products sequentially, and then decide which product to purchase from their consideration set. We propose a \emph{cascade browse model} to capture the consumers' browsing behavior, and use the Multinomial Logit (MNL) model as our choice model. We study two problems in this paper: in the first problem, we assume that each product has a fixed revenue and preference weight, the goal is to identify the best sequencing of products to offer so as to maximize the expected revenue subject to a cardinality constraint. We propose a constant approximate solution to this problem. As a byproduct, we propose the first fully polynomial-time approximation scheme (FPTAS) for the classic assortment optimization problem subject to one capacity constraint and one cardinality constraint. In the second problem, we treat the price of each product as a decision variable and our objective is to jointly decide a sequence of product and their prices to maximize the expected revenue. We propose a constant approximate solution to this problem.

cs.GT↗

arXiv2019

Online Pricing with Reserve Price Constraint for Personal Data Markets

The society's insatiable appetites for personal data are driving the emergency of data markets, allowing data consumers to launch customized queries over the datasets collected by a data broker from data owners. In this paper, we study how the data broker can maximize her cumulative revenue by posting reasonable prices for sequential queries. We thus propose a contextual dynamic pricing mechanism with the reserve price constraint, which features the properties of ellipsoid for efficient online optimization, and can support linear and non-linear market value models with uncertainty. In particular, under low uncertainty, our pricing mechanism provides a worst-case regret logarithmic in the number of queries. We further extend to other similar application scenarios, including hospitality service, online advertising, and loan application, and extensively evaluate three pricing instances of noisy linear query, accommodation rental, and impression over MovieLens 20M dataset, Airbnb listings in U.S. major cities, and Avazu mobile ad click dataset, respectively. The analysis and evaluation results reveal that our proposed pricing mechanism incurs low practical regret, online latency, and memory overhead, and also demonstrate that the existence of reserve price can mitigate the cold-start problem in a posted price mechanism, and thus can reduce the cumulative regret.

cs.CE↗

arXiv2019

Secure Federated Submodel Learning

Federated learning was proposed with an intriguing vision of achieving collaborative machine learning among numerous clients without uploading their private data to a cloud server. However, the conventional framework requires each client to leverage the full model for learning, which can be prohibitively inefficient for resource-constrained clients and large-scale deep learning tasks. We thus propose a new framework, called federated submodel learning, where clients download only the needed parts of the full model, namely submodels, and then upload the submodel updates. Nevertheless, the "position" of a client's truly required submodel corresponds to her private data, and its disclosure to the cloud server during interactions inevitably breaks the tenet of federated learning. To integrate efficiency and privacy, we have designed a secure federated submodel learning scheme coupled with a private set union protocol as a cornerstone. Our secure scheme features the properties of randomized response, secure aggregation, and Bloom filter, and endows each client with a customized plausible deniability, in terms of local differential privacy, against the position of her desired submodel, thus protecting her private data. We further instantiated our scheme with the e-commerce recommendation scenario in Alibaba, implemented a prototype system, and extensively evaluated its performance over 30-day Taobao user data. The analysis and evaluation results demonstrate the feasibility and scalability of our scheme from model accuracy and convergency, practical communication, computation, and storage overheads, as well as manifest its remarkable advantages over the conventional federated learning framework.

cs.LG↗

arXiv2019

Adaptive Robust Optimization with Nearly Submodular Structure

Constrained submodular maximization has been extensively studied in the recent years. In this paper, we study adaptive robust optimization with nearly submodular structure (ARONSS). Our objective is to randomly select a subset of items that maximizes the worst-case value of several reward functions simultaneously. Our work differs from existing studies in two ways: (1) we study the robust optimization problem under the adaptive setting, i.e., one needs to adaptively select items based on the feedback collected from picked items, and (2) our results apply to a broad range of reward functions characterized by $ε$-nearly submodular function. We first analyze the adaptvity gap of ARONSS and show that the gap between the best adaptive solution and the best non-adaptive solution is bounded. Then we propose a approximate solution to this problem when all reward functions are submodular. Our algorithm achieves approximation ratio $(1-1/e)$ when considering matroid constraint. At last, we present two heuristics for the general case. All proposed solutions are non-adaptive which are easy to implement.

cs.LG↗

arXiv2019

Price of Dependence: Stochastic Submodular Maximization with Dependent Items

In this paper, we study the stochastic submodular maximization problem with dependent items subject to packing constraints such as matroid and knapsack constraints. The input of our problem is a finite set of items, and each item is in a particular state from a set of possible states. After picking an item, we are able to observe its state. We assume a monotone and submodular utility function over items and states, and our objective is to select a group of items adaptively so as to maximize the expected utility. Previous studies on stochastic submodular maximization often assume that items' states are independent, however, this assumption may not hold in general. This motivates us to study the stochastic submodular maximization problem with dependent items. We first introduce the concept of \emph{degree of independence} to capture the degree to which one item's state is dependent on others'. Then we propose a non-adaptive policy that approximates the optimal adaptive policy within a factor of $α(1-e^{-\fracκ{2}+\fracκ{18m^2}}-\frac{κ+2}{3mκ})$ where the value of $α$ is depending on the type of constraints, e.g., $α=1$ for matroid constraint, $κ$ is the degree of independence, e.g., $κ=1$ for independent items, and $m$ is the number of items. We also analyze the adaptivity gap, i.e., the ratio of the values of best adaptive policy and best non-adaptive policy, of our problem with prefix-closed constraints.

cs.SI↗

arXiv2019

No Time to Observe: Adaptive Influence Maximization with Partial Feedback

Although influence maximization problem has been extensively studied over the past ten years, majority of existing work adopt one of the following models: \emph{full-feedback model} or \emph{zero-feedback model}. In the zero-feedback model, we have to commit the seed users all at once in advance, this strategy is also known as non-adaptive policy. In the full-feedback model, we select one seed at a time and wait until the diffusion completes, before selecting the next seed. Full-feedback model has better performance but potentially huge delay, zero-feedback model has zero delay but poorer performance since it does not utilize the observation that may be made during the seeding process. To fill the gap between these two models, we propose \emph{Partial-feedback Model}, which allows us to select a seed at any intermediate stage. We develop two novel greedy policies that, for the first time, achieve bounded approximation ratios under both uniform and non-uniform cost settings.

cs.SI↗

arXiv2018

Fine-Grained User Profiling for Personalized Task Matching in Mobile Crowdsensing

In mobile crowdsensing, finding the best match between tasks and users is crucial to ensure both the quality and effectiveness of a crowdsensing system. Existing works usually assume a centralized task assignment by the crowdsensing platform, without addressing the need of fine-grained personalized task matching. In this paper, we argue that it is essential to match tasks to users based on a careful characterization of both the users' preference and reliability. To that end, we propose a personalized task recommender system for mobile crowdsensing, which recommends tasks to users based on a recommendation score that jointly takes each user's preference and reliability into consideration. We first present a hybrid preference metric to characterize users' preference by exploiting their implicit feedback. Then, to profile users' reliability levels, we formalize the problem as a semi-supervised learning model, and propose an efficient block coordinate descent algorithm to solve the problem. For some tasks that lack users' historical information, we further propose a matrix factorization method to infer the users' reliability levels on those tasks. We conduct extensive experiments to evaluate the performance of our system, and the evaluation results demonstrate that our system can achieve superior performance to the benchmarks in both user profiling and personalized task recommendation.

cs.IR↗

arXiv2018

Stochastic Coupon Probing in Social Networks

In this paper, we study stochastic coupon probing problem in social networks. Assume there is a social network and a set of coupons. We can offer coupons to some users adaptively and those users who accept the offer will act as seeds and influence their friends in the social network. There are two constraints which are called the inner and outer constraints, respectively. The set of coupons redeemed by users must satisfy inner constraints, and the set of all probed users must satisfy outer constraints. One seeks to develop a coupon probing policy that achieves the maximum influence while satisfying both inner and outer constraints. Our main result is a constant approximation policy for the stochastic coupon probing problem for any monotone submodular utility function.

cs.SI↗