arXiv subjects

Kwan Hui Lim

Publications and source records attributed to Kwan Hui Lim.

46 records · Page 3Linked to original sources

arXiv2020

Understanding Public Sentiments, Opinions and Topics about COVID-19 using Twitter

The COVID-19 pandemic has caused widespread devastation throughout the world. In addition to the health and economical impacts, there is an enormous emotional toll associated with the constant stress of daily life with the numerous restrictions in place to combat the pandemic. To better understand the impact of COVID-19, we proposed a framework that utilizes public tweets to derive the sentiments, emotions and discussion topics of the general public in various regions and across multiple timeframes. Using this framework, we study and discuss various research questions relating to COVID-19, namely: (i) how sentiments/emotions change during the pandemic? (ii) how sentiments/emotions change in relation to global events? and (iii) what are the common topics discussed during the pandemic?

cs.SI↗

arXiv2020

Urban Heat Islands: Beating the Heat with Multi-Modal Spatial Analysis

In today's highly urbanized environment, the Urban Heat Island (UHI) phenomenon is increasingly prevalent where surface temperatures in urbanized areas are found to be much higher than surrounding rural areas. Excessive levels of heat stress leads to problems at various levels, ranging from the individual to the world. At the individual level, UHI could lead to the human body being unable to cope and break-down in terms of core functions. At the world level, UHI potentially contributes to global warming and adversely affects the environment. Using a multi-modal dataset comprising remote sensory imagery, geo-spatial data and population data, we proposed a framework for investigating how UHI is affected by a city's urban form characteristics through the use of statistical modelling. Using Singapore as a case study, we demonstrate the usefulness of this framework and discuss our main findings in understanding the effects of UHI and urban form characteristics.

cs.SI↗

arXiv2020

Urban Crowdsensing using Social Media: An Empirical Study on Transformer and Recurrent Neural Networks

An important aspect of urban planning is understanding crowd levels at various locations, which typically require the use of physical sensors. Such sensors are potentially costly and time consuming to implement on a large scale. To address this issue, we utilize publicly available social media datasets and use them as the basis for two urban sensing problems, namely event detection and crowd level prediction. One main contribution of this work is our collected dataset from Twitter and Flickr, alongside ground truth events. We demonstrate the usefulness of this dataset with two preliminary supervised learning approaches: firstly, a series of neural network models to determine if a social media post is related to an event and secondly a regression model using social media post counts to predict actual crowd levels. We discuss preliminary results from these tasks and highlight some challenges.

cs.SI↗

arXiv2020

Mining Influentials and their Bot Activities on Twitter Campaigns

Twitter is increasingly used for political, advertising and marketing campaigns, where the main aim is to influence users to support specific causes, individuals or groups. We propose a novel methodology for mining and analyzing Twitter campaigns, which includes: (i) collecting tweets and detecting topics relating to a campaign; (ii) mining important campaign topics using scientometrics measures; (iii) modelling user interests using hashtags and topical entropy; (iv) identifying influential users using an adapted PageRank score; and (v) various metrics and visualization techniques for identifying bot-like activities. While this methodology is generalizable to multiple campaign types, we demonstrate its effectiveness on the 2017 German federal election.

cs.SI↗

arXiv2020

Self-Evolving Adaptive Learning for Personalized Education

Primary and secondary education is a crucial stage to build a strong foundation before diving deep into specialised subjects in colleges and universities. To excel in the current education system, students are required to have a deep understanding of knowledge according to standardized curriculums and syllabus, and exam-related problem solving skills. In current school settings, this learning normally occurs in large classes of 30-40 students per class. Such a ``one size fits all'' approach may not be effective, as different students proceed on their learning in different ways and pace. To address this problem, we propose the Self-Evolving Adaptive Learning (SEAL) system for personalized education at scale.

physics.ed-ph↗

arXiv2020

EPIC30M: An Epidemics Corpus Of Over 30 Million Relevant Tweets

Since the start of COVID-19, several relevant corpora from various sources are presented in the literature that contain millions of data points. While these corpora are valuable in supporting many analyses on this specific pandemic, researchers require additional benchmark corpora that contain other epidemics to facilitate cross-epidemic pattern recognition and trend analysis tasks. During our other efforts on COVID-19 related work, we discover very little disease related corpora in the literature that are sizable and rich enough to support such cross-epidemic analysis tasks. In this paper, we present EPIC30M, a large-scale epidemic corpus that contains 30 millions micro-blog posts, i.e., tweets crawled from Twitter, from year 2006 to 2020. EPIC30M contains a subset of 26.2 millions tweets related to three general diseases, namely Ebola, Cholera and Swine Flu, and another subset of 4.7 millions tweets of six global epidemic outbreaks, including 2009 H1N1 Swine Flu, 2010 Haiti Cholera, 2012 Middle-East Respiratory Syndrome (MERS), 2013 West African Ebola, 2016 Yemen Cholera and 2018 Kivu Ebola. Furthermore, we explore and discuss the properties of the corpus with statistics of key terms and hashtags and trends analysis for each subset. Finally, we demonstrate the value and impact that EPIC30M could create through a discussion of multiple use cases of cross-epidemic research topics that attract growing interest in recent years. These use cases span multiple research areas, such as epidemiological modeling, pattern recognition, natural language understanding and economical modeling.

cs.SI↗

arXiv2020

Strategic and Crowd-Aware Itinerary Recommendation

There is a rapidly growing demand for itinerary planning in tourism but this task remains complex and difficult, especially when considering the need to optimize for queuing time and crowd levels for multiple users. This difficulty is further complicated by the large amount of parameters involved, i.e., attraction popularity, queuing time, walking time, operating hours, etc. Many recent works propose solutions based on the single-person perspective, but otherwise do not address real-world problems resulting from natural crowd behavior, such as the Selfish Routing problem, which describes the consequence of ineffective network and sub-optimal social outcome by leaving agents to decide freely. In this work, we propose the Strategic and Crowd-Aware Itinerary Recommendation (SCAIR) algorithm which optimizes social welfare in real-world situations. We formulate the strategy of route recommendation as Markov chains which enables our simulations to be carried out in poly-time. We then evaluate our proposed algorithm against various competitive and realistic baselines using a theme park dataset. Our simulation results highlight the existence of the Selfish Routing problem and show that SCAIR outperforms the baselines in handling this issue.

cs.AI↗

arXiv2020

CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding

Classification of crisis events, such as natural disasters, terrorist attacks and pandemics, is a crucial task to create early signals and inform relevant parties for spontaneous actions to reduce overall damage. Despite crisis such as natural disasters can be predicted by professional institutions, certain events are first signaled by civilians, such as the recent COVID-19 pandemics. Social media platforms such as Twitter often exposes firsthand signals on such crises through high volume information exchange over half a billion tweets posted daily. Prior works proposed various crisis embeddings and classification using conventional Machine Learning and Neural Network models. However, none of the works perform crisis embedding and classification using state of the art attention-based deep neural networks models, such as Transformers and document-level contextual embeddings. This work proposes CrisisBERT, an end-to-end transformer-based model for two crisis classification tasks, namely crisis detection and crisis recognition, which shows promising results across accuracy and f1 scores. The proposed model also demonstrates superior robustness over benchmark, as it shows marginal performance compromise while extending from 6 to 36 events with only 51.4% additional data points. We also proposed Crisis2Vec, an attention-based, document-level contextual embedding architecture for crisis embedding, which achieve better performance than conventional crisis embedding methods such as Word2Vec and GloVe. To the best of our knowledge, our works are first to propose using transformer-based crisis classification and document-level contextual crisis embedding in the literature.

cs.CL↗

arXiv2020

IPOD: An Industrial and Professional Occupations Dataset and its Applications to Occupational Data Mining and Analysis

Occupational data mining and analysis is an important task in understanding today's industry and job market. Various machine learning techniques are proposed and gradually deployed to improve companies' operations for upstream tasks, such as employee churn prediction, career trajectory modelling and automated interview. Job titles analysis and embedding, as the fundamental building blocks, are crucial upstream tasks to address these occupational data mining and analysis problems. In this work, we present the Industrial and Professional Occupations Dataset (IPOD), which consists of over 190,000 job titles crawled from over 56,000 profiles from Linkedin. We also illustrate the usefulness of IPOD by addressing two challenging upstream tasks, including: (i) proposing Title2vec, a contextual job title vector representation using a bidirectional Language Model (biLM) approach; and (ii) addressing the important occupational Named Entity Recognition problem using Conditional Random Fields (CRF) and bidirectional Long Short-Term Memory with CRF (LSTM-CRF). Both CRF and LSTM-CRF outperform human and baselines in both exact-match accuracy and F1 scores. The dataset and pre-trained embeddings are available at https://www.github.com/junhua/ipod.

cs.CL↗

arXiv2020

A Large-scale Industrial and Professional Occupation Dataset

There has been growing interest in utilizing occupational data mining and analysis. In today's job market, occupational data mining and analysis is growing in importance as it enables companies to predict employee turnover, model career trajectories, screen through resumes and perform other human resource tasks. A key requirement to facilitate these tasks is the need for an occupation-related dataset. However, most research use proprietary datasets or do not make their dataset publicly available, thus impeding development in this area. To solve this issue, we present the Industrial and Professional Occupation Dataset (IPOD), which comprises 192k job titles belonging to 56k LinkedIn users. In addition to making IPOD publicly available, we also: (i) manually annotate each job title with its associated level of seniority, domain of work and location; and (ii) provide embedding for job titles and discuss various use cases. This dataset is publicly available at https://github.com/junhua/ipod.

cs.SI↗