arXiv subjects

Maciej Beręsewicz

Publications and source records attributed to Maciej Beręsewicz.

15 recordsLinked to original sources

arXiv2026

Data integration of non-probability and probability samples with deterministic predictive mean matching

We study deterministic predictive mean matching mass imputation estimators to integrate data from probability and non-probability samples. We consider two approaches: predicted-to-predicted (PMM~A) and predicted-to-observed (PMM~B) matching. We prove the consistency of mean estimators, derive a variance decomposition, and propose estimators of variance. We establish consistency of the PMM~A estimator under model misspecification and underline key differences from the nearest neighbour method. Our PMM~B approach can be employed with non-parametric regression techniques, such as kernel regression, and the analytical expression for variance applies to nearest neighbour matching for non-probability samples. Extensive simulation studies compare properties of the proposed estimators with existing alternatives and examine the effects of model misspecification. The paper concludes with an empirical study on the integration of job vacancy survey and vacancies submitted to public employment offices (admin and online data). Open-source software is available.

stat.ME↗

arXiv2026

BlockingPy: approximate nearest neighbours for blocking of records for entity resolution

Entity resolution (probabilistic record linkage, deduplication) is a key step in scientific analysis and data science pipelines involving multiple data sources. The objective of entity resolution is to link records without common unique identifiers that refer to the same entity (e.g., person, company). However, without identifiers, researchers need to specify which records to compare in order to calculate matching probability and reduce computational complexity. One solution is to deterministically block records based on some common variables, such as names, dates of birth or sex or use phonetic algorithms. However, this approach assumes that these variables are free of errors and completely observed, which is often not the case. To address this challenge, we have developed a Python package, BlockingPy, which uses blocking using modern approximate nearest neighbour search and graph algorithms to reduce the number of comparisons. The package supports both CPU and GPU execution. In this paper, we present the design of the package, its functionalities and two case studies related to official statistics. The presented software will be useful for researchers interested in linking data from various sources.

stat.AP↗

arXiv2025

nonprobsvy -- An R package for modern methods for non-probability surveys

The following paper presents nonprobsvy -- an R package for inference based on non-probability samples. The package implements various approaches that can be categorized into three groups: prediction-based approach, inverse probability weighting and doubly robust approach. In the package, we assume the existence of either population-level data or probability-based population information and leverage the survey package for inference. The package implements both analytical and bootstrap variance estimation for the proposed estimators. In the paper we present the theory behind the package, its functionalities and case study that showcases the usage of the package. The package is aimed at scientists and researchers who would like to use non-probability samples (e.g.big data, opt-in web panels, social media) to accurately estimate population characteristics.

stat.ME↗

arXiv2025

Multilingual hierarchical classification of job advertisements for job vacancy statistics

The goal of this paper is to develop a multilingual classifier and conditional probability estimator of occupation codes for online job advertisements in accordance with the International Standard Classification of Occupations (ISCO) extended with the Polish Classification of Occupations and Specializations (KZiS), which is analogous to the European Classification of Occupations. In this paper, we utilise a range of data sources, including a novel one, namely the Central Job Offers Database, which is a register of all vacancies submitted to Public Employment Offices. Their staff members code the vacancies according to the ISCO and KZiS. A hierarchical multi-class classifier has been developed based on the transformer architecture. The classifier begins by encoding the jobs found in advertisements to the widest 1-digit occupational group, and then narrows the assignment to a 6-digit occupation code. We show that incorporation of the hierarchical structure of occupations improves prediction accuracy by 1-2 percentage points, particularly for the hand-coded online job advertisements. Finally, a bilingual (Polish and English) and multilingual (24 languages) model is developed based on data translated using closed and open-source software. The open-source software is provided for the benefit of the official statistics community, with a particular focus on international comparability.

stat.AP↗

arXiv2025

Quantile balancing inverse probability weighting for non-probability samples

The use of non-probability data sources for statistical purposes and for official statistics has become increasingly popular in recent years. However, statistical inference based on non-probability samples is made more difficult by nature of their biasedness and lack of representativity. In this paper we propose quantile balancing inverse probability weighting estimator (QBIPW) for non-probability samples. We apply the idea of Harms and Duchesne (2006) allowing the use of quantile information in the estimation process to reproduce known totals and the distribution of auxiliary variables. We discuss the estimation of the QBIPW probabilities and its variance. Our simulation study has demonstrated that the proposed estimators are robust against model mis-specification and, as a result, help to reduce bias and mean squared error. Finally, we applied the proposed methods to estimate the share of job vacancies aimed at Ukrainian workers in Poland using an integrated set of administrative and survey data about job vacancies.

stat.ME↗

arXiv2025

singleRcapture: An R Package for Single-Source Capture-Recapture Models

Population size estimation is a major challenge in official statistics, social sciences, and natural sciences. The problem can be tackled by applying capture-recapture methods, which vary depending on the number of sources used, particularly on whether a single or multiple sources are involved. This paper focuses on the first group of methods and introduces a novel R package: singleRcapture. The package implements state-of-the-art single-source capture-recapture (SSCR) models (e.g.~zero-truncated one-inflated regression) together with new developments proposed by the authors, and provides a user-friendly application programming interface (API). This self-contained package can be used to produce point estimates and their variance and implements several bootstrap variance estimators or diagnostics to assess quality and conduct sensitivity analysis. It is intended for users interested in estimating the size of populations, particularly those that are difficult to reach or measure, for which information is available only from one source and dual/multiple system estimation is not applicable. Our package serves to bridge a significant gap, as the SSCR methods are either not available at all or are only partially implemented in existing R packages and other open-source software.

stat.AP↗

arXiv2024

Survey calibration for causal inference: a simple method to balance covariate distributions

This paper proposes a~simple, yet powerful, method for balancing distributions of covariates for causal inference based on observational studies. The method makes it possible to balance an arbitrary number of quantiles (e.g., medians, quartiles, or deciles) together with means if necessary. The proposed approach is based on the theory of calibration estimators (Deville and Särndal 1992), in particular, calibration estimators for quantiles, proposed by Harms and Duchesne (2006). The method does not require numerical integration, kernel density estimation or assumptions about the distributions. Valid estimates can be obtained by drawing on existing asymptotic theory. An~illustrative example of the proposed approach is presented for the entropy balancing method and the covariate balancing propensity score method. Results of a~simulation study indicate that the method efficiently estimates average treatment effects on the treated (ATT), the average treatment effect (ATE), the quantile treatment effect on the treated (QTT) and the quantile treatment effect (QTE), especially in the presence of non-linearity and mis-specification of the models. The proposed approach can be further generalized to other designs (e.g. multi-category, continuous) or methods (e.g. synthetic control method). An open source software implementing proposed methods is available.

stat.ME↗

arXiv2023

A note on joint calibration estimators for totals and quantiles

In this paper, we combine calibration for population totals proposed by Deville and Särndal (1992) with calibration for population quantiles introduced by Harms and Duchesne (2006). We also extend the pseudo-empirical likelihood method proposed by Chen, Sitter, and Wu (2002). This approach extends the calibration equations for totals by adding relevant constraints on quantiles of continuous variables observed in the data. The proposed approach can be easily applied to handle non-response and data integration problems, and results in a single vector of weights. Furthermore, it is a multipurpose solution, i.e. it makes it possible to improve estimates of means, totals and quantiles for a variable of interest in one step. In a limited simulation study, we compare the proposed joint approach with standard calibration, calibration using empirical likelihood and the correctly specified inverse probability weighting estimator. Open source software implementing the proposed method is available.

stat.ME↗

arXiv2021

COVID-19 and the gig economy in Poland

We use a dataset covering nearly the entire target population based on passively collected data from smartphones to measure the impact of the first COVID-19 wave on the gig economy in Poland. In particular, we focus on transportation (Uber, Bolt) and delivery (Wolt, Takeaway, Glover, DeliGoo) apps, which make it possible to distinguish between the demand and supply part of this market. Based on Bayesian structural time-series models, we estimate the causal impact of the first COVID-19 wave on the number of active drivers and couriers. We show a significant relative increase for Wolt and Glover (15% and 24%) and a slight relative decrease for Uber and Bolt (-3% and -7%) in comparison to a counterfactual control. The change for Uber and Bolt can be partially explained by the prospect of a new law (the so-called Uber Lex), which was already announced in 2019 and is intended to regulate the work of platform drivers.

econ.GN↗

arXiv2021

The gig economy in Poland: evidence based on mobile big data

In this article we address the question of how to measure the size and characteristics of the platform economy. We propose a~different, to sample surveys, approach based on smartphone data, which are passively collected through programmatic systems as part of online marketing. In particular, in our study we focus on two types of services: food delivery (Bolt Courier, Takeaway, Glover, Wolt and transport services (Bolt Driver, Free Now, iTaxi and Uber). Our results show that the platform economy in Poland is growing. In particular, with respect to food delivery and transportation services performed by means of applications, we observed a growing trend between January 2018 and December 2020. Taking into account the demographic structure of apps users, our results confirm findings from past studies: the majority of platform workers are young men but the age structure of app users is different for each of the two categories of services. Another surprising finding is that foreigners do not account for the majority of gig workers in Poland. When the number of platform workers is compared with corresponding working populations, the estimated share of active app users accounts for about 0.5-2% of working populations in 9 largest Polish cities.

econ.GN↗

arXiv2021

Estimating the number of entities with vacancies using administrative and online data

In this article we describe a study aimed at estimating job vacancy statistics, in particular the number of entities with at least one vacancy. To achieve this goal, we propose an alternative approach to the methodology exploiting survey data, which is based solely on data from administrative registers and online sources and relies on dual system estimation (DSE). As these sources do not cover the whole reference population and the number of units appearing in all datasets is small, we have developed a DSE approach for negatively dependent sources based on a recent work by Chatterjee and Bhuyan (2020). To achieve the main goal we conducted a thorough data cleaning procedure in order to remove out-of-scope units, identify entities from the target population, and link them by identifiers to minimize linkage errors. We verified the effectiveness and sensitivity of the proposed estimator in simulation studies. From a practical point of view, our results show that the current vacancy survey in Poland underestimates the number of entities with at least one vacancy by about 10-15%. The main reasons for this discrepancy are non-sampling errors due to non-response and under-reporting, which is identified by comparing survey data with administrative data.

stat.AP↗

arXiv2020

Estimation of the number of irregular foreigners in Poland using non-linear count regression models

Population size estimation requires access to unit-level data in order to correctly apply capture-recapture methods. Unfortunately, for reasons of confidentiality access to such data may be limited. To overcome this issue we apply and extend the hierarchical Poisson-Gamma model proposed by Zhang (2008), which initially was used to estimate the number of irregular foreigners in Norway. The model is an alternative to the current capture-recapture approach as it does not require linking multiple sources and is solely based on aggregated administrative data that include (1) the number of apprehended irregular foreigners, (2) the number of foreigners who faced criminal charges and (3) the number of foreigners registered in the central population register. The model explicitly assumes a relationship between the unauthorized and registered population, which is motivated by the interconnection between these two groups. This makes the estimation conditionally dependent on the size of regular population, provides interpretation with analogy to registered population and makes the estimated parameter more stable over time. In this paper, we modify the original idea to allow for covariates and flexible count distributions in order to estimate the number of irregular foreigners in Poland in 2019. We also propose a parametric bootstrap for estimating standard errors of estimates. Based on the extended model we conclude that in as of 31.03.2019 and 30.09.2019 around 15,000 and 20,000 foreigners and were residing in Poland without valid permits. This means that those apprehended by the Polish Border Guard account for around 15-20% of the total.

stat.AP↗

arXiv2020

Scanner data in inflation measurement: from raw data to price indices

Scanner data offer new opportunities for CPI or HICP calculation. They can be obtained from a~wide variety of~retailers (supermarkets, home electronics, Internet shops, etc.) and provide information at the level of~the barcode. One of~advantages of~using scanner data is the fact that they contain complete transaction information, i.e. prices and quantities for every sold item. To use scanner data, it must be carefully processed. After clearing data and unifying product names, products should be carefully classified (e.g. into COICOP 5 or below), matched, filtered and aggregated. These procedures often require creating new IT or writing custom scripts (R, Python, Mathematica, SAS, others). One of~new challenges connected with scanner data is the appropriate choice of~the index formula. In this article we present a~proposal for the implementation of~individual stages of~handling scanner data. We also point out potential problems during scanner data processing and their solutions. Finally, we compare a~large number of~price index methods based on real scanner datasets and we verify their sensitivity on adopted data filtering and aggregating methods.

stat.AP↗

arXiv2019

Enhancing the Demand for Labour survey by including skills from online job advertisements using model-assisted calibration

In the article we describe an enhancement to the Demand for Labour (DL) survey conducted by Statistics Poland, which involves the inclusion of skills obtained from online job advertisements. The main goal is to provide estimates of the demand for skills (competences), which is missing in the DL survey. To achieve this, we apply a data integration approach combining traditional calibration with the LASSO-assisted approach to correct representation error in the online data. Faced with the lack of access to unit-level data from the DL survey, we use estimated population totals and propose a~bootstrap approach that accounts for the uncertainty of totals reported by Statistics Poland. We show that the calibration estimator assisted with LASSO outperforms traditional calibration in terms of standard errors and reduces representation bias in skills observed in online job ads. Our empirical results show that online data significantly overestimate interpersonal, managerial and self-organization skills while underestimating technical and physical skills. This is mainly due to the under-representation of occupations categorised as Craft and Related Trades Workers and Plant and Machine Operators and Assemblers.

econ.GN↗

arXiv2019

Estimation of the size of informal employment based on administrative records with non-ignorable selection mechanism

In this study we used company level administrative data from the National Labour Inspectorate and The Polish Social Insurance Institution in order to estimate the prevalence of informal employment in Poland. Since the selection mechanism is non-ignorable we employed a generalization of Heckman's sample selection model assuming non-Gaussian correlation of errors and clustering by incorporation of random effects. We found that 5.7% (4.6%, 7.1%; 95% CI) of registered enterprises in Poland, to some extent, take advantage of the informal labour force. Our study exemplifies a new approach to measuring informal employment, which can be implemented in other countries. It also contributes to the existing literature by providing, to the best of our knowledge, the first estimates of informal employment at the level of companies based solely on administrative data.

stat.AP↗