Predicting Social Inequality in Poland Using Price Dispersion on the Real Estate Market

The real estate market reflects the social and economic changes that take place in the economy on a national and global scale. The real estate market is constantly changing, resulting from the relationships and dependencies between its participants, which include primarily investors and lenders, as well as developers, renters, tenants, etc. as discussed by Banai et al. (2017). The real estate market is quite specific, since the price is determined not only by the features directly characterizing a given property. It is also affected by a variety of factors related to the economic, legal and social environment (Beltratti & Morana, 2010). On the one hand, the real estate market is strongly influenced by the processes in the economics, but on the other hand, it, as a one of the fundamental markets in the economy, has also influence on the macroeconomic situation (Meen, 1995).

In recent decades, in most of the OECD countries, housing prices have increased remarkably. At the same time, in those countries, a simultaneous growth in income inequality can be observed. Numerous studies confirm that the rise in prices and income inequality is not a coincidence (Goda et al., 2021). The theoretical models suggest two possible mechanisms that link income inequality to house prices. On the consumption side, with increasing inequality, households are more likely to pay a higher price for a property. Secondly, premises as an element part of the financial market are treated as an investment good, especially for high-income households. Moreover, expectations about future property prices based on trends can contribute to the formation of speculative bubbles. Furthermore, the rising prices of housing can make it unaffordable. It especially concerns the most productive urban areas and low-income households Goda et al. (2021).

Research indicates that the access to the housing market is more restricted for low-income householders, especially in European countries with higher income inequality (Dewilde & Lancee, 2013). Considering potential negative social effects, researchers have paid more attention to the problem of income inequality in the context of its influence on the housing market.

Income inequality is not a new phenomenon, and it exists in societies on every stage of socio-economic development. However, income inequalities are a serious challenge for the socio-economic coherency. The increase in income inequality is also a threat to sustainable development (Leszczyńska, 2015). There are two main streams of research on income inequality. The first focuses on causal analysis and seeks to explain income disparities through economic theory. Within this area, there is also research on the relationship between income inequality and selected social problems such as unemployment, crime or early school-leaving (Lange, 2015). The second area of research on income inequality focuses on its measurement (Leszczyńska, 2015). In this article, we focus on the latter stream of research.

Income inequality is only one side of the general concept of social inequality. Social inequalities have become a significant challenge facing many countries in the world. It is viewed as a serious issue not only in the public debate but also in academic research. Studies concerning especially income inequality have significantly intensified after The Global Financial Crisis of 2008–2009 (Peterson, 2017). Social inequality is a complex concept that can be analysed from many different perspectives. It is not straightforward to measure quality of life, wealth, and inequalities. However, there is a need to quantify them and to identify their key factors. Such measures are often used as a benchmark in the context of comparisons between regions or countries. Usually, social inequalities are often analysed through the prism of economic or income discrimination (Minkner et al., 2019, p. 179).

The main reason for using price dispersion measures in the real estate market is that, as shown above, they can be used as an additional, valuable, and informative dimension of the social inequality assessment. It is empirically confirmed by Goda et al. (2021) who show the positive dependence between income inequality measures, such as the Gini coefficient and income variance, and house prices in OECD countries. Therefore, in this context, it is noteworthy to acquire a comprehensive understanding of the formation of the price dispersion. In our paper, we propose to measure social inequality in Polish macroregions through measuring price dispersion existing on the housing market. For this purpose, we employ four quantile-based dispersion measures of the property price distribution and make predictions based on longitudinal historical data and a statistical model that explains price fluctuations across time and space. Therefore, the goal addressed in the article is to predict future price dispersion measures in Polish macroregions. It allows us to identify macroregions with the highest and lowest levels of spatial variation in the transaction values of the housing market, not only for historical data but also, based on forecasts, for future periods. We propose a forecasting procedure based on price modelling using individual data (not the values of the metrics), the mixed model allowing to include not only explanatory variables but also spatial or temporal correlation, and the class of PLUG-IN predictors.

Our additional goal is to estimate the ex ante prediction accuracy. It is due to the fact, that one issue is the forecasted value of a dispersion measure, but another, equally important, is its reliability. It is assessed based on the differences between unknown future real values of the considered measure and its forecasts. From this point of view, the assessment of the ex post accuracy of forecasts, discussed for example by Papastamos et al. (2015) in the context of the real estate market, is not sufficient. To estimate the prediction accuracy, we are not going to limit to the classic measure known as the prediction Root Mean Squared Error. We will also use our proposal of an ex ante prediction accuracy measure called the Quantile of Absolute Prediction Error (QAPE). It provides a detailed description of the relationship between the value of the prediction error and the probability of its occurrence. From our perspective, it can be highly advantageous in the context of predicting price dispersion on the real estate market, displaying not only the forecasted dispersion but also the distribution of its prediction errors, thereby facilitating the discussion of diverse market scenarios. For forecasts computation and estimation of prediction accuracy, we use the prepared R package qape (Wolny-Dominiak & Ża̧dło, 2023).

Finally, it is worth noting that the proposed approach in measuring social inequality solves the problem of incomplete income data in publicly available databases. In Poland, the public administration institution accountable for public statistics is the Central Statistical Office. Through the Local Data Bank, it provides a diverse range of data on the economy, society, and environment across various dimensions, including territorial breakdowns. Furthermore, the published information includes data on wages and salaries, which can be used to assess the extent of social inequalities in different regions. The Local Data Bank offers access to two sources of wage and salary data. The first source provides information on monthly average wages, while the second source provides information on average wages over the course of a year. However, there are certain limitations to using these resources. The first source only covers entities in the enterprise sector employing at least 10 people, thereby making it impossible to assess inequalities on a national scale. The second source, which provides annual data, includes wages not only from enterprises employing at least 10 people but also from public sector entities regardless of the number of employees. However, a limitation of this second data source is that the data is published with a significant delay, and furthermore, the data is available at the voivodeship level. This prevents the examination of disparities in smaller administrative units or in other geographical subdivisions beyond those defined by territorial boundaries. The limited utility of the data on the income of the populations in the publicly accessible databases has provided an additional argument for considering the concept of social inequality analysis from a different perspective—using information on price volatility in the real estate market.

The layout of this paper is as follows. In Sect. 2 the considered quantile-based price dispersion measures on the housing market are defined. Section 3 provides a description of the methodology proposed in the context of predicting dispersion measures. Firstly, the formal description of the considered prediction problem is presented, including the definition of predicted price dispersion measures and the considered mixed model. Secondly, the class of the predictors used to forecast the population and subpopulation housing price dispersion is presented. Finally, the considered prediction accuracy measures, together with the residual bootstrap algorithm used to estimate them, are introduced. In Sect. 4, the results of the prediction for the Polish real estate market are described. The conclusion of this study is presented in Sect. 5.

It has been recognized that traditional measures of income inequality do not capture the full picture of how wealth and income are distributed within a society. The importance of adopting a multidimensional approach to addressing social inequalities is emphasized (Parente, 2019). Particularly, the measurement of housing price dispersion can provide valuable insights into the distribution of wealth and income. Price dispersion in the housing market has recently received significant attention, with increasing concern over its potential impact on social and economic outcomes. Several measures have been developed to quantify and evaluate housing price dispersion, with the aim of informing policy and practice. The typical approach for assessing price dispersion involves the computation of either the coefficient of variation or the standard deviation (Leung et al., 2005). Our approach involves using quantile-based indicators to measure price dispersion, similarly to Burger and van Beuningen (2020) and Josa and Aguado (2020). Lach (2002) also employed quantile indices to examine the issue of price dispersion.

Let \(q_{\tau }\) be the \(\tau \)-quantile, where \(\tau \in (0,1)\). The first considered dispersion measure is the widely used Interquartile Range defined as follows:where \(q_{0.75}\) and \(q_{0.25}\) are quartiles, respectively, the third and the first. The interquartile range characterise the middle 50% of the distribution. If the interquartile range is substantial, it implies that the middle 50% of observations are widely dispersed.

The Quartile Dispersion Ratio (or interquartile ratio) is defined as follows:The Decile Dispersion Ratio (or inter-decile ratio) has the following form:where \(q_{0.9}\), \(q_{0.1}\) are deciles, respectively, the ninth and the first. It is also possible to construct more sophisticated indices, which are ratios or differences between quantiles of other orders. Price distribution analysis using these metrics allows a comparison between \(\tau \) % of the highest priced units and \(\tau \) % of the lowest priced units. Dispersion ratios focus on the tails of the distribution, in contrast to the Gini coefficient, which is also often used to measure inequality. The dispersion ratio appears to outperform Gini, which is oversensitive to changes in the middle part of the distribution of the variable (Jȩdrzejczak & Pekasiewicz, 2018).

In the analysis of housing price dispersion, it is common to apply also the coefficient of variation (CV), i.e. standard deviation of the variable relative to its mean (Chiang et al., 2019). Due to the lack of robustness to outlier measures such as the mean and standard deviation, we used the Quartile Coefficient of Dispersion, which is promoted as an alternative to the CV:In order to predict the measures (1)–(4) for Polish macroregions as well as to estimate the prediction accuracy we propose the special procedure. It covers the following steps: The detailed description of the methodology, including the considered assumptions, is presented in Sect. 3. The application of this four-step procedure is presented in Sects. 4.1–4.4.

We consider a longitudinal population dataset in m periods and the problem of prediction for future periods \(m + 1, m + 2, \dots , M\). Values of the variable of interest for observed cases are assumed to be realizations of the random vector \({\textbf {Y}}_s = \begin{bmatrix} Y_{11}&\dots&Y_{it}&\dots&Y_{Nm} \end{bmatrix}^T\), where N is the population size. Random variables for future periods form the following vector \({\textbf {Y}}_r = \begin{bmatrix} Y_{1 m+1}&\dots&Y_{NM} \end{bmatrix}^T\). Let \(\textbf{Y}= \begin{bmatrix} \textbf{Y}_s^T&\textbf{Y}_r^T \end{bmatrix}^T\). We also assume that there are available two known matrices of auxiliary variables which form two matrices \(\textbf{X}= \begin{bmatrix} \textbf{X}_s^T&\textbf{X}_r^T \end{bmatrix}^T\), and \(\textbf{Z}= \begin{bmatrix} \textbf{Z}_s^T&\textbf{Z}_r^T \end{bmatrix}^T\), where matrices \(\textbf{X}_s\), \(\textbf{X}_r\), \(\textbf{Z}_s\), \(\textbf{Z}_r\), all known, are of sizes \(Nm \times p\), \(N(M-m) \times p\), \(Nm \times h\), and \(N(M-m) \times h\), respectively. Since quantiles are used in our price dispersion analysis, it should be noted that in the prediction approach, the quantiles are defined via the finite population distribution function \(F_{NT}(h)\). Formally, the distribution function of the variable of interest for the future period T, evaluated at some point h and is defined by Valliant et al. (2000) p. 378 as follows:where I(.) is the indicator function.

Let us consider the problem of prediction of the finite population \(\tau \)-quantile of the response variables for a future period \(T \in \{m+1, m+2, \dots , M\}\), which forms the population vector \(\textbf{Y}_T\). As presented by Valliant et al. (2000) p. 378, it is defined as follows:It should be underlined, that in the prediction approach both (5) and (6) are functions of random variables and hence, the predicted dispersion measures, given by (1)–(4), where quantiles are given by (6), are random as well.

In order to predict the price dispersion measures, let us generalize the problem of quantile prediction for any given function (7) of the population vector \(\textbf{Y}_T\) of the response variable for future period \(T \in \{m+1, m+2, \dots , M\}\):The considered price dispersion measures (1)-(4) are special cases of (7), where quantiles are defined by (6). For example, (1) in future period T can be written as \(\theta (\textbf{Y}_T)=IQR(\textbf{Y}_T) = q_{0.75}(\textbf{Y}_T) - q_{0.25}(\textbf{Y}_T)\). It means that our objective is not to estimate the parameters of the distribution in the future period, but to predict a function of random variables in this period. It is called the prediction approach, and it is considered, inter alia, by Valliant et al. (2000).

Lest us consider the vector of the variable of interest \(\textbf{Y}\) after any transformation function f(.), such that its inverse exists. Let the transformed vector be denoted by \(\textbf{Y}^{(trans)}=f(\textbf{Y})\). We assume that the longitudinal data obeys the assumptions of the following mixed model (e.g. Rao & Molina, 2015, p. 98):where vectors of random effects \(\textbf{v}\) and random components \(\textbf{e}\) are assumed to be independent. Hence, the vector of parameters in (8) can be defined as \(\varvec{\psi } = [\varvec{\beta }^T, \varvec{\delta }^T]^T\), where \(\varvec{\beta }\) is a vector of fixed effects of size \(p \times 1\) and \(\varvec{\delta }\) is a vector of variance components. To obtain an estimator \(\hat{\varvec{\psi }}\) of \(\varvec{\psi }\) we use the Restricted Maximum Likelihood Method (REML), which can be used for non-normal data as shown by Jiang (1996).

Generally, to predict (7) different classes of predictors can be used including Empirical Best Linear Unbiased Predictors – EBLUPs (see e.g. Henderson, 1950 and Royall, 1976), PLUG-IN predictors (see e.g. Boubeta et al. 2016, Chwila & Ża̧dło 2022, Hobza & Morales, 2016) and Empirical Best Predictors—EBPs (see e.g. Molina & Rao, 2010). Because EBLUPs can be used only to predict linear combinations of \(\textbf{Y}\) (here: \(\textbf{Y}_T\)), they cannot be used to predict the considered dispersion measures. The EBPs can be used to predict any given function of \(\textbf{Y}\) (including \(\textbf{Y}_T\)), but they require strong assumptions regarding the form of distribution of the variable of interest. That is why, the plug-in predictor of \(\theta \) is considered. It does not require additional assumptions except (8), but in general, it is not optimal. However, it was shown by Monte Carlo simulation that its accuracy is similar or even higher comparing with the Empirical (or Estimated) Best Predictors, where the best predictors minimize the prediction mean squared errors (see Boubeta et al., 2016, Chwila & Ża̧dło, 2022, Hobza & Morales, 2016). The plug-in predictor of (7) is defined as:where \(\mathbf {\hat{Y}}_T^{(trans)}\) is the vector of predicted for period T values of the transformed variable of interest under the assumed model and \(f^{-1}(.)\) is the inverse function of f(.) presented in (8).

To assess the prediction accuracy, we use two measures. The first one is the Root Mean Squared Error:where \(\theta \) and \(\hat{\theta }\) are given by (7) and (9), respectively, and \(U=\hat{\theta }-\theta \) is the prediction error.

The squared prediction error distribution is typically characterised by a strong positive asymmetry. Because the mean is not advised as an appropriate measure of the central tendency in such distributions, the quantile-based prediction accuracy measures QAPE are also calculated, see Ża̧dło (2013), Wolny-Dominiak and Ża̧dło (2022). The measure QAPE represents the pth quantile of the absolute prediction error |U| and it is given by the formula:This measure informs that at least p100% of observed absolute prediction errors are smaller than or equal to \(QAPE_p(\hat{\theta })\) while at least \((1-p)100\%\) of them are higher than or equal to \(QAPE_p(\hat{\theta })\). Quantiles describe the relationship between the error value and the probability of its occurrence. With the QAPE, it is possible to make a comprehensive description of the distribution of prediction errors instead of using the average (shown by the RMSE). Furthermore, the MSE is the mean of positively (typically very strongly) skewed squared prediction errors, whereas the mean should not be used to assess the central tendency of positively skewed distributions.

To estimate (10) and (11) we use the following estimatorsandwhere \(u^{*(b)}\) for \(b=1,2,...B\) are residual bootstrap prediction errors obtained using the algorithm presented in the Appendix A, B is the number of bootstrap iterations and \(q_p(.)\) is the quantile of order p.

We apply the methodology presented in the previous section for prediction of price dispersion measures based on real data from the real estate market. It will include: the description of the dataset and potential independent variables explaining price changes, the form of the chosen model, forecasts and estimated prediction accuracy measures. We use the prepared R package qape (Wolny-Dominiak & Ża̧dło, 2023) to perform the calculations.

Our findings have the potential to provide valuable insights to a diverse range of stakeholders, including policymakers, investors, and households. Policymakers can identify regions that exhibit significant dispersion in real estate prices. It is possible to improve housing affordability by implementing targeted policies, such as incentivizing affordable housing development or implementing income-based housing assistance programs. Investors can make informed decisions regarding the optimal location and investment horizon. By examining variations in price dispersion, both policymakers and developers can aim for more balanced regional development. The prediction of the price dispersion allows to detect speculative bubbles. The bubbles, studied, i.e. by Ohnishi et al. (2013), are not defined as a sharp rise in prices but as an increase in price dispersion. Moreover, as shown in the study presented by D’Ambrosio et al. (2020), real estate is the main factor of the permanent wealth. According to the authors’ recommendations, the accumulation of wealth focusing on real estate should be one of the aims of public policies concentrating on increasing life satisfaction and reducing social inequalities.

In our research, the new approach of predicting price dispersion in the real estate market is proposed. It is applied to Polish macroregions giving the additional dimension of social inequality assessment. We propose to use the plug-in predictor under the longitudinal mixed model. What is more, we use an innovative approach to evaluate the reliability of predicted dispersion measures. The analysis of prediction accuracy includes the estimation of not only the widely known measure RMSE, but also the QAPE. The second accuracy measure seems more appropriate to analyse the distribution of prediction errors, especially when the prediction errors distribution is skewed.

However, our study also has some limitations, such as data availability and the reliance on certain assumptions. The results of our research were obtained based on aggregated housing market data at the county level. It could be possible to derive more reliable conclusions by utilizing individual-level data on the housing market, which regrettably is not accessible in publicly accessible databases. Further research could explore the use of data at a lower level of aggregation and infer the degree of dispersion in other spatial or social dimensions.