Data Processing Pipeline
-
1
Collect
Fetch weather, CCI, retail and macroeconomic data from public sources.
-
2
Merge
Join sources by country and year_month into a 480-row panel.
-
3
Clean
Keep target-specific missingness explicit instead of hiding source gaps.
-
4
Engineer
Add COVID dummy, cyclical month features, inflation log and weather PCA in notebooks.
-
5
Evaluate
Use hypothesis tests, regression, tree ensembles, classification and clustering.
Missing Value Diagnostics
| Column | Missing Values | Missing Percent |
|---|---|---|
| retail_index | 120 | 25.00 |
| unemployment_rate | 51 | 10.62 |
| inflation_yoy | 49 | 10.21 |
Cleaning Decision
Retail has 120 missing rows because UK retail observations are unavailable in the merged target. Retail models use available Germany, Spain and Turkey observations; CCI analysis keeps all four countries.
Outlier Diagnostics
| Column | IQR Outliers |
|---|---|
| covid_dummy | 112 |
| inflation_yoy | 42 |
| CCI | 29 |
| retail_index | 27 |
| precipitation_sum | 10 |
| rain_sum | 9 |
| precipitation_hours | 2 |
| sunshine_hours | 0 |
| temperature_2m_min | 0 |
| daylight_hours | 0 |
| temperature_2m_mean | 0 |
| temperature_2m_max | 0 |
| unemployment_rate | 0 |
| month_num | 0 |
| month_sin | 0 |
| month_cos | 0 |
| year | 0 |
Feature Engineering
covid_dummy
Encodes March 2020 to June 2022 as a structural shock period.
month_sin / month_cos
Captures seasonality while keeping December and January close.
inflation_log
Reduces high-inflation leverage while preserving information.
weather_PC1 / weather_PC2
Compresses correlated weather variables into orthogonal components.
Feature Engineering and Dimensionality Reduction Figures
