DSA210 / Models

A score needs a reference.

Every number below comes from held-out calendar years, 2019–2024. Higher R² is better; lower RMSE and MAE are better. Negative R² means the model underperforms a test-set-mean reference.

Consumer confidence

Four countries · 288 held-out observations

Model	N	RMSE	MAE	R²
Country mean	288	2.944	2.273	-0.886
Persistence	288	0.492	0.351	0.947
RF combined	288	2.915	2.300	-0.849
RF combined + lag	288	1.020	0.695	0.773
RF macro	288	2.931	2.317	-0.869
RF weather	288	2.942	2.249	-0.884
Ridge + weather PCA	288	2.999	2.285	-0.957

Retail spending

Three countries · 216 held-out observations · no UK retail

Model	N	RMSE	MAE	R²
Country mean	216	22.188	13.293	-0.516
Persistence	216	3.592	2.213	0.960
RF combined	216	13.724	9.139	0.420
RF combined + lag	216	10.376	5.693	0.668
RF macro	216	13.724	9.068	0.420
RF weather	216	22.180	13.201	-0.515
Ridge + weather PCA	216	16.042	10.297	0.207

Ablation is the attribution test

RF weather uses weather + calendar + country. RF macro uses macro + calendar + country. RF combined uses both. The same RF settings and test rows are used. RF combined + lag also receives the previous observed target. A high combined score alone is not evidence that sunshine explains that performance.

Can the model predict next-month direction?

0.571logistic ROC AUC

52.1%classification accuracy

46.1%training-prior baseline accuracy

284held-out labelled months

AUC alone does not establish useful trading or spending forecasts. Four terminal labels are excluded. Fixed logistic model, two training-only weather PCs, macro/calendar controls and country encoding. This is a descriptive validation result without a claimed significance test.

Country-level checks

Target	Country	Model	RMSE	R²
CCI	Germany	Persistence	0.284	0.954
CCI	Spain	Persistence	0.547	0.939
CCI	Turkey	Persistence	0.489	0.871
CCI	UK	Persistence	0.592	0.948
CCI	Germany	RF combined	1.928	-1.135
CCI	Spain	RF combined	3.077	-0.923
CCI	Turkey	RF combined	2.437	-2.194
CCI	UK	RF combined	3.855	-1.224
CCI	Germany	RF combined + lag	0.385	0.915
CCI	Spain	RF combined + lag	0.937	0.822
CCI	Turkey	RF combined + lag	1.163	0.272
CCI	UK	RF combined + lag	1.336	0.733
retail_index	Germany	Persistence	2.733	0.269
retail_index	Spain	Persistence	3.754	0.486
retail_index	Turkey	Persistence	4.140	0.979
retail_index	Germany	RF combined	6.904	-3.663
retail_index	Spain	RF combined	5.947	-0.289
retail_index	Turkey	RF combined	21.955	0.400
retail_index	Germany	RF combined + lag	3.290	-0.059
retail_index	Spain	RF combined + lag	3.545	0.542
retail_index	Turkey	RF combined + lag	17.308	0.627

Fold-level results

Every annual result is shown, including weak periods.
Target	Test year	Model	RMSE	R²
CCI	2019	Country mean	1.752	-0.076
CCI	2020	Country mean	3.357	-4.677
CCI	2021	Country mean	1.767	0.046
CCI	2022	Country mean	5.173	-5.442
CCI	2023	Country mean	2.604	-2.894
CCI	2024	Country mean	1.005	0.009
CCI	2019	Persistence	0.192	0.987
CCI	2020	Persistence	0.565	0.839
CCI	2021	Persistence	0.515	0.919
CCI	2022	Persistence	0.654	0.897
CCI	2023	Persistence	0.586	0.803
CCI	2024	Persistence	0.249	0.939
CCI	2019	RF combined	1.698	-0.011
CCI	2020	RF combined	3.276	-4.407
CCI	2021	RF combined	2.534	-0.963
CCI	2022	RF combined	4.767	-4.471
CCI	2023	RF combined	2.028	-1.362
CCI	2024	RF combined	2.023	-3.015
CCI	2019	RF combined + lag	0.924	0.701
CCI	2020	RF combined + lag	0.932	0.563
CCI	2021	RF combined + lag	0.836	0.787
CCI	2022	RF combined + lag	1.800	0.220
CCI	2023	RF combined + lag	0.678	0.736
CCI	2024	RF combined + lag	0.360	0.873
CCI	2019	RF macro	1.667	0.026
CCI	2020	RF macro	3.262	-4.360
CCI	2021	RF macro	2.521	-0.943
CCI	2022	RF macro	4.749	-4.431
CCI	2023	RF macro	2.063	-1.442
CCI	2024	RF macro	2.225	-3.857
CCI	2019	RF weather	1.751	-0.075
CCI	2020	RF weather	3.319	-4.549
CCI	2021	RF weather	1.844	-0.039
CCI	2022	RF weather	5.117	-5.304
CCI	2023	RF weather	2.680	-3.125
CCI	2024	RF weather	1.041	-0.064
CCI	2019	Ridge + weather PCA	1.563	0.144
CCI	2020	Ridge + weather PCA	3.433	-4.939
CCI	2021	Ridge + weather PCA	2.702	-1.230
CCI	2022	Ridge + weather PCA	5.022	-5.071
CCI	2023	Ridge + weather PCA	2.349	-2.167
CCI	2024	Ridge + weather PCA	1.297	-0.652
retail_index	2019	Country mean	4.742	0.663
retail_index	2020	Country mean	9.808	-0.174
retail_index	2021	Country mean	11.973	-8.451
retail_index	2022	Country mean	17.718	-4.162
retail_index	2023	Country mean	30.429	-1.502
retail_index	2024	Country mean	38.106	-0.935
retail_index	2019	Persistence	1.094	0.982
retail_index	2020	Persistence	6.635	0.463
retail_index	2021	Persistence	3.977	-0.043
retail_index	2022	Persistence	2.310	0.912
retail_index	2023	Persistence	2.853	0.978
retail_index	2024	Persistence	1.698	0.996
retail_index	2019	RF combined	4.632	0.678
retail_index	2020	RF combined	10.349	-0.307
retail_index	2021	RF combined	10.825	-6.726
retail_index	2022	RF combined	17.559	-4.070
retail_index	2023	RF combined	19.861	-0.066
retail_index	2024	RF combined	13.474	0.758
retail_index	2019	RF combined + lag	2.375	0.915
retail_index	2020	RF combined + lag	5.968	0.565
retail_index	2021	RF combined + lag	3.987	-0.048
retail_index	2022	RF combined + lag	7.774	0.006
retail_index	2023	RF combined + lag	19.190	0.005
retail_index	2024	RF combined + lag	12.653	0.787
retail_index	2019	RF macro	4.209	0.734
retail_index	2020	RF macro	10.329	-0.302
retail_index	2021	RF macro	10.652	-6.481
retail_index	2022	RF macro	17.645	-4.119
retail_index	2023	RF macro	20.018	-0.083
retail_index	2024	RF macro	13.421	0.760
retail_index	2019	RF weather	4.814	0.653
retail_index	2020	RF weather	9.796	-0.171
retail_index	2021	RF weather	11.834	-8.233
retail_index	2022	RF weather	17.573	-4.078
retail_index	2023	RF weather	30.305	-1.482
retail_index	2024	RF weather	38.281	-0.953
retail_index	2019	Ridge + weather PCA	5.329	0.574
retail_index	2020	Ridge + weather PCA	9.223	-0.038
retail_index	2021	Ridge + weather PCA	10.820	-6.719
retail_index	2022	Ridge + weather PCA	10.928	-0.964
retail_index	2023	Ridge + weather PCA	23.659	-0.513
retail_index	2024	Ridge + weather PCA	25.186	0.154