Here is what I got from running 10000 iterations....
see http://www.cs.ubc.ca/~poole/cs532/2017/as1/triv_learn2.py
Sum of Squares Error
Method |
n=1 |
n=2 |
n=3 |
n=4 |
n=5 |
n=10 |
n=20 |
n=100 |
n=1000
|
Predict 0.5 |
25 |
25 |
25 |
25 |
25 |
25 |
25 |
25 |
25
|
empirical ave |
33.359 |
24.986 |
22.261 |
20.830 |
20.0244 |
18.364 |
17.545 |
16.797 |
16.718
|
median |
33.239 |
25.050 |
29.861 |
24.975 |
28.669 |
24.919 |
25.091 |
24.960 |
24.972
|
thresholded ave |
33.329 |
25.022 |
22.249 |
20.762 |
20.072 |
18.334 |
17.438 |
16.871 |
16.629
|
Laplace (pseudo-count 1) |
22.199 |
20.8249 |
19.957 |
19.463 |
19.0139 |
18.036 |
17.409 |
16.819 |
16.659
|
pseudo-count 2 |
22.679 |
21.308 |
20.413 |
19.765 |
19.291 |
18.163 |
17.453 |
16.835 |
16.680
|
pseudo-count 0.5 |
22.859 |
21.297 |
20.331 |
19.692 |
19.217 |
18.127 |
17.419 |
16.833 |
16.678
|
Sum of Absolute Error
Method |
n=1 |
n=2 |
n=3 |
n=4 |
n=5 |
n=10 |
n=20 |
n=100 |
n=1000
|
Predict 0.5 |
50 |
50 |
50 |
50 |
50 |
50 |
50 |
50 |
50
|
empirical ave |
33.48 |
33.28 |
33.40 |
33.33 |
33.40 |
33.36 |
33.41 |
33.36 |
33.389
|
median |
33.39 |
33.32 |
29.99 |
29.86 |
28.62 |
27.27 |
26.21 |
25.23 |
24.99
|
thresholded ave |
33.45 |
33.41 |
33.37 |
33.39 |
33.34 |
33.38 |
33.26 |
33.30 |
33.30
|
Laplace (pseudo-count 1) |
44.48 |
41.68 |
39.97 |
38.87 |
38.09 |
36.01 |
34.85 |
33.65 |
33.35
|
pseudo-count 2 |
46.66 |
44.46 |
42.88 |
41.65 |
40.76 |
38.14 |
36.13 |
33.90 |
33.43
|
pseudo-count 0.5 |
41.68 |
38.79 |
37.42 |
36.64 |
36.18 |
34.75 |
34.21 |
33.53 |
33.37
|
Log Loss
Method |
n=1 |
n=2 |
n=3 |
n=4 |
n=5 |
n=10 |
n=20 |
n=100 |
n=1000
|
Predict 0.5 |
100 |
100 |
100 |
100 |
100 |
100 |
100 |
100 |
100
|
empirical ave |
33376.10 |
17022.11 |
10061.59 |
6603.60 |
4888.64 |
1624.61 |
525.45 |
93.89 |
72.30
|
median |
33387.90 |
16273.63 |
30082.50 |
20187.84 |
28589.00 |
22705.33 |
23753.52 |
24556.73 |
24588.09
|
thresholded ave |
332.53 |
199.40 |
149.93 |
124.83 |
110.31 |
86.35 |
77.68 |
72.88 |
72.01
|
Laplace (pseudo-count 1) |
91.76 |
88.03 |
84.55 |
82.28 |
81.14 |
77.51 |
74.79 |
72.68 |
72.38
|
pseudo-count 2 |
92.98 |
89.01 |
85.72 |
84.04 |
82.32 |
78.61 |
76.21 |
72.24 |
71.49
|
pseudo-count 0.5 |
94.48 |
90.97 |
86.04 |
84.02 |
82.83 |
77.85 |
75.69 |
72.68 |
72.47
|