Course:CPSC532:StaRAI:2017:As1:Q1:dp

From UBC Wiki

Here is what I got from running 10000 iterations....

see http://www.cs.ubc.ca/~poole/cs532/2017/as1/triv_learn2.py

Sum of Squares Error

Method n=1 n=2 n=3 n=4 n=5 n=10 n=20 n=100 n=1000
Predict 0.5 25 25 25 25 25 25 25 25 25
empirical ave 33.359 24.986 22.261 20.830 20.0244 18.364 17.545 16.797 16.718
median 33.239 25.050 29.861 24.975 28.669 24.919 25.091 24.960 24.972
thresholded ave 33.329 25.022 22.249 20.762 20.072 18.334 17.438 16.871 16.629
Laplace (pseudo-count 1) 22.199 20.8249 19.957 19.463 19.0139 18.036 17.409 16.819 16.659
pseudo-count 2 22.679 21.308 20.413 19.765 19.291 18.163 17.453 16.835 16.680
pseudo-count 0.5 22.859 21.297 20.331 19.692 19.217 18.127 17.419 16.833 16.678

Sum of Absolute Error

Method n=1 n=2 n=3 n=4 n=5 n=10 n=20 n=100 n=1000
Predict 0.5 50 50 50 50 50 50 50 50 50
empirical ave 33.48 33.28 33.40 33.33 33.40 33.36 33.41 33.36 33.389
median 33.39 33.32 29.99 29.86 28.62 27.27 26.21 25.23 24.99
thresholded ave 33.45 33.41 33.37 33.39 33.34 33.38 33.26 33.30 33.30
Laplace (pseudo-count 1) 44.48 41.68 39.97 38.87 38.09 36.01 34.85 33.65 33.35
pseudo-count 2 46.66 44.46 42.88 41.65 40.76 38.14 36.13 33.90 33.43
pseudo-count 0.5 41.68 38.79 37.42 36.64 36.18 34.75 34.21 33.53 33.37

Log Loss

Method n=1 n=2 n=3 n=4 n=5 n=10 n=20 n=100 n=1000
Predict 0.5 100 100 100 100 100 100 100 100 100
empirical ave 33376.10 17022.11 10061.59 6603.60 4888.64 1624.61 525.45 93.89 72.30
median 33387.90 16273.63 30082.50 20187.84 28589.00 22705.33 23753.52 24556.73 24588.09
thresholded ave 332.53 199.40 149.93 124.83 110.31 86.35 77.68 72.88 72.01
Laplace (pseudo-count 1) 91.76 88.03 84.55 82.28 81.14 77.51 74.79 72.68 72.38
pseudo-count 2 92.98 89.01 85.72 84.04 82.32 78.61 76.21 72.24 71.49
pseudo-count 0.5 94.48 90.97 86.04 84.02 82.83 77.85 75.69 72.68 72.47