Course:CPSC532:StaRAI:2017:As1:Q1:dp
Appearance
Here is what I got from running 10000 iterations....
see http://www.cs.ubc.ca/~poole/cs532/2017/as1/triv_learn2.py
Sum of Squares Error
| Method | n=1 | n=2 | n=3 | n=4 | n=5 | n=10 | n=20 | n=100 | n=1000 |
|---|---|---|---|---|---|---|---|---|---|
| Predict 0.5 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |
| empirical ave | 33.359 | 24.986 | 22.261 | 20.830 | 20.0244 | 18.364 | 17.545 | 16.797 | 16.718 |
| median | 33.239 | 25.050 | 29.861 | 24.975 | 28.669 | 24.919 | 25.091 | 24.960 | 24.972 |
| thresholded ave | 33.329 | 25.022 | 22.249 | 20.762 | 20.072 | 18.334 | 17.438 | 16.871 | 16.629 |
| Laplace (pseudo-count 1) | 22.199 | 20.8249 | 19.957 | 19.463 | 19.0139 | 18.036 | 17.409 | 16.819 | 16.659 |
| pseudo-count 2 | 22.679 | 21.308 | 20.413 | 19.765 | 19.291 | 18.163 | 17.453 | 16.835 | 16.680 |
| pseudo-count 0.5 | 22.859 | 21.297 | 20.331 | 19.692 | 19.217 | 18.127 | 17.419 | 16.833 | 16.678 |
Sum of Absolute Error
| Method | n=1 | n=2 | n=3 | n=4 | n=5 | n=10 | n=20 | n=100 | n=1000 |
|---|---|---|---|---|---|---|---|---|---|
| Predict 0.5 | 50 | 50 | 50 | 50 | 50 | 50 | 50 | 50 | 50 |
| empirical ave | 33.48 | 33.28 | 33.40 | 33.33 | 33.40 | 33.36 | 33.41 | 33.36 | 33.389 |
| median | 33.39 | 33.32 | 29.99 | 29.86 | 28.62 | 27.27 | 26.21 | 25.23 | 24.99 |
| thresholded ave | 33.45 | 33.41 | 33.37 | 33.39 | 33.34 | 33.38 | 33.26 | 33.30 | 33.30 |
| Laplace (pseudo-count 1) | 44.48 | 41.68 | 39.97 | 38.87 | 38.09 | 36.01 | 34.85 | 33.65 | 33.35 |
| pseudo-count 2 | 46.66 | 44.46 | 42.88 | 41.65 | 40.76 | 38.14 | 36.13 | 33.90 | 33.43 |
| pseudo-count 0.5 | 41.68 | 38.79 | 37.42 | 36.64 | 36.18 | 34.75 | 34.21 | 33.53 | 33.37 |
Log Loss
| Method | n=1 | n=2 | n=3 | n=4 | n=5 | n=10 | n=20 | n=100 | n=1000 |
|---|---|---|---|---|---|---|---|---|---|
| Predict 0.5 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
| empirical ave | 33376.10 | 17022.11 | 10061.59 | 6603.60 | 4888.64 | 1624.61 | 525.45 | 93.89 | 72.30 |
| median | 33387.90 | 16273.63 | 30082.50 | 20187.84 | 28589.00 | 22705.33 | 23753.52 | 24556.73 | 24588.09 |
| thresholded ave | 332.53 | 199.40 | 149.93 | 124.83 | 110.31 | 86.35 | 77.68 | 72.88 | 72.01 |
| Laplace (pseudo-count 1) | 91.76 | 88.03 | 84.55 | 82.28 | 81.14 | 77.51 | 74.79 | 72.68 | 72.38 |
| pseudo-count 2 | 92.98 | 89.01 | 85.72 | 84.04 | 82.32 | 78.61 | 76.21 | 72.24 | 71.49 |
| pseudo-count 0.5 | 94.48 | 90.97 | 86.04 | 84.02 | 82.83 | 77.85 | 75.69 | 72.68 | 72.47 |