Course:CPSC532:StaRAI:2017:Results
Results for Paper for Predicting Gender from Movie Ratings
Here are the publishable results for the paper
- make sure you use the definitions of the errors in the paper
- make sure that you set hyperparameters by 5-fold cross validation
- you may only train on the specific data given (the ratings before 884673930 and the gender of the people who rated before 880845177), and report on the accuracy on the test set (which is the same as the assignment)
- the descriptions should be suitable for inclusion in a paper. In particular someone should be able to reproduce the results from the description.
| Method | ml-60k ASE | ml-60k Log loss | ml-1m ASE | ml-1m Log Loss | Yelp ASE | Yelp Log Loss |
|---|---|---|---|---|---|---|
| Predict 0.5 | 0.25 | 1 | 0.25 | 1 | 0.25 | 1 |
| training average | 0.2159 | 0.9004 | 0.2043 | 0.8637 | 0.2364 | 0.9604 |
| training average (pseudocount=50) | 0.2139 | 0.8932 | 0.2044 | 0.8642 | 0.2364 | 0.9605 |
| Naive Bayes/Normal Distribution/Rating > 3 | 0.2160 | 0.9006 | ? | ? | ? | ? |
| BPMF(5) with Logistic Regression/Ratings | 0.1950 | 0.8223 | ? | ? | ? | ? |
| BPMF(5) with Logistic Regression/Rating > 3 | 0.2046 | 0.8673 | 0.1896 | 0.8083 | 0.2363 | 0.9600 |
| BPMF(5) with Logistic Regression/One Start With User Gender Int Var/Rating > 3 | 0.1927 | 0.8235 | 0.1871 | 0.8015 | ? | ? |
| MF and LR (k = 5) /Rated | 0.2155 | 0.8995 | 0.2055 | 0.8683 | 0.2361 | 0.9595 |
| MLN LR/Rating (wgts per item+stop early) | 0.1896 | 0.8284 | 0.1356 | 0.611 | 0.2364 | 0.9604 |
| MLN LR/Rating (wgts per item+select #iter by cv) | ? | ? | ? | ? | ? | ? |
| Markov Logic Network with Hidden/Rating > 3 | 0.2124 | 0.8882 | 0.2048 | 0.8652 | 0.2344 | 0.9538 |
| Logistic Regression + limited parents (Rated) | 0.1934 | 0.8387 | 0.1505 | 0.6772 | 0.1885 | 0.8013 |
| Logistic Regression + limited parents (Rating > 3) | 0.1994 | 0.8489 | 0.1479 | 0.6683 | 0.1981 | 0.8368 |
| MF and LR (k = 5)/Rating > 3 | 0.1998 | 0.8440 | 0.1929 | 0.8240 | 0.2358 | 0.9586 |
| RDN-Boost | 36.85 | 153.70 | ? | ? | ? | ? |
| Alchemy, no hidden units (rated) | 0.2349 | 1.2434 | 0.1597 | 0.8267 | 0.1931 | 0.8569 |
| Alchemy, no hidden units (rating) | 0.2202 | 1.1486 | 0.1586 | 0.8528 | 0.1984 | 0.8707 |
Method Descriptions
For each method, please provide a description of the method (as may appear in a research paper), and a link to the code that produces these results. Make sure your code specifies its copyright.
Training average
The training average is the