Course talk:CPSC522/Improving the accuracy of Affect Prediction in an Intelligent Tutoring System

Hi Abed,
I liked the concept of your project and I also think you have structured your page very well. It has a very neat and clean layout. Some suggestions and queries that I had are outlined below:
1. I think there is some typing error in your hypothesis. You were probably meaning to say that PCA and wrapper features have previously been used but now you try to increase the accuracy of the model by using Logistic with L1 and ensemble methods?
2. I am unable to access any of your references. You might want to double check your links. 3. You could probably add a line or two on Intelligent Tutoring Systems(ITSs) and make this abbreviation explicit. 4. When you mention that the you try to avoid the use of sensors to make the study as unobtrusive as possible so that the user is not distracted, don’t you think when you ask the user to take regular self-report prompts by clicking on the radio button; that is more distracting and might even annoy the test takers? 5. How was the data collected from the undergraduate students? was it like a questionnaire or was it like a radio button they had to click while gazing at a screen? 6. Its not very clear to me why both emotions boredom and curiosity are treated separately? Because how can a user be bored and curious at the same time? 7. I don’t know if its possible, but you might want to show some snapshots of what your dataset looks like. How did you attribute numerical values to features like boredom and curiosity? Was it the number of people who clicked on boredom in some time interval were accounted for? Also have you considered levels of boredom’ for instance ‘a little bored, very bored, dying to get out bored’; maybe as a future work. 8. Why does voting ensemble method have the highest accuracy for boredom but Stacked RBF for curiosity.
Also, maybe if you could provide us with a little bit more intuition on your statistical analysis or some background on the terms in one or two lines, because even though you have provided with links to read them; there are too many and without which the whole section is a little hard to understand.
Thanks for introducing me to the research work going on in the field of ITSs.
Regards,
Ritika

Hi Ritika, Thank you for your detailed feedback. Regarding your feedback I have the following things to add 1. I left out some important details while trying to make the hypothesis less verbose. Thank you for pointing that out. I have updated it. Please, take a look at it and let me know whether it works 2. The links in the page work for me for the most part. I randomly tried a couple of them and they seem to work. I would double check all of them for the final draft. 3. Since, this is a continuation of my previous work; but yes, I would try to add some more details 4. As I mentioned, to train the system you need labels and data was taken over a period of 14 minutes; so this was the best available solution compared to use more obtrusive methods that cause continuous distractions 5. The data was taken using an eye-tracker where a label was taken every 14 minutes. The details are in the paper for MetaTutor Study 6. They can overlap. That's why the same dataset was divided into two separate datasets, one with boredom label[bored or not], another with curiosity label[curious or not]. In the original dataset, they did coincide, but as I mention we are taking all of the information and simply dividing them based on label 7. Various subjective denominations are used. Actually they used a Likert scale, and the dataset I work on uses thresholding where 3 and above is taken as yes and no otherwise. 8. Why a classifier gives better results in one dataset and not on the other and vice versa is a bit tough to define, empirically always a bunch of model is tried[within the boundaries of sanity!!!] and then the best is chosen. I will try to give some insight from my own understanding. As far as detecting a certain emotion, the true model for curiosity might be very different then what we get for curiosity then what we get for boredom; and the model found by RF in one dataset might be closer to one 'true model' and for other dataset something else might work. But the classifiers shown here work well in practice in binary classification problems[Also in this particular domain!!!] 9. Also the analysis has external links. I would try to add some intuition behind them in the page itself. Let me know whether I was able to answer your queries. Also feel free to mention other queries and/or concerns

Thanks for your clarifications Abed. I am still a little uncomfortable with how the user can be bored and curious at the same time and I don't see why they should overlap, but I guess those are just model parameters which you chose to use this way, which is perfectly fine.
As for the links, I am able to go from within your wiki page to the references and vice versa but not able to go to the actual page on the web. For instance where you have sited the papers, I cannot access those papers from your wikipage. I think the web url is missing in the references. You could refer to my page, 'Analysing online dating trends using Weka'; the reference section to see how to link it to the pages outside of wiki.
Rest all looks good!
Ritika

Thanks Ritika for your feedback. Users were seen to have overlap in multiple emotions in the data set which might sound a bit counter-intuitive but was shown to be practical in the papers. As far as the links, I am not sure whether you are talking about external links or papers citations. For the cited papers, I have just provided the name and authors; as I have done in my previous pages; as far as I know putting external links for cited work is not a necessity, but I would try to do that if I am able to do so. Thanks again for your feedback

Abed

Hi Abed,
Nice work! I It fits in perfectly from where you left in the previous assignment. You also did a good job presenting your work today. It made understanding your page a lot easier after seeing you present.

Here are a few comments/suggestions:

• The Statistical analysis section could be improved by providing the reader a bit more intuition behind what is happening. For instance, you could briefly explain the ANOVA table for curiosity by showing how we interpret the values.
• Rather than just having a single level for boredom and curiosity, breaking them down into smaller categories like “Extremely bored” and “Moderately bored” might give us a more accurate analysis. You could probably try that it out in the future.
  

Keep up the good work.

Best Regards,

Adnan

Hi Adnan, Thank you for your feedback and your suggestions. I would try to update the page to the best of my ability

• I agree that the analysis section can be updated a bit, specially with giving intuition behind what is happening. I would try to add a section there
• I forgot to mention in the page that some amount of thresholding was done in the original paper which lead to the dataset being binary. The actual dataset contains data in a 5 point likert scale. This is definitely a future work prospect.

Thanks Abed

Nice Page. The "Results Analysis" section seems hard to understand for people with no background information.

Hi YanZhao, Thank you for your feedback. I agree the results analysis can be a bit confusing for people who don't know much. I hoped the external links would be enough. I would definitely try to add more information/explanation of the things done for analyzing the results.

Hi Abed,

I am really glad to read your page. Your topic is similar to mine. Both of what we have done are feature selection. I will try to apply the method you metioned in this page in the data sets I collected and compare the performance of your method with mine. And I have only one suggestion for you. You can add a conclusion section to explicitly answer whether your hypothesis is true.

Sincerely,

Ke Dai

Hi Ke Dai, Thank you for your feedback. I also took a look at your page and I liked it. I can definitely take some pointers from your page. As far as the conclusion section, I mentioned it in the discussion that I do not have substantial information to give a definitive answer but enough to mention that this is a interesting development to further explore. I do agree I could have been a bit more clear. I would definitely try to improve it.