Course talk:PSYC305/2013ST2/ClassProject/5.4 Discussion - Limitations

Hi all,

I was wondering does sexual orientation effect the results of gender diagnosticity? That is to say, when compared a butch (a masculine lesbian) with a flamer (gay men who are more femme), is the butch more masculine on both behaviours and thoughts than the flamer? As well as when we compared a femme (a feminine Lesbian) with a butch, or even with a flamer, is the femme much more feminine than butch and flamer? I think when we collect the data and analyze the results, this might be a factor that should be concerned..

In one study, researchers find that tomboy as a gendered social identity can provide masculine-typed behavior in girls and women, and can be a protective Identity (Craig & LaCroix, 2011). In my view, when to determine gender diagnosticity we need also consider the participants sexual oritation and gender identity role in society.

Is this a limitation of the research, or a further discussed topic related to gender diagnosticity? Any thought or help? Thanks.

Hi Sunny, I think sexual orientation is actually a good indicator to differentiate gender-related issues. Also, in the reading of Gender Diagnosticity, it says “Gender Diagnosticity measurement do not reify gender-related individual differences or freeze them into specific constructs”. So a butch could score higher on masculine than flamer. Hope this helps

We didn't ask participants' sexual orientation. So this could be a limitation. I can see how it could be a limitation if we wanted to do a diagnosticity between, say, heterosexual men and heterosexual women in the class. That wasn't our goal though, we wanted to a gender diagnosticity between all men and women in the class, so I think it's OK.

I noticed that someone added sexual orientation as a limitation. However, I don't think it is, considering that we only wanted to do a diagnosticity between men and women, as opposed to heterosexual men and heterosexual women (see above post). Being a particular orientation does not affect this measure. I propose that we edit out this portion of the limitations- any objections?

we cannot say a man with feminine-orientation can be included in the group of female. of course they are different. inborn and environment both have influences on how to form a gender. So, I think sexual orientation is a good indicator to differentiate fender-related issues.

I think someone has edited on the limitation but carelessly. originally it was two paragraphs, and it was combined into one however, it is not nice to leave it unfluid, such as "Each culture have unique emphasis, and this may allow to inhibit or express certain personality traits, and challenged the certainty of the relationship between gender and personality traits. That is to say, religious belief, as a question...." It is not connecting at all, and I have to edit it again.

(It's probably too late now, but) I think there is a way to undo changes.

Actually there is a way to review the changes been made. by clicking on the change been made (the contribution link) through the history button, and click on the change someone made on the relative topic, and there is a button called 'diff' and we can compare the former & later version.

Though the demographic age of the sample varies from 18-49, most of the questions are geared towards those who have not yet entered the work force or have completed their post-secondary education. Questions orientations such are socialization habits, career choices and dating choices seem to be more geared towards those in the lower ages of the sample. Though these topics may hold some bearing with those who are at more advanced age, the emphasis seems to be, on the surface at least, for those of a younger age.

Yes, so while we know that the questionnaire works on an undergraduate university sample, the limitation is that we don't know how well our questionnaire would work on, say, older adults.

Hi all!

I just want to talk about the language section about the participant's level of English comprehension. This is an upper level psychology course and to be able to enroll into this course it requires the students to meet psychology prereqs, and these psychology prereqs usually requires a pretty good level of interpretation and understanding of English. Furthermore UBC students are required to pass the LPI and the LPI is a measurement of one's English proficiency so I think we can safely establish that we all have a pretty good level of English comprehension. I understand that there are variations in everyone’s English comprehension however the survey was designed in pretty simple English and I don’t think people had a lot of problems understanding it. Even if there are individuals that had problems completing the surveys I believe that to be a very small number so maybe we can mention it in some other parts of the survey and not dedicate a major section about it.

I could be wrong! Let me know what you guys think!

Hi!

I personally don't agree with you just because from personal experience, I disagree that being in an upper level psychology course means that ones's English proficiency is at a good level of understanding. I'm not trying to make assumptions that people in this class do not have a high level of English proficiency; I'm merely stating that I agree with the contributor(s) ideas that English comprehension could have been a possible factor and what you consider to be "simple English" may not be interpreted the same way by other classmates. The contributor(s) did specifically mention that this limitation might only be pertaining to several questions that have words/terms with suggested connotations - and not the whole study in general. The two points listed in the section were written up by different people which means that this issue was considered to be a limitation by more than one person and should probably be treated as a possibility.

Although there were some evident limitations in this study, there are others mentioned on this page that are just possibilities and may or may not have made much impact even if they were corrected; these factors have as good of a chance of being a limitation as not. In any case, I do believe that English comprehension being a factor in this study is a possibility but again, this is just my personal opinion and I'd love to see others reply too! :)

I really do sincerely apologize if my reply is interpreted in any sort of offensive way!

Hey! I agree with Spkwong. The fact that this is an upper level psychology course does not automatically mean that all students are familiar with the connotations behind certain terms that were used in the questionnaire. Whether or not all students in PSYC305 have high English proficiency, I think it is important to note that English comprehension could have been a factor in the way that some people answered the questionnaire. Better to point out all the possible limitations than to miss one, right?

I think you are all correct. There is nobody in the class with really bad English skills - they wouldn't have been able to get past the LPIs if they did. On the other hand, it makes sense that there are varying levels of english proficiency in this class. As well as being something that might have affected people's responses, it might have been an issue with some of the question development as well.

Hi guys, I have added some information on this part. In order to connect to the excellent work you guys have done, I have to change or delete a few words, I hope it is ok and apologize for that. And please don't be hesitated to edit my work if anyone hold different opinions. cheers~

So much excellent information on this page! A lot of good points. I have run through and done some general editing on this page to make certain points more clear.

Hi all, Just added some citations and edits to what I contributed! Thanks!

Hi, as I was filling out the questonnaire I noticed that while theer were questons related to preference for having children in the future, there were few or no questions related to my own demographic as a parent (father).

Perhaps we should include this as an oversight.

Also how about something about a more controlled environment for filling out the questionnaire? Would filling it out in a given time and place ie a group sitting create a more homogeneous condition for completion?

It brings to mind the final application of our questionnaire: do we intend it to be filled out by individuals in unmonitored circumstances (like we did, on our own via internet) or for more controlled clinical applications?

Not sure if this is relevant but it just occurred to me.

It's quite common to have internet questionnaire that people fill out in their own time and seems to be relatively accepted. Nevertheless if you folks think this warrants further discussion then this section is the place to do it in!

I thought it would be interesting to build on the confounding differences between university students and the rest of the population. Specifically, how our social roles change the way we view gender and personality as we age, and as our social roles become more important as we age (ie work roles, family roles). Specifically in how increased participation in the workforce for women increase their sense of agency vs. their involvement in social roles. As students, we seem to be more defined in term of personal attributes (as the 20 statements test shows) as opposed to our occupational roles as we get older. I hope this is relevant to the study!

I think there only being ten males is probably the biggest limitation here.

I just tacked on a bit about that continuing in section 2.

Hi, these are what I have in mind (some of them might already have been mentioned) 1. sample size was too small and the gender distribution was uneven (10 males and 50 females), therefore, the results may show low in generalizability and decrease the external validity.

2. The differences between the demographic characteristics of participants may have introduced confounding factors into the studies. (The GD scale is really a universal measurement).

3. The use of a correlational study (especially using questionnaire) to conduct investigation and the inability to demonstrate a causal relationship between the research variables, because of the use of a correlational design instead of longitudinal/experimental study.

4. Since the participants of the study were the same group who made the questionnaire. Moreover, we knew the concept , prediction, and the purpose of the study, thus, it may possibility of creating both experimenter bias and participant bias (we may not get the true reaction from the subjects).

5. The participants were not random selected, which can't represent the large population. (the best it can represent is the university psychology major students only)

Hi there, you're completely right about the external validity and generalizability concerns. When thinking about the rest of what you've written, I think it's good to keep in mind the goal of the project (to design a questionnaire).

Hi,

  Reading through limitation 12 and 13 I noticed some similarities between the two. Both address how randomness/selection can have an effect on the generalizability of the questionaire to the pubic. Therefore would it be appropriate to combine them as one limitation instead ? Since I did not write these limitations, I think its best if I receive input from others, or if possible from the contributors, if I were to make an edit to their contribution.

Sorry not sure why my reply was cut off...Here it is again. Reading through limitation 12 and 13 I noticed some similarities between the two. Both address how randomness/selection can have an effect on the generalizability of the questionaire to the pubic. Therefore would it be appropriate to combine them as one limitation instead ? Since I did not write these limitations, I think its best if I receive input from others, or if possible from the contributors, if I were to make an edit to their contribution.

Hi Karen, I also noticed some repetition in some of the latter limitations that were posted and I actually asked Dr. Veale this same question today. She told me that we are allowed to tidy up the page and merge redundant posts. I'm just about to do a bit of that tidying-up now but it seems like you have some idea of what you feel should be merged as well, so feel free to make/edit my arrangements!

I am just wondering is it necessary to have a control group in a study like this. Because it is not a experimental study, but a self-report questionnaires study. If a control group is needed, how? Also, I think there is a bidirectional effect between religious belief and personality, so it may actually impact our findings on gender diagnosticity. Does anyone agree with me? Thanks

Hi YuchengLiang, I do not think that we need a control group in our study, unless we wanted to manipulate some sort of variable in an experimental group and compare it with a control group. I think that since we do not have a control group, we simply cannot propose that some of the relationships that we found were caused by one another; rather, we can only say they are correlated. I am not sure if others have mentioned a control group in the sense that we should have a 'faking good' profile group in order to reduce the social desirability bias.

If we were to have a control group, I think that that would further our study in the sense that we could look into the relationships between two specific variables, for example people's choice in occupation and gender and see which influences which. Let me know what you think!

Hi PhoebeDychinco, that is what I am thinking as well. Initially, I misunderstood the limitation someone had mentioned, which is in our study, we lack of a control group. I completely agree with you, in order to find some causality between variables such as occupation and gender, experiment need to be designed. Also in designing a comprehensive research, the combination of LOST data is best persuasive.

Hi everyone,

I noticed that some of the more recent posts had topics that were either already discussed or briefly touched on above. I’ve taken the liberty of doing some tidy-up and merged some of the posts that had similar meanings and/or redundancies to ones previously listed. I tried to keep as much of the original contributions as possible but I did have to make some edits in order to establish fluidity – it is a bit difficult when the authors are different.

I’m making this post so there are no misunderstandings with lost posts (because they aren’t lost, they’ve just been re-arranged!) and I apologize in advance if anyone is concerned or upset with the changes that I made.

Thanks

yea i was thinking about the same thing too, cuz i saw some of the limitations were slightly overlapped. The editing you had done looks good to me! =]

I agree that merging where we can will help to make this section more accessible to people - good thinking.

I cut and pasted a bit from Survey to item #13 here regarding dual roles. I hope this is not a repetition; itf it is, please edt as necessary.

Even though #8 and #13 are not exactly the same I was thinking to combine them to organize this section more clearly but I just left them as it was. Since #14 also talks about dual roles and if I put all the ideas about dual roles in one point, it would be more confusing. How about if we change the order of the points so it is more logically organized and easier to reference to the previous points?

Hi,

I also think that there is definitely overlap with regards to dual roles; I agree that they should be re-arranged to be closer together. I've also noticed some recent posts that touch on some topics that have been mentioned above as well, such as generalizability and sampling. I don't know if you guys saw Dr. Veale's post down there but she would prefer that we eliminate list format and arrange these limitations into paragraphs instead.

I attempted to cluster the different limitations into different categories and this is what I came up with:

Participant Background: #9, #14 / Participant Bias: #1, #5, #6, #12, #16 / Sample Issues + Generalizability: #2, #15 / Method: #13, #8 / Content: #3, #4, #10 / Language: #7, #11

This is just a suggestion based on my attempt to clump similar topics together. How does everyone feel about this? I haven't actually made any changes on the page yet so anyone can feel free to give it a go! I know there are a number of repetitions to edit out and some sentences to make in order to enhance the flow - so it'll definitely be a challenge.

Hi,

I'm working on editing limitations based off what Jaimie has said which is to try and eliminate the numbers (list) and make it more flow as a paragraph format. this is my most current edit in quotations. I am wondering who put the quote from Larsen & Buss because I changed the word likable to appealing since likable sounded awkward.However if it is a direct quote then I shouldn't change it and if it is a direct quote it also has to be in quotations. One limitation with using the self-report method is people may be biased and untruthful when responding to questions regarding sensitive behaviors and private details. Generally, people tend to have the bias of social desirability, which is a tendency to answer items in a way that is socially appealing(Larsen & Buss). By responding with this bias, people may produce a falsely positive impression of themselves by responding with what they believe to be socially accepted in the Western culture.

Hi Nicola, It is not a direct quote, and the term "socially likeable" comes from the last paragraph of p.39 of the textbook.

Hi,

I'm working on editing limitations based off what Jaimie has said which is to try and eliminate the numbers (list) and make it more flow as a paragraph format. this is my most current edit in quotations. I am wondering who put the quote from Larsen & Buss because I changed the word likable to appealing since likable sounded awkward.However if it is a direct quote then I shouldn't change it and if it is a direct quote it also has to be in quotations. One limitation with using the self-report method is people may be biased and untruthful when responding to questions regarding sensitive behaviors and private details. Generally, people tend to have the bias of social desirability, which is a tendency to answer items in a way that is socially appealing(Larsen & Buss). By responding with this bias, people may produce a falsely positive impression of themselves by responding with what they believe to be socially accepted in the Western culture.

There are some very good outlines of limitations here. It also seems there is some good work going on with refining these right now. I would suggest that we need to change it to paragraph format rather than a numbered list. The first step to doing this would be to sort these limitations into ones that go together (even if they are not completely the same). The ones that go together could go together in each paragraph. Good luck!

Hi Dr. Veale, It looks like your thread was pushed to the bottom so I made a post in the "Merging Posts" thread above regarding your request to make paragraphs. Although I haven't made any changes per se, I made an attempt to cluster the items together and I've listed them in the post as well. Cheers.

I made some minor tweaks to the introduction sentence at the beginning of limitations, it sounded repetitive so hopefully it is a little more clear now.

Hi all! First of all, nice work guys! You guys covered a lot of the limitations of the study and also pointed some really some thoughtful points. From speaking to Dr, Veale and other classmates in the Validity session, I think that it may be better if we could keep anything that relates to validity to that session, and any other types of limitations to this session so that the information wont overlap. Although I believe generalization could be further expanded in this session rather than in the validity because generalizing is not just having high external validity. Cheers, William

Hi, I'm just wondering if there is a specific tense we should be using in this section of the paper. I'm seeing a mix of both present and past tense and I'm just wondering if there is a correct one. I think I've been doing past (and conditional) tense for my contributions so far - but in any case, if there's one that is preferred, I'll gladly make some edits to my contributions (and to others, where necessary!). I don't know how important this is but I just thought I'd just ask as I was finishing up some of my modifications.

Hi, according to the website provided by Dr. Veale (http://www.essex.ac.uk/psychology/department/A-Z_files/GUIDE%20TO%20WRITING%20RESEARCH%20REPORTS.pdf), here's the explanation of your concern:

2. Use of tenses

Tenses can be very difficult to use correctly. These guidelines can only be very general rules of thumb. Basically, anything that is history should be written in the past tense. When you write up your work, even your method and results will be history, and should be described in the past tense. The conclusions of previous workers are history, however yours are still current and should be described in the present tense. The theories and models that were derived from the results and conclusions still make predictions today (even if they are the wrong ones) and their predictions thus should be described using the present tense.

Thus, for a previous piece of work that you are describing:“Smith et al. (1970) found that… they concluded that…and developed the XYZ model. This predicts that…”

If you were discussing the results of your experiment: “It was found that… and thus we conclude that…. the ABC model predicts that…"

PS, I had made some corrections according to this.

Thank you for addressing my question!

Yes thank you Emma this is good information, thanks for making it accessible here.

Glad it helps! =]

Perfect :)