Previous Measures : How can we improve?

Hi all, I would like to address again the importance of addressing the limitations of this study in order to provide solutions for future research. When considering future research you are considering the possible improvements that you can make to the study in order to increase both the reliability as well as the validity of the study. Below, I have reworded a couple of the limitations that have been collaborated by the class, and included possible resolutions to these limitations which could aid us in future research.

As mentioned, the overlapping limitation of this study was that a self-report was the only procedure of measurement utilized. Although the use of a self-reports provides ease as well as well as access to information that is reserved only by the individual, its limitations should be considered and the study should be revised for future research. Through the revision of the limitations which this study was hindered by, one is able to develop a better and more accurate study through which gender diagnosticity can be examined. The limitations that should be considered in future research are as follows:

1. When questionnaires ask questions which are private for a particular individual, that individual may respond in a “socially desirable” manner instead of truthfully. To answer a questionnaire in a socially desirable manner is also known as faking good. There are however possible solutions to this limitation that could be considered in future research. It is important to note however that anonymous questionnaires don’t often pose the problem of individuals answering truthfully due to the privacy of the individual being assured.

• One way in which one is able to overcome the problem of individuals answering the questionnaire in a socially desirable manner is to impose a “faking good profile” (Larsen & Buss 2010). This profile is constructed to use in comparison to the responses of an individual to determine whether he or she is “faking good.”

2. This study was representative of ethnicity, religious belief, level of education, and personality, but lacked in the representation of gender. The ratio of individuals who reported their genders were 10 males to 50 females. Due to the limitation of equal gender representation this study is only reliable when distributed to a higher population of BOTH females to males. Due to the lack in reliability and due to the study not measuring what it claims to measure, this study cannot be claimed as reliable or valid.

• In order to define this study as both valid and reliable we need to consider two things 1) that the study measures what it claims to be measuring (validity), and 2) that the study will yield the same results when tested on other populations or sample groups (reliability)
• these two points can be resolved by a sample size that is 1) not biased (for example the sample should not consist of the psychology students constructing the study) 2) representative of both females and males (equal or close to equal number of males and females within the sample) 3) randomly selected to control for other factors such as ethnicity, religious beliefs, educational level, etc.

The first step in future research is resolving these as well as other limitations to the current scale. Once the limitations have been resolved the many interesting and important factors that everyone has suggested could be researched in the future. Once you have a reliable and valid scale on which gender diagnosticity can be measured one will be able to add specific measures to investigate, such as how “gender and personality traits may have great influences on mental disorders,” or determining the correlation between “life expectancy[,] gender and personality traits,” and the rest of the suggestions which have been contributed to this page.

Hi everyone, As mentioned before, an aspect of future research that should be considered is how to revise a study in order to raise its validity and reliability, which will give us more accurate results. I have some additional ideas on solutions to fixing the limitations that were mentioned by GennaFerreira.

1. Solutions to reduce social desirability I found the following methods in the following article: Nederhof, A. (1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15, 263-280.

• Neutral Questions
  • We should try to make questions that are as neutral as possible, so that it may not be seen as something that a person should answer in a specific way (a socially desirable way) by many different social groups.
• The Bogus Pipeline
  • Participants are told that either an attitude or emotion is being analyzed by a machine, or by a polygraph. However, the wires that are attached to the participant are not real. Since the participant does not know this, they will most likely answer the questions honestly.

2. Solutions to having a more valid and reliable study

• The sample size needs to be much larger; having a few hundred will be more effective than having less than a hundred. This will help eliminate individual differences. Moreover, by having a larger sample, we will be able to have a more equal number of females to males.
• As GennaFerreira mentioned, having random sampling will create less differences between individuals. Therefore, we should be using the random sampling technique to include not only Psychology students but the general public as well.
• In order to increase reliability, this questionnaire should be distributed more times to different samples to see if the results are still the same between different groups (ex. between students, parents etc.). We can also ask multiple questions about the same idea, in order to see if participants are answering it in the same way throughout the questionnaire (I believe we did have a few of these throughout the questionnaire).

3. Solutions to Experimenter/Confirmation Bias and Content

The overarching solution to confirmation bias/content is to develop questions from different perspectives (as in, consider culture, geographic location, SES, age/cohorts, etc) to provide the most objective, all-encompassing questionnaire. The questions were presumably developed by individual students. Although there was some convergence in ideas, the questionnaire could have been streamlined more efficiently. Instead, we could develop questions using either (1) a panel consensus method, where questions are considered as a group or (2) a structured delphi method, where individuals in a panel develop their own questions, which are then consolidated and refined repeatedly until ideas are stabilized and the questionnaire is where we want it to be (eg. objective, representative of different perspectives, provides a variety of possible responses).

4. Language

Aside from translating the questionnaire to multiple languages, I think either question development method as mentioned above could potentially address the language barrier and English comprehension issue.

Objectivity also plays into the neutrality of the questions, as PhoebeDychinco stated would address the social desirability/negative connotations issue.