Describing and Evaluating the Research Methodology of A Level Studies - An Illustration

I, Jyotika Varmani, tutor students of Psychology at all levels. I reside in Mumbai and tutor students online. You can contact me personally on my e-mail id jyotikapsychology@gmail.com or call/message me on 9892507784 for enquiries.

---

AS and A-level studies are to be prepared with respect to their descriptions, the issues and debates they involve and the description and evaluation of the research method they adopt. As concerns the last type of preparation, I have had a number of queries by students which have all suggested to me that they simply do not engage in this type of preparation explicitly. That is, students assume that preparing for the description of studies with the syllabus guidelines will automatically prepare them to describe and evaluate the research methods employed in the studies. This is not the truth and in order to show how far taken away this notion is from reality, in this post, I use a part of the study by Laney et.al. to illustrate how a description and evaluation for any AS study must be prepared using a step-wise approach. Let's begin, keeping in mind that this post is not intended to provide ready answers to any questions whatsoever but instead to provide a sample of preparation for research methods-

Description and Evaluation of the Research Method Used in the Laney et.al. Study

Laney et.al. (2008) conducted two studies to investigate whether false memories could be implanted in individuals for a beneficial purpose, both of which were laboratory or controlled experiments. In this lesson, we take a detailed view of how the method was employed in the first study and review how effectively it was employed.

Description of the Use of Laboratory Experimentation in Laney et.al.'s Study 1

Any laboratory experiment is distinguished from other research methods by its use of a highly controlled setting. This means that it employs several controls by keeping a check of extraneous variables that could tamper the relationship between the independent variable(s) and the dependent variable(s). Before taking a look at the controls employed in the present study, we must recognize its independent and dependent variable(s) correctly.

In the first Laney et.al. study, the first independent variable was the inducement of a love for asparagus amongst participants, which was manipulated at two levels - 

(1) inducement of the love for asparagus (experimental condition); and 

(2) no inducement of the love for asparagus (control condition). 

Operationally, this independent variable would be defined as the presentation of false computerized feedback to participants that they had loved eating cooked asparagus as children, manipulated at two levels - 

(1) presentation of the false feedback (experimental condition); 

(2) no presentation of such feedback (control condition). 

This variable was randomized across participants, that is, different participants were made to undergo different levels of this variable.

The second independent variable was time, which was manipulated at two levels - 

(1) before presentation of false feedback; and 

(2) after presentation of false feedback. 

Operationally, this independent variable would be defined as the time point of collecting responses from the participants, at two levels - (1) on session 1; and (2) on session 2. 

This variable was repeated within participants, that is, all participants of the experiment were made to undergo both levels of this variable.

The dependent variables were the development of a love for asparagus among participants; and (three) related consequences. 

Operationally, the first dependent variable would be defined as confidence ratings given by participants for the critical item on the Food History Questionnaire, "Loved asparagus the first time you tried it." 

The second dependent variable would be operationally defined as the ratings given by participants for their desire to eat asparagus on the Restaurant Questionnaire.

The third dependent variable would be operationally defined as ratings given by participants for their preference for asparagus on the Food Preferences Questionnaire.

The fourth dependent variable would be operationally defined as the price option selected by participants for the price they would be willing to pay for a pound of asparagus.

A number of controls were maintained for the accurate determination of the relationship between these independent variables and the dependent variable. These are listed below:

• In order to control demand characteristics from influencing the performance of participants, the true purpose of the experiment, that is, of inducing false memories, was not revealed to them in any manner. 

• In accordance with this end, several different questionnaires were presented for answering to participants, not only the questionnaire containing the critical item of false memory for it would make the purpose of the experiment obvious. Furthermore, even within the questionnaire containing false memory several distractor items were included to present the target item subtly;

• A check for socially desirable responses was included in the form of the  Marlowe-Crowne Social Desirability Scale along with the other questionnaires;

• To make the false feedback given to participants appear credible and to prevent them from guessing the true purpose of the experiment which could, again lead to demand characteristics, the false feedback was presented to participants with the information that it had been generated by a computer, and as if it had been individually tailored to their responses;

• To ensure that all participants had attended to, and had understood the feedback given to them, participants were asked to respond to a brief questionnaire which reflected mental elaboration of the feedback they had received;

• A control group was employed which underwent the same procedure as the experimental group, except that they were not given the critical feedback, "You loved to eat cooked asparagus". Comparison with this group enabled researchers to check whether the critical item and its elaboration influenced experimental participants' liking for asparagus, or whether it was a result of mere exposure to the experimental stimuli;

• To account for individual differences in susceptibility to accepting false information, a 'Memory or Belief?' questionnaire was administered to participants, in which they had to indicate whether a memory for or just a belief of having loved asparagus as children. Responses on this questionnaire were subject to an inferential statistical analysis comparing individuals who were found to have a memory and considered to be susceptible; and to those who had a belief and were considered to be not susceptible. This control enabled researchers to determine the extent to which their manipulation was successful, given the extraneous or participant variable of 'susceptibility to manipulation' which was influential on the dependent variable;

Design

The design for this study is certainly very complex for AS students. Nevertheless, I have presented it in all its complexity for the interested learner. At the end of the section, I have mentioned how it can be interpreted in a simplified manner for the AS level.

The design employed in the study was a highly complex design. 

A mixed factorial design was used for measuring the first dependent variable and part of the second one. Specifically, the interactional effect of both independent variables - inducement of love for asparagus, which was randomized across groups; and time, which was repeated within groups - was studied for the dependent variables of confidence ratings for the critical item and ratings given by participants for their desire to eat asparagus.

This factorial design is presented below schematically:

IV 1 - Love for Asparagus  / IV 2 - Time	Induced (EG)	Not Induced (CG)
Pre-manipulation (before being given feedback)	Factorial Condition 1 - Pre-manipulation of EG	Factorial Condition 2 - Pre-manipulation of CG
Post-manipulation  (after receiving feedback)	Factorial Condition 3 - Post manipulation of EG	Factorial Condition 4 - Post-manipulation of CG

A quasi design was employed for the rest of the second independent variable. Specifically, participants were identified as being either believers or non-believers and then compared with respect to their ratings for their preference for asparagus; and for the price they would be willing to pay for a pound of asparagus.

For the A-level, going by the course companions available, it is assumed that there is a single independent variable in the study - the first one outlined above, that of either inducement or no inducement of a love for asparagus. The dependent variable is grouped into a single one - that measuring a preference for asparagus post manipulation or its absence, through the use of five different questionnaires. With this simplification of the independent and dependent variables, the design becomes a simple random groups design with one independent variable manipulated at two levels.

Assignment

Participants were randomly assigned into either the control or experimental group for the manipulation of the first independent variable.

For the comparison of believers and non-believers on their preference for asparagus and price they would be willing to pay for the same, participants were identified as belonging to either of the groups on the basis of a questionnaire administered to them.

Quantitative Analysis

Since various measures of participant performance were employed in the experiment, a number of suitable descriptive and inferential statistics were employed for their analysis. These are outlined below:

• For the analysis of confidence ratings given by participants for the critical item on the test, mean ratings were calculated for the ratings given by the experimental and control groups for both sessions for each; and any change in the mean ratings for each group from the first to the next session was calculated. A repeated-measures ANOVA was then conducted to infer the significance of the differences in ratings of the two groups, ratings across time and interaction of the two. Partial eta-squared was also calculated as a measure of the size of the effect of the differences found.

• For the analysis of ratings given by participants for their desire to eat asparagus on the Restaurant Questionnaire, an identical ANOVA procedure as outlined in the previous point was conducted.

• For the analysis of ratings given by participants for their preference for asparagus on the Food Preferences Questionnaire, means and standard deviations of the ratings given by the believer and non-believer groups were calculated. The results were then subject to an independent-samples' t-test for inferring the significance of the difference calculated between the means. A point-biserial correlation coefficient was computed to estimate the size of the effect of the difference between means.

• For the analysis of the price participants were willing to pay for a pound of asparagus, the non-parametric Mann-Whitney 'U' test was computed to compare differences between the price choices of the believer and non-believer groups; since the options were presented on an ordinal rather than an interval scale. The results of the test were then transformed into a z-statistic for inferring the significance of differences. Finally, the option of 'never buying asparagus' was comparitively weighed by counting the number of participants in each group who chose the option. A chi-squared analysis was performed to infer the significance of this difference between groups.

Evaluation of Use of Laboratory Experimentation in Laney et.al.'s Study 1

Once the research methodology of any study is systematically and thoroughly outlined, as above, it is possible to evaluate the method without missing any aspect, as demonstrated below:

The most critical point to be kept in mind while evaluating the use of a research method in a particular study is to continuously present the points of evaluation in the context of the study in question. The answers on the paper should reflect the student's ability to evaluate the methodology of any study and not simply his knowledge of what the strengths and limitations of various research methods are.

Strengths of the Study

• High Internal Validity - As outlined above, the large number of controls employed in the present study lent it high internal validity. Particularly for the case of inducing a memory into an individual, the information given to him should appear highly authentic for its acceptance. Such authenticity is difficult to achieve as it is prone to confounding from sources such as the credibility of the individual giving the information, the context in which the information is presented, the amount of information given, etc. Such factors were carefully considered and controlled from influencing the dependent variable by means of giving computerized feedback to participants, including the information about the critical item along with that of other food items, providing minimal information about the 'memory' to prevent participants from becoming suspicious, etc.

• Possibility of Replication - The systematic and clearly outlined procedure lent the study a high degree of replicability. As an example, the standardized questionnaires used in the study have been clearly named and the questionnaires developed for the explicit purpose of the present study have been thoroughly described alongwith the procedure with which they were employed. As another example, the feedback given to participants by the computer has been clearly outlined. As yet another example, the details of the participants of the study - their number, educational status, gender ratio, etc. have been presented in the research paper. All of this information is very useful to any researcher who wishes to repeat the study as it is for verification or by making modifications to it after identifying gaps or scope for further investigation.

• Collection and Analysis of Quantitative Data - Quantitative data is more objective and also easier to verify than qualitative data. In the present study, data was collected in the form of ratings and selections from among options; and was subject to inferential statistical procedures such as ANOVA, t- and chi-squared analyses; all of which are amenable to verification in future and also to further statistical investigation.

• Ethical Control - It was possible for researchers to gain consent, to provide explicit instructions to participants and to fully debrief them, as outlined in the paper, because the research was conducted in a controlled setting. This gave a high ethical standard to the study. Also note that though deception was employed in the study, it was inevitable, given the nature of the aim of research and would have to have been employed for a study of this nature conducted using any other research method. 

Limitations of the Study

• Low Ecological Validity - All laboratory experiments suffer from artificiality, that is, the environment in a controlled setting never matches a real setting, which makes it difficult to determine whether findings would generalize to the real world. In the present study, though authenticity was attempted in terms of providing computerized feedback to participants and in assessing their liking for asparagus on a realistic-looking restaurant questionnaire, it cannot be automatically assumed that people would accept such feedback given to them by, say, for example, teachers who wish to encourage them to study and so claim that they have computerized, objective evidence that they liked studying as children. In the real world, it is easier to gage the motives of people who wish to encourage us towards certain acts and therefore, whether these results will hold true even in such cases needs to be determined with more ecologically valid studies. Further, in an actual restaurant, when a person kniws that he really has to eat the food he chooses, unlike in the study, and in which he is influenced by the choices of those eating with him, the presentation of food and so forth, whether he would choose asparagus needs further, realistic exploration.

---

*The following is a brief discussion illustrating how the strengths or limitations of any research method must not be blindly applied to any study without due consideration of its relevance to the study at hand-

When I discuss the Laney et.al. study research methodology with students, they frequently bring up 'demand characteristics' as a limitation, given that they have jotted it down as a limitation of laboratory experiments in their notebook. When I ask them to relate it to the study, however, they have a hard time.

The fact is that the experimenters had given due consideration to the possibility that experimental participants would have simply acted in accordance with the feedback they received, not out of the development of a false memory or belief but merely to comply with the feedback given to them by the computer. Accordingly, they never presented asparagus items in isolation, in neither the feedback nor questionnaires, making it nearly impossible that participants had guessed that the chief aim of the experiment was to manipulate their liking of asparagus; and also that their favorable response only to asparagus amongst all the food items was only a response to the demand of the experiment, highly unlikely. Relatedly, they thoroughly analysed the responses of participants on the social desirability scale for significance and found no significant differences between the experimental and control groups when compared on their desire to give socially favorable responses. This showed that their responses were not just an attempt to present themselves favorably to the researcher.

This study, therefore, demonstrates well how the applicability of any point of evaluation to a given study needs to be scrutinized carefully before being listed down. Students must make careful note of the fact that not every point of evaluation, be it a point of issues and debates or of research methodology, is unconditionally applicable to every study in the course.

---

Of course, there are many more strengths and limitations of the given study. However, those emerge when the issues and debates of the study are analysed. Would you like to know how you should go about analyzing those as well? Do let me know in the comments below.

(Search - Research Method for A level, A-level research methodology, Laboratory Experiments in A-level Psychology, Controlled Experiments in A level psychology, Paper 1 A-level psychology, Paper 3 in A level Psychology, A-level Psychology, CIE AS and A Psychology, IGCSE Psychology, Psychology 9990, Laboratory Experiments in Psychology, Laney et al, Asparagus - a Love Story, false memories, AS Psychology Exam Notes, Learning for A level Psychology)