Tutor Marked Summative Evaluation Proposal: Research Design And Bias

Research Bias in Adopting One Group Pretest-Posttest Design

Write about the Tutor Marked for Summative Evaluation Proposal.

Research is an instrumental part of understanding social issues. However, conducting sociological investigations is a challenging task because of the dynamic attribute of the society. According to Rubin and Babbie (2016), it is vital for researchers to adopt research designs that aim at eradicating partiality. Although Rubin and Babbie (2016) point out that it is impossible to have a perfect research design, it is essential to ensure that the research outcomes and interpretations are valid and reliable. This summative evaluation proposal analyzes the research bias in adopting the one group pretest-posttest design proposed and suggests a suitable approach to attain the director’s directives.

The one group pretest-posttest design is widely applied because of its simplicity and time consciousness. According to Rubin and Babbie (2016), this methodology entails measuring the dependent variable in a single group in a pretest, then, administering the trial stimuli and post-measuring the dependent variable in the posttest. The image below is an illustration of how a one group pretest-posttest method operates.

Source: (Rubin & Babbie, 2016,p. 254)

However, scholars have disparaged this procedure claiming that it is biased. Particularly, the one group pretest-posttest framework has been faulted for its bias and threats to internal validity. According to Rubin and Babbie (2016), the one group pretest-posttest design fails to consider other factors that affect research outcome other than the stimuli utilized in the study. Logically, the society is not static. In this respect, there is a high possibility of externalities altering the observations. For instance, in the case of the community program, externalities like prior knowledge, an interaction between study subjects, and employers’ preferences can affect the employability skills evaluation and time taken to secure a job.

Additionally, the method compromises internal validity. According to Rubin and Babbie (2016), internal validity in this methodology is affected by maturation, testing, and instrumentation. Maturation refers to the alteration in observations due to changes in time. There are some employability skills which develop with time. For example, a skill like communication evolves with time. In this respect, since the one group pretest-posttest method lacks a control group, it is impossible to rule out the effect of maturation on the research outcomes. On the other hand, the act of giving the study group a pretest can have a significant implication on the posttest results.  For example, taking an employability skills test for the second time can cause an improved performance which might not be due to the effectiveness of the training program. A person taking a test for the second time is likely to score higher than one who is tackling it for the first time.

Recommended Design

Instrumentation is another factor that imposes bias and lowers internal validity when using a one group pretest-posttest method. The instruments used to make the observations are essential in exhibiting the effect of the stimuli on the study subject. For instance, in the case of the community assessment, the people who will be offering the training are not the ones giving out the employment. In this respect, the variation in skills required by different employers can affect the results of the study. Since the one group pretest-posttest method lacks randomization, it entails assignment bias. Consequently, this research design will not be sufficient in addressing the two research objectives outlined for the community assessment study.

The posttest-only control group research design is the best method for this assessment. According to Rubin and Babbie (2016), the posttest-only control group method provides optimum control for threats to internal validity by randomly assigning the study participants to experimental and control groups. Notably, the control group was lacking in the one group pretest-post test approach. The essence of having a control group is to make comparisons of the effects of the treatment applied to the study subjects. Additionally, the random allocation of the study subjects minimizes the selection bias.

For example, in the community study, participants will be randomly allocated to the groups. The reason for randomizing group allocation is to come up with varied study participants. In fact, the posttest-only control group design addresses almost all the bias and internal validity concerns raised while using the one group pretest-posttest strategy. Firstly, Rubin and Babbie (2016) point out that the posttest-only control group framework solves the maturation issues. According to Rubin and Babbie (2016), if the maturation can influence the study observations, then, it should be influential in both the experimental and the control groups. Since the maturation effect is observed in both groups, its effects on the study outcomes and interpretation become insignificant.

Additionally, randomization eliminates any significant differences between the experimental and control groups. In this respect, both groups are regarded as equal before the application of the training program. Moreover, Rubin and Babbie (2016) posit that since random assignment is used in the posttest-only control group research design, statistical regression effects are suppressed. In the posttest-only control group design, bias introduced by the pretest is eliminated. As earlier elaborated, pretesting can have a significant impact on how the study subjects respond to the posttest. In this case, the posttest-only control group method ensures that all the changes observed in the study emanate from the effects of the stimuli applied. Furthermore, for the community study objectives, it is impossible to pretest some of the activities. For instance, it is challenging to pretest time used to find employment. Consequently, using the posttest-only control group design eradicates these complications.

Assumptions in Implementing the Posttest-only Control Group Design

In any research study, it is difficult to control all the factors that can alter the outcomes. Particularly, in social studies which occur in uncontrolled environments, researchers must make some assumption in their studies. Unlike laboratory- based experiments where all conditions can be controlled, the society is more complex to manage. In the posttest-only control group study, it is assumed that the random assignment of subjects removes any significant initial variations between the experimental and control groups (Rubin & Babbie, 2016). Remember, all research studies are supposed to be impartial, and randomization is used to pursue this course.Additionally, in this method, its is assumed that any interactions between the two study groups will not have significant implications on the result. In a community set-up, one cannot completely restrict the interaction between the study subjects.

When dealing with individuals from varying cultural backgrounds, it is vital to observe the appropriate cultural competency and ethical standards. According to Rubin and Babbie (2016), cultural competence implies being conscious of and responding appropriately to how cultural variations and factors influence research and its interpretations. The person conducting an investigation should appreciate that cultures differ and people attach value to their cultures. In this respect, all the tools and procedures that are utilized in conducting social research must illustrate high levels of cultural competence and ethics.

According to Rubin and Babbie (2016), involving sufficient and representative participants from minority and oppressed population is one of the ways of illustrating cultural competency. Researchers should not discriminate a given section of a population. Instead, researchers should focus on attaining equality when selecting their study subjects. With scientists striving at attaining both internal and external validity in their studies, it is essential to use study subjects that reflect the diverse attributes of the community. In this respect, this evaluation study will ensure that its participants reflect the status of the community. Additionally, the assessment will avoid any elements ethnocentrism. The use of cultural stereotypes introduces bias and affects the validity of research outcomes.

Additionally, before conducting the evaluation, it will be vital to seek permission from the community leaders and members.  Seeking the consent of the participating community is a crucial way of ensuring that a study agrees with ethical standards (Rubin & Babbie, 2016).  Apart from ensuring equality and seeking the consent of the community, this evaluation will also use approved methodologies to address its objectives. According to Rubin and Babbie (2016), ethically oriented studies should use methods that are acceptable in the field of practice. In this respect, the evaluation will utilize internationally recognized research methods and maintain the confidentiality of involved subjects.  

The evaluation will take several actions to ensure that it is conducted in a way that is sensitive and appropriate for low-income families. Firstly, the participants will be adequately compensated during the research period. Secondly, transportation will be provided for all the participating individuals during the assessment period. Additionally, the ministry will source for employers where the members can apply for employment after the evaluation period. The evaluation will also be designed in a way that it does not harm the participants healthwise. Finally, the ministry will offer additional insight to the community members on ways of increasing their income. Importantly, all these strategies are directed towards improving the economic position of the participating community.

Conclusion

Conclusively, it is essential to use research methods that optimize validity and reliability. Despite the one group pretest-posttest approach using limited resources, it compromises internal validity. In this respect, using the posttest-only control group design will ensure that the observations are valid. Furthermore, the evaluation will comply with all the ethical requirements in conducting social research and ensure that it is sensitive to the needs of the participating low-income families.

References

Rubin, A., & Babbie, E. R. (2016). Empowerment Series: Research Methods for Social Work (9th ed.). Boston: Cengage Learning.