Quantitative Techniques Used in a Journal Article: Analysis and Evaluation

This document is your formal analysis of the quantitative methods used by the author of and economic journal article. 

What methods and data were used for the article?  Were proper quantitative procedures followed by the author? 

What key or critical information was not addressed by the author to support his/her conclusion(s)? 

What improvement could the author have made to the quantitative method to improve the results and conclusion(s)? 

How could the study be improved to promote economic policy implications?  Below is the research evaluation suggested points

What is the clear purpose of the research?

Who are the researchers and do they have a quantitative background?

What is the researcher’s hypothesis? Is it focused and clear?

What are the data variables analyzed?  What is the objective or response (Y) variable? What are the independent or X variables in the analysis?

Was the target population of the objective variable infinite or finite? Was the study experimental or observational?

Was the data collected cross-sectional or time series? Are the study data Ratio or Interval, Ordinal or Nominative?

What Sampling technique was used?

1. Stratified Random Sampling

2. Cluster Sampling

3. Systematic Sampling

Was the sampling probability or convenience sampling?

Was there sampling under-coverage relative to population members excluded if surveys were used what type. Were they Phone, Mail, Web, Interview?

Was there significant non-response in sampling? What was done to prevent potential data sampling response bias and sampling or recording error?

Was a random sample done with or without replacement? Was the sample size adequate to determine analysis reliability?

What data analysis was performed to determine data characteristics and distribution? E.g. time series plots, histograms, bar charts, ogive charts?

Does the researcher describe objective data sample mean, median or mode? Did the researcher determine if the objective data was skewed and not normally distributed?

Did the analyst describe the modality of the data is it multimodal, bimodal? Does the analyst describe the Range of the sample data, variance of the sample or the standard deviation of the sample data?

Do the variables in the researcher’s hypothesis reliably describe the objective variable/s variation?  If so at what confidence level?

Does the researcher mention potential data outliers? What are the covariance and correlations of the sample independent variables to the objective dependent variable?  Are they provided?

What are the directions of the relationship between each of the sample independent variables (Xs) and the objective (Y) variable?

Does the researcher describe the linear relationship between each X variable and the objective variable Y? Does the researcher provide the slope and intercept of the linear relationship?  Note this relationship may be determined to be non-linear and the researcher must state this if it is so.

Did the analyst use factor analysis in the research. Did the researcher quantitatively determine the accuracy of the analysis (Mean squared error, Root Mean Squared Error, Mean absolute percent error)?

Did the researcher provide information on the analysis error terms relative to information omitted from the model or research algorithm? (E.g. what was omitted in their quantitative analysis model or algorithm.)

In what manner did the researcher determine analysis reliability or confidence in the analysis results? What statistics were used to determine this? Were the results reasonable from the perspective of others besides the researcher him/herself? What others were involved in inspecting the results for reasonableness?

Do you believe the research is the best that could be conducted to reach the researchers conclusion? What effect will the research conclusion have on society or the economy?

What would you change in the analysis to improve the analysis and support a better conclusion? (E.g. use more data observations, or better choice of independent variables, or have more conclusive analysis of algorithm or model error (residuals).