[Social Science Research 101] Quantitative research questions and model building

My steps to do quantitative research

In this article, I will not talk about mathematics and formulas but rather how to set up a hypothesis. 

1. Start with your problem statement, your story, and everything else (it could be a summary of your novel)
2. Transform them into a set of concepts 
3. Transform them into variables 
4. Make connections between variables (=Hypothesis) 
5. Identify the most appropriate statistical methods 
6. Analyze the hypothesis with statistical software 
7. Interpret the results 
8. Put your findings in writing 

Here is an example from my recent paper that used regressions with an interaction effect. In the (1) problem statement stage, it is totally fine to be vague and the questions could be non-academic terms! Please do not limit your inquiry at this stage. You will figure out (2) how to transform it into concepts and (3) how to operationalize them by reading the relevant literature (find the appropriate terms in the literature). To do this, you can go directly to the measurement section of the paper when you do the literature review.

Step	Example for the Research Question
1. Problem Statement	I see lots of parent(s) lost their jobs and experienced serious financial shock during the COVID-19 pandemic. This shock may have a negative impact on mental health among adolescents (related to the life-course perspective too), but is that association different for boys and girls?
2. Transform into Concepts	Concepts: Household financial decline, mental health outcomes, gender.
3. Transform into Measurable Variables (Operationalization)	Variables: – Independent: Household financial decline (measured as a Likert scale) – Gender (e.g, categorical: male, female) – Dependent: Mental health outcomes (e.g., measured as a continuous variable using PHQ-9 and GAD-7)
4. Make Connections Between Variables (Hypothesis)	Hypothesis: 1. There is a positive association between household financial decline and mental health outcomes. 2. There is a moderating effect of gender on the relationship between household financial decline and mental health outcomes, such that the relationship is stronger for one gender compared to the other.
5. Identify Appropriate Statistical Methods	Statistical methods: To test moderation, a multiple regression analysis with an interaction term between household financial decline and gender can be used. Additionally, descriptive statistics may be used to characterize the data.
6. Analyze Hypothesis with Statistical Software	Use statistical software (e.g., Stata, R) to perform the multiple regression analysis and calculate interaction effects between household financial decline and gender on mental health outcomes.
7. Interpret the Results	Interpret the regression coefficients and interaction effects to determine whether gender moderates the relationship between household financial decline and mental health outcomes.
8. Put Findings in Writing	Write a research report summarizing the problem, methodology, results, and conclusions. Present findings in tables and graphs, and discuss implications and limitations.

To set up a problem statement (1), connect variables (4), and identify the most appropriate statistical method (5), it is essential to know what statistical methods are available. Research in social science relies heavily on collaboration, so you may collaborate with someone who has strong quantitative skills. 

Despite this, sometimes a researcher’s scope of imagination is limited by their toolbox. Therefore, it is always beneficial to “know” research methods to ask research questions.

Developing quantitative skills is also very accumulative, and the learning curve is getting steeper. At first, it is very challenging, but as you become more proficient, you will learn the next methodology much faster (Trust me!). 

Multiple Regression Model Building

To build the simple multiple regression models, you first need to think about the independent and dependent variables you want to test the relationships. Let’s say I want to test the relationship between the years lived in the US (IV) and perceived social support (DV). Then, I need to test the control variables with that key IV. The conceptual model for this multiple regression model is as follows:

Here are some common control variables in social science research. Depending on your research questions, they can be the key variables to test in your model.

1. Demographic Variables: Age, gender, race/ethnicity, nativity, marital status, …
2. Economic Variables: Employment status, level of education, income, subjective SES, household wealth, etc.
3. Geographic Variables: Urban/rural location, region…

[ADVANCED] Moderation and Mediation Effect

Here is a brief comparison of mediator and moderator since lots of students are interested in using them in their research design. Simply speaking, the mediator is about “how and why,” and the moderator is about “for whom and when.”

Concepts/design	Mediator	Moderator
Nature of the variable	a state, which is a temporary condition of mentality or mood, transitory level of arousal or drive, and currently evoked activities, behavior, or process	a trait, which is a relatively stable characteristic, innate attribute, enduring process, or disposition. It can also be a background or contextual variable
	Emotionality (i.e., physiological reaction)	Subject of the test (e.g., Math) or gender
Function in a causal relationship	A third variable that links a cause and an effect	A third variable that modifies a causal effect
Type of questions	How and why cause leads to effect	For whom and when cause and effect occurs
Analogy	Dominos	Dimmer switch for lighting
Sequence of operation	Follow the independent variable and precede dependent variable	Precedes both independent variable and dependent variable
	E.g. Test difficulty precedes emotionality, which precedes test anxiety	E.g. Subject of the test precedes test difficulty and test anxiety
Relationship with the independent variable	Correlated with the independent variable	Uncorrelated with the independent variable
	E.g. Emotionality is correlated with test difficulty	E.g. The subject of test (math/science) is not correlated with test difficulty (i.e., designing the tests to be equally difficult for math and science before the experiment)
Design control	Manipulated or observed	Typically observed
	E.g. Emotionality can be either measured or manipulated depending on the role and design	E.g. There is no need to manipulate the subject of the test, the subject of the test is simply observed

Three types of Moderating effects

1. Enhancing effect (+): The presence of a moderator enhances the effect of the predictor on the outcome variable.
2. Buffering effect (-): The presence of a moderator reduces the effect of the predictor on the outcome variable
3. Antagonistic effect: The presence of a moderator reverse the effect of the predictor on the outcome variable

▶️ Moderation Example: The relationship between the experience of immigration and depression is moderated by social support. (FYI: social support is an extremely common moderator (buffering effect) used in mental health literature)

▶️ Mediation Example: The relationship between socioeconomic status and life expectancy is mediated by access to healthcare.

!! For your information, testing the mediation effect using cross-sectional data requires very heavy theoretical support since the cross-sectional data do not offer evidence for temporal precedence between variables.

For the modeling moderation and mediating effects, the slides by HS Wu are really helpful and well-described! Please check out the slides here: https://www.bgsu.edu/content/dam/BGSU/college-of-arts-and-sciences/center-for-family-and-demographic-research/documents/Workshops/2011-workshop-How-to-Model-Mediating-and-Moderating-Effects.pdf

Last but not least, it is always great to draw your own model like I did above! You can use Powerpoint or Google Slides, as I did above!

Based on what I have written in this post, I am sharing my paper proposal template. You can click the link to download it in the Word Format or your own paper proposal. The information needed in this proposal format is the information you need when you write the methods section in the journal article. So, it is really great to have a concrete plan in the ideation stage!