research guide – R3ciprocity Blog

In the sciences, often we are faced with the difficult problem of not being able to explicitly measure what we theorize about. For example, it is impossible to measure things like ‘love,’ ‘altruism,’ and ‘hate.’ These difficult to measure higher order constructs often are referred to as latent variables in the context of a study. Scientists must operationalize-- basically convert-- these constructs into something more measurable. To do operationalization well, one must think about how to measure variables in as many ways as possible and capture the construct as directly as possible. https://www.youtube.com/watch?v=kkyB3zAtZxY Check out this video if you need more help with operationalization! This post was written by Dr. Stephanie A. Bosco-Ruggiero (PhD in Social Work from Fordham University Graduate School of Social Service) on behalf of Dr. Dave Maslach for the R3ciprocity project (check out the YouTube Channel or the writing feedback software). R3ciprocity helps students, faculty, and researchers by providing an authentic look into PhD and academic life, and how to be a successful researcher. For over four years the project has been offering advice, community, and encouragement to students and researchers around the world. Examples of operationalization from the social sciences To explain just a bit more about the concept of operationalization, think about a higher order construct that is difficult to measure like the ones noted above, or even something in physics like, what is a particle or what is gravity? Scientists of all kinds, including in the social sciences, must get at a way of measuring what something is in concrete terms so that others understand what they are studying. Scientists might study how a latent variable impacts a phenomena or how much a latent variable is present. https://www.youtube.com/watch?v=-RaNjHT5K2M You might watch this video for understanding of latent and manifest variables. Here are some additional examples of constructs that need to be operationalized so they can be studied. What is a successful program? Well, how are we going to measure the construct, “success?” If it is a social service program meant to keep kids out of trouble after school we might operationalize success as the number of kids participating in the program who got in trouble with the law versus the number of kids who didn’t participate in the program who got in trouble, if it’s a randomized experiment. You could determine the success of this program in keeping kids out of trouble in the short-term (e.g., 6 months) or long-term (2 years after completing the program). Time can be another element of a construct. Success may also be operationalized in terms of participation. Did kids want to participate in this program? Did it have high enrollment? It may have had high enrollment but the kids themselves reported that it wasn’t very fun or valuable to them. So, success of a program is not just about how many participated but also perhaps how participants rated the program in terms of its value to them and in keeping them out of trouble. Did they get something out of the program, or did someone say they had to participate? In a related study, we might want to operationalize the concept and term “juvenile delinquent." We've heard that term in the media and elsewhere, but in a study we have to measure this term in a tangible way so we know who we are talking about and studying. Is a juvenile delinquent a kid who has been in juvenile detention, or is it a kid who has shoplifted once? Is it a kid who assaulted someone? Is it a kid who has broken the law once, or several times? We have to be precise in our measurement of terms and concepts so scientists and others understand precisely what we are studying. Research studies that are published in peer reviewed journals and presented to the field must include clearly operationalized constructs, otherwise others in the field will have difficulty comparing their results to yours and to results pf other studies. If it is not clear what a juvenile delinquent is in the context of my study, how can other scientists use my results to create additional research questions? How can professionals in the field of juvenile justice use my study if I have operationalized juvenile delinquent in a different way than most other researchers? You will obtain a lot of information about how your variables were operationalized in previous studies from your literature review. And while we are on the subject of literature reviews, here is a blog about how to write a good literature review. Now there may be some utility to using the outcomes of my study when I have operationalized juvenile delinquent differently from how it is usually operationalized. My operationalization may be more accurate or useful in some way to people on the ground doing the work day-to-day. My operationalization may show other researchers that they have to be more precise in their own operationalization of a construct or look at it or measure it slightly differently. But if my operationalization is really different from how it was done in previous research with no explanation, my research may be viewed as not being as valid or useful to scientists and those working in the field. Measuring latent variables that have been operationalized This brings us to how you measure a construct that has been operationalized. Different researchers may operationalize a construct similarly but measure it differently. For example, two researchers may agree that children who can be characterized as leaders exhibit certain traits, but they may use different approaches to measuring the presence of a trait or the effect of one latent variable on another. One may have the child or parent complete a survey that measures leadership in children using items that operationalize the construct, while another researcher may observe the child in a classroom or sports environment. The two researchers may identify the same children as having leadership traits and measured the construct of leadership similarly but identify the latent variable in two different ways. Different ways of measuring a construct: Surveys: Many researchers use surveys to measure the presence of a latent variable. They have a person complete a set of questions to determine if a construct is present. A survey instrument must be as precise as possible in getting at the construct. This can be tricky to do, and internal and external validity (see below) are important to think about when measuring a construct this way. If a survey has one question to capture a construct the survey probably doesn’t measure it well. You have to ask a lot of different questions to measure a construct. It can be annoying for responders to have to answer so many questions that seem to be getting at the same thing, but that is the point; you have to make sure, using many items, that you are measuring and capturing the concept accurately. Check out this blog post on how to analyze a survey! Observation: Observing a latent variable is another way to measure a concept and collect data about it. A social scientist may distribute surveys to measure the “quality” of teaching in a school, but may also, or instead, observe teachers in the classroom. When making observations to measure a construct, it is important to be precise in how the latent variable is being operationalized. It is important to consistently measure the construct in a way that gets at the meaning of “quality” as you have defined it. Best practices in operationalization: Be as specific as possible. A survey instrument measuring the concept of sadness with 40 items is better than one with 10, as long as the 40 items measure some aspect of the construct you are interested in. If you want a more general understanding of the construct, 10 items might be fine. A good operationalization often is consistent with how other researchers have operationalized a construct, but you can come up with novel operationalization, as long as you point out it is new or different in some way. Experiments: Operationalization is used in looking at the influences of independent variables on dependent variables. How are you operationalizing your independent variables? Are you manipulating independent variables in a way that help you consistently measure that construct’s influence on the dependent variable? When it comes times to analyze your data, think about how you operationalized your construct and what the data are telling you about relationships between variables (experimental) or the presence of latent variables (observational). By the way, here is a blog I wrote about analyzing survey data for a large research project. Reliability in operationalizing latent variables Validity and reliability are very important to understand when it comes to operationalizing latent variables. Dave has several vlogs that discuss both. It is important when conducting high quality research that you have the most reliable and valid constructs and measurement instruments as possible. Here is Dave discussing reliability and validity in inferences: https://www.youtube.com/watch?v=5lmUFIxfuAs Surveys can be imprecise in measuring a construct, and cause problems with reliability or validity of your findings. Problems with reliability of a survey, for example, may arise when survey respondents answer items meant to measure a construct in the same way, in unexpected ways. As noted, it is important to have many items in a survey to measure a latent variable. One reason is that if you ask the same thing, in several different ways, if the latent variable is present, the respondent should answer each item consistently (e.g., saying no to all items in a subscale if the construct is not present for them). If they answer inconsistently, their response and data may not be reliable. For example, if a survey aims to measure whether students who are first in their generation to go to college experience alienation when attending an elite private school, the survey should aim to measure alienation precisely using a subscale, perhaps, comprised of a number of items. Let’s say alienation is measured by a subscale comprised of six questions, and high alienation is represented by high numbers such as 5 “strongly agree;” if the student does experience alienation, he or she should consistently choose responses with a higher number. But if a number of students who do experience alienation choose 2 for some items and 5 for other items in the alienation subscale, with 2 indicating lower levels of alienation, either they are not interpreting the question correctly or the researcher has not worded items in the subscale clearly. The researcher should doubt whether alienation has been reliability measured in his survey if many students have chosen responses that indicate low alienation and high alienation. The responses contradict each other, so did he or did he not experience alienation? Here is Dave talking a bit more about common problems with research surveys: https://www.youtube.com/watch?v=-voq7AzxTIM Validity in operationalizing latent variables There are many types of validity, and you can learn more about validity from Dave’s vlog, but I want to briefly mention here how validity relates to the operationalization of variables. There is internal validity and external validity. If you measure a concept reliability, you are accurately measuring a concept and therefore your project has high internal validity (with regards to measurement). Where a problem can arise is in those instances when the way you have operationalized a latent variable is different from how other researchers have operationalized the variable. If you cannot compare the results of your research with another’s’ results because of a discrepancy in operationalization, your study may end up having low external validity. It doesn’t measure the concept as others do and may not be accurately measuring the construct at all. Theory As Dave discusses in his vlog, there can be a big divide between thinking about a theory and measuring the concepts embedded in that theory. Theories can be grand and overarching. They help guide our thinking about what needs to be known, what might happen, and how to go about operationalizing and studying phenomena. It can be difficult to operationalize elements of a theory or apply that theory to your area of research. For example, in social work we tend to look at things from an ecological standpoint. What influences a person to be as they are, at the micro, mezzo, and macro level? If we want to operationalize this theory to look at risk for child maltreatment, we will need to define risks and protective factors as each level of their social ecology. Is funding for parent training at the macro level good or bad? Is a history of the offending parent being abused themselves in the past as a child a risk or protective factor? How are we defining good and bad influences and their presence? What have other researchers looked at? Are these the right things to look at when applying ecology theory to the problem of child maltreatment? The bottom line about operationalization Operationalizing constructs and latent variables in research is tricky. It takes practice, but you should learn more about it in your introductory research classes. Your professor may begin with a very simple example of how to operationalize a variable such as depression. We can use a tried and true evaluation instrument to operationalize the persistent sadness and physical symptoms associated with depression, which would probably lend our study a lot of external validity. Or we can try to operationalize it through observing people’s affect, moods, manner of speaking, and manner or completing tasks. If your respondents are not consistent in choosing items equated with the presence of depression, and they are depressed, what do we do? How do we know if they are depressed? In this instance there may be a problem with how we operationalized depression or with the survey instrument we are using. We then have to go back to the drawing board and make sure our questions are clear and consistently measure the construct we are studying, so we can also say our study and mode of operationalizing a variable has high internal reliability. There are several places were you can learn more about operationalizing variables including from your research textbooks, professor, or free courses and videos such as this one from Coursera. You must make sure you understand elementary research terms and concepts such as operationalization early in the coursework phase of your PhD. You cannot do good research without understanding operationalization, and you will need to understand the basic mechanics of research for your comprehensive exams. If you don’t know what a comprehensive exams is, or have questions, check out my last blog on this sometimes intimidating phase of doctoral studies. If you enjoyed this blog, check out other posts at https://blog.r3ciprocity.com