I had a chance to talk with Catherine E. Harnois about her ground-breaking book: “Feminist Measures in Survey Research”. She shared what she’s been thinking since writing the book and some tips and tools to help embed equity in survey-based data products.
When I was a graduate student, I was studying mathematical statistics and was interested in ways to use my work as a feminist. I took several women’s studies research methods classes and asked a variety of professors how to use quantitative statistics from a feminist perspective. Without exception, I was told it wasn’t possible. That “regression was deductive” or “quantitative statistics turns everyone into one patriarchal average” etc. knew this wasn’t all there was and I kept looking. Then I found Dr. Harnois’ book “Feminist Measures in Survey Research” and it felt like an oasis in the desert. I was so happy to find someone interested in some of the same questions. I was thrilled to find someone who knew some answers. Over the years, I’ve purchased many copies of this book and refer many of my students and colleagues to it. Now I’ve gotten the chance to talk with Catherine who kind enough to do a Q&A with us.
Heather: We really like your examples if multiplicative models as a way to do intersectional analysis. This works will for the large national surveys in your example. What about with smaller surveys? We run out of statistical power fast. Suggestions?
Catherine: I really like this question, and all the others too! With this one, I think it is useful to think about the relationship between any particular study (small or large scale), and all of the other existing research and research possibilities in the future.
The way I think about this is that a smaller survey project is often able to involve more nuanced measures – measures meant to really get at a particular concept or experience or behavior among a particular group or groups. An example I really like is Evelyn Simien’s National Black Feminist Study (N=500, which is still on the large side but bear with me). She creates measures of black feminism and womanism and includes lots of attitudinal questions that attend to race and gender issues at the same time. So, extremely helpful for understanding black feminism and the intersection of race and gender more broadly. But the study includes only African American respondents, so we’re left wondering how these items, and the relationships among them, might differ, or not from other social groups.
So, applying this to small samples, I would say that one especially valuable aspects of projects with small samples is that the researchers can be really careful with their measures, develop contextually specific / relevant questions to assess the dimensions of life that they see as relevant, and then test their specific hypotheses or conduct their exploratory analyses. But then the question is, what makes this isolated, contextually specific analysis intersectional??
And there I would say that such studies can be intersectional in many ways. First, when developing the research question, hypotheses, measures, sample etc., the intersectional researcher will have considered the ways in which inequalities come together in this particular context. That intersectional theoretical framework informs the research design. Then, once analyzed, that same intersectional framework informs the interpretation of the specific findings and the way they are communicated. And, last but not least, if we consider this particular small-scale study in relation to other existing work in other contexts, we can see what patterns are similar, which are different, and again use an intersectional framework to contextualize not only the researcher’s present findings. The results from this single study can also provide a starting point for other existing studies, that may have inadvertently put forth general claims that in reality are specific only to a particular social group or context.
It is my view that a small-scale survey, administered to a small and even relatively homogenous group can be intersectional, as long as it is designed and interpreted through an intersectional framework. Curious to know if others thinking about these things agree!
Heather: How do you recommend we deal with dynamic sociodemographics such as gender and sexual orientation?
Catherine: From a sociological point of view, almost all sociodemographics are potentially dynamic, and also contextually specific 🙂 Think age, education, certainly income and work force participation, parental status etc. That said, I think there’s several different ways of acknowledging and/or modeling the dynamism of gender and sexuality. A few different literatures are relevant.
First is research on race and ethnicity. As you likely know, there’s an ongoing, complex global argument about whether surveys should even ask individuals about their race or ethnic statuses. One position is that doing so essentializes and reproduces socially constructed categories (this is the position taken by France and Australia, among other states, organizations and individuals, that tend not ask questions re. racial/ethnic identity in their surveys). The contrasting position (articulated by the American Sociological Association among other organizations, states, and people) is that if we don’t ask people about their racial/ethnic identities, then it’s very difficult to track inequalities.
With respect to the dynamism of gender identities, Aliya Saperstein (Stanford) and Laurel Westbrook (Grand Valley State) have been doing very interesting work on this issue. They’ve been investigating various approaches for measuring not only gender identity, but also gender performances. See Hart et al (2019 in Journal of Health and Social Behavior) for a good example. There is also quite a bit of older quantitative work that documents variation in gendered behaviors, attitudes and ideology too. Sandra Bem’s work is one example. Elizabeth Cole’s (Michigan) work also is excellent for intersectional (gendered*racial) identities and attitudes.
With respect to sexual orientation in surveys, some foundational work here has been done by public health scholars (and of course earlier on by Kinsey), who saw early on that it was important to draw a distinction between sexual identities and sexual practices. This is because identities might be more closely associated with some factors of social life more than behaviors (identities might predict activist behavior and involvement with social movements or community organizations), but sexual behaviors might sometimes be better predictors of “high risk behaviors” or particular health outcomes. This is especially true when we’re talking about stigmatized or marginalized identities.
A cool theme throughout all of this research is (race/ethnicity, gender, sexuality) is thinking about the circumstances in which these identities are likely to change, and the implications of that change for behaviors, or health or whatever dependent variable one might be interested in 🙂 Longitudinal research here can be especially revelatory! There’s a well documented phenomenon on racial/ethnic identities in Brazil, showing that as people move up the class ladder, they’re more likely to identify as white. There’s a tiny bit of research in the US (Penner & Saperstein) that shows this may happen in the US too.
Getting back to the bones of the question, I would just say that the way to deal with dynamic identities and statuses is to (1) acknowledge that they are dynamic (2) measure them in a way that makes sense for the research question (3) be clear about why the measurement is appropriate and (4) be clear about what you are implying by using this measure and what you are not implying.
I would like to add that sexual identities seem to be changing and expanding particularly rapidly, so it’s extra hard for large-scale surveys to keep pace and to assess the meanings and outcomes associated with very particular identities. And in large-scale surveys, often only a few people claim a particular identity. For example, the American National Election Study now includes a third gender option, but only a handful respondents choose this option. I think this is another place where smaller-surveys studies can be especially valuable.
Heather: How can we include sociodemographic identity in a model as a social institution, not a personal characteristic?
Catherine: So that I don’t totally confuse everyone, let me start by making a conceptual distinction between social statuses and identities. I’ll use the term “social status” to refer to a socially recognized category and its associated meanings / norms in a particular social context. Here I’m drawing from Judith Lorber’s Paradoxes of Gender. She defines gender statuses as “the socially recognized genders in a society and the norms and expectations for their enactment behaviorally, gesturally, linguistically, emotionally, and physically.” (1994, 30). She goes on to define “gender identity” as an “individual’s sense of gendered self.” She sees gender statuses as being part of the social institution of gender, and identities as part of gender at the level of the individual.
I think one good thing to remembers is that all levels of social life are interconnected and mutually influential. (See Risman’s 2017 article for a very nice discussion of this!) So, in my mind, it’s not really an either/or situation. When we use a categorical variable like gender status or sex in a statistical model, we are never just modeling a personal characteristic, but the combination of social institutions and cultural practices, institutional arrangements, that have produced particular gender categories along with meanings, expectations, rewards and opportunities attached to them. It’s never just the individual.
Now let me loop in research from sociological social psychology. This literature emphasizes that when the focus is individual identities, as opposed to social statuses, these identities have to be understood in relation to macro-level aspects of society. Identities are socially created, historically and contextually specific. When we think of ourselves in terms of this category or that category, this term or that term, it is at least partly because of the social created meanings associated with those categories and terms. Even if we disavow a particular identity, the identity that we are distancing ourselves from, and the meanings and hierarchical position of that identity, is culturally rooted.
An example of using a status-based variable but linking it to social institutions and interpersonal interactions : João Luiz Bastos (Federal University of Santa Catarina, Brazil) and I wrote a paper about workplace gender discrimination and sexual harassment, and its relation to gendered health disparities. We used data from the General Social Survey (GSS; 2002-2012) and used a really simplistic measure of gender statuses (the variable SEX), along with variables assessing perceived gender discrimination and sexual harassment at work. A few interesting points: First, I believe the SEX variable isn’t really even a survey “question” –the interviewers were told to “code respondents’ sex”. Second, it seems likely to me that interviewers were recording the respondents’ gender status, not their sex. So, some serious limitations! But, obviously the GSS is a really valuable survey for analyzing all sorts of things. So, we used this variable to look at gender gaps in health, and argued that:
“The present study illustrates one way in which quantitative analyses can clarify the processes through which gender inequality is embodied at the level of the individual. Rather than taking “a dichotomous classification of bodies as a complete definition of gender” (Connell 2012:1675), here we have conceptualized gender categories as socially constructed statuses, and we have interpreted them within the context of gendered processes (e.g., discrimination and harassment) and within gendered social institutions (e.g., the workplace). We contend, then, that the “gender effect,” to which we referred above, materializes in the form of a statistically significant regression coefficient due to capturing these and other complex social processes occurring at multiple societal levels. Were it not for the social construction of gender, based on social, cultural, and biological meanings of what makes gender, the aforementioned “gender effect” on health-related outcomes would likely not be consistent or even detectable through our analyses.”
Sandya Hewamanne (Essex, UK) and I have a forthcoming article at the journal Gender Issues called “Categorical Variables Without Categorical Thinking? A Relational Reading of the Sri Lankan Demographic and Health Survey” (DOI: 10.1007/s12147-020-09252-5) where we tackle this issue directly. One of our big points is that you can interpret survey data in ways that recognize that social statuses like gender are socially created and relational. Another theme is seeing survey research, especially global survey research in the context of historic global inequalities and colonial legacies.
All that said, if we are really interested in complex identities, and here I don’t mean only the gender statuses that we identify with but the more psychologically complex ways in which we might identify as more or less masculine and/or feminine, or varying combinations of these things, or not at all, or how these identifications might change in various situations, then we could go back to Bem, and also to Saperstein and Westbrook to think about how to measure what we might call a “gendered sense of self” (e.g., masculine / feminine / androgymous/ non-binary) as opposed to gendered “social identities” of gender (e.g., man, trans, genderqueer, woman).
Heather: In your book you discuss measures of discrimination – single item or multi-item. How do we deal with the self-rating issue that’s present in so muchof our research and surveys? Abused kids don’t usually know that they are abused kids.
Catherine: Depends on what we’re trying to measure, and how much space we have in the survey. If we are trying to assess whether or not someone feels that they have experienced “discrimination” AND thinks about it in those terms, then maybe a single item question is appropriate. But if we are trying to assess whether or not people have ACTUALLY experienced discrimination, regardless of whether they think about it in those terms, then we are probably in the land of multi-item. Not only because we need to ask about several types/forms of “mistreatment” (Unfairly fired? Unfairly denied a promotion? Unfairly not hired? etc.), and probably we need to also ask about why it occurred. But even with these latter questions, which we could call as asking about “perceived mistreatment” we are still in the realm of perception.
What you say about abused kids is right, but it’s true for adults too. Women are often paid less because of their gender and but in many cases do not know it because our culture discourages talking about salaries / wages (and I believe some workplaces explicitly policies forbid it). When people aren’t hired for a job – which obviously happens all the time – they almost never know why they weren’t hired. Self-reports are limited. Research about how question wording affects reports of sexual assault is really illustrative here.
Big picture, I would just reiterate that no one study, or even a single methodological approach can do it all. Methodological pluralism is really needed for understanding / analyzing these complex social issues.
Also, I would like to make a plug for research on the cognitive aspects of survey methodology (CASM), for example this book, which is all about how to ask questions that solicit the information that we are intending to ask about, how to identify when things have gone wrong, and how things like response options, question order and interview dynamics influence survey results. One important take-away is that things often go wrong, even when asking questions about relatively straightforward issues!
Heather: Do you have a successful way of communicating these complex results of your intersectional models? Our audiences – who range from policymakers to school district executives, to foundation directors – don’t understand the results from the models a lot of the time.
Catherine: I think this may be the hardest question for me to answer. In terms of writing, what I do is write for someone I care about, but who has no background in statistics or even the social sciences. For me it’s usually my mom, who is very smart and insightful and cares about social justice issues, but who is not a sociologist. So, I try to make my writing clear (reduce jargon and passive voice, grounding abstract claims with concrete examples) so that she and others will have a better reading experience.
I think images help a lot. Graphs as well as images of arguments / concepts / processes. I believe that interactive visualizations may be increasingly central here, particularly for the intersectional models, because users can see for themselves how things change when one or more characteristic changes. Alternatively, or additionally, making visual representations of two or three cases based on, for example, predicted probabilities or something similar. If we’re thinking about sharing findings with policy makers, foundation directors, we’re probably talking about smart people without a lot of time. So, my recommendation would be to visualize the key findings, which could easily highlight differences across groups, and which could make a strong impact in a short amount of time. Then have the details on hand if people want more information.
I think when writing about issues of gender and other social inequalities it is also very important to spell out the processes and institutional arrangements that lead to differences, rather than just reporting differences. This idea is emphasized in Maxine Baca Zinn and Bonnie Thornton Dill’s (1996) piece, “Theorizing difference from multiracial feminism” (Feminist Studies), which is among my favorite intersectional articles of all times. When we report differences, or really any findings, it’s important to provide readers with the context, conceptual tools, and theoretical arguments to make sense of them. Without these, it’s easy to walk away with static, essentialistic, and/or individualistic ideas of difference and inequality.
Another thing which is probably obvious, but I’ll say anyway, is that it is important to know up front who it is that you are writing for. And if you are seeking to share your results with multiple audiences, you should probably be prepared to communicate in multiple ways. This can be time consuming.
