This discussion appears in a larger discussion along with the Figure discussed on a personal website entry that can be found here: https://wp.me/P5jqJm-1

Review body

This paper is well-written and engages with an unaddressed question: namely: Is there an increase in the crimes females fear in formerly sex-separated spaces with the passage of 'gender-inclusive' policies (aka, gender self-ID). There does not appear to be evidence on this question. Unfortunately, due to a number of limitations, especially related to the data especially a small sample size (due in large part to non-responses) as well as a vague description of key methodological decisions, evidence from the comparisons is shaky, at best. In my reading, despite several strengths, this study has several serious shortcomings that raise questions about the utility of the study and its findings and for these reasons should not be used to inform public policy.

Below, I discuss the framing of this study, several problems or limitations, and how the dearth of information provided on the methods makes it difficult to ascertain the actual implications of the results. I will note I emailed the first and second authors of the study to ask for clarification on the sample size and study comparisons in late 2019. I did not receive a response to my emails, so this discussion represents my effort to understand the study in light of what was presented in the article.

The Hasenbush et al. (2019) paper is well-described by its title: “Gender Identity Nondiscrimination Laws in Public Accommodations: A Review of Evidence Regarding Safety and Privacy in Public Restrooms, Locker Rooms, and Changing Rooms”. The description in the abstract is also rather clear, so I shan’t try to reword it: “Opponents of gender identity inclusive public accommodations nondiscrimination laws often cite fear of safety and privacy violations in public restrooms if such laws are passed, while proponents argue that such laws are needed to protect transgender people and concerns regarding safety and privacy violations are unfounded. No empirical evidence has been gathered to test such laws’ effects. This study presents findings from [3?] matched pairs analyses of localities in Massachusetts with and without gender identity inclusive public accommodation nondiscrimination ordinances. Data come from public record requests of criminal incident reports related to assault, sex crimes, and voyeurism in public restrooms, locker rooms, and dressing rooms to measure safety and privacy violations in these spaces. This study finds that the passage of such laws is not related to the number or frequency of criminal incidents in these spaces. Additionally, the study finds that reports of privacy and safety violations in public restrooms, locker rooms, and changing rooms are exceedingly rare. This study provides evidence that fears of increased safety and privacy violations as a result of nondiscrimination laws are not empirically grounded” (p. 70).

Hasenbush et al. (2019) see the issue of bathrooms, locker rooms, and other sex-separated provisions as being contested due to physical safety concerns on the part of females versus concerns about the safety of transwomen. As I have noted elsewhere, physical safety is a concern (although less for ‘bathrooms’ specifically), but other valid (as reasonable and understandable) concerns for females include those related to privacy, dignity, and psychological well-being, ranging from discomfort, unease, and even fear caused by sharing sex-separated spaces with male-bodied and male-socialized persons. The presence of male people (regardless of gender identity) can make female people feel uneasy not because of transphobia, but because gender identity is not observable. Thus, women have no way of ascertaining whether a male person is safe or a sincere transwoman just going on with their lives. Moreover, due to female socialization and past experiences, some girls and women may read male-patterned behaviors as unsafe when they are just male-patterned. Moreover, we remain a very prudish species; only in recent years have all 50 US states allowed breast feeding in public in at least some conditions (but not all). Thus, while Hasenbush et al. (2019) frame the sex-separated spaces debate as one of physical safety, psychological well-being is also an important concern that is neglected.

The question Hasenbush et al. (2019) seek to answer is whether the criminal acts that people fear might follow from gender self-ID increase following the passage of gender self-ID legislation (viz., assault, sex crimes, and voyeurism in public restrooms, locker rooms, and dressing rooms). The authors compare the rates of these relevant behaviors using open records requests from law enforcement agencies across localities in Massachusetts with different gender identity regulations. As the authors note, since 2011 Massachusetts has had some gender identity protections. In early 2011, Governor Patrick banned discrimination against transgender people in the state government and contractors. Later that year, the state enacted legislation prohibiting ‘discrimination based on gender identity’ in employment, credit, union practices, and housing—but not public accommodations. In contrast to the Equality Act, Massachusetts did not define or protect a gender identity based on in the moment self-declaration, but instead extended right-of-access to sex-based provisions to those who had a gender identity that “may be shown” and included an “improper purpose clause” that explicitly prohibits the use of gender identity right-of-access for improper purposes. Specifically, the statute states:

“Gender identity” shall mean a person's gender-related identity, appearance or behavior, whether or not that gender-related identity, appearance or behavior is different from that traditionally associated with the person's physiology or assigned sex at birth. Gender-related identity may be shown by providing evidence including, but not limited to, medical history, care or treatment of the gender-related identity, consistent and uniform assertion of the gender-related identity or any other evidence that the gender-related identity is sincerely held as part of a person's core identity; provided, however, that gender-related identity shall not be asserted for any improper purpose” (see: https://www.mass.gov/info-details/massachusetts-law-about-gender-identity-or-expression).

Massachusetts subsequently passed a gender identity non-discrimination law for public accommodations (including bathrooms) in 2016. The Hasenbush el al. study focused on the time period prior to the passage of this statewide non-discrimination; specifically, they “sought to empirically assess whether reports of safety or privacy violations in public restrooms, locker rooms, and dressing rooms change in frequency in localities that have gender identity inclusive public accommodations nondiscrimination ordinances (GIPANDOs) as compared to matched localities without GIPANDOs” (p.73).

I will note that I found the methods section of the paper difficult to follow; I could not find a statement about the sample size, which is an important piece of information. From the information presented, we can see that the authors identified 7 localities in Massachusetts that contained GIPANDOs; two other locales close to the other GIPANDO localities “were also identified as having some gender identity protections, though their coverage was not as extensive as the seven other localities” (p. 74). These two localities are Cambridge, which was noted to include gender identity protections but not for bathrooms, and Brookline, which “contains a gender identity resolution,” but the authors noted, “it is unclear the extent to which this resolution resulted in actionable changes or any enforcement mechanism within the locality” (p.74). It is not made clear to the reader whether the resolution in other localities resulted in such “actionable changes or enforcement mechanism”; if so, that might have been mentioned, if not, the justification for the exclusion of Brookline is lacking.

Later the authors note that three additional localities might have been included but were not: “While Cambridge and Brookline were categorized as limited GIPANDO localities and selected because of their geographic proximity to the clear GIPANDO localities, a few other localities in the state that were not in close geographic proximity to the clear GIPANDO localities also had gender identity ordinances that were unclear or limited as related to restroom access. These were: Northampton, Amherst, and Worcester, and they were not included in this analysis, since it was originally designed to include a boundary regression discontinuity that would have required localities to share physical borders” (p.75). Hasenbush et al. further noted: “Amherst, Massachusetts prohibits the denial of any rights based on gender identity (Human Rights Bylaw, 2009). However, the town’s director of human resources and human rights, who investigates complaints of discrimination, was unable to confirm whether such rights include access to public accommodations and/or public restrooms (Radway, 2015).” One could (and I do) reasonably question whether we could expect the general resident to understand the specifics of the GIPANDO rules when the city human resource officer does not understand the scope (i.e., whether they provide access to opposite-sex bathrooms based on gender self-identification). As before it is not clear whether the city officials in other localities were asked to confirm that their regulations extended explicitly to bathrooms.

The authors “used quantitative models to identify localities within Massachusetts to match for comparison”; localities were matched based on geographic proximity and “demographic and other characteristics that may relate to the likelihood that locations would pass a GIPANDO as well as characteristics that may be predictive of criminal incidents (or a lack thereof) in public restrooms” (p.74). From these different crime characteristics, which the authors noted were highly intercorrelated, the authors created a composite factor measure, and used “a covariate balancing approach to create a propensity score to identify the most fitting matched localities for each of the GIPANDO localities,” citing (Imai & Ratkovic 2014). Hasenbush et al. (2019) note: “The fitted propensity score was extracted and used in addition to the covariates in a genetic matching procedure (Diamond & Sekhon, 2013; Sekhon, 2011). The genetic matching procedure identifies appropriate comparison localities by examining the full distribution of covariates, which may improve the matching procedure that other matching processes may worsen” (p. 75).

I am not familiar with this genetic matching procedure, and Hasenbush et al. (2019) do not provide much explanation, perhaps because this is a common method in their field. Standard information about the sample size (i.e., how many localities were examined) or details about the model fit or matching scores were not provided. My own (admittedly brief) inquiry into this procedure using the cited references for guidance suggested to me that the lack of information provided about the match fit statistics was, if not unusual, not good practice. The authors did not note the size for the matching; they do not define locality in their paper, but a quick search revealed that the state of Massachusetts has 351 municipalities (and no unincorporated land).

Although readers are not provided with the details, Hasenbush et al. (2019) state: “Matched localities were selected for all of the localities with clear GIPANDOs that applied to public restrooms. The final GIPANDO localities and their matched localities for two different analyses are listed in Table 2”, which I reproduce below with my annotations. Based on their discussions, I have concluded that the matched comparisons for the analyses were made with the three yellow highlighted GIPANDO localities in the left column along with the two matched in the middle column. (Note I say two, because Beverly is used as the match for both Medford and Melrose). Brookline, although discussed as a locality with GIPANDO protections with unclear enforceability (in column 1) is, without justification, used as a non-GIPANDO match for Newton in column 2.

In my reading, then, two GIPANDO localities (Medford and Melrose) were compared with Beverly, and Newton is compared with Brookline, which has a GIPANDO but without clear enforceability. I do not see how this provides us with anything like a defensible comparison of GIPANDO effects across matched localities. No descriptive statistics are provided for any of the localities (only averages across the unspecified sample size), which is somewhat surprising given that with an apparent n of 5, it would be quick work to show the locality descriptive statistics.

For their analyses, the authors compared the GIPANDO to their ‘non’-GIPANDO matched controls and compared the changes in the rates of these crimes over time. The reasoning being that if the GIPANDO laws made females less safe, we would see higher crime in the GIPANDO areas vs. ‘non-GIPANDOs’ and increases in crime in the GIPANDO areas after the passage of the law. Again, it bears repeating that given that even the city administrators are not clear about the laws, it seems unlikely that potential male predators understand the implications (i.e., that the law grants access via gender self-ID to women’s spaces). Equally relevant is the fact that Massachusetts explicitly includes the ‘improper purpose clause’ as well requires some proof of gender identity for access to opposite-sex provisions, which would impede after-the-fact justification for male predators. Moreover, all of the GIPANDO localities were clustered around Boston, which had a GIPANDO statute since 2002.

As for their results, the authors found no differences between the GIPANDO localities and the matched localities in the rate of reported incidents in public restrooms and locker rooms over the time span, which was not made clear. The authors concluded: “we found no evidence that privacy and safety in public restrooms change as a result of the passage of GIPANDOs” (p.78). While that appears to be a true statement, I do not think it is one that is robust enough to be useful for policy implications. For the reasons noted above, I believe the study’s methodology vitiates the utility of its findings, especially their generalizability to areas that are not adjacent to a large, progressive city that has had a GIPANDO rule for a decade (Boston).

The following summarizes the limitations of this paper and why I do not believe this study can be used as evidence for the potential effects of gender self-ID legislation:

1) The idea that we would observe a substantial increase in these crimes following the passage of GIPANDO laws that are poorly understood, even by some human rights officials (e.g., Amherst), in three localities abutting a large metropolitan area (Boston) that has had a GIPANDO law since 2002, and whose media outlets dominate the reporting and news in the area, provides at best a weak test not clearly generalizable to most US areas. Moreover, given that the rationale for including transwomen in women’s spaces is to protect transwomen from male predation, we would expect that crime against transwomen would decrease, potentially offsetting any increased predation by men against women in women’s spaces. The results suggest that these three counties did not experience a unique shift in these crimes, which seems in line with what we would expect. Even so, that, in general, tells us little about whether, if, or how the sex-separated spaces were differently experienced by female persons and how this did or did not affect their well-being, and/or how this might be experienced elsewhere.

2) Methods/Analyses: The information provided was not sufficient to allow me to ascertain the quality of the findings of the results conducted on what appears to be a sample size of 3 localities matched to 2 other localities (one, Beverly used twice—with no clear explanation as to how this was reflected in the average). Again, we can conclude only that no rash increase in these offenses were reported to the police after the passage of GIPANDO laws in these jurisdictions, but information such as whether the public was aware of these changes, how the existence of a GIPANDO in Boston for over a decade (since October 2002) affected the practices in these localities is not clear.

As evidence for what might follow from the US Equality Act and or other state or locality regulations about gender self-ID, I believe this study in current form is not generalizable. The authors descriptions suggest that GIPANDO laws were not well-understand by the Massachusetts population, and without such understanding, methodological information, and other contextualization the question—although an interesting one to ask—remains unanswered.

Reviewer Limitations

As noted, I am not an expert in the matching procedures used in this study.

Overall categorization

• Significant Limitations, Potentially Red-Flagged.