Weather and Society Watch
Testable Hypotheses and the Societal Impacts Discussion Board
It is an interesting phenomenom that after every severe weather event the Societal Impacts Program Discussion Board (http://www.rap.ucar.edu/forums/phpBB2/) is filled with discussions about why people reacted they way that they did, about what they should have done differently, about whether or not they received the warning or forecast, about if they did receive it why didn't they respond, about if they didn't receive it why they didn't, about ….
There is inevitably a passionate discussion about what the weather community can do to get people to do what they "should do" when there is a severe weather event, about what role social media plays, about how sirens do or don't work, about whether or not people should have shelters, about how to improve the spatial scale of warnings, about …
And many of the same issues, questions, concerns come up again and again with every new event.
With the devastating tornado, flood, and wildfire season we've experienced so far this year, there has been no shortage of similar discussions on the Board. While I admit I haven't been able to keep up with all of the posts, I have seen enough to know that there has been much enlightening and thoughtful discussion.
I feel sometimes, though, that we are covering the same ground and that a certain level of frustration exists that we don't have answers to many of the discussion points.
I would like therefore to offer one approach that may help move some discussions forward.
I can't take credit for the idea but borrow an approach George Youngs used at the Red River and Devils Lake Integrated Warning Team workshop in Fargo, N.D., this June. George is a sociologist with the Department of Emergency Management at North Dakota State University.
During the first day of the workshop, George listened to what people were saying about their agencies, the problems they face, and issues related to flooding in the Red River and Devils Lake area. He identified a number of common themes and issues that came up during that first day. The second morning of the workshop, he presented these themes and issues—but George had transformed these "discussion points" into hypothesis that could be tested using social science research and methods.
Why did this make sense? Because by identifying problems and issues that were presented as assertions, anecdotes, frustrations, concerns, or problems and presenting them as hypotheses, he offered a way to actually deal with them. He offered the scientific method as embodied in the social sciences as a way to develop explanatory understanding that would allow decision makers to base their decisions on "knowledge" rather than continuing to feel frustrated with not knowing why people reacted they way that they did, what they should have done differently, whether or not they received the warning or forecast, if they did receive it why didn't they respond, or if they didn't receive it why they didn't.
When reading some of the recent postings on the Discussion Board, I felt that a similar approach could be taken with many of the comments there.
In a quick scan of these discussions, I have seen virtually no citations of research supporting or refuting the assertions made. Note: I am not saying that having a citation proves something is true or not but it does move it to a level where issues can be discussed based on evidence and scientific practice of evaluating evidence (i.e., the scientific process).
So … here is my attempt to "George Youngs" some of these comments. I skimmed several of the comments and did not choose any particular issue to address or anyone in particular to pick on or support but simply chose some statements that caught my eye. Many of these are assertions that may be based on extensive personal experience or anecdotal evidence and may or may not be true. My point is not that they are or aren't correct but that we may not know whether or not they are—and we can't make sound decisions based on anecdotes.
Also, I don't know the extent to which some of these hypothesis have already been tested and valid and reliable social science research has already answered them. I would encourage more discussion on the Board about this research where it is available. I also note that many of the comments below could generate many different hypotheses, so feel free to make up some of your own!
Okay, I chose some of the comments to raise issues of the degree to which assertions—even if they are testable as hypotheses–may involve generalizations. On the other hand, if this last assertion were true, that would indicate a significant problem for the future communication of accurate and reliable weather warning information!
As you may expect, I could pull out many, many more assertions from the posts on the Discussion Board. Most of these just happened to relate to media communication.
The point is that making assertions again and again doesn't add to knowledge—in fact, it may add to frustration. By recognizing that these are assertions—perhaps based on experience and observation—and moving to formulate these as hypotheses, we take the first step toward developing knowledge. One description of four steps of the "scientific method" is:
Let's recognize that many of the Discussion Board assertions are, at most, "characterization[s] from experience and observation." Building on these characterizations let's take a next step to developing hypotheses. As defined in Wikipedia "A hypothesis is a suggested explanation of a phenomenon, or alternately a reasoned proposal suggesting a possible correlation between or among a set of phenomena." This would be a first step into the world of the social sciences from the world of "societal impacts."
Remember the quote from Salmon: "It is explanatory knowledge that provides scientific understanding of the world."
*Jeff Lazo (firstname.lastname@example.org) is the director of the Societal Impacts Program (SIP) at the National Center for Atmospheric Research (NCAR).
Salmon, Wesley C., Four Decades of Scientific Explanation, University of Minnesota Press, Minneapolis, MN, 1990.