Baseballs and Caps - Messaging Uncertainty

Imagine yourself at the forecast desk, or about to brief partners, or working up a “Key Takeaways” graphic for social media. You’re looking through various model guidance and you see a very consistent trend for a very concerning environment over the next 24 hours, characterized by extreme instability (4000-5000 j/kg MLCAPE), modest shear (30-35kt effective), weak inhibition (-25 j/k CIN), and very steep lapse rates (8-9 C/km). It’s the kind of environment you tell your spouse to keep the flashlight and basement handy. But, there’s a caveat. You can’t find any appreciable large-scale forcing for ascent and low-level convergence is weak, at best. Oh, and there’s a little ridging building in aloft. It’s one of those very conditional, but non-zero threats. The thing is, we’re not talking about a conditional threat of pea size hail and 30 mph wind gusts. This is a conditional threat of baseballs, 80 mph wind gusts, and the occasional EF-3 tornado. It may be that only one storm forms, but the impact would be high. But, from a probabilistic standpoint, it’s likely that people will see puffy clouds and experience a hot, muggy, storm-free day. What do you do?

It’s a not-to-uncommon question faced by Meteorologists on the Plains, especially in the transition from spring to summer patterns, where instability is trending higher, but the stronger flow aloft is shifting away from the area and forcing is weak. It’s a target of opportunity that probably deserves some attention, but how much?

This week I begin my fourth year forecasting the weather in the Central Plains, and I still have a lot to learn. These conditional high-impact days are intriguing, a challenge to forecast, and carry a lot of weight. If you tell people severe weather is possible, will they treat it like it’s likely? Does the crying wolf syndrome come into play? What if you skip messaging the low probability of occurrence, but then it happens? Which is worse, crying wolf or “it hit without warning”? Maybe the answer varies by region of the country? Maybe there isn’t a one-size-fits-all-regions answer.

There are a lot of factors at play here, some of which social science probably gives us the best avenue moving forward in an increasingly probabilistic realm of messaging.

I recently asked some fellow Mets, on Twitter, what they thought this scenario (if interested, you can read the thread to see the discussion). Something in that discussion that struck me was our ability, as Meteorologists, to message the environment. We focus a lot on messaging the forecast, but I’m seeing more and more that the message can’t stop there. Dr. Julie Demuth suggests that we “stop thinking of a given forecast as the end-point and start thinking of it as the starting-point for a conversation.”. In our scenario above, our users are seeing a low probability of storms in our forecast, but do they understand the magnitude of severe weather that could occur if storms do form? I wonder if targets of opportunity extend beyond just our forecast grids. Maybe the environment, itself, is a target.

We may not change our forecast, but we might consider changing our messaging.

Three years ago this month, I worked a shift that featured a conditional, high-impact environment.  I ended up working a 13 hour shift that day…a day that included baseball size hail and an EF-3 tornado. Three years later, I worked a shift with a similar conditional threat with a very similar environment. That shift ended on time, and did not include any baseballs or tornadoes. Two shifts with a similar, high-impact potential, but with drastically different outcomes.

Probabilistic forecasting isn’t exactly new for Meteorologists, or even our users. However, it appears that we are moving into an advanced realm of forecasting and messaging that has a higher ratio of probabilistic to deterministic info, with a learning curve both for us and for our users. I get the sense that there isn’t one right answer for all scenarios, but I still am not sure where to land on these “fringe” events, so-to-speak. I’m hoping continued discussion, group-think, trial and error, user feedback, and social science findings will help us as we continue to seek effective messaging of low-probability events.

If you have any ideas, suggestions, or comments on this topic, I’d love to hear them! Comment on the post, shoot me a message, or add to the Twitter discussion. We’re better off if we learn together.

Probabilistic vs Deterministic Messaging

This past Winter, our office (TFX) participated in the NWS' prob snow experiment. For those who might not be familiar with what that is or what it involved, it was a way to experiment with utilizing snowfall probability information within operations and decision-support activities. Perhaps in another post I'll ponder the good and the bad about the experiment, itself. But, for now, I wanted to take it a different direction.

Related to that experiment, a question was posed...is probabilistic or deterministic information better when it comes to messaging? At its core, this question is part of a larger and ongoing debate related to the effective communication of weather hazards. That debate is a fascinating and challenging one, but is probably too long for one post. For now, I'll just address the one piece of the puzzle that focuses on probabilistic vs deterministic messaging.

When it comes to snowfall amounts, what do we often see? Ranges. And, we seem to gravitate toward certain ranges at that. 1-3", 3-6", 6-12". If you are one of those rebel types, you might even use 2-5" or 3-7". Oh the humanity...

The interesting (note I said interesting and not necessarily bad) part about the end-user's use of ranges is the seemingly automatic focus on the high number. Knowing the worst-case scenario isn't a bad thing in of itself, but how it's used can be. Just before a winter storm a couple months ago, a friend of mine texted me and said, 'Hey! I heard we are supposed to get 7" of snow'. The winter product from our office said something to the effect of 2-4" with isolated amounts up to 7", if I remember correctly. My friend read that as we are getting 7" of snow. I doubt he was alone in that assessment.

I often give ranges when messaging upcoming snowfall events and I am not here to argue against that. My end-game is to think through the different possibilities. Recently, I decided to give the ole probability method a try. Prior to a winter event, a caller asked how much snow we expected for her area. With experimentation on the brain, I boldly informed her that there was an 80% chance of exceeding 4" at her house. To which she replied, 'So, do you think we might get a foot?'.

The sample size on my little experiment is incredibly small. But, how much would you be willing to wager against her response representing a large part of the population? One thing that stuck out to me in her response was the 12" amount. After talking with her more, I got the sense that 12" is when she starts having problems in her world. It's the point when her daily plans change. I believe that is why her mind immediately jumped to a foot. For her, my arbitrary percentage-greater-than-x-amount didn't help. Now, had I given her the probability of exceeding 12", well that could have been a different story. Would she have been able to interpret it effectively? I don't know.

When it comes to the general public, the thresholds for when action is taken is all over the place. That lady's threshold was 12". A recent transplant from the South would probably have a different response. So where does that leave us as Meteorologists? In a very challenging position. We have a responsibility to message hazardous weather, but to a group of people who don't even share a common breaking point.

On the flip side, we have individuals or groups (DOT, emergency managers, etc) that often DO have specific thresholds that we can know. I watched an enlightening presentation recently that looked at the potential effectiveness of probability information for decision-makers like the DOT. I get the sense that probability messaging works great for them. Honestly, I believe it could work great for the general public as well. The challenge is our inability to know each and every person's breaking point.

One part of the prob snow experiment that I really liked was that it gave probability information for several breaking points (2", 4", 6", 8", 12", 16"). We may not be able to know all thresholds, but we can certainly try to cover as many as possible in our messaging, within reason. But, that's just snow. What about rain, hail size, tornadoes, tornado strength, etc? Do we say "this storm will produce up to golf ball size hail" or "there is an 80% chance of exceeding quarter size hail?". I'm not sure a warning product is the place to put a lot of probability wording, if nothing else but for the sake of time/understanding. Imagine The Weather Channel scrolling the probability of multiple thresholds, or hearing those probabilities being read over Weather Radio broadcasts/statements?

My answer to the question of probabilistic or deterministic? The verdict is still out, but I imagine it involves some sort of a mix that relates to the known users, the product, and the event. I don't know if there will come a time when all of our messages are completely understood, used correctly, and heeded, but working through and experimenting with this piece of the puzzle is beneficial to the larger discussion regarding effective communication. In the spirit of probabilistic messaging, I will inform you that there is a 100% chance that I will blog more about effective communication down the road...