Since 2002, the Indian state of Odisha has been undertaking a grassroots awareness campaign on "dos and don'ts" during heat wave conditions through the Disaster Risk Management (DRM) program. The selection criteria for DRM districts were earthquake, flood, and cyclone incidence, but subsequently heat wave awareness also received intense attention in these districts. We present quasi-experimental evidence on the impact of the program, taking DRM districts and periods as treatment units and the rest as controls. We analyze the impact on the death toll from heat stroke for the 1998 to 2010 period, using difference-in-difference (DID) regressions with district level panel data and a set of control variables. We find indications of program effectiveness with initial DID specifications, but results are not strongly robust. We then take into account a statewide heat wave advertising program, to which the poor have limited exposure but which may also provide spillover benefits, using a triple differencing approach. These results suggest the heat wave awareness programs may have complementary impacts. We examine research strategies for much-needed improvement in the precision of impact evaluation results for innovative programs of this type.