Data Rescuer

After the 2016 presidential election, Graduate Fellows from the Penn Program in Environmental Humanities (PPEH) came to the University of Pennsylvania’s Van Pelt Library to consult Laurie Allen, the director for digital scholarship. They feared environmental and climate data on government websites would disappear under the new administration. What could they do?

Allen knew that before the new administration came in digital specialists were already downloading and storing publicly available government data, most notably the Internet Archive (which maintains the Wayback Machine) and the End of Term Archive, which since 2009 has done an end-of-term harvest that captures federal websites at risk of changing or disappearing. Both efforts needed help to capture the large quantities of information involved.

Allen and Bethany Wiggin, director of PPEH, and the Fellows decided the library would host a code-a-thon on January 13 and 14, 2017. Volunteers would secure copies of climate and environmental data housed on federal websites. After surveying scientists in the Committee for Concerned Scientists on the most at-risk data, the codirectors chose to start with the Environmental Protection Agency (EPA) and National Oceanic and Atmospheric Administration (NOAA) websites. They created an online repository: www.datarefuge.org.

That weekend, dozens of hackers, scientists, and librarians turned up at Van Pelt. For the next six months, people participated in more than 50 events nationwide. “Volunteers—numbering in the many thousands—came together to support the end-of-term harvest by feeding federal webpages to the Internet Archive,” says Allen. They “scraped and wrote scripts to harvest data from pages that wouldn’t easily flow into the Wayback Machine. These data were processed, generally by volunteer librarians, and deposited in Data Refuge.”

Data Refuge made copies of tens of thousands of websites and 395 data sets—in all, many “terabytes of data,” Allen says. “But the outcome of this work is probably not best measured in numbers.... The greater impact was in forging new partnerships and helping many people think differently about the ways we value and care for information for the long term.”

Now, Data Refuge’s Data Stories, a partnership between PPEH and the Union of Concerned Scientists, documents how environmental and climate data is used in real communities. Examples include portraits of data rescuers, social media field notes of rescue events, and tales of people who use open federal environmental and climate data locally.

“Data Stories,” says Allen, “is a great way for libraries of all sizes and types to get involved [in] their own communities.”