Research data is essential to facilitate scientific progress, yet, many valuable datasets are hidden on web sites and small repositories or are hard to find due to insufficient metadata. Only a fraction of researchers pro-actively share dataset metadata through public portals, and curation of such metadata collections is costly. Unknown Data will provide means to automatically discover, extract, and publish metadata about research data that is hidden on the Web or in scholarly publications. Thus, the project’s goal is to improve findability and re-usability of research data by (a) improving metadata quality, in particular with respect to authority and use of existing datasets and (b) uncovering datasets that are not yet reflected in public data repositories and registries.

Our approach

1. utilises data citations from scholarly articles and web pages to collect metadata about relevant datasets,
2. discovers datasets and their context by crawling web pages,
3. consolidates metadata by linking information from domain-specific databases,
4. facilitates high metadata quality by establishing a discipline-specific curation process, and (5) ensures long-term availability of original data sources by archiving relevant web pages.

Obtaining metadata about research data directly from web sources and publications is a novel approach, increasing the visibility of “long tail” datasets while at the same time providing crucial insights into the actual use and impact of (known) datasets. The results of this project will benefit two disciplines, computer science and the social sciences, through use case pilots. The DBLP computer science bibliography (https://dblp.org) and the GESIS portals, accessible through GESIS Search (https://search.gesis.org/), are among the most prestigious and widely-used metadata collections in their respective fields.

All metadata collected in this project will be made publicly available as linked open data and through REST APIs beyond the project’s duration. In doing so, we are actively contributing to making research data findable, accessible, interoperable, and re-usable for both machines and researchers (i.e., following the FAIR Data Principles). All software products from this project will be made available as open source and methods and code can be adapted to further disciplines.

Creative Commons BY 4.0

Marcel R. Ackermann