Ag Data Commons Harvest Policy
We consider the following criteria when deciding whether to accept a repository for automated metadata harvesting:
- We prioritize repositories with datasets that were USDA-funded, including collaborations, grant funding, and USDA employee participation.
- It must be possible to filter the content to specifically find datasets (as opposed to scholarly publications and reports) relevant to USDA (as opposed to other disciplines)
- There must be a minimum of metadata available (e.g. OAI-PMH fields)
- Data are consistent with our current collection policy
- There must be an endpoint with machine-readable metadata that our harvester can access (often, this is an API or web service, does not need to be CSW or data.json or OAI-PMH but it would help)
- There must be some evidence of data quality (e.g. association with peer reviewed literature, citations, curation or validation processes, etc.)
- There is a large quantity of target datasets now or an expectation of growth over time.
If you feel you meet these criteria, please contact us at NAL-ADC-Curator@ars.usda.gov
We do not currently harvest data (resource files). We only harvest metadata describing datasets. If all of the above criteria are not met, we may recommend manual submission of metadata depending on which criteria are not met.
We do not currently issue DOIs for datasets whose metadata we harvest because we are not the publisher or managers of those datasets. If you are interested in DOIs, please contact us to discuss the possibility of harvesting resource files from you.