-
Notifications
You must be signed in to change notification settings - Fork 7
Selecting the evaluation services to apply
- Getting started lets you find the right series of documentation pages based on what you're looking to get out of DataFAQs.
- Selecting the datasets to analyze, the other half of configuring an evaluation epoch.
The evaluation services to apply are selected using the following Core Services. Details for each are provided at DataFAQs Core Services.
- select-datasets/identity
- select-datasets/via-sparql-query
- select-datasets/by-ckan-group
- select-datasets/by-ckan-installation
- select-datasets/by-ckan-tag
This page describes how to write an epoch configuration, which specifies evaluation services to apply and [datasets to analyze](Selecting the datasets to analyze) during an evaluation epoch. Here, we focus on selecting the evaluation services.
An epoch configuration prescribes the process that should be used to select evaluation services and datasets for an evaluation epoch. The configuration is described in RDF using the PROV-O vocabulary, among others. All FAqT Service Selectors follow the same simple pattern. First, they accept an RDF graph via HTTP post. Next, they return an RDF description of the FAqT Services to use in the evaluation epoch.
config
graph goes in
selector invocation
graph comes out
The structure of the RDF that goes to each FAqT Service Selector can vary depending on what the selector service does, but all of the services' output is structured in the exact same way. Using this simple pattern allows the core DataFAQs engine to generically perform the evaluation epoch. Anyone can implement new FAqT Service Selectors and they can be used without changing the core engine -- simply mention it in the epoch configuration and it will be used!
To illustrate the common output, we'll show two configurations that use two different FAqT Service Selectors to obtain a list of FAqT Services. After determining the evaluation services, the datasets to analyze will also be determined, but that is described [elsewhere](Selecting the datasets to analyze).
- Selecting the datasets to analyze, the other half of configuring an evaluation epoch.
- DataFAQs Core Services provides details for each core services' implementation.