Skip to content

New batch framework, supporting suspending of the batch process.

License

Notifications You must be signed in to change notification settings

adam-vessey/islandora_batch

Repository files navigation

Islandora Batch Build Status

Introduction

This module implements a batch framework, as well as a basic ZIP/directory ingester.

The ingest is a two-step process:

  • Preprocessing: The data is scanned, and a number of entries created in the Drupal database. There is minimal processing done at this point, so it can complete outside of a batch process.
  • Ingest: The data is actually processed and ingested. This happens inside of a Drupal batch.

Requirements

This module requires the following modules/libraries:

Additionally, installing and enabling Views will allow additional reporting and management displays to be rendered.

Installation

Install as usual, see this for further information.

Configuration

N/A

Documentation

Further documentation for this module is available at our wiki

Usage

The base ZIP/directory preprocessor can be called as a drush script (see drush help islandora_batch_scan_preprocess for additional parameters):

drush -v -u 1 --uri=http://localhost islandora_batch_scan_preprocess --type=zip --target=/path/to/archive.zip

This will populate the queue (stored in the Drupal database) with base entries.

For the base scan, files are grouped according to their basename (without extension). DC, MODS or MARCXML stored in a *.xml or binary MARC stored in a *.mrc will be transformed to both MODS and DC, and the first entry with another extension will be used to create an "OBJ" datastream. Where there is a basename with no matching .xml or .mrc, some XML will be created which simply uses the filename as the title.

The queue of preprocessed items can then be processed:

drush -v -u 1 --uri=http://localhost islandora_batch_ingest

A fuller example, which preprocesses large image objects for inclusion in the collection with PID "yul:F0433", is:

drush -v -u 1 --uri=http://digital.library.yorku.ca islandora_batch_scan_preprocess --content_models=islandora:sp_large_image_cmodel --parent=yul:F0433 --parent_relationship_pred=isMemberOfCollection --type=directory --target=/tmp/batch_ingest

then, to ingest the queued objects:

drush -v -u 1 --uri=http://digital.library.yorku.ca islandora_batch_ingest

Customization

Custom ingests can be written by extending any of the existing preprocessors and batch object implementations. Checkout the example implemenation for more details.

Troubleshooting/Issues

Having problems or solved a problem? Check out the Islandora google groups for a solution.

Maintainers/Sponsors

Current maintainers:

Development

If you would like to contribute to this module, please check out CONTRIBUTING.md. In addition, we have helpful Documentation for Developers info, as well as our Developers section on the Islandora.ca site.

License

GPLv3

About

New batch framework, supporting suspending of the batch process.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published