Welcome!
Thank you for stopping by! We're glad you're interested in trying out the Dataverse-Archivematica demonstration sandbox hosted by Scholars Portal.
Scholars Portal sponsored Artefactual Systems Inc. to enable Archivematica to receive packages from Dataverse instances in two phases between 2015 and 2018. The first phase produced a proof-of-concept workflow and scoped out the basic functionality of the integration. The second phase brought the integration into a public release of Archivematica (v. 1.8) in November 2018.
This page contains information on how to access the sandbox, notes on its limitations, and a description of the workflow to use it.
Please note that individuals at institutions that are members of the Ontario Council of University Libraries (OCUL) can access the sandbox as-is with the credentials below. If you are from an institution outside of OCUL, please fill out the linked request form and access will be granted.
We are currently seeking feedback on the integration to identify areas for future development. Please send your feedback to dataverse@scholarsportal.info. Have questions, or are experiencing technical issues? Send them to dataverse@scholarsportal.info too!
Accessing the Sandbox
The sandbox is available at: https://archocul.scholarsportal.info/
Username: test / password: testtest
Members of OCUL can access the sandbox without further setup - you're good to go!
All users outside of OCUL schools must fill out a short request form to gain access to the instance.
The request form asks for your name, e-mail, institutional affiliation and IP address. The IP address will be used to open the instance up to your network address and will not be otherwise retained. If possible, please use the IP address from your usual workstation connected to your usual network. If you change your network settings, such as connecting to wifi in a different location, you may lose access if the IP address changes. You can use this website (or just typing "what is my IP" in a Google search) to retrieve your IP address.
Notes on the Sandbox
- The sandbox is connected to a test Dataverse repository in Scholars Portal's demonstration Dataverse.
- Users may not test with their own data, but they may submit a dataset for review. An account on the test Dataverse site is required to do so.
- The sandbox rebuilds nightly. If you wish to download any stored data, please do so immediately.
- The following additional fixes will be included in a future release of Archivematica:
- Multiple authors are not captured in the Dataverse METS - only the first author listed is.
- When using the Dataverse transfer type, it is not possible to delete packages after extraction if the package contains derivatives. The processing configuration in the Archivematica-Dataverse demo instance has been configured to take this into account.
- Additional known issues are listed in Archivematica's Dataverse wiki page.
Want to Learn More?
Need a quick intro to Archivematica? Check out the Overview guide in Archivematica's documentation.
Visit the Dataverse page on Archivematica's wiki, as well as the Dataverse documentation for Archivematica and the Archivematica storage service.
Workflow
This workflow is specific to the Dataverse integration. OCUL users can also request instructions on testing other kinds of data by e-mailing permafrost@scholarsportal.info.
The sandbox is integrated with a set of sample datasets in Scholars Portal's demo Dataverse.
A. Starting a Transfer
- Log into Archivematica at the URL and with the credentials provided above.
- Near the top of the page, you’ll see a transfer initiation pane as below.
3. Under ‘Transfer type’ select "Dataverse" as pictured above.
4. Enter a transfer name. You can leave "Accession no." and "Access system ID" blank.
5. Hit the 'Browse' button.
6. A window will pop up showing the available applicable transfers in the transfer source. Click on the dropdown menu that shows "Transfer Source in Swift via SVFS" and select instead "Archivematica Test on Demo Dataverse." The three sample datasets will appear as pictured below.
7. Select one of these transfers by clicking on it.
8. Click the blue ‘Add’ button. The transfer will be added to the top of the pane. If you add additional transfers at this stage, they will be processed separately.
9. Click the green “Start transfer” button and you’re off to the races! You may have to wait a few hot seconds until the transfer begins processing, so please be patient. Note: if the "Approve automatically" checkbox is clicked under the "Browse" button as pictured above, your transfer will begin running up to the file identification step. If the box is not checked, you will have to approve the transfer to initiate it.
B. Processing a Transfer
The transfer steps are determined based on a standard configuration with some option-based stops along the way. It also does not make use of the backlog/appraisal functions, but you are welcome to do so. Consult the appropriate documentation to do so here.
- Approve transfer: If the "Approve automatically" checkbox is clicked under the "Browse" button as pictured under step 6 above, your transfer will begin running up to the file identification step (#2 below). If the box is not checked, you will have to approve the transfer to initiate it. You can choose approve or reject (you can reject if you want to start over for some reason or another). Please note that the
- Select file format identification command: recommended options are choosing between Siegfried or Fido - both perform the same function, though Siegfried will be generally be quicker based on our experience to date.
- A number of services will run. At the end, you have the option of creating a single SIP and continuing processing. The general case is to select "create single SIP." If you want to use the Appraisal tab, select "Send to backlog." For information on this function, please consult Archivematica's documentation here.
- The SIP will move to the Ingest page. You have to click on the Ingest tab to continue. You'll see a number pop up indicating an action is required. Under ‘Ingest’ a number of services will run.
- The processing will pause at normalization. Select "Normalize for preservation" to create an AIP only. If you want to create additional access copies (i.e., a DIP), you can select “Normalize for preservation and access.” You can also choose not to normalize by selecting "Do not normalize."
- After normalization, you can review and approve normalization by clicking on the little report icon:
7. You’ll see a report about whether normalization was successful or not and can choose to approve, reject or redo. Assuming normalization was successful, select ‘approve.’
8. If you chose to normalize for access, the Upload DIP option will come up first, followed by the Store AIP option. It's best practice to deal with the AIP first, so wait for this option to arrive and process the AIP before the DIP. The rationale is that if there's some error in the AIP, you don't want to replicate it in the DIP.
9. You’ll have the option to store or reject the AIP. The normal case is to store, but it’s possible you might want to pause at this point or start over. After a few more automatic steps, the AIP will be stored - by default it will be on the Ontario Library Research Cloud, Scholars Portal's storage cloud. You can search for and download it from the Archival Storage tab in Archivematica
- For the DIP, select "Do not Upload DIP" first, unless you have a previously-configured system like AtoM to upload the DIP to. You will then be prompted with the option to store the DIP. When the option to Store DIP is available, select "Store DIP" or reject it by selecting "Do not store," if you want. By default, the DIP will be stored on the OLRC. It will be accessible there - not through the Access tab in Archivematica, which controls only DIPs uploaded to a connected access system like AtoM. See the instructions for Accessing DIPs below.
You're done!- The default is to compress AIPs and DIPs. You'll need a program capable of extracting 7z files to open it. We recommend The Unarchiver, although 7Zip is another common method that works well in Windows. However, you can choose to turn off compression in consultation with SP.
H. Accessing AIPs
You can search download AIPs via the Archivematica interface.
- Click on the "Archival storage" tab.
- From here you can search for datasets using the search field at the top.
3. To access a dataset, click on its name or UUID.
4. To download an AIP, click on the "Download" button (circled in purple).
I. Accessing DIPs
Accessing stored DIPs is not offered as part of the sandbox, as doing so requires navigating to storage.