Throughout 2020-21, the Scholars Portal team has been upgrading the hardware and software that make up the OLRC to new versions, or "OLRC 2.0". This page outlines some of the key new features and changes in this upgrade, including new options for transferring content and changes to user management.

Hardware & Software

The OLRC 2.0 includes a major upgrade to the OpenStack software that powers the OLRC. The software will be upgraded from OpenStack version "Kilo" (April 2015) to "Wallaby" (April 2021).

Hardware upgrades to the storage nodes include:

  • CPUs increased from 40 to 72
  • RAM increased from 256GB to 384GB
  • Size increased from 240TB to 500TB

Hardware upgrades to the proxy nodes include:

  • CPUs increased from 40 to 48
  • RAM increased from 128GB to 256GB RAM

Domains

One of the major changes in the OLRC 2.0 is the addition of domains. Domains will give administrators at individual institutions more control over the management of their users and projects. Each institution will have its own domain, and each domain can contain multiple projects. Domains can be managed within the Horizon dashboard or via the Swift API.

Data Transfer Options

After the migration to OLRC 2.0, you will have new data transfer options available, including DuraCloud and the Horizon dashboard. You don't have to choose a single option for your data. You can choose to have some containers managed by DuraCloud, and some managed through Horizon or APIs, depending on your requirements.

Feature Matrix


DuraCloudNon-DuraCloud
Swift APIRead-onlyYes
S3 APIRead-onlyYes
Swift BrowserNoYes
DuraCloud APIYesNo
DuraCloud InterfaceYesNo
Bit-integrity reportsYesNo
Audit logsYesNo
DuraCloud Sync ToolYesNo
Sync to other cloud storageYesNo
Chronopolis integrationYesNo
Archive-It integrationYesNo
DSpace integrationYesYes*

*via S3 API. Additional testing required.

DuraCloud

Diagram showing that the user can connect to the OLRC with Swift and S3 APIs, and can connect to Duracloud with the Duracloud API, and that Duracloud then connects to the OLRC

With this option, data will be managed via the DuraCloud API, and therefore it will become part of the DuraCloud ecosystem. You have the option of having some or all of your data managed through Duracloud.

The features of DuraCloud include:

  • DuraCloud API: The DuraCloud HTTP REST API supports all the standard API requests for reading/writing data to and from the OLRC.
  • Content Synchronization: DuraCloud can copy your content onto several different cloud storage providers and ensures that all copies are kept synchronized.
  • Health Reporting: DuraCloud performs regular bit-level integrity checks and health reports, which are available for download.
  • Content Transfer Tools: The DuraCloud Sync Tool makes it easier to continuously synchronize data from a local desktop or server to the OLRC.

It may initially appear clear that you should use DuraCloud, and in most cases we would agree, but it is worth noting these disadvantages:

  • Read-only Native APIs: Once ingested into DuraCloud, you will have read-only access to your objects using the Swift or S3 APIs, but can only write using the DuraCloud API.
  • It's Slower: Because of all the background tasks that occur when writing to DuraCloud, it is necessarily slower.

There is no additional charge to use DuraCloud.

Horizon

Horizon is Openstack's default dashboard where you can manage users, browse, upload and download files. Support for uploading files is limited, but we are actively developing improvements.

A screenshot of a container in the OpenStack Horizon dashboard

APIs

Diagram showing that the user can connect to the OLRC with Swift and S3 APIs

Data transfer directly through APIs will continue to be available. If you decide to use DuraCloud, you will also have the option of using DuraCloud's API.

  • Swift API: The native Swift API, which is the fastest interface to the OLRC.
  • S3 API: The AWS S3-Compatible API, which is supported by a wide-range of third party software. To learn more about which S3 operations are supported, see the OpenStack S3/Swift API comparison documentation.

SwiftBrowser

SwiftBrowser, the Scholars Portal developed web interface, will continue to be available as a data transfer option in OLRC 2.0.

Contact

If you have any questions about the OLRC, contact the Scholars Portal team.


  • No labels