Blog from July, 2014

When the ODLRC is fully operational, it will consist of four or more data storage centres at different OCUL locations. These data centres, or nodes, will be constantly talking to each other, making sure data is being replicated between them for preservation and ease of access. 

In order to make an informed decision about where the nodes should live, the ODLRC steering committee needs to get some evidence of how the servers will perform in various possible circumstances and configurations. What’s the most efficient way to set up this network? How much bandwidth will it need? How will the node 1 react if node 4 goes offline?

For the past month, Scholars Portal and the University of Toronto have been working with York University and Ryerson University on a pilot project to answer these questions. Using a shared 10GB connection through ORION, with routers extended through all three schools, we’ve started testing network capacity for content uploading, replication, and downloading. 

During testing, the proxy node reached maximum network capacity during the upload of 30 TB of test content, while the data notes were working at 50-60% capacity. We’ll next test increasing proxy node capacity (we started with 1 GB), and measure the affect of adding additional nodes.

A graph showing network traffic at Ryerson

Next up, we’re going to look at the way these nodes are talking to each other, how viewing and downloading of content is affected during uploads of large datasets, and do some disaster scenario testing.

Within the next few weeks we will have a much better idea of the network’s needs, and can begin making plans to add nodes outside of Toronto.