Click here to download OCUL MultiSearch Early Adopters Final Report to the Scholarly Information Resources Working Group (DRAFT March 30, 2007)

OCUL MultiSearch Early Adopters Final Report to the Scholarly Information Resources Working Group

DRAFT March 30, 2007

About MultiSearch

MultiSearch is a federated search tool offered by CSA.  It uses the MuseGlobal metasearch engine, as do some other federated search tools such as SIRSI's SingleSearch.  A strong competitor is Ex Libris's MetaLib.  MetaLib is a very well-regarded federated search tool, because it has a well maintained knowledge base with the vast majority of its connectors based on Z39.50, XML gateways or SRW/U rather than the less reliable 'html parsing' prevalent in MuseGlobal.  MetaLib, however, offers only about one quarter the number of connectors provided by MuseGlobal. Scholars Portal Search, which runs on the CSA Illumina platform and has been operational since 2005, aims to provide the benefits of a federated search tool while avoiding the drawbacks:  a single search box can be used to search multiple databases, and those searches are performed locally on the CSA Illumina platform rather than across multiple systems, reducing issues such as broken connectors.  Scholars Portal Search, however, is limited to the databases for which OCUL negotiates local loading.

Rationale for Testing MultiSearch

MultiSearch was of interest to several OCUL libraries because of its potential to provide a single interface and/or subject clusters for resources not currently aggregated in Scholars Portal Search.  MultiSearch was attractive because its interface looks similar to Illumina and because of the potential to build a connection between Illumina and MultiSearch: it was thought that perhaps a user would begin searching in Illumina and then broaden the search to MultiSearch without re-entering search details.

Participating Libraries

Ten OCUL libraries subscribed to MultiSearch as 'early adopters' either in December 2005 or shortly thereafter:  Carleton, Guelph, McMaster, Nipissing/Canadore, Ottawa, Toronto, Queen's, Western, Windsor, York.  The OntarioCollege of Art & Design also subscribed in 2006 but was not involved in early adopter group discussions.  Some of the early adopter libraries chose to use MultiSearch to create a multi-disciplinary search that covers more disciplines than possible in Scholars Portal Search; some used it to create subject clusters.  Guelph, McMaster, Queen's and Western featured MultiSearch on their websites.  Guelph and Queen's conducted user testing. Queen's full report is available here: http://library.queensu.ca/libdocs/spswg-report-2007jan.pdf  

Benefits and Concerns

MultiSearch allowed the participating libraries to present subject clusters and multi-disciplinary searches that were not possible in Scholars Portal Search, which lacks databases in some disciplines.  As well, it allowed the inclusion of resources that are quite different from the index/abstract tools on Scholars Portal Search, such as library catalogues and Google Scholar.  Feedback indicated that users liked the idea of a multi-disciplinary search, despite a variety of frustrating technical limitations.  It is interesting to note that when users were asked to compare MultiSearch and Scholars Portal Search atGuelph, most preferred the latter because it returns results faster, in date order, and the peer review limit is useful. The benefit of using a federated search tool that resembled Illumina was not clear.  CSA has not pursued the development of a connection between Illumina and MultiSearch. The technical issues encountered were numerous.  Initially, each library had to invest considerable time in working with CSA to resolve proxy and authentication issues.  Every institution complained about the slow and cumbersome admin module, and although CSA promises there will be improvements there were none during the testing period.  In the Fall, it was discovered that SFX and RefWorks were not working with MultiSearch; CSA resolved these particular problems in December.  In early 2007 there were still outstanding issues that appeared impossible to resolve:  users were frustrated by the slow return of search results and various interface issues; the "view source" option was unreliable because it required CSA to keep a proxy up to date for every connector, and reported problems were not fixed for weeks; broken connectors took months to be repaired. The problem of broken connectors is not unique to MultiSearch.  At the Atlantic Scholarly Information Network, which uses SIRSI's SingleSearch and therefore the MuseGlobal search engine, there is 1 FTE assigned to ensuring that problems with connectors are detected and repaired. The turnaround time on repairs appears to be much better than with MultiSearch.  In early 2007, CSA indicated that they would be backing away from this support and customers would deal directly with MuseGlobal to resolve connector issues.

Renewal Decisions

As the renewal date for MultiSearch drew near in December 2006, the participating libraries asked for an extension on their renewal decision, reasoning that the technical issues with the product had prevented full implementation and testing.  CSA granted that extension to March 31, 2007.  In mid-February, the participating libraries discussed whether there was sufficient interest in MultiSearch to warrant the planned installation of the system on a Scholars Portal server, and decided there was not. In early March, due to the amalgamation of ProQuest and CSA, participating libraries were given the option of renewing with MultiSearch or migrating their subscriptions to Serials Solutions' CentralSearch.  None of the libraries felt they had sufficient information or time to consider the CentralSearch option.  Many of the libraries who participated as early adopters felt that the opportunity to experiment with a federated search tool was well worth the resources invested: the exercise forced consideration of various 'mental models' of search.  In the end, however, the technical issues with MultiSearch outweighed the benefits, and none of the subscribing institutions renewed their MultiSearch license.

Next Steps

The OCUL libraries who participated as early adopters of MultiSearch suggested several next steps for Scholars Portal:

1. Facilitate a 'discovery exercise' to evaluate federated search tool options.  There is a perceived a benefit in OCUL schools working together on this, both in terms of potential price reduction and learning from each other.
2. At the same time, work on improving Scholars Portal Search to address the reasons that people are still searching for another tool:
   • Interface issues with Illumina, such as the inability to create subject clusters within the Illumina interface
   • Content lacking in specific disciplines
   • Quality management (e.g. clarifying discrepancies with native platform)
3. If the conclusion, after 1 and 2 above, is that the current opportunities and overall directions for Scholars Portal warrant a consortial purchase of a federated search tool, conduct a formal selection process.