I’m not into data really. Statistics don’t do it for me. Facts and numbers just aren’t my thing. And archiving is definitely not something I get excited about. The suggestion of archiving conjures up for me images of fusty libraries with tomes of dust covered ledgers; the Maesters in Game of Thrones, cloistered away for years on end, hoping to find the secret to youth in those leather- bound books or amidst the jackets of ancient manuscripts. I have however, through the journey I’m about to share with you, met quite a few people who really do get animated about archiving. And they are generally very nice, quite normal people! For them the mere whisper of a data cleansing plan, the whiff of anonymised quants, or a hint of cumulative comparative processes, has them chomping at the bit to get down and dirty with the data.
Whilst data may not light my fire, I am unequivocally passionate about sharing information. My friends tell me I don’t know when to stop! I get frustrated when we reinvent wheels; I am irritated when we don’t adequately learn lessons, and I find it inexcusable that we hold insights close to our chests for fear of diminishing their value, when actually, opening them up for others to view and scrutinise would increase the knowledge multiple times over. I may not be good on detail, but the concept of an effective archiving process does resonate with me as something which is fundamentally important.
Of course, when we started out on this journey ten years ago, we hadn’t thought far enough ahead to know that we would want or need an archiving process; we just knew that we wanted to ensure that whatever we learned, whatever data we gathered, and whatever implications these had for policy and practice, would be disseminated thoughtfully, transparently, accessibly and honestly. Simple!
The Childhood Development Initiative (CDI) was formally established in 2007, after a three year consultation and assessment process with those living and working in Tallaght West (TW). Funded through the Department of Children and Youth Affairs (DCYA), and The Atlantic Philanthropies (Atlantic), CDI set out to design, deliver and evaluate a suite of interventions aimed at improving outcomes for children and families. In our first year of operating, we began an EU wide tendering process to commission eight independent evaluations, including three randomised controlled trials and two quasi-experimental studies, with a total spend of just under €2 million committed to various academic institutions by the end of 2008. Since then, we have commissioned a number of further studies and spent €2.7 million, or almost 12.5% of our total budget on research related activities. This comprehensive and often complex process was enabled by leadership from our Board of Management, and an Expert Advisory Committee (EAC) overseeing and advising the research programme. In addition, Atlantic provided legal guidance in relation to copyright and intellectual property rights (IPR) which subsequently proved to be invaluable.
Atlantic required that all our contracts stated that IPR would be held by CDI, which would in turn grant the researcher:
A perpetual, transferable, non-exclusive, royalty-free licence (carrying the right to grant sub-licences including to the Department of Children and Youth Affairs and The Atlantic Philanthropies (Ireland) Limited) to use for any purpose any such Intellectual Property Rights.
In essence, this clause ensured that CDI would own any intellectual property, including qualitative and quantitative data, produced through or arising from, research and evaluations funded by us. In 2008, it was extremely unusual for IPR to be held by anyone other than the researcher, and even more unusual for contracts to explicitly state IPR ownership in favour of the commissioning agent. The implications of this were not immediately apparent to the various research teams with whom we worked, and it wasn’t for a couple of years that we realised ourselves how important this clause was.
Unravelling the implications
Given the number of evaluations commissioned, and the fact that there were so many academic institutions and Principle Investigators involved, for the first three years or so, CDI brought the eight evaluation teams together on a quarterly and then half-yearly basis. These meetings proved to be extremely useful for the purposes of scheduling data collection, identifying synergies between approaches and maximising the utilisation of the information being collected. They were also informal networking opportunities and created a sense of camaraderie or shared purpose amongst many of the research teams.
It was at one of these meetings, in early 2009, that the implications of the contractual commitment first became apparent for many of the research teams. In discussing the eventual use of the data, and the potential dissemination plans, one academic suggested that this would require the consideration of their ethics committee, as the potential for archiving data had not been agreed as part of their ethics approval process. CDI noted that our ownership of the IPR determined that the dissemination process was, in effect, in our hands. The ensuing discussion highlighted a very significant difficulty for some of the evaluation teams because the process of agreeing, scrutinising and signing off on the contract with CDI had been undertaken by people who had no connection to or dialogue with those who oversaw the ethical approval process. Likewise, the ethics committees did not review contracts or consider their content in any way. There was an almost universal separation of roles in terms of the legal considerations and those relating to ethical approval. And so, we had a situation where individual evaluation teams had signed a contract with CDI which contradicted the terms of their internal ethical approval agreements.
Inevitably, unravelling these complications took some time, and to date they have not been universally resolved. Establishing a legal context in which CDI was able to drive a significant and comprehensive dissemination strategy required considerable time, effort and expertise, but we were fortunate: we had money! We bought in expertise to undertake the following tasks:
– Review the IPR and legal context of CDI’s research and evaluation data; identify those which could potentially meet the required standards for submission to a national archive, and set out a plan for progressing the archiving of each available dataset;
– Write an Archiving Toolkit based on CDI’s experience, to enable others to replicate the process (http://www.twcdi.ie/wp-content... uploads/2016/11/CDI-Sharing_Social_ Research_web.pdf)
– Engage with and support each individual research team to archive relevant data.
In a similar, parallel process, we were invited to engage in a project with the Irish Qualitative Data Archive (IQDA), which entailed being the pilot site for archiving qualitative data, and supporting the documentation of the process and lessons learned (https://www.twcdi.ie/resources... publications/). Resulting from a connection with a member of our EAC, this project really cemented our commitment to engaging in and driving effective archiving of our own research and maximising the capacity of others to also do so.
Commissioning Dr Brid McGrath and Robin Hanen to support the archiving of internally held data, and CDI owned data held by independent evaluation teams was critical to progressing this initiative. Their expertise and detailed understanding of the sometimes very
technical tasks to be completed (such as the management of imputed data, or developing a consistent approach to metadata which could be consistently applied across all archives) were invaluable. However, their passion for the work, their ability to get excited by the complexity of numerous seemingly disparate datasets, and their ability to communicate the potential impact of effectively sharing this information was not only energising but transformative. I was converted!
Being surrounded by others who had the knowledge, skills and feelings to support a long, slow, sometimes very frustrating process was also vital. The required knowledge related to a thorough understanding of research and data, but also of how organisations work, and the kinds of dynamics which can enable collaboration or mitigate against it. The skills needed were those relating to being solution focused, and having the capacity to work around obstacles. The relevant feelings were to do with having the commitment, enthusiasm and motivation to not only do the right thing but also to do it right.
Finding the right people for a particular job can sometimes be a matter of luck, but the chances of doing so are significantly increased if you know what you’re looking for. In this case, through our engagement with the IQDA, coupled with the expertise within our EAC, and drawing on our experience with the evaluation teams to date, we were able to draw up a fairly concise and precise tender document. This undoubtedly informed our selection of the archiving supports.
In addition to the above, for an organisation to commit to sharing its data, and having all its research products and findings shared and laid open, requires a particular ethos and mind-set. Not all organisations will readily agree to share the data, irrespective of the research findings or conclusions. This is not a process which can be readily adapted deepening on the stomach for it so up-front discussion about ‘what if…’ is needed. What if…the research doesn’t find any change? What if…the findings reflect poorly on us? What if…the evaluation indicates that our work didn’t do what it set out to do?
Dialogue and reflection throughout the organisation to consider these possibilities is therefore an essential component of an archiving strategy.
The central mechanism to enable effective archiving is to build in the possibilities for this from the outset. When the study is complete, the numbers crunched and conclusions drawn, it may be agreed that there is little of value to be archived; that the anonymisation process is too complex to undertake, or would produce relatively meaningless information; the findings may be indeterminate or dull, and the available effort to disseminate them reduced accordingly. None of this can be predicted, so start with the assumption that you will want to archive as much as possible. This will shape your ethical approval and consent processes; it may even inform your methodology in terms of using standardised surveys or the balance of qualitative vs quantitative data.
Clarity of ownership is also vital, and this should be explicit in any service level agreement or contract, and ideally be made clear from the outset of the procurement process.
Finally, as with pretty much any development, surround yourself with the best! Source people with relevant expertise, and bear in mind that often this doesn’t require funding. Most people are delighted to share their insights, especially if this will support a process aimed at knowledge transfer.