Process and analyse

Data processing and analysing is an important stage in the research data cycle. Data Analysis is the process of systematically applying statistical and/or logical techniques to describe and illustrate, condense and recap, and evaluate data (Responsible Conduct in Data Management). Data processing is the manipulation and conversion of data by computer in order to format or transform raw data into machine-readable data (Britannica Academic).

It is recommended that working data (copy of your master copy) is stored for processing and analysis in other locations such as QCIF but ensure that your master copy is stored in a secure location such as QUT's Research Data Storage Service.

Easy metadata for working data

A simple way to capture metadata during the working phase of your research is to create a readme.txt type file, a collection of simple metadata that describes the details of the datasets and improves the long-term usability of the data. Save the readme.txt file with the data files in the same folder within your research storage.

Download a Readme.txt template from Cornell University's guide.

In some cases, metadata can be generated or extracted from digital files automatically. For example, a digital camera records the date, time, exposure setting, and file format. Software programs sometimes allow structured metadata such as title, author, organisation, subjects or keywords to be added via 'Properties'. See more tools for understanding digital files.

Documentation

Methods of processing must be rigorously documented to ensure the utility and integrity of the data.

Documentation at the processing and analysing stage should include:

  • derived data created, with code, algorithm or command file used to create them
  • weighting and grossing variables created and how they should be used
  • data list describing cases, individuals or items studied, for logging qualitative interviews
  • all structured, tabular data should have cases or records and variables well documented
  • other documentation may be contained in user guides, reports, publications, working papers and laboratory books.
Example of a spreadsheet of metadata, visible columns for 'Instrument detail' worksheet are: site, Location, type, instrument number, From. Other visible worksheet names are: Field schematics, Beckingham, Bushley, Cuddyarch, Idle.
Embedding metadata in an MS Excel spreadsheet. (Source UK Data Archive, 2017).