Tools, resources and publications
NBN Record Cleaner
NBN Record Cleaner is a free software tool to help people improve the quality of their wildlife records and databases.
Whether you are an individual recorder or work in an organisation such as Local Environmental Records Centre or a Recording Scheme, the NBN Record Cleaner is designed to help you spot common problems in your data. The goal is to aid the process of data cleaning and ensure the quality of any datasets you pass on to others.
It is designed to access biological records stored in a wide variety of formats such as text files (CSV, tab delimited, etc), Excel spreadsheets and databases – including those in biological recording packages such as Recorder and MapMate.
“validation is the process of checking if something satisfies a certain criterion”
The tool first “validates” your data – checking the format against a set of built-in rules. This includes spotting bad dates (e.g. 31st February) or spatial references (e.g. TL123) and checking the spelling of items like species and vice county names. You can correct any problems on screen or change the original source and reload before proceeding.
“verification: confirmation or additional proof that something that was believed (some fact or hypothesis or theory) is correct”
You then choose the “verification rules” you want to apply. These essentially check whether the data is credible and give you warnings about records that are unusual in some way and need further investigation. These checks can include issues such as a record of a terrestrial animal which is in the sea, or outside its currently known range, or occurring at a time of the year when it is not expected.
Verification requires additional information that is supplied in the form of “rule files” which are maintained by relevant experts. You choose which rule sets you wish to use and they are downloaded and installed from the internet. The application automatically notifies you about which rule-sets are available, or have been updated.
As well as presenting the records with the potential problems highlighted, the tool also allows you to map your records. This helps you to spot misplaced records.
The tool does not change your original data. It produces reports of the items that were queried, but you must apply any required changes using whatever tools you normally use to manage your data.
The latest version of the NBN Record Cleaner (V184.108.40.206) was released in February 2013.
Note that when installing this on Windows Vista or Windows 7 please ensure you are running as an administrator.
NBN Record Cleaner demonstration
View the powerpoint presentation in video format which demonstrates how Record Cleaner works.
Improving data quality – creating verification rules
The reports below are all concerned with improving data quality and have all been produced under the NBN Trust’s contract with Defra. They outline how verification rules were created and what verification processes should be used – essentially, what to do with records flagged up by Record Cleaner.