NBN Record Cleaner is a free software tool to help people improve the quality of their wildlife records and databases.

Whether you are an individual recorder or work in an organisation such as Local Environmental Records Centre or a Recording Scheme, the NBN Record Cleaner is designed to help you spot common problems in your data. The goal is to aid the process of data cleaning and ensure the quality of any datasets you pass on to others.

It is designed to access biological records stored in a wide variety of formats such as text files (CSV, tab delimited, etc), Excel spreadsheets and databases – including those in biological recording packages such as Recorder and MapMate.

Automatic checks

validation is the process of checking if something satisfies a certain criterion

The tool first “validates” your data – checking the format against a set of built-in rules. This includes spotting bad dates (e.g. 31st February) or spatial references (e.g. TL123)  and checking the spelling of items like species and vice county names. You can correct any problems on screen or change the original source and reload before proceeding.

verification: confirmation or additional proof that something that was believed (some fact or hypothesis or theory) is correct

You then choose the “verification rules” you want to apply. These essentially check whether the data is credible and give you warnings about records that are unusual in some way and need further investigation. These checks can include issues such as a record of a terrestrial animal which is in the sea, or outside its currently known range, or occurring at a time of the year when it is not expected.

Verification requires additional information that is supplied in the form of “rule files” which are maintained by relevant experts. You choose which rule sets you wish to use and they are downloaded and installed from the internet. The application automatically notifies you about which rule-sets  are available, or have been updated.

Outputs

As well as presenting the records with the potential problems highlighted, the tool also allows you to map your records. This helps you to spot misplaced records.

The tool does not change your original data. It produces reports of the items that were queried, but you must apply any required changes using whatever tools you normally use to manage your data.

Downloads

The latest version of the NBN Record Cleaner (V1.0.8.4) was released in February 2013.

Note that when installing this on Windows Vista or Windows 7 please ensure you are running as an administrator. 

NBN Record Cleaner demonstration

View the powerpoint presentation in video format which demonstrates how Record Cleaner works.

Improving data quality – creating verification rules

The reports below are all concerned with improving data quality and have all been produced under the NBN Trust’s contract with Defra.  They outline how verification rules were created and what verification processes should be used – essentially, what to do with records flagged up by Record Cleaner.

Verification rule sets

Final Report on the Verification Rules for NBN Record Cleaner

British Arachnnological Society & Spider Recording Scheme Improving the quality of spider records

Dragonfly Recording Network Record Cleaner Verification rule sets for NBN Record Cleaner and recommendations on species whose records should be treated as sensitive

Improving the quality of bryophyte data on the NBN Gateway

Mammal records verification rule sets for NBN Record Cleaner and recommendations on species whose records should be treated as sensitive

Marine Biodiversity Data Flow – Marine Biological Association

Improving the quality of Lepidoptera records available via the NBN Gateway (Butterfly Conservation) – revised

British Trust for Ornithology – Enhancing data quality of bird records for NBN_120711

Botanical Society of the British Isles – Improving the quality of botanical records available via the NBN Gateway