NBN Record Cleaner

NBN Record Cleaner is a free software tool to help people improve the quality of their wildlife records and databases.

Whether you are an individual recorder or work in an organisation such as Local Environmental Records Centre or a Recording Scheme, the NBN Record Cleaner is designed to help you spot common problems in your data. The goal is to aid the process of data cleaning and ensure the quality of any datasets you pass on to others.

It is designed to access biological records stored in a wide variety of formats such as text files (CSV, tab delimited, etc), Excel spreadsheets and databases – including those in biological recording packages such as Recorder and MapMate.

Automatic checks

“validation is the process of checking if something satisfies a certain criterion”

The tool first “validates” your data – checking the format against a set of built-in rules. This includes spotting bad dates (e.g. 31^st February) or spatial references (e.g. TL123) and checking the spelling of items like species and vice county names. You can correct any problems on screen or change the original source and reload before proceeding.

“verification: confirmation or additional proof that something that was believed (some fact or hypothesis or theory) is correct”

You then choose the “verification rules” you want to apply. These essentially check whether the data is credible and give you warnings about records that are unusual in some way and need further investigation. These checks can include issues such as a record of a terrestrial animal which is in the sea, or outside its currently known range, or occurring at a time of the year when it is not expected.

Verification requires additional information that is supplied in the form of “rule files” which are maintained by relevant experts. You choose which rule sets you wish to use and they are downloaded and installed from the internet. The application automatically notifies you about which rule-sets are available, or have been updated.

Outputs

As well as presenting the records with the potential problems highlighted, the tool also allows you to map your records. This helps you to spot misplaced records.

The tool does not change your original data. It produces reports of the items that were queried, but you must apply any required changes using whatever tools you normally use to manage your data.

Downloads

The latest version of the NBN Record Cleaner (V1.0.8.5) was released in February 2013. The download above is for an earlier version (V1.0.8.3) and users need to run the software update
link on opening the tool after installing the desktop application in their local machine. This updates the application to the latest version (V1.0.8.5).

Note that when installing this on Windows Vista or Windows 7 please ensure you are running as an administrator.

An updated master species list for NBN Record Cleaner can be downloaded from the NBN Forum. The list is based on the latest version of the UK Species Inventory (July 2021).

The software updates link in NBN Record Cleaner doesn’t work on Windows 10, which is why this manual update is necessary. The NBN Record Cleaner application will be updated so that the species dictionary can be updated using software updates link for those people using Windows XP and Windows 7.

NBN Record Cleaner demonstration

View the powerpoint presentation in video format which demonstrates how Record Cleaner works.

Improving data quality – creating verification rules

The reports below are all concerned with improving data quality and have all been produced under the NBN Trust’s contract with Defra. They outline how verification rules were created and what verification processes should be used – essentially, what to do with records flagged up by Record Cleaner.