The future of the NBN Gateway
Since the NBN was formed 16 years ago, our community has shared over 127 million biological records; creating the largest single biodiversity database for any single country in the world! The NBN Gateway, managed by Centre for Ecology & Hydrology (CEH) and Joint Nature Conservation Committee (JNCC) has played a fundamental role in enabling our Network to share these records, both between members of our Network, and externally with others who require access to these data.
As the needs of our network increase, we must ensure that the infrastructure underpinning this world class network is fit for purpose, in turn ensuring that our biodiversity data holdings are put to their maximum use.
With this in mind, as the volume of data we share continues to increase, the complicated access controls we use to manage our data holdings grow and running costs increase. Furthermore, the scope to expand the technological functionality offered by the current system is limited, so it has unfortunately become unviable to continue to support and develop the NBN Gateway as our data sharing infrastructure.
CEH has confirmed that it will host the NBN Gateway for 2016-2017, however, as a result of many different factors, extending its lifetime past April 2017 will incur significantly increased costs.
Technical Review of the NBN Gateway
As many of you will be aware, with your help, the NBN Secretariat undertook a technical review of the NBN Gateway in 2015. During this review we heard that;
- There is a complex technical infrastructure in the UK for capturing, curating, analysing and disseminating biological data and information.
- The creation of the NBN Gateway and the aggregation of more than 100 million biological records is an extraordinary achievement and demonstrates the value of collaboration.
- There is a diverse array of users of NBN systems and meeting all their needs will be a challenge and will undoubtedly require compromise. It is unlikely that all needs of all users can be met by one system. There are many potential audiences that are not being reached at present.
- The existing data infrastructure is not fit for purpose and alternative solutions or significant work is required to ensure the needs of both data providers and users are met.
- There is tremendous support within the National Biodiversity Network to collaborate to improve the data infrastructure, to share biological data and to help both data providers and users.
- The UK Species Inventory is an important system that needs investment and promotion.
- Supporting data providers will require:
- Ensuring data flow processes are simple and efficient.
- Ensuring data access request processes are simplified and streamlined.
- Greater promotion of data providers.
- Support for storage of diverse data types (images and habitat, environmental layers etc.)
- Supporting data users will require:
- Improved user interface.
- Improved security and stability of the infrastructure.
- Improved tools and systems for viewing, accessing, analysing and downloading data.
- Most data are not shared and more work is needed by all biological data custodians to provide a collective view of UK biodiversity.
Taking all this into account the NBN Secretariat and the Atlas of Living Australia team, with the help of you, our data partners, are now building a new biodiversity infrastructure which will replace the NBN Gateway.
The NBN Atlas
The NBN Atlas is based on the Atlas of Living Australia infrastructure. There will be separate Atlas front-ends for each of the four countries of the UK, as ways of accessing the underlying common database, as well as the overarching NBN Atlas. The individual country front-ends will be customisable and the steering groups and stakeholders will be involved in these decisions. Indeed, the NBN Atlas Scotland is already live and is continuing to be developed, much of the development being as a result of user feedback.
The NBN Atlas provides a user-friendly interface for viewing, downloading, and analysing data, and creating information products. Photos and links to external ecological information will enhance the ability to put data and information into context. A broad range of biodiversity and environmental data types can be utilised and outputs can be combined with locally-held data using powerful analysis and query tools.
Opening up access to data
During the technical review of the NBN Gateway, we also heard that the main motives for data partners supplying data to the NBN Gateway was to “provide data for the common good” and to make “data available to all”. Furthermore, responses to the Strategy consultation clearly stated that the most important strategic aim was to “make biological data and information available to everyone”, and there was support for increasing the visibility and openness of data across the NBN. There was also the support for ensuring that records are made easily available as this could inform conservation on the ground. Respondents to the consultation also stated that the NBN Gateway did not meet data partners’ needs. In particular, restrictions on access to data and difficulty in gaining access to enhanced data were commonly cited as reasons for frustration.
Taking the views of the range of NBN members into account the NBN Trust, led by its Board of Trustees, is committed to the NBN Atlas being open by default, except access to sensitive species which will be restricted. Initially, we would like, with your permission, to transfer the records that you have on the NBN Gateway, at their current public resolution, to the Atlas. Data providers will be contacted in due course, to agree revised data partner permissions and which creative commons licences should be applied to datasets (see below). These datasets can be used as discovery datasets which will link to your own member page on the Atlas, where you can provide more information about your data holdings and the process for applying directly to you for enhanced access. This should reduce the number of spurious requests for data that are already available publicly or are for a different geographical area. Furthermore, you can customise your member page to showcase the services you provide.
Given the NBN’s objective of making biodiversity data more visible, as well as freely and openly available, we would like to work closely with you to produce a road map for increasing openness of data over the next few years. If you would be interested in being involved with a working group on this subject, please get in touch.
Licences
Every record that is held on the NBN Atlas will be licensed with one of three Creative Commons Licences or an Open Government Licence (OGL). The Creative Commons Licences offered are:
- Creative Commons Zero (CC0)
- Creative Commons with Attribution (CC BY)
- Creative Commons, with Attribution, Non-commercial (CC BY-NC)
You can find out more information on Creative Commons licences.
The OGL Licence is for government datasets and is more or less equivalent to the CC BY license. The purpose of the licences is to ensure complete clarity over the permitted uses of data, and to ensure that all NBN data are compatible with GBIF’s licensing standards. The NBN is the UK node for GBIF and encourages data to be shared further than just within the UK. All data partners will be consulted about further data sharing if it is applicable.
A standard suite of Terms and Conditions will govern the use of all NBN Atlases in addition to the terms of the selected data licence. The Terms for the NBN Atlas Scotland are available for information.
As a Network, we must continue to work together to make the most of this opportunity to fully implement this new infrastructure across the UK, ensuring that our Network remains world leaders in collecting, aggregating, sharing, using and analysing biodiversity data.
We look forward to working with you as we move away from our hard working NBN Gateway and towards a new era of biodiversity data infrastructure delivered through the NBN Atlas platforms.
Q&As
Impact on me?
- Clearer and simpler terms and conditions for data use through standardised application of Creative Commons and Open Government Data licences.
- New system layout and design with added functionality
- Change of URL away from https://data.nbn.org.uk/
- Change from REST web services. The NBN Secretariat will offer support to help update web services to those affected
- Access to open data
- No data access controls – enhanced access directly through the data provider
Will my data be affected?
- Data will only be available at one resolution of your choice, unless the record is for a sensitive species. Data held in the Atlas can either be displayed as point or grid resolution data. The available resolutions include; Point data, 100m, 1km, 2km or 10km
- Data will be uploaded, stored and accessed in Darwin Core format, compatible with many other biological databases around the world. In the future, Data Partners will be able to ‘push’ data directly to the NBN Atlas at the time of their choosing (daily, monthly, yearly).
- Dataset metadata will be automatically transferred over from the NBN Gateway
- Data will be under a creative commons licence of your choice (CC0, CC-BY, CC-BY-NC) or Open Government Licence (OGL)
What do I need to do?
- We will be sending out data partner agreements to all data partners when we are ready to receive your datasets for the relevant NBN Atlas.
- Send us your feedback on the current NBN Atlas Scotland to help us improve the site in advance of the NBN Gateway being turned off
- Introduce others to the new NBN Atlases
More information will follow as, together, we develop the road map for moving away from the NBN Gateway to the new NBN Atlas infrastructure. We will keep everyone updated as this project progresses and we will also be introducing a specific NBN Atlas blog in due course.
In the meantime, the NBN Forum is the place for feedback and discussion on the NBN Atlas.