Following the launch of the NBN Gateway 5 we have encountered and dealt with a number of issues. There are however, some new and some ongoing technical issues which we have been working to resolve. We have also received feedback and comments on the new system, which have been very useful. These have already been actioned, added to our user requirements list or followed up directly stating why certain requests cannot be implemented.
As ever, we want to keep you abreast of developments and to communicate openly to all users. We therefore hope that the information below will answer a number of queries and will also alleviate some concerns.
Performance and uptime (availability) of the NBN Gateway
We had an issue with spatial indexes, which meant that maps performed very badly when zoomed in. That persisted until early February, but has now been resolved. There are a number of searches and displays which are slow, but for the most part these are searches that were not possible to perform in NBN Gateway 4, as neither the performance nor the technology was available. One example is that all Diptera records can now be displayed, which wasn’t previously possible.
Performance continues to be a major concern and we are endeavouring to make this the best it can possibly be. We are the only system in the world to have the sort of access controls that the Gateway supports, and so we consistently sit at the “bleeding edge” of technology.
We are already looking into technologies to allow further performance increases, and localised caching for heavy users and those with increased support demands. This will mean that a local copy of NBN Gateway data that you have access to can be kept, and usage reported back to data providers. This will allow organisations to have access to data when the NBN Gateway suffers issues. Whilst we cannot give you any more detail at the moment, this is how we are looking to the future for the NBN Gateway.
The Joint Nature Conservation Committee is in the process of letting a contract to review the Geographic Information strategy around the NBN Gateway. This is mostly focussed around Web Feature Services (WFS) and data presentation which we hope will make our mapping systems easier and clearer to use.
Last week we released an update to change a problematic backend caching mechanism, and the database replication system has been made more robust. There is still a late night (midnight to 5am) stability issue we are looking into, and hope to have a solution for next week. Getting to the root of these issues has been difficult, and we are lucky to have a highly skilled developer in Matt Debont to find the root causes.
We are reliant on a layer of server software to run our code (like webservers and map servers), which essentially makes the system work. These are open source pieces of software with robust development communities. Unfortunately one of these, Tomcat, has been an issue. As we are running at the ‘bleeding edge’ of technology we are triggering undiscovered faults in Tomcat and the supporting Java ecosystem which causes the server to fall over.
We are planning to put up a ‘status page’ to assist us in communicating uptime and performance metrics to end users, which we hope will be helpful.
Features
Despite the consultation, we are aware that users dislike a number of new features and wherever possible these will be fixed, in due course.
These include:
- We will review the datasets panel on the Interactive Mapping Tool (IMT) to improve usability. This work will occur during the next iteration of the IMT due to happen before April.
- Commenting on Records. This functionality will return in the 2014-15 financial year. We are currently aligning it with the consultation run around record commenting in iRecord and other issues around record quality. The first part of this work is due by April.
- Reading Grid References. A change in underlying technology has made this difficult to re-implement. We will however, look into the feasibility of restoring this function.
As part of the consultation over access controls a number of compromises, workarounds and agreements were made with data providers and data users in relation to viewing records, viewing record details and data security. These were discussed with the community on a number of occasions. They were never going to please all parties, but the consultation did lead to the design decisions you see today.
Of course, it is almost impossible to have a system that suits every user's needs and we are juggling requirements as best we can. Some of the key data access changes requested are:
- Any method of retrieving raw records would be required to be tracked by the user logging in
- Acquiring large quantities of records would require a download, which in turn would require a reason being lodged by the downloader.
- Users needed to be able to download data they could see
- Time limits on data access.
- One of the lost pieces of functionality is the ability to assign intermediate resolution access to a user. The default is now Public or Full access
Browser Incompatibility
Whilst we endeavour to be compatible with the vast majority of browsers, and take development steps to ensure it, it is not possible to suit every browser/Operating System(e.g. Windows, iPad, Android etc.) combination. Our priority is to those browsers with highest market share, and those various organisations are forced to use (Internet Explorer 6/7). For instance, Opera represents less than 2% of market share and is not a corporate desktop standard. However, the Gateway is mostly Opera compatible as it based on the jQuery Javascript library, but there are some issues within the Interactive Mapping Tool.
If you are having problems with your browser, we would appreciate you sending us the issues you have encountered, including error messages, so that we can add these to our fix list. Please ensure you are running the latest version.
Easimaps
A new version of Easimaps was being tested when we discovered some missing functionality that had not been included in the public specification for the Easimaps API (Application Programming Interface). The development work on this has been completed and once testing is finished a new version of Easimaps that will interface with the new database will be rolled out.
We want this to be as seamless a transition as possible for users of Easimaps, so are taking our time to ensure we do not issue an incomplete version.
GBIF
One of the major issues with Global Biodiversity Information facility (GBIF) is that it allows download of data without logging. One of the workarounds that we came to during the access controls consultation was to ensure all downloads were logged in return for removing the ‘view only’ control. We gave GBIF a copy of publically downloadable data from NBN Gateway 4 prior to July and the switch over. We have not subsequently given them any data whilst we tackle the very complex issues around this functionality.
We apologise for not responding to this issue sooner and for not communicating the status.
Direct URLs
With the redesign of the site, a number of fundamental changes were required. The old NBN Gateway exposed a number of internal keys in the URLs and these could not be preserved to the new design. The URL design now follows best practice in the field following the REST specifications. On reflection this could have been communicated more clearly.
Interactive Mapping Tool (IMT) Defaults
The new IMT has far more functionality than the original interactive mapper. The design is intended to be accessible to users who only need to quickly view data, whilst making the advanced functionality discoverable for power users. We are collecting usage metrics to enable us to improve the experience.
Designation History
The data for historical designations had a number of fundamental issues leading to inaccuracies and errors. They are not maintained to keep them in track with changing taxonomy. A decision was made to remove them, in conjunction with Joint Nature Conservation Committee.
Inconsistency in Designated Sites List
The inconsistency in site names comes from the original data providers. We rely on the Country Agencies and other bodies to label sites and return what they provide. Some add the designation type, others do not.
Dataset Upload
JNCC is currently focussing on reducing the backlog of data to be loaded. We apologise for the delay and refer you to our previous update. Progress on data loading can be seen here on the NBN Gateway.
NBN Gateway 4
A few users have asked why we cannot return to the previous version of the Gateway. This is not possible for a number of reasons, which are beyond our control:
- It would not scale beyond the current volume, so would not allow it to hold any more records. Unlike NBN Gateway 4, version 5 is scalable into the future.
- It is based off ESRI ArcIMS, a technology that is no longer supported and will likely cease to be licensed at the end of 2014-15 FY.
Responding to queries
We try to deal with enquiries and feedback as quickly as possible. Some queries go straight through to the technical team and others go to the Secretariat depending on the contact method used. Whilst we try to respond to issues as they arise, please be aware that we can only provide full support during office hours.
Some of the priority issues that we are currently dealing with are:
- Uploading new and updated datasets.
- Late night stability issues.
- Providing new record commenting facilities.
We will deal with all issues in due course, but please be aware that resourcing is tight and so we may not be able to respond as quickly as you would like us to or as we would hope. We hope you understand and we thank you for your continued patience and support.
Please send any further comments or queries to access@nbn.org.uk
Previous information on the upgrade can be found here and here as well as specific feedback to queries on the NBN Forum