|Hutton Roof Crags|
Friday, 26 August 2016
Wednesday, 24 August 2016
Yet of the huge amount of geological data that BGS holds from borehole or surface and subsurface sampling, we actually acquire a relatively small of the data ourselves, the rest is deposited with BGS for statutory reasons. Therefore a new geological model might incorporate data collected by any (or even all) of the water, energy, minerals and construction industries, each delivered in their own formats to their own standards.
For the last 25 years, BGS has been holding data in normalised structured databases. Normalised data means holding every element of your records separately and once only, with keys to then link these fields together. This is highly efficient, it means that whenever you move house your credit card company only needs to change one address field and not every transaction record. The disadvantage of such an approach can be slow response time and complex data structures for scientists to interrogate. Creating a 3D model might involve interrogate 17 major databases, with 50 datatypes containing millions of records. Until now, each dataset has to be searched separately using different tools and then laboriously reformatted. So making a “first-look” 3D model could involve several days work before any interpretation can occur.
To solve this we’ve adapted another idea from the finance insurance industry and built PropBase the world’s first true geo-data warehouse. A data warehouse takes a copy of the original data (thereby ensuring its integrity), then reformats the data back together in a standardised structure, and outputs them in common formats. Given that all data used in 3D models has a broadly common structure (a location in 3D, then the datatype, its value and any qualifiers) they can be imported in a common way. Therefore PropBase outputs these data standardised into a common set of multiple output formats from each record (e.g. a GIS shape file, a CSV for importing into modelling packages or webservices for machine-to-machine interrogation) by simply flicking a “switch” to toggle between them. The key advantages are massively improved data response times to querying and standardised outputs so data can be imported much more quickly by modelling software, these same ideas are now being used as templates for more complex datatypes such as real-time streaming of sensor outputs.
|PropBase Explorer tool showing spatial search for physical property data within the area of interest defined by a |
Our new publication in Computing and Geosciences defines this new data structure and applies a model for how scientists can effectively access and serve complex multiple spatially enabled structures. If you find it useful please cite it. (Please note that this is behind a paywall, researchers who cannot access this should contact me).
Monday, 22 August 2016
After such a great time in sunny Builth Wells, it’s now time to roll out the questionnaire to the rest of the UK with the help of project partners from the veterinary sector. We’re hoping we have as positive response from these farmers as we did at the show!
Friday, 19 August 2016
|A weathered sulphide chimney encountered during a |
HyBIS dive on new mound.
|Inside the RD2 control container during seafloor drilling operations.|
Thursday, 18 August 2016
Open data is at the heart of our work in Informatics and we are constantly developing ways of making that data available to the wider community.
We have just launched a new search application which gives access to all open data deposited with the National Geological Repository (NGR) and National Geoscience Data Centre (NGDC) (including NERC-Funded Grants and UK Carbon Capture and Storage projects) and, in addition, holds all the Scottish data which has been relocated as a result of the move to the new Heriot Watt campus.
You can search using keywords or a geographical interface. Items that are held digitally can be opened and viewed directly from the results screen. Physical items are available on request through our enquiry service.
All data can be accessed by clicking here.
Please note some data may be held with restricted access, in these cases a justification is supplied and a contact is given if you wish to gain further information.
Contacts for more information:
Information Management: Ali Fernie
Data Collection and NERC Grant Data: Sue Roper
UKCCS Projects: Mary Mowatt
Monday, 15 August 2016
|Liam and Grace in the Stable Isotope Facility|
For me the week has been mind-blowing and so technical - working with mass spectrometers and vacuums! Scientific equipment that I have never even seen before, this made the week even more intriguing for me as I have been doing things that I have never ever dreamt of! To begin with myself and Liam (another student on work experience) kick-started the week with vacuum extractions of soil samples, whilst using liquid nitrogen! Wow! I felt like a real life scientist, working with intriguing gases and chemicals. On Tuesday I was given the job of micro-carbonate preparation. This entailed weighing micrograms (very very small amounts) of sample material and carefully putting the sample into tiny vials that were to be later put into the mass spectrometer where their isotopes would be measured.
On Wednesday I did carbonate preparation for isotope analysis in the morning, and in the afternoon we measured isotopes in water. The water samples came all the way from the Antarctic as ice cores from the ice cap!! It was interesting discussing with one of the geoscientists, Carol, about how ice cores from the Antarctic ice sheet are used in her research on climate change from 750,000 years ago to present. The water put into the mass spectrometer was measured for oxygen isotope ratios (18O/16O) and this data is converted into past temperatures. Carol has been able to show that Antarctic temperatures have changed over time and the western Antarctic is one of the fastest warming parts of the world. On Thursday Liam and I did some more carbonate extractions of carbon and oxygen isotopes where we had to prepare and extract the samples using a vacuum line and later used the mass spectrometer to find out the different carbon and oxygen values. Later that day we also went around all of the labs with Mel to help with a safety audit in order that she could keep a check on safety in the laboratories.
|Grace working on a vacuum line which extracts and |
collects gases for analysis
Grace Nicholls and her fellow student Liam Curtis are sixth form student at Rushcliffe School.
Thursday, 11 August 2016
Monday, 8 August 2016
|Pokémon invade the BGS|
One lunchtime last week, my colleagues introduced me to this on their phones. It was quite an experience and I found myself spinning a 166-year-old statue of Hercules for Pokéballs, chasing an angry-looking squirrelly-thing behind a building I didn’t even know you could get behind (which actually proved to be quite a lovely patch of garden), and talking Pokémon to a group of teenagers at a football club up the road from the office. Three things I certainly wasn’t expecting to happen that day and I doubt Hercules has had this much attention since the fig leaf incident in 1883. Don’t ask.
Then we were suddenly in ‘a Lure’ and surrounded by confetti and we released pink incense and more Pokémon appeared. It was all tremendously bewildering and I laughed my socks off. Although I could have done without the ratty fellow. He was quite a bitey chap.
|Roll up! Roll up! Get your Pokéballs here! The nearest BGS Pokéstops|
|L to R: Pokémon at our Herriot Watt office; our Clive with a worm sporting a party hat; a Fearow on a bench.|
|L to R: Bitey Ratty Chap after your biscuits; our Wil with an angry bird in the office; Hercules can't believe his eyes|
|L to R: Party hat worm in the Geology Shop; a fuzzy thing in our library; |
the bitey ratty chap in the office; party hat worm on the Cromer Sheet?!!
|L to R: Angry squirelly thing on the Geological Walkway; Surprised crab on the steps outside |
the James Hutton Building; An Alfred Hitchcock situation outside the William Smith Building.
Friday, 5 August 2016
|We have around 125,000 high resolution core photos|
It would be nice if we could reassemble the core so it appears as if it is laid out in a long line on the floor. This could help with visualising longer runs of core, for logging, for instance. The catch is that there is some variation between the images which to the human eye look “the same”. This can be easily demonstrated by overlaying a few images and adjusting the transparency on them:
|A few of the core images overlaid with adjusted transparency|
Can we hack it? Yes we can!Like all good problems, we broke the system down into smaller components. Basically, these were:
- Load the image from our large image server, which delivers all of our public facing core photos
- Find the corners of the core boxes in the image, or (even better), the edges of the actual core
- Cut away the non-data pixels
- Match the top and bottom depths to the start and end of the core run in the image
- Join the core end-to-end
- Repeat until you have a complete core
How did we do it?Looking at the stages outlined above, it was clear that finding the corners of the boxes would be the greatest challenge. We looked into image analysis, and whilst we were able to run basic filters, we found it was taking quite some time to run, and, perhaps due to our lack of experience with the software, we were not going to be able to tweak it to produce reliable results in the time available.
|The results of running the Canny edge detect filter on an image|
Splitting up into three pairs, our team designed the three components that we needed to successfully complete the task:
- Andy and Roman built Oracle database tables to hold the positions of the points, so we could re-use them at any later point in the processing
- Paul and Simon looked into the viability of using image processing software, and wrote a simple script for ImageMagick which would perform the cropping and stitching
Running over WiFi, and on a laptop, the whole process took about 60-90 seconds. However of this only around 10 seconds is user input time, and the remaining processing could be stored until later. We also realised that we could “cascade” the cropping information to the next image, simply requiring the user to confirm the crop or perhaps adjust it as necessary.
If in the future we are able to find the corners using entirely automated image analysis, we can simply drop the code into the workflow without too much hassle. For the purposes of the hack, however, I think it was a valid choice to use the method which allowed us to demonstrate the process in the time available.
We also still have to work out how each sub-run is joined together to form the whole core, and write some more script which adds depth markers. There are some definite speed improvements that we can make, for example creating the JPEG preview from the JP2 in advance, or using the large image server to generate it on the fly.
What could we do differently?We’d still like to get the image analysis working, if not on the existing images then on any new images we take. There were enticing suggestions in the form of utilities such as “visgrep” and “zbar-img” that we could use to identify the type of tray in use and also the position in the image, and from there apply a standard crop.
|The judging panel|
I thoroughly enjoyed the hack – often I feel that I know that something can be done, but am clueless to how to actually achieve it. Working directly with people whose skills complemented mine was an ideal and effective way to solve this problem.
I feel that, given the time available, we were able to show that a process we previously believed to be impossibly time consuming to achieve could in fact be broken down into smaller chunks and made achievable. The next step is to consult some of the many users of the core images, and see what their thoughts on the matter are.
Simon Harris (BGS Conservator / Hack participant) and the hack team of Paul Denton, Brian Hamilton, Andy Riddick, Roman Roth and Paul Williams.
Wednesday, 3 August 2016
The word Olympic literally means 'of Olympus or 'of Olympia' with Olympus being famous in mythology as the home of the Greek gods, so it's no wonder that there are a number of locations across the world (and beyond) that have chosen to share that name. We've decided to have a look at these places and see if their geological heritage meets the 'gold medal' standard.