one of the buoy loggers misset to local Cayman/Miami time

[This post was created in January/2015 but back-dated to the time of the incidents described.]

On November 7th, 2014, I noticed that the CCMI2 buoy's two dataloggers were running on different time zones.  The "met" logger continued to (correctly) operate in UTC but the "main" logger was running five hours behind, which coincided with local time at the Little Cayman Research Centre and here at AOML in Miami.  Regarding the only other servers known to connect to these loggers, those of YSI, I'm not sure where they are located or what time zone they run in.

The reset happened on Wednesday, November 5th, at some time between 2:20pm and 2:30pm local Cayman/Miami time.

The most obvious explanation for this problem would be if someone had connected to the main datalogger from a computer running in this -0500 time zone whose local loggernet settings were configured to automatically "correct" the logger clock if it were found to be out of sync with the server.  This is not a default setting on loggernet but neither is it an uncommon setting for people to choose.

However, we were unable to identify any computer that might have reset this datalogger's clock.  Consider:

• Laura Wright reported, on November 12th, that their land-based servers were completely unable to connect to the station in any way:  "The antennae cable/connector end is literally broken. I'm waiting on parts to fix. There is only one computer on station that connects to the buoy via the antennae cable which is currently not connecting due to the cable issue and was the only computer connecting since July 14, 2014 when Rob transferred everything over before that previous computer died."
• The AOML loggernet server runs exclusively in UTC and anyway its settings were verified as not auto-correcting any loggers clocks even if they were found to be out of sync.
• Jon Fajans, on November 8th, suggested the possibility that the main datalogger might have reset its clock based on a time signal provided by the modem, since the modem likely "knows" what local Cayman time is.  However there is no known mechanism for the datalogger and modem to trade time information in this way.  The one bit of clock-sync code that used to exist, that which every hour reset the "met" logger clock using the "main" logger time, was in fact deleted in the last reprogramming because of problems it had caused at one of the Tobago sites.
• On November 12th, Matt Previte of YSI reported: "Verified that our Loggernet is not set to sync clock for Cayman."  He suggested asking CCCCC while saying that he didn't recall setting up the logger to call out to any servers but AOML's and YSI's.  I checked the programming and confirmed that there were no other callouts except for AOML and YSI, and did not pursue the matter further with CCCCC.

In the absence of any explanation for how the clock was reset in the first place, we decided that AOML would correct the time on the main logger through our IP/modem connection.  This was done at 2014/11/12 17:56:46 UTC.  Since then (as of January 2015) we have not noticed any further problems with either logger's clock.

The data from November 5th - 12th are all archived but all of their timestamps are wrong so they have not been loaded or integrated in any way.  Eventually I will write a simple script to correct the timestamps on those records so that this hole in the dataset can be patched.

Just to reiterate, we still have no theory about how the main logger's clock could have been changed on November 5th.

(signed)
Mike Jankulak

2014 Annual Maintenance

The 2014 annual maintenance of the Crews Buoy was completed this October by CCMI staff members Jon Clamp, Laura Wright and Lowell Forbes along with remote assistance from NOAA tech Mike Jankulak and field assistance from C-ARMS principal Jon Fajans.

 

The buoy was released from its mooring and towed to Salt Rock Dock on October 14 where it was lifted out onto our truck with the help of CB Power and Light. Once at the station the buoy was lifted off the truck and set down for maintenance. This included a thorough power washing and fresh bottom paint for the float ring. New zincs were attached as needed along with zip ties as well. The CTD was replaced after some programming issues were updated remotely by Mike Jankulak and was then launched and tested by Jon Fajans in Little Cayman (October 24 at 17:03 UTC). Our Bics were cleaned and left in place for another season due to their reliability. These will be replaced in 2015 or if and when needed during the coming year. A new RM Young Temp/RH sensor was also attached as this had failed earlier in the year (June 6). Jon Fajans completed an instrument test and all was pronounced operational.

 

Finally the buoy was given a pressure test on the battery and electronics housing and given the green light for redeployment. We launched the buoy on October 29 in a fresh Easterly breeze and had to manage the re-attachment of the buoy in less than optimal conditions however no problems were encountered. We were also glad to have the help on this leg by CCMI long time volunteer Greg Locher.

 

Although NOAA is receiving the data stream, CCMI has not received any data updates since redeployment.  We are currently working to resolve this issue.

Please view the video clip of the removal and still photos of the redeployment.

CREWS buoy redeployed, data feeds resume

From examination of the data record, here is a timeline of buoy refurbishment operations:

• 2014-10-14 1800 UTC: buoy towed from its deployment location
• 2014-10-15 1400 UTC: buoy removed from water
• 2014-10-24 1700 UTC: programming updated on both dataloggers
• 2014-10-29 1800 UTC: buoy once again moored at deployment site and fully operational

As of October 31st, 2014, all buoy data feeds are once again operational at AOML.  This includes the near-realtime web reports, the feed to the CHAMP database (as accessible via the CHAMP Portal), and the feed to NDBC's page for this buoy.

(signed)
Mike Jankulak

Data Sharing: CoRIS and NDBC

This is a followup announcement to an earlier post of mine, the one that talked about fully integrating the Little Cayman CREWS station's ocean and light data into CHAMP processing.  These data appear on our web page reports, are loaded into our expert system for ecoforecasting analysis, and are queryable from our CHAMP Portal web site.  Please see this prior post for details on the original causes of the missing ocean/light data and how we worked around it.

One thing that has changed since last month's post is that the concurrency problem with the CHAMP Portal is now resolved.  While there may still be some inefficiencies in serving very large data queries, there is no longer any user concurrency problem.  The CHAMP Portal can be found here:

http://www.coral.noaa.gov/champportal/

Another development is that the CREWS data and metadata reports were submitted to NOAA's Coral Reef Information System (CoRIS) on September 30th, 2014.  This submission covered data collected during the 2013 calendar year, and includes the first report of data from CCMI2 (for a period covering October 23rd through December 31st).

One way to access these data reports is to visit the CoRIS web site:

http://www.coris.noaa.gov/

Once at this site, click on "CoRIS Geoportal Search Tool" (prominently announced in the middle of the page, noted as "New!"), and then as a search term enter "CREWS AND Little Cayman AND 2013" (note that the capitalization of AND is required).

As of this writing this query brings back exactly one record, which is the submitted report for the CCMI2 site for 2013:

http://www.coris.noaa.gov/geoportal/catalog/search/resource/details.page?uuid=%7B8CB71F6D-1371-43E3-9CC7-C812E95BFAAC%7D

Other 2013 (and 2012) CREWS metadata/data submissions may be found on the CHAMP web site as follows:

http://www.coral.noaa.gov/noaa/coris/2013/
http://www.coral.noaa.gov/noaa/coris/2012/

On another topic, the feed of Little Cayman data from AOML to NDBC has begun, dating back to September 24th.  As is the convention with buoys, the Little Cayman station has been given an all-numeric designation by NWS.  This designation is 42089, although we will continue to use CCMI2 informally.  The NDBC data site can be found here:

http://www.ndbc.noaa.gov/station_page.php?station=42089

Please note that all of these data submission and feeds -- the CoRIS reports, the NDBC feed, the online CHAMP hourly and daily web reports, and the CHAMP Portal's data archives -- they all include the complete parameter set from the Little Cayman buoy, including the ocean and light data.

(signed) Mike Jankulak

Routine Cleaning

It is time once again for the CREWS buoy routine cleaning.  As you can see in the photos, we stir up quite a mess but it's all in the name of science!  Nothing out of the ordinary was observed and all reported data remains in the expected ranges.

Integration of Ocean and Light Data

It's not gone unnoticed that the light and ocean data from this buoy have until now not been included in AOML's reports and database.  This was due to a miscommunication between AOML, which processes the output of the buoy, and YSI, which created the buoy programming.  AOML's near-real time processing of CREWS data depends on having one hourly summary table with data from all station instruments, but the buoy's current program only provides 10-minute data summaries for light and ocean data and spreads these data summaries across three data tables, one per instrument.

I am in the process of altering the buoy's CRBasic programming to correct this problem.  Once these changes are complete the programs will be handed off to CCMI/LCRC for upload to the buoy.  Very likely this programming update will be timed to coincide with the buoy's annual refurbishment, when it will be brought to shore for cleaning, repainting, and instrumentation swapout.

We always knew that new routines would have to be created to process the first year's worth of buoy data after the fact.  Those routines have now been created, and with their creation it now becomes possible (through a minorly hackish kludge) to process ocean and light data in near-real time as well.

Complete data reports, including light and ocean data, are now available here for the first time:

http://www.coral.noaa.gov/static/data_ccmi2_Web_12.html

See also the links at the bottom of that page for "Detail" reports that include every parameter reported by the buoy, including diagnostics, and also the "Ten-Minute" reports that show some of the higher-frequency data that are summarized in the "Hourly" reports.

The light+ocean data have also been integrated into the "Ecoforecast" expert system that is running on our Gensym/G2 server:

http://ecoforecast.coral.noaa.gov/index/0/CCMI2/station-home

And the light+ocean data are also queryable through our beta-versioned CHAMP Portal product:

http://www.coral.noaa.gov/champportal/

Note that the CHAMP Portal currently has known issues with user concurrency -- only one visitor at a time may run a query, while others who attempt to connect will see a progress message that will remain fixed in place until the first visitor's query has finished.  Developers are hard at work on this issue and we hope to release a fix very soon.

As a final note about CCMI2 data integration, I have now set up a feed of data from this station from AOML to the National Data Buoy Center (NDBC).  As of this writing the feed is undergoing quality assurance on the NDBC end.  I will post an update (including a link) once the feed has gone live on the NDBC web site.

cheers,
Mike J+

Installation of anodes

On our last visit to the buoy we noticed the chains had started to pitt and the anodes were worn.  We have ordered new anodes to be replaced and today we added two 1" shaft anodes to each chain.  We plan to add more anodes to the main structure once they arrive.

Warning of mass bleaching of hard corals

Ecological forecast received from NOAA’s Coral Reef Early Warning System (CREWS) warns of mass bleaching of hard corals.

The warning was issued on Wednesday, 2014-Aug-13 as a result of high sea surface temperatures (SSTs) recorded at the ICON Reef station in Little Cayman (LCIY2) just offshore from the Central Caribbean Marine Institute.

PALE BUT NOT BLEACHED: Researchers today documented individual colonies with yellow and pale yellow colorations along the inner shelf of Grape Tree Bay (<3m depth) near the CREWS buoy.  Early indicator species are the Orbicella (formerly Montastraea) faveolata, O. annularis, and the agariciids. A single colony of Pseudodiploria (formerly Diploria) strigosa with pale coloration was observed adjacent to a colony with normal coloration (see picture). Although individual corals were pale, neither mass bleaching nor whole colony bleaching (i.e. white tissue) were observed.

DARK SPOTS: Individual Siderastrea siderea colonies were mottled or beginning to show signs of dark spots.
It is estimated that 5-10% of the above mentioned species were discolored.  All other species observed appeared normal in color.  Dive computer temperatures indicated 86°F at Grape Tree Bay.

Calm waters surrounding the Little Cayman CREWS this week.  Our new OA lab is screaming busy with three simulation experiments running around the clock. (in the background)

Routine Maintenance

Routine maintenance occurred today.  There was a small blockage in one of the CTD inlets and once that was removed the salinity readings returned to normal.  All other instrumentation was cleaned and continues to report accurate information.

February Routine Maintenance

A spontaneous heavy rain storm on Thursday afternoon forced us to postpone the routine maintenance on CCMI2 until Friday, February 14th. Friday's morning waters were extremely flat calm, slowly growing rougher over the course of the day. The afternoon was an ideal time for Sam and Tom to share the Valentine's love and revisit CCMI2 for an inspection and clean.

There has been an important development over the past few weeks. Last time we visited the buoy,  the West-facing support buoy was closer to the surface than the East-facing support buoy. We assumed that was caused by the NE winds that day, giving the West-facing lines and chains more slack. However, over the past few weeks, we have noticed that the West-facing support buoy was partially emerged out of the water. Friday's visit was during a period of W winds, and the buoy was still sitting at the surface. This now confirms to us that the East-facing line, particularly the black elastic line that attracts gooseneck barnacles between the main buoy and the support buoy, has been ultimately stretched as a result of the predominantly NE winter winds. A decision regarding the need for replacing the line has not yet been made.

On the bright side, there were many fewer gooseneck barnacles to be plucked. The cleaning of the lines and structure dealt with mostly the same reddish-brown turf algae as before, but in slightly thicker abundance due to the warming weather. With low currents, the barnacles and clumps of algae that were sent slowly drifting down to the bottom quickly surrounded us in a murky cloud and caught the attention of many hungry queen triggerfish and black durgons.

All connection points and chains showed no signs of damage or change, with the exception of the duct tape surrounding the loop of rubber sheath on the lines near the anchor points. In this photo from early January, you can see that the duct tape has started to come off. Now, nearly two months later, it's almost entirely unravelled, but we'll continue to monitor it for any effect it may have on the splice.

Storm Damage Check & Clean-up

Today, a team of three divers visited the CCMI2 buoy during a short window of calm seas. Since January 16th, sustained winds have been predominantly from the north and in excess of 8-10 mps (15-19kts) with gusts up to 12mps (23kts), giving the CCMI2 plenty of pitch and roll. All sensors have been continually logging and transmitting, making the main focus of the dive to check for any structural damage and give the entire subsurface structure a good cleaning.

Tom and I began work cleaning the attachment lines, as algae and barnacles had been noted on the January 2nd inspection. The lines from the sea floor to subsurface attachment buoys were easy to clean and mostly only covered in a light layer of reddish-brown turf algae. The lines from the subsurface attachment buoys to the chains (which then attach to the buoy substructure) are partially made of elasticized rope, and appear to be preferred by gooseneck barnacles. Approx. 60 barnacles as well as more turf algae were removed from the lines.

Jon inspected the attachments, shackles and chains on the seafloor, noting that everything was in sound condition. The abrasion on the chain between the two attachment points seen on the Jan 2nd dive has not worsened.

I was then able to scrub the outsides of the metal substructure and clean the subBIC arm of more algae, removing a handful of gooseneck barnacles along the way. The eye of the subBIC was much less biofouled than on Jan 2nd, indicating that a 2-3 week cleaning frequency is likely optimum. Sacrificial anodes still have adequate material and the conductivity cell on the CTD was gently cleaned although relatively unaffected by algae.

All cables and connections topside appeared in place and unaffected. The solar panels are still clean and all MET instruments were operational upon visual inspection.

Until mid-February! See photos below.

Tom cleaning the main rope - scrubbing makes the rope go back to bright white. 

Sam picking barnacles off the black elasticized rope after being scrubbed for algae.

Sam cleaning the subBIC arm and metal structure of the buoy.