On November 7th, 2014, I noticed that the CCMI2 buoy's two dataloggers were running on different time zones. The "met" logger continued to (correctly) operate in UTC but the "main" logger was running five hours behind, which coincided with local time at the Little Cayman Research Centre and here at AOML in Miami. Regarding the only other servers known to connect to these loggers, those of YSI, I'm not sure where they are located or what time zone they run in.
The reset happened on Wednesday, November 5th, at some time between 2:20pm and 2:30pm local Cayman/Miami time.
The most obvious explanation for this problem would be if someone had connected to the main datalogger from a computer running in this -0500 time zone whose local loggernet settings were configured to automatically "correct" the logger clock if it were found to be out of sync with the server. This is not a default setting on loggernet but neither is it an uncommon setting for people to choose.
However, we were unable to identify any computer that might have reset this datalogger's clock. Consider:
- Laura Wright reported, on November 12th, that their land-based servers were completely unable to connect to the station in any way: "The antennae cable/connector end is literally broken. I'm waiting on parts to fix. There is only one computer on station that connects to the buoy via the antennae cable which is currently not connecting due to the cable issue and was the only computer connecting since July 14, 2014 when Rob transferred everything over before that previous computer died."
- The AOML loggernet server runs exclusively in UTC and anyway its settings were verified as not auto-correcting any loggers clocks even if they were found to be out of sync.
- Jon Fajans, on November 8th, suggested the possibility that the main datalogger might have reset its clock based on a time signal provided by the modem, since the modem likely "knows" what local Cayman time is. However there is no known mechanism for the datalogger and modem to trade time information in this way. The one bit of clock-sync code that used to exist, that which every hour reset the "met" logger clock using the "main" logger time, was in fact deleted in the last reprogramming because of problems it had caused at one of the Tobago sites.
- On November 12th, Matt Previte of YSI reported: "Verified that our Loggernet is not set to sync clock for Cayman." He suggested asking CCCCC while saying that he didn't recall setting up the logger to call out to any servers but AOML's and YSI's. I checked the programming and confirmed that there were no other callouts except for AOML and YSI, and did not pursue the matter further with CCCCC.
The data from November 5th - 12th are all archived but all of their timestamps are wrong so they have not been loaded or integrated in any way. Eventually I will write a simple script to correct the timestamps on those records so that this hole in the dataset can be patched.
Just to reiterate, we still have no theory about how the main logger's clock could have been changed on November 5th.
(signed)
Mike Jankulak
