Channel status updates "created_at" field spuriously showing a time in the future

Hi,

I'm querying status updates of a channel as part of my workflow. Each of these status updates comes with a "created_at" timestamp, so when I query for the last one I expect to get the latest.

However, twice during March I got an status update with a "created_at" timestamp several weeks in the future, which completely messes my workflow. Because that same message always get returned as the latest for several weeks.

I could code a workaround for this, but I would rather understand why this happen and how to avoid it.

Attached is a CSV export from that channel, and you can see how all dates make sense and suddenly, the entry_id 502 (row 503), reports being created on March 29 2023, 20 days ahead of when it was actually created.

Some background to my question:

The reason why the status updates are so important in my workflow, is because I'm sending data in binary format (bitpacked) to ThingSpeak from a satellite modem (RockBlock), were a React scripts decode it and send it to another Channel for visualization.

One of my encoded messages looks like "6ae5c7c895ec7803abc602c0ffffffffff07a8"

To get this encoded messages, I read the status updates of the channel. Due to the way the satellite system works, in the status update I can find the IMEI of the modem, a message ID and the encoded message, therefore one status entry at the status updates feed of the channel looks like this:

300434066310190,1121,6ae5c7c895ec7803abc602c0ffffffffff07a8

So I use these status updates to retreive the binary messages.

[Christopher Stapels]

Can you show the way you are updating the channel? Where does the data come from to get into the channel?

[Camilo Rada]

The data comes from the Rock7 service, they make the satellite modems and provide the satellite data service. They offer a data delivery option to ThingSpeak here is how the configuration look

Anyway, the "created_at" timestamp is data that comes with the data? Or it is created by ThingSpeak every time the status of a channel is updated?

[Christopher Stapels]

In our API secificaation for the Write data endpoint there is a created_at timestamp that you can provide or if you do not provide one, ThingSpeak will make one for you at the time of ingest. I see that is the address and Im glad you blocked out the API key, but I need to see the rest of the details of the POST to understand.

According to this document on Rock7, you provide the datetime details. I suspect the formula you are using for datetime has an error or the clock for the Rock7 is sending bad data that is shifting the date. Perhaps your data packing method is overfilling the Rock7 data buffer? ThingSpeak fields can carry 255 characters so it does not seem that you are overfilling that. The ThingSpeak server clock is very steady in practice. We have so many updates (billions and billions) that a clock error would be very apparent, but it has not been observed.

[Camilo Rada]

OK,

The post request is just that, I add no data I'm only providing the key.

As my data doesn't have the format expected:

datetime,field1,field2,field3,field4,field5,field6,field7,field8,latitude,longitude,elevation

So, maybe Rock7 is sending my binary data anyway, despite it is a binary message and not a text message (they are transmitted with different commands) and then ThingSpeak is trying to make sense of it as a datetime.

[Camilo Rada]

But in that case I don't undestand why only two out of 30,000+ messages turn out with a wrong timestamp, that both times happened to be a few weeks in the future.

Is there a filter on ThingSpeak that would reject timestamps in the past or too far in the future?

[Christopher Stapels]

A few things I would try if you have the bandwidth.

Set up another feed with the same data where you do explicitly set the datetime in Rock7, and see if that gives any errors.

Set up another feed with fixed data, like a single simple character. Your special characters might be getting interpreted unexpectedly somewhere that overflows a buffer, but ony when certain things are true about the date time (for example if the datetime does not have a fixed width)

Since we allow people to use times in the future and in the past, ThingSpeak does not see these dates as incorrect data and there is no built in filter. However, you can easily build your own filter in MATLAB analysis and use TimeControl to run the code regularly to write the filtered data into a new channel. I do this for the traffic monitor channel. It writes every 15 seconds, but I read the data every day and then calculate and write the houlry average into another channel so I can view the trend easier.