ESP8266 connection issues but

I have a few ESP8266 running various sensor tasks, and sending GET or POST requests to Thingspeak via api.thingspeak.com. They all connect to the household router.

Recently, I became aware of a problem with the devices occasionally timing out when trying to communicate with Thingspeak. For example, here's a bit of code from a basic water level sensor:

String url = "http://api.thingspeak.com/update?api_key=xxxxxxxxxxxxx&field1=";
url += getdistance();
url += "&field8=";
url += batt;
http.begin(client, url);
int httpcode = http.GET();
Serial.println(httpcode);
String payload = http.getString();
Serial.println(payload);
http.end();

Fairly often, the connection will fail with a -1 httpcode. Sometimes it will run for weeks with no issues and then suddenly stop connecting. If I type the URL in a browser, it will always work immediately. If I connect the ESP to a mobile hotspot, it will also connect immediately with no issues. I can't find any issues at all with the router, and I've fully reset it etc. I've also confirmed the wifi connection between ESP and router is not the problem.

Sometimes one ESP will be running fine, and another will be having connection issues. The next day, the first one will have issues and the other will be fine.

So I added a bit of code to see if I could time the connection:

server = "api.thingspeak.com"
timer= millis();
while(client.connect(server,80)){
    delay(10)}
    t=millis()-timer;
    Serial.print(t);

The devices now connect every time. But the connection can take up to 180 seconds. Some ESP's updating to older channels are much more reliable and connect within a second or two; it seems as though the more recently created channels take longer to connect to.

I'm reaching out here because I can't really think of where to begin trying to fix this. Why does a browser succeed immediately updating a channel with the api.thingspeak url but the ESP takes up to 180 seconds? Why do I not have any problems connecting the ESP via a mobile hotspot? Why does the same device work for weeks then suddenly play up?

I think I've isolated the problem to either the router or the thingspeak api, and I can't find anything wrong with the router.

Might be worth noting that I have an ESP in a heat pump which updates every 15 seconds to thingspeak by MQTT and it hasn't skipped a beat in months so maybe I should upgrade them all to MQTT.

Is the api just that flakey? I've seen other posters here with the same issue but it never seems to get resolved.

Answer 1

[Christopher Stapels]

I've seen ESP8266 with long connection times of 10-15 s, but nothing quite like what you see. I know you have the MQTT devices as a comparison, but everything else seems to point to wifi strength. Have you moved the devices or tried using an external antenna?

I have some devices in my yard, about 200' from the house or more that connect to the home wifi sporadically due to the large distance. There is definitely a temperature and humidity effect.

The API is definitely not flakey though. I have tens of other devices that update over wifi regularly for years at a time with periods of seconds, hours, and days.. We also monitor the number of requests over time, and would most likety be able to observe if there was a sudden connection issue casued by ThingSpeak, since thousands of clients would not be able to connect simultaneously.

I appreciate the work you have dont to study the problem thus far. Please let us know if you find a solution.

[Perky Whale]

Thanks for the answer, after some more experimenting I think it's a combination of the router and specific ESP devices. The devices that have the problem are the (Chinese) 'D1 Mini Pro', with the ceramic "Rainsun" antenna. They have no issue connecting to wifi but will take forever connecting to the API. Connection to the API will happen immediately if I use any other hotspot - even with signal strength down to -73 dBm. But regardless of signal strength (even as good as -38 dBm), the main router will always present long API connection times. But I can replace the ESP with a different one and run the same sketch connecting to the same router from the same place and it will connect to the API immediately. It's just quite bizarre, for some reason it seems that the router is happily accepting wifi connections but somehow delaying specific ESP devices from connecting to the API. I don't even know where to begin to look to figure out why that's happening.

[Christopher Stapels]

Definitely appreciate you sharing what you learned. Its interesting, I wonder if there is something in the DNS lookup firmware for the ESP library that is causing the delay. Are you using Arduino IDE to program? Perhaps you could upgrade or downgrade the device libraries and see if that makes a difference.

[Perky Whale]

Let me add another bit of info which I think may be important. I've noticed that the API connection times will be variable (and long) if the ESP has recently been connected to a power supply, either directly to the PC or by charging the battery via TP4056 board. If I unplug and leave it alone just on battery power for 20 minutes, often the connection time will drop substantially and it will stabilise out. I wonder if there is a floating pin which needs to be pulled high or low that might be interfering with the connection somehow. Can you think of any pins on the 8266 that might be a culprit here?