Data isn't showing up in the channel

I have used the following code in Raspi 4. When I run the code it displays field1 and field2 on the screen. However my channel is not receiving any data.

Writing Payload = field1=6.6&field2=17.2 to host: mqtt3.thingspeak.com clientID= xxxxxxxxxxxxxxxxxxxx User xxxxxxxxx PWD xxxxxxxxxxxxxxxx

import paho.mqtt.publish as publish
import psutil
import string

# The ThingSpeak Channel ID.
# Replace <YOUR-CHANNEL-ID> with your channel ID.
channel_ID = "xxxxxxxx"

# The hostname of the ThingSpeak MQTT broker.
mqtt_host = "mqtt3.thingspeak.com"

# Your MQTT credentials for the device
mqtt_client_ID = "xxxxxxxxxxxxxxxx"
mqtt_username  = "xxxxxxxxxxxxx"
mqtt_password  = "xxxxxxxxxxxxx"

t_transport = "websockets"
t_port = 80

# Create the topic string.
topic = "channels/" + channel_ID + "/publish"

while (True):
    
    # get the system performance data over 20 seconds.
    cpu_percent = psutil.cpu_percent(interval=20)
    ram_percent = psutil.virtual_memory().percent
    
    # build the payload string.
    payload = "field1=" + str(cpu_percent) + "&field2=" + str(ram_percent)
    
    # attempt to publish this data to the topic.
    try:
        print ("Writing Payload = ", payload," to host: ", mqtt_host, " clientID= ", mqtt_client_ID, " User ", mqtt_username, " PWD ", mqtt_password)
        publish.single(topic, payload, hostname=mqtt_host, transport=t_transport, port=t_port, client_id=mqtt_client_ID, auth={'username':mqtt_username,'password':mqtt_password})
        except (keyboardInterrupt):
        break
        except Exception as e:
        print (e) 

[Christopher Stapels]

Have you created an MQTT device? If yes, I suggest trying the desktop client as practice. If you have MATLAB you can use that, or there is an example for MQTTx in the doc. Once you get that working, then you can be sure the device is set up correctly and then i would look into the python code further.

[Manira]

I tried the desktop client and it's working fine. Would you please look at my code now?

[Christopher Stapels]

Your code looks fine to me. Things I would try:

1. Publish a fixed number to see if your sensors are failing.

 payload = "field1=" + str(cpu_percent) + "&field2=" + str(ram_percent)
 # change to 
  payload = "field1=10"
  

2. See if you can publish using the REST API, to test the wifi connection for your device.

3. Write everything out to the command line in your python program to make sure it is executng as expected.

4. final thing to try is to make sure the space for the MQTT command buffer is large enough, This used to be an arduino problem, I dont think its ever been an Rpi problem.

[Manira]

1. Tried the fixed number, the program works fine and prints the fixed number. However, it doesn't update the channel fields.
2. My Rpi is connected to the WiFi as I am accessing my Rpi remotely through VNC viewer.
3. My program is executing just fine.
4. I am not sure how to check the MQTT command buffer. Any help would be appreciated

Answer 1

[Manira]

Christopher, your code is just fine. It's working fine for me now. Channel fields are getting updated accordingly.

Answer 2

[Christopher Stapels]

Can you try this code on you pi for the REST API? Do you get a value in your channel?

import requests
import threading

event=threading.Event()
writeKey='YOUR WRITE API KEY'

while True:

    url='https://api.thingspeak.com/update?api_key=%s&field1=10&status=%s'% (writeKey,'success!')
    try:
        y=requests.get(url)
        print("wrote" + url)
        y.raise_for_status()
    except requests.exceptions.HTTPError as errh:
        print ("Http Error:",errh)
    except requests.exceptions.ConnectionError as errc:
        print ("Error Connecting:",errc)
    except requests.exceptions.Timeout as errt:
        print ("Timeout Error:",errt)
    except requests.exceptions.RequestException as err:
        print ("OOps: Something Else",err)
    #pause for a while

    event.wait(16)

[Manira]

Yes, it's updating my channel.

[Christopher Stapels]

Ill have to retest the MQTT code above. There may be library differences, or a need to change pthon versions.

[Manira]

Please keep me posted. I just wanted to let you know that with the above MQTT code last night my channel got connected for couple of minutes, got around 200 entries. After a while the filed chart got stuck although the entries were updating. Finally everything went silent again. Really interesting!

[Christopher Stapels]

That may provide a hint. As a free user, your update rate is limited to once every 15 seconds per channel. if you were able to generate 200 entries in a couple of minutes, perhaps you are posting or attempting to post faster than the allowed limit. You might want to try adding additional delay to the MQTT code. For example from the second code I gave:

event.wait(16)

Im glad to hear you got it connected! That seems to show that it hasnt anything to do with the python version or the library. Generally sporadic updates are due to rate limiting or wifi connectivity. Perhaps you could use the REST code to do a long term study on the reliability of the connection.

[Manira]

You do have a very good point. I will add the delay and post the updates.

[Manira]

With the above mentioned MQTT code if I change the payload like

payload = "field2=20" and add a dealy of 30s everything goes fne and channel gets updated.

However, if I change the payload to

payload = "field1=" + str(cpu_percent) + "&field2=" + str(ram_percent)

with same delay, entries in the channel field get updated, nothing shows up in field chart though.

[Christopher Stapels]

Great! Use the Read API or thingSpeakRead to se the data in you channel. If it doesnt show up on the plot, ThingSpeak is probably interpreting your post as string data. You can see the read API format on the API keys tab of your channel view. Also you can click export recent data on the private view of your channel. I suspect you will see special characters or other things that confuse thingSpeak and we can repair the issue.