ESP32 Greengrass

system · December 12, 2018, 5:53pm

FreeOurToes wrote on December 12, 2018:

Hi,

I have problems connecting to the greengrass device.

I have functional greengrass core with multiple devices on it and successfully sending data between each other.

I wanted to connect esp32 to it.
My steps were: Created new thing inside greengrass group by adding device. Used 1 click configurations etc. I downloaded certificates. Used CertificateConfigurator.html to create aws_clientcredential_keys.h . Setup endpoint and wifi settings. All setup was done using a cloned freertos repo.

I redeployed greengrass core which included new esp32 device thing.

I successfully, did MQTTEchoDemo. Device was publishing to the cloud.

But when I comment out mqttechodemo and un-commented greengrass discovery demo. It fails to connect.
Did I missed any steps? I am not sure where things are going wrong.
Thanks

This is the output from serial after WiFi connection established.


3 303 [main] WiFi Connected to AP. Creating tasks which use network...
4 304 [main] Write code signing certificate...
I (3140) PKCS11: Initializing SPIFFS
I (3150) PKCS11: Partition size: total: 52961, used: 2761
5 306 [main] Write device private key...
I (3270) PKCS11: Initializing SPIFFS
6 323 [main] Key provisioning done...
7 323 [IoT_GGD] Attempting automated selection of Greengrass device
I (3460) PKCS11: Initializing SPIFFS
I (3460) PKCS11: Initializing SPIFFS
I (3560) PKCS11: Initializing SPIFFS
I (3560) PKCS11: Initializing SPIFFS
I (3570) PKCS11: Initializing SPIFFS
I (4340) wifi: pm start, type:0

I (5440) PKCS11: Initializing SPIFFS
8 1248 [IoT_GGD] About to close socket.
9 1250 [IoT_GGD] Socket closed.
10 1250 [IoT_GGD] Stack high watermark for discovery helper task: 2620.
11 1750 [IoT_GGD] About to close socket.
12 1750 [IoT_GGD] Socket closed.
13 1750 [IoT_GGD] Stack high watermark for discovery helper task: 2092.
14 2254 [IoT_GGD] About to close socket.
15 2254 [IoT_GGD] Socket closed.
16 2254 [IoT_GGD] Stack high watermark for discovery helper task: 2092.
17 2761 [IoT_GGD] About to close socket.
18 2761 [IoT_GGD] Socket closed.
19 2761 [IoT_GGD] Stack high watermark for discovery helper task: 2092.
20 2761 [IoT_GGD] GGD - Can't connect to greengrass Core
21 2761 [IoT_GGD] Auto-connect: Failed to retrieve Greengrass address and certificate.
22 2761 [IoT_GGD] ----Demo finished----

system · December 12, 2018, 5:58pm

FreeOurToes wrote on December 12, 2018:

Or slightly different output but still no success


3 303 [main] WiFi Connected to AP. Creating tasks which use network...
4 304 [main] Write code signing certificate...
I (3140) PKCS11: Initializing SPIFFS
I (3150) PKCS11: Partition size: total: 52961, used: 2761
5 311 [main] Write device private key...
I (3320) PKCS11: Initializing SPIFFS
6 323 [main] Key provisioning done...
7 323 [IoT_GGD] Attempting automated selection of Greengrass device
I (3410) PKCS11: Initializing SPIFFS
I (3420) PKCS11: Initializing SPIFFS
I (3520) PKCS11: Initializing SPIFFS
I (3520) PKCS11: Initializing SPIFFS
I (3520) PKCS11: Initializing SPIFFS
I (4350) wifi: pm start, type:0

I (5360) PKCS11: Initializing SPIFFS
8 1227 [IoT_GGD] About to close socket.
9 1273 [IoT_GGD] Socket closed.
10 1273 [IoT_GGD] Stack high watermark for discovery helper task: 2668.
I (13350) PKCS11: Initializing SPIFFS
I (13350) PKCS11: Initializing SPIFFS
I (13460) PKCS11: Initializing SPIFFS
I (13460) PKCS11: Initializing SPIFFS
I (13460) PKCS11: Initializing SPIFFS
I (14360) PKCS11: Initializing SPIFFS
11 1598 [IoT_GGD] About to close socket.
12 1599 [IoT_GGD] Socket closed.
13 1599 [IoT_GGD] Stack high watermark for discovery helper task: 236.
14 1599 [IoT_GGD] Greengrass device discovered.
15 1599 [IoT_GGD] Establishing MQTT communication to Greengrass...
I (16110) PKCS11: Initializing SPIFFS
I (16110) PKCS11: Initializing SPIFFS
I (16220) PKCS11: Initializing SPIFFS
I (16220) PKCS11: Initializing SPIFFS
I (16220) PKCS11: Initializing SPIFFS
I (17050) PKCS11: Initializing SPIFFS
16 2318 [IoT_GGD] Disconnecting from broker.
17 2320 [IoT_GGD] Disconnected from the broker.
18 2320 [IoT_GGD] Deleted Client.
19 2320 [IoT_GGD] Heap low watermark: 103232. Stack high watermark: 236.
20 2320 [IoT_GGD] ----Demo finished----

system · December 13, 2018, 6:45pm

FreeOurToes wrote on December 13, 2018:

Thanks,

Policy attached to the thing’s certificate allows all actions on greengrass and IoT basics.

What should be subscription requirements for this demo?
I selected my thing as source, IoT cloud as target and filtering freertos/demos/ggd topic.

Greengrass core is online and deployments with subscription changes gets successfully deployed within seconds.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Publish",
        "iot:Subscribe",
        "iot:Connect",
        "iot:Receive",
        "iot:GetThingShadow"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "greengrass:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

wanggord · December 13, 2018, 6:01pm

gordon-AWS wrote on December 13, 2018:

Hello,

It looks like in the first log, you are unable to connect to any of the addresses returned by AWS IoT. Can you check that the GG core is online?

In the second log, it looks like you can connect, but get disconnected upon initiating an MQTT connection.
Can you check that GG subscriptions and your AWS IoT policy are configured correctly?

https://docs.aws.amazon.com/greengrass/latest/developerguide/config_subs.html
https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html

wanggord · December 13, 2018, 10:30pm

gordon-AWS wrote on December 13, 2018:

For subscriptions, can you try # (all topics), just to ensure that the subscriptions are not the issue.

A few other things to check on your core:
In your greengrass/ folder, is there a crash.log file? This means your core crashed some time ago.
In greengrass/configuration/group, make sure that group.json is correct. The ARN must match the group ARN on AWS IoT.
Also make sure that greengrass/configuration/config.json is correct.

An example config.json:


{
    "coreThing": {
        "caPath": "root-certificate.pem",
        "certPath": "xxxx-certificate.pem.crt.txt",
        "keyPath": "xxxx-private.pem.key",
        "thingArn": "YOUR THING ARN",
        "iotHost": "YOUR ENDPOINT HERE",
        "ggHost": "greengrass.iot.us-west-2.amazonaws.com"
    },
    "runtime": {
        "cgroup": {
            "useSystemd": "yes"
        }
    }
}

system · December 14, 2018, 4:33pm

FreeOurToes wrote on December 14, 2018:

Thanks,

So I couldn’t find any crash log, and I would like to believe that greengrass is functioning ok, because I have other devices connected to that publishing and subscribing works. And new greengrass deployments go down successfully. config.json seems to be good too.

The only other detail I could tell that Core connectivity was set to find endpoints automatically, and inside Core endpoints I have multiple IP addresses. But from debugging it seems that it finds the core.

for Subscription i did “#” for the topics. And deployed this update, rebooted esp32 and still same results.

config.json


{
  "coreThing" : {
    "caPath" : "root.ca.pem",
    "certPath" : "xxx.cert.pem",
    "keyPath" : "xxx.private.key",
    "thingArn" : "arn:aws:iot:us-east-1:xx",
    "iotHost" : "xxx.iot.us-east-1.amazonaws.com",
    "ggHost" : "greengrass-ats.iot.us-east-1.amazonaws.com",
    "keepAlive" : 600
  },
  "runtime" : {
    "cgroup" : {
      "useSystemd" : "yes"
    }
  },
  "managedRespawn" : false,
  "crypto" : {
    "principals" : {
      "SecretsManager" : {
        "privateKeyPath" : "file:///greengrass/certs/xxx.private.key"
      },
      "IoTCertificate" : {
        "privateKeyPath" : "file:///greengrass/certs/xxx.private.key",
        "certificatePath" : "file:///greengrass/certs/xxx.cert.pem"
      }
    },
    "caPath" : "file:///greengrass/certs/root.ca.pem"
  }
}

Still same results


2 306 [IP-task] vDHCPProcess: offer a0b0314ip
3 307 [main] WiFi Connected to AP. Creating tasks which use network...
4 308 [main] Write code signing certificate...
I (3181) PKCS11: Initializing SPIFFS
I (3181) PKCS11: Partition size: total: 52961, used: 2761
5 310 [main] Write device private key...
I (3301) PKCS11: Initializing SPIFFS
6 322 [main] Key provisioning done...
7 322 [IoT_GGD] Attempting automated selection of Greengrass device
I (3401) PKCS11: Initializing SPIFFS
I (3411) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (4381) wifi: pm start, type:0

I (5411) PKCS11: Initializing SPIFFS
8 1249 [IoT_GGD] About to close socket.
9 1251 [IoT_GGD] Socket closed.
10 1251 [IoT_GGD] Stack high watermark for discovery helper task: 2588.
I (13131) PKCS11: Initializing SPIFFS
I (13131) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (16441) PKCS11: Initializing SPIFFS
11 1816 [IoT_GGD] About to close socket.
12 1818 [IoT_GGD] Socket closed.
13 1818 [IoT_GGD] Stack high watermark for discovery helper task: 108.
14 1818 [IoT_GGD] Greengrass device discovered.
15 1818 [IoT_GGD] Establishing MQTT communication to Greengrass...
I (18391) PKCS11: Initializing SPIFFS
I (18391) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (19411) PKCS11: Initializing SPIFFS
16 2609 [IoT_GGD] Disconnecting from broker.
17 2611 [IoT_GGD] Disconnected from the broker.
18 2611 [IoT_GGD] Deleted Client.
19 2611 [IoT_GGD] Heap low watermark: 103232. Stack high watermark: 108.
20 2611 [IoT_GGD] ----Demo finished----

system · December 14, 2018, 5:36pm

FreeOurToes wrote on December 14, 2018:

So,
I think I discovered some ridiculous stuff. I enabled
#define mqttconfigENABLE_DEBUG_LOGS 1
inside aws_mqtt_config.h and demo was succesful, I published 3 messages to the cloud via greengrass. But the moment I remove debug log it goes back again to failing. And most of the time it’s the same but once every 3rd try I get errors like


GGD - Can't connect to greengrass Core
21 2834 [IoT_GGD] Auto-connect: Failed to retrieve Greengrass address and certificate.

or


Greengrass device discovered.
15 1964 [IoT_GGD] Establishing MQTT communication to Greengrass...
16 2465 [IoT_GGD] ERROR: Could not connect to the Broker.
17 2465 [IoT_GGD] Heap low watermark: 103180. Stack high watermark: 212.
18 2465 [IoT_GGD] ----Demo finished----


8 1224 [IoT_GGD] ERROR: Handshake failed with error code -29312 
9 1224 [IoT_GGD] About to close socket.
10 1224 [IoT_GGD] Socket closed.
11 1224 [IoT_GGD] Stack high watermark for discovery helper task: 2820.
12 1224 [IoT_GGD] JSON request could not connect to end point
13 1224 [IoT_GGD] Auto-connect: Failed to retrieve Greengrass address and certificate.
14 1224 [IoT_GGD] ----Demo finished-----

My conclusion was that with debugging enabled it was success most of the time. but but with debugging disabled it was 0% success, maybe it’s all just a coincidence.

I write firmware with “make erase_flash all flash monitor -j9” command

Sometime it just fails to connect to the AP. I would like to write off most of these errors to bad wifi. We are using CloudTrax to mesh wifi around the office, but it’s horrible and we planning to replace it.
I did not change anything else in the code but the enable metrics or debugging and just reset device like a million times.

Now even with debugging enable I just get.


19 2888 [IoT_GGD] Stack high watermark for discovery helper task: 2088.
20 2888 [IoT_GGD] GGD - Can't connect to greengrass Core
21 2888 [IoT_GGD] Auto-connect: Failed to retrieve Greengrass address and certificate.
22 2888 [IoT_GGD] ----Demo finished----

Could #define mqttconfigENABLE_METRICS ( 1 )
or
#define mqttconfigENABLE_DEBUG_LOGS 0
be affecting this or it was just a coincidence?
Could all this be just our wifi problems?

No matter what I do I can’t get anything stable.

system · December 18, 2018, 4:23pm

FreeOurToes wrote on December 18, 2018:

I have tried it on a different network and it connected the first time and since then all other connection attempts were a fail. Failed to retrieve certificates from greengrass.

system · December 18, 2018, 9:16pm

SarenaAtAws wrote on December 18, 2018:

Hello FreeOurToes,

Your original output:


2 306 [IP-task] vDHCPProcess: offer a0b0314ip
3 307 [main] WiFi Connected to AP. Creating tasks which use network...
4 308 [main] Write code signing certificate...
I (3181) PKCS11: Initializing SPIFFS
I (3181) PKCS11: Partition size: total: 52961, used: 2761
5 310 [main] Write device private key...
I (3301) PKCS11: Initializing SPIFFS
6 322 [main] Key provisioning done...
7 322 [IoT_GGD] Attempting automated selection of Greengrass device
I (3401) PKCS11: Initializing SPIFFS
I (3411) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (3511) PKCS11: Initializing SPIFFS
I (4381) wifi: pm start, type:0
 
I (5411) PKCS11: Initializing SPIFFS
8 1249 [IoT_GGD] About to close socket.
9 1251 [IoT_GGD] Socket closed.
10 1251 [IoT_GGD] Stack high watermark for discovery helper task: 2588.
I (13131) PKCS11: Initializing SPIFFS
I (13131) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (13241) PKCS11: Initializing SPIFFS
I (16441) PKCS11: Initializing SPIFFS
11 1816 [IoT_GGD] About to close socket.
12 1818 [IoT_GGD] Socket closed.
13 1818 [IoT_GGD] Stack high watermark for discovery helper task: 108.
14 1818 [IoT_GGD] Greengrass device discovered.
15 1818 [IoT_GGD] Establishing MQTT communication to Greengrass...
I (18391) PKCS11: Initializing SPIFFS
I (18391) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (18501) PKCS11: Initializing SPIFFS
I (19411) PKCS11: Initializing SPIFFS
16 2609 [IoT_GGD] Disconnecting from broker.
17 2611 [IoT_GGD] Disconnected from the broker.
18 2611 [IoT_GGD] Deleted Client.
19 2611 [IoT_GGD] Heap low watermark: 103232. Stack high watermark: 108.
20 2611 [IoT_GGD] ----Demo finished----

Was a successful demo. When mqttconfigENABLE_DEBUG_LOGS is set to 0, the MQTT publishes are silent on success for the Greengrass demo. The Greengrass demo will print a message only if the MQTT client fails to publish.

system · January 4, 2019, 7:55pm

FreeOurToes wrote on January 04, 2019:

Thank you!