AFQ - Qualification Test [MQTT]

cyberthor wrote on July 03, 2019:

I am running AFQ MQTT test. When I unplug the network cable from my network switch, all the test items would also pass.

This is strange… I thought MQTT test is supposed to test for actual connection with the IoT Cloud? Why are all the test past even without network connection?

SarenaAtAws wrote on July 03, 2019:

Exactly which MQTT tests are you running and having concerns about?

In the latest release we have a few MQTT tests that can be run.

We have the following tests that DO NOT need the network:
MQTT_Unit_Validate
MQTT_Unit_Subscription
MQTT_Unit_Receive
MQTT_Unit_API
MQTT_Unit_Metrics

All other tests DO need the network:
MQTT_System
Full_MQTT_Agent_Stress_Tests
Full_MQTT_Agent
Full_MQTT_Agent_ALPN

cyberthor wrote on July 04, 2019:

Dear SarenaAtAWS,

I am using AFR_v1.4.8, and running AFQ for Renesas, following developer guide v1.1.6

The MQTT test I am running is: FULL_MQTT (by setting testrunnerFULL_MQTT_ENABLED to 1)

Please advise. Thank you

SarenaAtAws wrote on July 06, 2019:

It is advised that you use the latest version of Amazon FreeRTOS for qualification. You can find instructions: https://docs.aws.amazon.com/freertos/latest/portingguide/afr-porting.html.

In regards to version 1.4.8, the tests under Full_MQTT do not connect to AWS IoT. These tests verify functionality of library code with some mocked data. These tests have been deleted in the latest version.
If you would like to test your connect to AWS IoT, you must use:
testrunnerFULL_MQTT_AGENT_ENABLED
testrunnerFULL_MQTT_ALPN_ENABLED
testrunnerFULL_MQTT_STRESS_TEST_ENABLED