Introduction

Similar to the first demo, this example uses the MQTT protocol to receive notification of a pending OTA Update Job and to download the firmware image. This demo builds on top of the first one by using code signing to verify that the firmware image has not been tampered with and comes from a trusted sender.

See the third demo for an example that receives job documents over HTTP with code signing.

Notice: We recommend that you digitally sign your firmware so that the devices that receive and run it can verify that the firmware came from an authorized source and that the firmware hasn’t been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.

Instructions

These instructions show how to:

These steps require some introductory knowledge about over-the-air updates. Information on this topic can be found in the Introduction and Terminology sections. We also recommend that you perform the first demo before attempting to run this demo.

We will be using unique terminology while working with FreeRTOS Windows Simulator:

Setting up the Cloud Services

Get started with the AWS console

This demo application uses the AWS console to create an IoT thing, store a firmware image, and schedule an OTA job. An AWS account is required for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.

Prerequisites for using the OTA Update Manager Service

Create a Thing

An AWS IoT Thing, or simply “thing”, is the cloud representation of a device. By registering your device as a thing, you are able to store credentials for it, assign permissions to your device, and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.

Follow these steps to create an IoT thing, private key, and certificate for your device to register with AWS IoT. Take note of your thing name and AWS IoT endpoint, and download the certificate and key associated with your thing.

Your AWS IoT endpoint is displayed as the Endpoint. It should look like:

<account-specific-prefix>-ats.iot.region.amazonaws.com

Setting up the OTA Client

Source Code Organization

The demo project is called ota_code_signing.sln and can be found on Github in the following directory:

FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing

Configuring the Demo Project

After you download the source code, open the following project file in Visual Studio:

FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing/ota_code_signing.sln

All of the changes in the following steps should be done on this project.

Configure the AWS IoT Demo Profile

You must configure your AWS IoT credentials on the client side. The file aws_iot_demo_profile.h holds the information you need to connect your FreeRTOS project to AWS IoT, including endpoints, credentials, and keys set to the appropriate values. Follow these steps to set up the credentials using the Certificate Configurator:

1. In a browser window, open tools/aws_config_offline/CertificateConfigurator.html.
2. Under Thing Name, enter the Thing name you registered on AWS IoT.
3. Under AWS IoT Thing Endpoint, enter your AWS IoT endpoint.
4. Under Certificate PEM file, choose the <ID> -certificate.pem.crt that you downloaded from the AWS IoT console.
5. Under Private Key PEM file, choose the <ID> -private.pem.key that you downloaded from the AWS IoT console.
6. Choose Generate and save aws_iot_demo_profile.h, and then save the file in FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/include. This overwrites the existing file in the directory.

Application Version

For a client to accept an OTA update, the version number of the update it receives must be higher than the version of the firmware that it’s currently running.

The application version of the device software is set in the “aws_application_version.h” header file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default settings of these macros are 0, 9, and 2 respectively. These do not need to be changed for now, but they will be modified in a future step.

Code signature verification can be enabled or disabled using the “configOTA_ENABLE_CODE_SIGNATURE_VERIFICATION” macro that is located in the “aws_ota_agent_config.h” header file. For this demo, this has been enabled and does not need to be modified.

OTA Control Protocol

The OTA Control Protocol setting manages the protocol used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. For control operations, only MQTT is supported at this time.

This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “aws_ota_agent_config.h” header file. The default value for this macro is “OTA_CONTROL_OVER_MQTT” and does not need to be modified for this demo.

OTA Data Protocol

The OTA Data Protocol defines the format used to transfer data over-the-air to and from the device. Currently MQTT and HTTPS are supported.

This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the “aws_ota_agent_config.h” header file. The default setting for this macro is “OTA_DATA_OVER_MQTT” and does not need to be changed for this demo.

Configure the networking stack

This demo uses the FreeRTOS+ TCP/IP stack, so follow the instructions provided for the TCP/IP starter project. All of these steps should be performed in the OTA demo project, not the TCP/IP starter project referenced in the documentation. Completing the “prerequisites” section ensures that you:

Verify that the project builds and runs successfully

Before you continue, verify that you are able to build and run the project. You can do this by pressing F5 in the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”. The output should look like:

While the demo runs, it will continue to output the last line in the image. If you do not see this output, refer to the troubleshooting guide.

Note: The output will show that the device has received one job, even if you have not sent an OTA job yet. This is expected because the Client publishes to the same channel it is listening on. It can be ignored.

Prepare for creating the OTA Update Job

To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image, it needs to be a higher version than what is running on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow:

1. Set the application version number and build the project executable
2. Verify that the executable can build and run correctly
3. Upload the executable to S3
4. Set the application version number to something lower than what was used in step 1
5. Build the demo with the lowered version number
6. Run the updated demo and let it continue to run while waiting for an OTA Job

Note: An example of a production workflow would be:

1. Write functional firmware for the MCU
2. Integrate the OTA Client library into the project
3. Manually program the board with the initial firmware
4. Make changes to the software that are tested locally
5. Generate the binary for the new version of the firmware
6. Upload the new version to S3 and send it to the MCU with the an OTA Job

Set the application version

To simulate having a “new” firmware image, increment the version number. For this demo, update the following macro values that can be found in the “aws_application_version.h” as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 2

Build the “new” firmware image

Generate the firmware image by building the project. To do this, press “Ctrl+Shift+b” or navigate to the build tab and press build within the Visual Studio project. This will generate the following directory:

FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing/Debug

Because the demo runs on the FreeRTOS Windows Simulator, the “firmware image” in this case is the executable named “RTOSDemo.exe”.

Verify that the project builds and runs successfully

It’s best practice to verify firmware locally before you send it via an OTA update. See the “Verify that the project builds and runs successfully” section.

Upload the firmware image to the S3 bucket

1. Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
2. Click on the bucket created in the previous steps.
3. Click on the “Upload” button that is under the “Overview” tab.
4. Drag and drop “RTOSDemo.exe” to the bucket.
5. Click “Upload” to add the executable to the bucket.

Lower the application version

Lower the application version. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 1

Build and run the OTA client

Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.

Creating the OTA Update Job using the AWS IoT Console

At this point, you should have:

• Created an AWS IoT Thing with the AWS IoT Service.
• Setup the S3 bucket and managed permissions for the various services.
• Uploaded a “newer” firmware image to the S3 bucket.
• Completed the setup required for code signing.
• Configured the OTA client running on your device.

Create the OTA Update Job

With the OTA Client running and the cloud services set up, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.

1. In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.

   
   

2. Under Create a FreeRTOS OTA update job, choose Create OTA update job.

   
   

3. You can deploy an OTA update to a single device or a group of devices. Under Select devices to update, choose Select. Choose the Things tab to update a single device. Select the check box next to the IoT thing associated with your device and press Next to continue.

   
   

4. Under “Select the protocol for firmware image transfer” , choose MQTT

   
   

5. Under the “Select and sign your firmware image” section, leave the default option of “Sign a new firmware image for me” selected. Under the “Code signing profile” section, press the “Create button”. 

   
   

6. In the “Profile name” section, enter “winsim_codesigning”. In the “Device hardware platform” section, select “Windows Simulator”. Under the “Code signing certificate” section select import and select the certificate and certificate private keys that you generated earlier. If you were using the suggested command, they will be called “ecdsasigner.crt” and “ecdsasigner.key”. After choosing these, select import. Write the path to the “ecdsasigner.crt” certificate that was just imported into the section called “Pathname of code signing certificate on device”. Then press “Create” to make the code signing profile. After doing these steps, it should look like the following image:

   
   

7. Press select in the “Select your firmware image in S3 or upload it” section and navigate to the executable uploaded during the previous steps. Type “./NewRTOSDemo.exe” in the “Pathname of firmware image on device” section. This path is where the file downloaded during the OTA update will be saved. Select the IAM role created for the OTA process and then press “Next” to continue.

   
   

8. Enter a unique job ID.

   
   

9. Leave the Job Type as the default option (snapshot) and click Create to finish creating the OTA Update Job.

   
   

10. You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted. 

    
    

Functionality

The example demonstrates a firmware update over-the-air using an OTA job with code signing. The demo creates a single task that connects to the MQTT broker, initializes the OTA Agent, prints OTA statistics, manages reconnects, and restarts if the number of publish failures reaches the limit. It also demonstrates how to register an application callback to the OTA Agent and handle OTA complete events. The structure of the demos is shown here:

void vOTAUpdateDemoTask( void * pvParameters )

{

    OTA_State_t eState;

    OTA_ConnectionContext_t xOTAConnectionCtx = { 0 };



    /* Remove compiler warnings about unused parameters. */

    ( void ) pvParameters;



    /* One time initialization of the libraries used by this demo. */

    prvInitialiseLibraries();



    configPRINTF( ( “OTA demo version %u.%u.%u\r\n”,

                    xAppFirmwareVersion.u.x.ucMajor,

                    xAppFirmwareVersion.u.x.ucMinor,

                    xAppFirmwareVersion.u.x.usBuild ) );

    configPRINTF( ( “Creating MQTT Client…\r\n” ) );



    /* Create the MQTT Client. */



    for( ; ; )

    {

        configPRINTF( ( “Connecting to broker…\r\n” ) );



        /* Establish a new MQTT connection. */

        if( _establishMqttConnection( true,

                                      &xMQTTConnection ) == IOT_MQTT_SUCCESS )

        {

            configPRINTF( ( “Connected to broker.\r\n” ) );

            xOTAConnectionCtx.pvControlClient = xMQTTConnection;

            xOTAConnectionCtx.pxNetworkInterface = ( void * ) IOT_NETWORK_INTERFACE_FREERTOS;

            xOTAConnectionCtx.pvNetworkCredentials = &xNetworkSecurityCredentials;



            /* Set the base interval for connection retry.*/

            _retryInterval = otaDemoCONN_RETRY_BASE_INTERVAL_SECONDS;



            /* Update the connection available flag.*/

            _networkConnected = true;



            /* Check if OTA Agent is suspended and resume.*/

            if( ( eState = OTA_GetAgentState() ) == eOTA_AgentState_Suspended )

            {

                OTA_Resume( &xOTAConnectionCtx );

            }



            /* Check if OTA Agent is suspended and resume.*/

            if( ( eState = OTA_GetAgentState() ) == eOTA_AgentState_Suspended )

            {

                OTA_Resume( &xOTAConnectionCtx );

            }



            /* Initialize the OTA Agent , if it is resuming the OTA statistics


             * will be cleared for new connection.*/

            OTA_AgentInit( ( void * ) ( &xOTAConnectionCtx ),

                           ( const uint8_t * ) ( awsiotdemoprofileCLIENT_IDENTIFIER ),

                           App_OTACompleteCallback,

                           ( TickType_t ) ~0 );



            while( ( ( eState = OTA_GetAgentState() ) != eOTA_AgentState_Stopped ) && _networkConnected )

            {

                /* Wait forever for OTA traffic but allow other tasks to run


                 * and output statistics only once per second. */

                IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );



                configPRINTF( ( “State: %s  Received: %u   Queued: %u   Processed: %u   Dropped: %u\r\n”,

                                pcStateStr[ eState ],

                                OTA_GetPacketsReceived(),

                                OTA_GetPacketsQueued(),

                                OTA_GetPacketsProcessed(),

                                OTA_GetPacketsDropped() ) );

            }



            /* Check if we got network disconnect callback and suspend OTA Agent.*/

            if( _networkConnected == false )

            {

                /* Suspend OTA agent.*/

                if( OTA_Suspend() == kOTA_Err_None )

                {

                    while( ( eState = OTA_GetAgentState() ) != eOTA_AgentState_Suspended )

                    {

                        /* Wait for OTA Agent to process the suspend event. */

                        IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );

                    }

                }

            }

            else

            {

                IotMqtt_Disconnect( xMQTTConnection, false );

            }

        }

        else

        {

            configPRINTF( ( “ERROR:  Failed to connect to MQTT broker.\r\n” ) );

        }



        /* After failure to connect or a disconnect, delay for retrying


         * connection. */

        _connectionRetryDelay();

    }

}

OTA update demo sample code.

Where _establishMqttConnection is defined as:

static IotMqttError_t _establishMqttConnection( bool awsIotMqttMode,

                                                IotMqttConnection_t * pMqttConnection )

{

    IotMqttError_t connectStatus = IOT_MQTT_STATUS_PENDING;



    /* Set the members of the connection info not set by the initializer. */

    memset( &xConnectInfo, 0, sizeof( xConnectInfo ) );

    xConnectInfo.awsIotMqttMode = awsIotMqttMode;

    xConnectInfo.cleanSession = true;

    xConnectInfo.keepAliveSeconds = otaDemoKEEPALIVE_SECONDS;

    xConnectInfo.clientIdentifierLength = ( uint16_t ) strlen( awsiotdemoprofileCLIENT_IDENTIFIER );

    xConnectInfo.pClientIdentifier = awsiotdemoprofileCLIENT_IDENTIFIER;



    /* Establish the MQTT connection. */

    configPRINTF( ( “MQTT demo client identifier is %.*s (length %hu).”,

                    xConnectInfo.clientIdentifierLength,

                    xConnectInfo.pClientIdentifier,

                    xConnectInfo.clientIdentifierLength ) );



    connectStatus = IotMqtt_Connect( &xNetworkInfo,

                                     &xConnectInfo,

                                     otaDemoCONN_TIMEOUT_MS,

                                     pMqttConnection );



    if( connectStatus != IOT_MQTT_SUCCESS )

    {

        configPRINTF( ( “MQTT CONNECT returned error %s.”,

                        IotMqtt_strerror( connectStatus ) ) );

    }



    return connectStatus;

}

_establishMqttConnection function in OTA sample code.

 

Initializing the OTA Agent

The OTA Agent is initialized by calling the OTA_AgentInit function which initializes the OTA context, registers the callbacks, resets the statistics counters and creates the OTA Agent Task. It returns the state of the OTA Agent- if initialization is successful within xTicksToWait the OTA Agent state is eOTA_AgentState_Ready otherwise it is set as eOTA_AgentState_Stopped.

 

OTA_State_t OTA_AgentInit( void * pvClient,

                           const uint8_t * pucThingName,

                           pxOTACompleteCallback_t xFunc,

                           TickType_t xTicksToWait )

OTA agent init function signature.

 

• pvclient is the messaging protocol client context (e.g. an MQTT context).
• pucThingName is a pointer to a C string holding the Thing name.
• xFunc is callback function for when an OTA job is complete.
• xTicksToWait The number of ticks to wait until the OTA Task signals that it is ready.

The console will show the following when the OTA Agent is successfully initialized:

Getting OTA Statistics

The OTA demo/application task can get OTA statistics on the number of messages received, dropped, processed and queued. The statistics are reset when OTA_AgentInit is called again after initialization. The functions that are available to get these statistics are –

• OTA_GetPacketsReceived: Return the number of packets received.
• OTA_GetPacketsProcessed: Return the number of packets processed.
• OTA_GetPacketsQueued: Return the number of packets queued.
• OTA_GetPacketsDropped: Return the number of packets dropped.

The demo outputs these statistics while it’s receiving a download. See the image below for an example.

Suspend on network disconnect and resume after reconnect

If the device disconnects from AWS IoT in the middle of downloading a firmware image, it will pause the download until it reconnects. Below is the callback registered through MQTT to handle the disconnect:

static void prvNetworkDisconnectCallback( void * param,

                                          IotMqttCallbackParam_t * mqttCallbackParams )

{

    ( void ) param;



    /* Log the reason for MQTT disconnect.*/

    switch( mqttCallbackParams->u.disconnectReason )

    {

        case IOT_MQTT_DISCONNECT_CALLED:

            configPRINTF( ( “Mqtt disconnected due to invoking diconnect function.\r\n” ) );

            break;



        case IOT_MQTT_BAD_PACKET_RECEIVED:

            configPRINTF( ( “Mqtt disconnected due to invalid packet received from the network.\r\n” ) );

            break;



        case IOT_MQTT_KEEP_ALIVE_TIMEOUT:

            configPRINTF( ( “Mqtt disconnected due to Keep-alive response not received.\r\n” ) );

            break;



        default:

            configPRINTF( ( “Mqtt disconnected due to unknown reason.” ) );

            break;

    }



    /* Clear the flag for network connection status.*/

    _networkConnected = false;

}

prvNetworkDisconnectCallback function in OTA sample code.

 

The flag _networkConnected will be set to false when the disconnect callback is invoked. Then OTA_suspend is called to tell the OTA agent to pause the download. Once the device reestablishes the connection to AWS IoT, OTA_resume is called tell the OTA agent to resume the download. Below is a snapshot of the device output showing the OTA agent suspends when the MQTT connection disconnects, then resumes the same update when the MQTT connection reconnects:

In this demo, the reconnect logic uses an exponential-backoff-and-jitter to prevent the MQTT server from getting overwhelmed with connection requests if many devices in a fleet get disconnected and then attempt to reconnect at the same time. The reconnection function is shown below.

static void _connectionRetryDelay( void )

{

    unsigned int retryIntervalwithJitter = 0;



    if( ( _retryInterval * 2 ) >= otaDemoCONN_RETRY_MAX_INTERVAL_SECONDS )

    {

        /* Retry interval is already max.*/

        _retryInterval = otaDemoCONN_RETRY_MAX_INTERVAL_SECONDS;

    }

    else

    {

        /* Double the retry interval time.*/

        _retryInterval *= 2;

    }



    /* Add random jitter upto current retry interval .*/

    retryIntervalwithJitter = _retryInterval + ( rand() % _retryInterval );



    configPRINTF( ( “Retrying network connection in %d Secs “, retryIntervalwithJitter ) );



    /* Delay for the calculated time interval .*/

    IotClock_SleepMs( retryIntervalwithJitter * 1000 );

}

_connectionRetryDelay function in OTA sample code.

 

OTA Complete Callbacks

The OTA complete callback is called when the OTA job is finished and receives the activate, self-test or failure events during the update process. The OTA agent has either completed the update job or determined that we’re in self test mode. If the image passed is successfully verified, the demo will try to activate the new image. This typically means we must reset the device to run the new firmware. In the case of the windows simulator, this means closing the application and running the new executable.

If the update was rejected, the demo will return without doing anything and will wait for another job. If it is reported that we should start test mode, the demo will perform user defined system checks to make sure the new firmware does the basic things it’s expected to do. For the purposes of this demo, the callback sets the image state as “accepted” without any verification steps. In general, the accept function will vary depending on your platform.

The callback is defined as :

static void App_OTACompleteCallback( OTA_JobEvent_t eEvent )

{

    OTA_Err_t xErr = kOTA_Err_Uninitialized;





    /* OTA job is completed. so delete the MQTT and network connection. */

    if( eEvent == eOTA_JobEvent_Activate )

    {

        configPRINTF( ( “Received eOTA_JobEvent_Activate callback from OTA Agent.\r\n” ) );

        IotMqtt_Disconnect( xMQTTConnection, 0 );

        OTA_ActivateNewImage();



        /* We should never get here as new image activation must reset the device.*/

        configPRINTF( ( “ERROR: New image activation failed.\r\n” ) );



        for( ; ; )

        {

            IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );

        }

    }

    else if( eEvent == eOTA_JobEvent_Fail )

    {

        configPRINTF( ( “Received eOTA_JobEvent_Fail callback from OTA Agent.\r\n” ) );

        /* Nothing special to do. The OTA agent handles it. */

    }

    else if( eEvent == eOTA_JobEvent_StartTest )

    {

        /* This demo just accepts the image since it was a good OTA update and networking


         * and services are all working (or we wouldn’t have made it this far). If this


         * were some custom device that wants to test other things before calling it OK,


         * this would be the place to kick off those tests before calling OTA_SetImageState()


         * with the final result of either accepted or rejected. */

        configPRINTF( ( “Received eOTA_JobEvent_StartTest callback from OTA Agent.\r\n” ) );

        xErr = OTA_SetImageState( eOTA_ImageState_Accepted );



        if( xErr != kOTA_Err_None )

        {

            OTA_LOG_L1( ” Error! Failed to set image state as accepted.\r\n” );

        }

    }

}

OTA complete callback in OTA demo

 

When the OTA file download is complete the console will output the following logs:

Activating the OTA Image

When the download is complete, stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when you created the job, then the new executable will be located in the same directory as the visual studio solution. For example:

Run that executable, and at the top of the console you should see that the version number has been updated. For example:

If the device has successfully rebooted, the status of the job in the IoT console will say completed.

Troubleshooting

The following sections contain information to help you troubleshoot issues with OTA updates.  

Topics

• Set Up Cloudwatch Logs for OTA Updates
• Log AWS IoT OTA API Calls with AWS CloudTrail
• Get OTA Failure Codes with the AWS CLI

Next Steps

This demo has shown how to securely send OTA Update jobs using the MQTT protocol. For an example of how to receive notifications over MQTT and then download the file with HTTPS, see the third demo. The API reference and porting guide sections describe how to integrate this library into a FreeRTOS project.

Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.