Quality RTOS & Embedded Software

LIBRARIES
NOTE: The OTA library and documentation are part of the FreeRTOS LTS Roadmap. These libraries are fully functional, but undergoing optimizations or refactoring to improve memory usage, modularity, documentation, demo usability, or test coverage. They are available on GitHub or part of the LTS Development Snapshot download
 

Basic OTA Demo (with Code Signing)

Introduction

Similar to the first demo, this example uses the MQTT protocol to be notified of a pending OTA Update Job and to download the firmware image. This demo builds on top of the first one by using code signing to verify that the firmware image has not been tampered with and comes from a trusted sender.

See the third demo for an example that receives job documents over HTTP with code signing.

Notice: We recommend that you digitally sign your firmware so that the devices that receive and run it can verify that the firmware came from an authorized source and that the firmware hasn’t been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.

Instructions

These instructions will explain the process of:
  • Setting up the cloud services to be able to store the firmware image and send it to the device
  • Configuring the device and running the OTA client for receiving the update
  • Preparing and creating the OTA Update Job for sending the firmware to the device
These steps require some introductory knowledge about over-the-air updates. Information on this topic can be found in the Introduction and Terminology sections. We also recommend that you perform the first demo before attempting to run this demo. We will be using unique terminology while working with FreeRTOS Windows Simulator:
  • “device”: The FreeRTOS Windows Simulator running in the Visual Studio project
  • “firmware image”: Executable that is the demo. When working with a physical MCU, this would typically be a binary image.

Setting up the Cloud Services

Get started with the AWS console

This demo application uses the AWS console to create an IoT thing, store a firmware image, and schedule an OTA job. An AWS account is used for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.

Prerequisites for using the OTA Update Manager Service

Create a Thing

An AWS IoT Thing, or simply thing, is the cloud representation of a device. By registering your device as a thing, you are able to store credentials for it, assign permissions to your device, and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.

Follow the steps in creating an IoT thing, private key, and certificate for your device to register an AWS IoT. Take note of your thing name and AWS IoT endpoint, and download the certificate and key associated with your thing.

Your AWS IoT endpoint is displayed as the Endpoint. It should look like:
<account-specific-prefix>-ats.iot.region.amazonaws.com

Setting up the OTA Client

Source Code Organization

The demo project is called ota_code_signing.sln and can be found on Github in the following directory:

FreeRTOS-Plus\Demo\FreeRTOS_IoT_Libraries\ota\ota_code_signing

Configuring the Demo Project

After downloading the source code, open the following project file in Visual Studio:

FreeRTOS-Plus\Demo\FreeRTOS_IoT_Libraries\ota\ota_code_signing\ota_code_signing.sln

All of the changes in the following steps are intended to be done on this project.

Configure the AWS IoT Demo Profile

You need to configure credentials for AWS IoT credentials on the client side. The file aws_iot_demo_profile.h holds the information you need to connect your FreeRTOS project to AWS IoT, including endpoints, credentials, and keys set to the appropriate values. Setup the credentials through the Certificate Configurator by following the steps below:

  1. In a browser window, open tools/aws_config_offline/CertificateConfigurator.html.
  2. Under Thing Name, enter the Thing name you registered on AWS IoT.
  3. Under AWS IoT Thing Endpoint, enter your AWS IoT endpoint.
  4. Under Certificate PEM file, choose the <ID> -certificate.pem.crt that you downloaded from the AWS IoT console.
  5. Under Private Key PEM file, choose the <ID> -private.pem.key that you downloaded from the AWS IoT console.
  6. Choose Generate and save aws_iot_demo_profile.h, and then save the file in FreeRTOS-Plus\Demo\FreeRTOS_IoT_Libraries\include. This overwrites the existing file in the directory.
Application Version

For a client to accept an OTA update, the version number of the update it’s receiving needs to be higher than the version of the firmware that it’s currently running.

The application version of the device software is set in the “aws_application_version.h” header file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default setting of these macros are respectively 0, 9, and 2. These do not need to be changed for now, but they will be modified in a future step.

Code Sign Verification

Code signature verification can be enabled or disabled using the “configOTA_ENABLE_CODE_SIGNATURE_VERIFICATION” macro that is located in the “aws_ota_agent_config.h” header file. For the purpose of this demo, this has been enabled and does not need to be modified.

OTA Control Protocol

The OTA Control Protocol setting manages what protocol is used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. Only MQTT is supported at this time for control operations.

This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “aws_ota_agent_config.h” header file. The default value for this macro is “OTA_CONTROL_OVER_MQTT” and does not need to be modified for this demo.

OTA Data Protocol

The OTA Data Protocol defines the format that data is transferred over-the-air to and from the device. Currently MQTT and HTTPS are supported for this setting.

This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the “aws_ota_agent_config.h” header file. The default setting for this macro is “OTA_DATA_OVER_MQTT” and does not need to be changed for this demo.

Configure the networking stack

This demo uses the FreeRTOS+ TCP/IP stack, so follow the instructions provided for the TCP/IP starter project. All of these steps should be performed in the OTA demo project, not the TCP/IP starter project referenced in the documentation. Completing the “prerequisites” section should ensure that you:

Verify that the project builds and runs successfully

Before continuing, verify that you are able to build and run the project. You can do this by pressing F5 within the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”. The output should look like:

While the demo is running, it will continue to output the last line in the image. If you are not seeing this output, refer to the troubleshooting guide.

Note: The output will show that the device has received one job, even if you have not sent an OTA job yet. This is expected due to the Client publishing to the same channel it is listening on and can be ignored.

Prepare for creating the OTA Update Job

To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image, it needs to be a higher version than what is running on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow:
  • Set the application version number and build the project executable
  • Verify that the executable can build and run correctly
  • Upload the executable to S3
  • Set the application version number to something lower than what was used in step 1
  • Build the demo with the lowered version number
  • Run the updated demo and let it continue to run while waiting for an OTA Job
Note: An example of a production workflow would be:
  • Write functional firmware for the MCU
  • Integrate the OTA Client library into the project
  • Manually program the board with the initial firmware
  • Make changes to the software that are tested locally
  • Generate the binary for the new version of the firmware
  • Upload the new version to S3 and send it to the MCU with the an OTA Job
Set the application version

To simulate having a “new” firmware image, set the version number to a higher number. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:

#define APP_VERSION_MAJOR 0

#define APP_VERSION_MINOR 9

#define APP_VERSION_BUILD 2

Build the “new” firmware image

Generate the firmware image by building the project. This can be done pressing “Ctrl+Shift+b” or by navigating to the build tab and pressing build within the Visual Studio project. This will generate the following directory:

FreeRTOS-Plus\Demo\FreeRTOS_IoT_Libraries\ota\ota_code_signing\Debug

Due to the demo running on the FreeRTOS Windows Simulator, the “firmware image” in this case is the executable named “RTOSDemo.exe”.

Verify that the project builds and runs successfully

It’s best practice to verify firmware locally before sending it via an OTA update. See the “Verify that the project builds and runs successfully” section.

Upload the firmware image to the S3 bucket
  • Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
  • Click on the bucket created in the previous steps.
  • Click on the “Upload” button that is under the “Overview” tab.
  • Drag and drop “RTOSDemo.exe” to the bucket.
  • Click “Upload” to add the executable to the bucket.
Lower the application version

Lower the application version. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:

#define APP_VERSION_MAJOR 0

#define APP_VERSION_MINOR 9

#define APP_VERSION_BUILD 1

Build and run the OTA client

Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.

Creating the OTA Update Job using the AWS IoT Console

At this point, you should have:
  • Created an AWS IoT Thing with the AWS IoT Service
  • Setup the S3 bucket and managed permissions for the various services
  • Uploaded a “newer” firmware image to the S3 bucket
  • Completed the setup required for code signing
  • Configured the OTA client running on your device

Create the OTA Update Job

With the OTA Client running and the cloud services setup, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.
  • In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.
    • Create a job
  • Under Create a FreeRTOS OTA update job, choose Create OTA update job.
Create a job
  • You can deploy an OTA update to a single device or a group of devices. Under Select devices to update, choose Select. Choose the Things tab to update a single device. Select the check box next to the IoT thing associated with your device and press Next to continue.
Create a job
  • Under “Select the protocol for firmware image transfer“, choose MQTT
Create a job
  • Under the “Select and sign your firmware image” section, leave the default option of “Sign a new firmware image for me” selected. Under the “Code signing profile” section, press the “Create button”. 
  • In the “Profile name” section, enter “winsim_codesigning”. In the “Device hardware platform” section, select “Windows Simulator”. Under the “Code signing certificate” section select import and select the certificate and certificate private keys that you generated earlier. If you were using the suggested command, they will be called “ecdsasigner.crt” and “ecdsasigner.key”. After choosing these, select import. Write the path to the “ecdsasigner.crt” certificate that was just imported into the section called “Pathname of code signing certificate on device”. Then press “Create” to make the code signing profile. After doing these steps, it should look like the following image:
  • Press select in the “Select your firmware image in S3 or upload it” section and navigate to the executable uploaded during the previous steps. Type in “./NewRTOSDemo.exe” in the “Pathname of firmware image on device” section. This path is where the the file downloaded during the OTA update will be saved. Select the IAM role created for the OTA process and then press “Next” to continue.
  • Leave the Job Type as the default option (snapshot).
Create a job
  • Enter a unique job ID and click Create to finish creating the OTA Update Job.
  • You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted. 

Functionality

The example demonstrates a firmware update over-the-air using OTA job with code signing. The demo creates a single task that connects to the MQTT broker, initializes the OTA Agent, prints OTA statistics, manages reconnects, and restarts if the number of publish failures reaches the limit. It also demonstrates registering an application callback to the OTA Agent and handling OTA complete events. The structure of the demos is as shown below: 
void vRunOTAUpdateDemo( const IotNetworkInterface_t * pNetworkInterface,
                        void * pNetworkCredentialInfo )
{
    IotMqttConnectInfo_t xConnectInfo = IOT_MQTT_CONNECT_INFO_INITIALIZER;
    OTA_State_t eState;
    OTA_ConnectionContext_t xOTAConnectionCtx = { 0 };

    configPRINTF( ( "OTA demo version %u.%u.%u\r\n",
                    xAppFirmwareVersion.u.x.ucMajor,
                    xAppFirmwareVersion.u.x.ucMinor,
                    xAppFirmwareVersion.u.x.usBuild ) );
    configPRINTF( ( "Creating MQTT Client...\r\n" ) );


     /* Create the MQTT Client. */

    for( ; ; )
    {
        xNetworkConnected = prxCreateNetworkConnection();

        if( xNetworkConnected )
        {
            configPRINTF( ( "Connecting to broker...\r\n" ) );
            memset( &xConnectInfo, 0, sizeof( xConnectInfo ) );

            if( xConnection.ulNetworkType == AWSIOT_NETWORK_TYPE_BLE )
            {
                xConnectInfo.awsIotMqttMode = false;
                xConnectInfo.keepAliveSeconds = 0;
            }
            else
            {
                xConnectInfo.awsIotMqttMode = true;
                xConnectInfo.keepAliveSeconds = otaDemoKEEPALIVE_SECONDS;
            }

            xConnectInfo.cleanSession = true;
            xConnectInfo.clientIdentifierLength = ( uint16_t ) strlen(clientcredentialIOT_THING_NAME);
            xConnectInfo.pClientIdentifier = clientcredentialIOT_THING_NAME;

            
            /* Connect to the broker. */
            if( IotMqtt_Connect( &( xConnection.xNetworkInfo ),
                                 &xConnectInfo,
                                 otaDemoCONN_TIMEOUT_MS, 
                                 &( xConnection.xMqttConnection ) ) == IOT_MQTT_SUCCESS )
            {
                configPRINTF( ( "Connected to broker.\r\n" ) );
                xOTAConnectionCtx.pvControlClient = xConnection.xMqttConnection;
                xOTAConnectionCtx.pxNetworkInterface = ( void * ) pNetworkInterface;
                xOTAConnectionCtx.pvNetworkCredentials = pNetworkCredentialInfo;

                OTA_AgentInit( ( void * ) ( &xOTAConnectionCtx ), ( const uint8_t * ) 
                                          ( clientcredentialIOT_THING_NAME ), 
                                            App_OTACompleteCallback, ( TickType_t ) ~0 );

                while( ( eState = OTA_GetAgentState() ) != eOTA_AgentState_Stopped )
                {
                   /* Wait forever for OTA traffic but allow other tasks to run and 
                    * output statistics only once per second. */
                    vTaskDelay( myappONE_SECOND_DELAY_IN_TICKS );
                    configPRINTF( ( "State: %s  Received: %u   Queued: %u   
                                     Processed: %u   Dropped: %u\r\n", 
                                     pcStateStr[ eState ],
                                     OTA_GetPacketsReceived(), 
                                     OTA_GetPacketsQueued(), 
                                     OTA_GetPacketsProcessed(), 
                                     OTA_GetPacketsDropped() ) );
                }

                IotMqtt_Disconnect( xConnection.xMqttConnection, false );
            }
            else
            {
                configPRINTF( ( "ERROR:  MQTT_AGENT_Connect() Failed.\r\n" ) );
            }

           vMqttDemoDeleteNetworkConnection( &xConnection );
            /* After failure to connect or a disconnect, 
             * wait an arbitrary one second before retry. */
            vTaskDelay( myappONE_SECOND_DELAY_IN_TICKS );
        }
        else
        {
            configPRINTF( ( "Failed to create MQTT client.\r\n" ) );
        }
    }
}

Where prxCreateNetworkConnection is defined as :

static BaseType_t prxCreateNetworkConnection( void )
{
    BaseType_t xRet = pdFALSE;
    uint32_t ulRetriesLeft = otaDemoCONN_RETRY_LIMIT;
    uint32_t ulRetryIntervalMS = otaDemoCONN_RETRY_INTERVAL_MS;

    do
    {
        /* No networks are available, block for a physical network connection. */
        if( ( AwsIotNetworkManager_GetConnectedNetworks() 
              & otaDemoNETWORK_TYPES ) == AWSIOT_NETWORK_TYPE_NONE )
        {
            /* Block for a Network Connection. */
            configPRINTF( ( "Waiting for a network connection.\r\n" ) );
            ( void ) xSemaphoreTake( xNetworkAvailableLock, portMAX_DELAY );
        }


        /* Connect to one of the network type.*/
        xRet = xMqttDemoCreateNetworkConnection( &xConnection, otaDemoNETWORK_TYPES );

        if( xRet == pdTRUE )
        {
            break;
        }
        else
        {
            /* Connection failed. Retry for a configured number of retries. */
            if( ulRetriesLeft > 0 )
            {
                configPRINTF( ( "Network Connection failed, retry delay %lu ms, 
                                 retries left %lu\r\n", ulRetryIntervalMS, ulRetriesLeft ) );
                vTaskDelay( pdMS_TO_TICKS( ulRetryIntervalMS ) );
            }
        }
    } while( ulRetriesLeft-- > 0 );

    return xRet;
}
 

Initializing the OTA Agent

The OTA Agent is initialized by calling OTA_AgentInit function which initializes the OTA context, registers the callbacks, resets the statistics counters and creates the OTA Agent Task. It returns the state of the OTA Agent , if initialization is successful within xTicksToWait the OTA Agent state is eOTA_AgentState_Ready otherwise it is set as eOTA_AgentState_Stopped.

  


OTA_State_t OTA_AgentInit( void * pvClient,
                           const uint8_t * pucThingName,
                           pxOTACompleteCallback_t xFunc,
                           TickType_t xTicksToWait )
pvclient is the messaging protocol client context (e.g. an MQTT context).
pucThingName is a pointer to a C string holding the Thing name.
xFunc is callback function for when an OTA job is complete.
xTicksToWait The number of ticks to wait until the OTA Task signals that it is ready.

The console will show the following when the OTA Agent is successfully initialized:

Getting OTA Statistics

The OTA demo/application task can get OTA statistics on the number of messages received, dropped, processed and queued. The statistics are reset when OTA_AgentInit is called again after initialization. The functions that are available to get these statistics are -
  • OTA_GetPacketsReceived: Return the number of packets received.
  • OTA_GetPacketsProcessed: Return the number of packets processed.
  • OTA_GetPacketsQueued: Return the number of packets queued.
  • OTA_GetPacketsDropped: Return the number of packets dropped.

The demo outputs these statistics while it’s receiving a download. See the image below for an example.

OTA Complete Callbacks

The OTA complete callback is called when the OTA job is finished and receives the activate, self-test or failure events during the update process. The OTA agent has either completed the update job or determined that we're in self test mode. If the image passed is successfully verified, the demo will try to activate the new image. This typically means we resetting the device to run the new firmware. In the case of windows simulator, this means closing the application and running the new executable.

If the update was rejected, the demo returns without doing anything and will wait for another job. If it reported that we should start test mode, it will perform user defined system checks to make sure the new firmware does the basic things it’s expected to do. For the purposes of this demo, the callback just sets the image state as accepted without any verification steps. In general, the accept function will vary depending on your platform.

The callback is defined as :

static void App_OTACompleteCallback( OTA_JobEvent_t eEvent )
{
    OTA_Err_t xErr = kOTA_Err_Uninitialized;


    /*OTA job is completed. so delete the MQTT and network connection. */*//*
    if( eEvent == eOTA_JobEvent_Activate )
    {
        configPRINTF( ( "Received eOTA_JobEvent_Activate callback from OTA Agent.\r\n" ) );
        IotMqtt_Disconnect( xConnection.xMqttConnection, 0 );
        #if defined( CONFIG_OTA_UPDATE_DEMO_ENABLED )
            vMqttDemoDeleteNetworkConnection( &xConnection );
        #endif
        OTA_ActivateNewImage();
    }
    else if( eEvent == eOTA_JobEvent_Fail )
    {
        configPRINTF( ( "Received eOTA_JobEvent_Fail callback from OTA Agent.\r\n" ) );
        /* Nothing special to do. The OTA agent handles it. */
    }
    else if( eEvent == eOTA_JobEvent_StartTest )
    {
        /* This demo just accepts the image since it was a good OTA update and networking
         * and services are all working (or we wouldn't have made it this far). If this
         * were some custom device that wants to test other things before calling it OK,
         * this would be the place to kick off those tests before calling OTA_SetImageState()
         * with the final result of either accepted or rejected. */
        configPRINTF( ( "Received eOTA_JobEvent_StartTest callback from OTA Agent.\r\n" ) );
        xErr = OTA_SetImageState( eOTA_ImageState_Accepted );

        if( xErr != kOTA_Err_None )
        {
            OTA_LOG_L1( " Error! Failed to set image state as accepted.\r\n" );
        }
    }
}

When the OTA file download is complete the console will output the following logs:

Activating the OTA Image

When the download is complete, stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when creating the job, then the new executable will be located in the same directory as the visual studio solution. For example:

Run that executable, and at the top of the console you should see that the version number has been updated. For example:

If the device has successfully rebooted, the status of the job in the IoT console will say completed.

Troubleshooting

The following sections contain information to help you troubleshoot issues with OTA updates.  

Topics

Next Steps

This demo has shown how to securely send OTA Update jobs using the MQTT protocol. For an example of how to receive notifications over MQTT and then download the file with HTTPS, see the third demo. See the API reference and porting guide sections to see how to integrate this library into a FreeRTOS project.

Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.