LibrariesinsideAWS IoT OTA

Updated Mar 2025

Running OTA over MQTT

With the coreMQTT Agent and Code Signing

Preamble

While this demo uses the AWS IoT OTA update service, FreeRTOS is generic MIT licensed open source software, and can be used with any OTA mechanism that works for you. We do however recommend you digitally sign your firmware whichever OTA method you choose. That way the devices that receive the new executable image can verify it came from an authorized source and has not been been modified. You can use Code Signing for AWS IoTexternal_link to sign your firmware or you can sign with your own code-signing tools.

Introduction

This example uses the MQTT agent to receive notification of a pending OTA Update Job and to download the new firmware image. Use of the MQTT agent enables the OTA update functionality to run in the background, simultaneously with other tasks that are sharing the same MQTT connection.

The example also demonstrates the use of code signing to verify that the downloaded firmware image has not been tampered with and comes from a trusted sender.

See the comments at the top of each C file in the Source Directoryexternal_link or the Functionality section for additional information.

Instructions

Getting started

Start by connecting one of the simple MQTT agent examples described on the main MQTT agent documentation page to the AWS IoT MQTT broker. That will ensure connectivity to AWS IoT is working correctly before moving on to set up the OTA service.

Once connectivity is working with a non OTA demo the following instructions show you how to:

  • Set up the cloud services to be able to store the firmware image and send it to the device.
  • Configure the device and run the OTA client to receive the update.
  • Prepare and create the OTA Update Job to send the firmware to the device.

Setting up the cloud services

Get started with the AWS console

This demo application uses the AWS console to create an IoT Thing, store a firmware image, and schedule an OTA job. An AWS account is required for you to be authorized to perform these operations. Click here to get started with creating and configuring an accountexternal_link.

Prerequisites for using the OTA Update Manager Service

Create a Thing

An AWS IoT Thing, or simply "Thing", is the cloud representation of a device. By registering your physical device as a Thing, you are able to store security credentials for it, assign permissions to your device (what it can and cannot do or access in the cloud), and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator "device" will be represented by an IoT Thing and will be sent an OTA job from the AWS Consoleexternal_link.

You will have created a Thing when connecting one of the non OTA demos to AWS IoT, so there are no additional steps here.

Setting up the device (the OTA client)

If, as recommended above, you have already used one of the other MQTT agent examples to connect with AWS IoT then you already have the source files, and have configured the network and AWS IoT connection.

Application Version

For a client to accept an OTA update, the version number of the update it receives must be higher than the version of the firmware that it’s currently running.

The application version of the device software is set in the "

demo_config.h
" source file with the "APP_VERSION_MAJOR", "APP_VERSION_MINOR", and the "APP_VERSION_BUILD" macros. The default settings of these macros are 0, 9, and 2 respectively. These do not need to be changed for now, but they will be modified in a future step.

OTA Control Protocol

The OTA Control Protocol setting manages the protocol used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. For control operations, only MQTT is supported at this time.

This setting is controlled by the "

configENABLED_CONTROL_PROTOCOL
" macro that can be found in the "
ota_config.h
" header file. The default value for this macro is "
OTA_CONTROL_OVER_MQTT
" and does not need to be modified for this demo.

OTA Data Protocol

The OTA Data Protocol defines the format used to transfer data over-the-air to and from the device. Currently MQTT and HTTPS are supported.

This option is controlled by setting the "

configENABLED_DATA_PROTOCOLS
" macro that can be found in the "
ota_config.h
" header file. The default setting for this macro is "
OTA_DATA_OVER_MQTT
" and does not need to be changed for this demo.

Code Signing

Code signing is used to confirm the author of the code and guarantee that it was not tampered with or corrupted since it was signed. As part of the OTA process, the signature of the file is verified after being completely downloaded.

The certificate used to verify the file needs to be set before you perform an over-the-air update. If you want to use a certificate stored on the device to verify the file, you must enter the path to the certificate in a later step. Otherwise, to configure the code signing certificate, set the "

signingcredentialSIGNING_CERTIFICATE_PEM
" variable in the "
aws_ota_codesigner_certificate.h"
header file.

Other OTA Library Configurations

Please refer to other OTA library configurationsexternal_link. These configuration values do not need to be changed for this demo.

Verify that the project builds and runs successfully

Before you continue, verify that you are able to build and run the project. You can do this by pressing F5 in the Visual Studio demo project, or by navigating to the "Debug" tab and clicking "Start Debugging".

Prepare for creating the OTA update job

To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The AWS IoT OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image the received image must have a higher version number than the image already being executed on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow (documented below the bullet list):

  1. Set the application version number and build the project executable.
  2. Verify that the executable can build and run correctly.
  3. Upload the executable /build/VisualStudio/Demo/RTOSDemo.exe to S3.
  4. Set the application version number to something lower than that used in step 1.
  5. Build the demo with the lowered version number.
  6. Run the updated demo and let it continue to run while waiting for an OTA Job.

Note: An example of a production workflow would be:

  1. Write the MCU firmware, integrating the OTA Client library source code.
  2. Program the device hardware (the MCU) with the initial firmware.
  3. Make changes to and test the firmware locally.
  4. Generate the binary for the new version of the firmware.
  5. Upload the new version to S3 and send it to the MCU with the an OTA Job.

Set the application version

To simulate having a "new" firmware image, increment the version number. For this demo, update the following macro values that can be found in the "

demo_config.h
" source file as shown here:

1#define APP_VERSION_MAJOR 0
2#define APP_VERSION_MINOR 9
3#define APP_VERSION_BUILD 2

Build the "new" firmware image

Generate the firmware image by building the project. To do this, press "Ctrl+Shift+b" or navigate to the build tab and press build within the Visual Studio project. This will generate the following executable: "/build/VisualStudio/Demo/RTOSDemo.exe".

Because the demo runs on the FreeRTOS Windows Simulator, the "firmware image" in this case is the Windows executable.

Verify that the project builds and runs successfully

It’s best practice to verify firmware locally before you send it via an OTA update. See the "Verify that the project builds and runs successfully" section.

Upload the firmware image to the S3 bucket

  1. Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/external_link.
  2. Click on the bucket created in the previous steps.
  3. Click on the "Upload" button that is under the "Overview" tab.
  4. Drag and drop "RTOSDemo.exe" to the bucket.
  5. Click "Upload" to add the executable to the bucket.

Lower the application version

Lower the application version. For this demo, update the following macro values that can be found in the "demo_config.h" source file as shown here:

1#define APP_VERSION_MAJOR 0
2#define APP_VERSION_MINOR 9
3#define APP_VERSION_BUILD 1

Build and run the OTA client

Press the "Local Windows Debugger" button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.

Creating the OTA update job using the AWS IoT console

At this point, you should have:

  • Created an AWS IoT Thing with the AWS IoT Service.
  • Setup the S3 bucket and managed permissions for the various services.
  • Uploaded a "newer" firmware image to the S3 bucket.
  • Completed the setup required for code signing.
  • Configured the OTA client running on your device.

Create the OTA update job

With the OTA Client running and the cloud services set up, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT consoleexternal_link.

  1. In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press Create Job.

    Create a Job - click to enlarge

  2. Choose Create FreeRTOS OTA update job, then press Next.

    Create OTA Update Job - click to enlarge

  3. On the OTA job properties page, enter a Job name for the FreeRTOS OTA update job (for example, "win_sim_update"). You can optionally enter a Description and add Tags to your job. Then press Next to continue.

    OTA Job Properties - click to enlarge

  4. You can deploy an OTA update to a single device or a group of devices. On the OTA file configuration page, under Devices to update, choose the things or thing groups associated with the devices you want to update. Under Select the protocol for file transfer, select the check box next to MQTT.

    Select a Device to update - click to enlarge

  5. Under Sign and choose your file, keep the default option of Sign a new file for me selected. Under Code signing profile, press the Create new profile button which is next to Existing code signing profile.

    Select the Files to OTA - click to enlarge

  6. On the Create a code signing profile page, under Profile name, enter "winsim_codesigning". Under Device hardware platform, select "Windows Simulator". Under Code signing certificate, change the default and select the Select an existing certificate checkbox. Under Certificates, select the certificate and certificate private keys that you generated earlier. If you followed the suggestion, these will be named "ecdsasigner.crt" and "ecdsasigner.key". Then press the Import button.

    Under Path name of code signing certificate on device, enter the path to the "ecdsasigner.crt" certificate that you just imported. However, if you set the variable "

    signingcredentialSIGNING_CERTIFICATE_PEM
    " during a previous step, you should enter "/" for the path here.

    Finally, press the Create button to make the code signing profile.

    Create a Code Signing Profile - click to enlarge

  7. Back on the OTA file configuration page, under File, change the default and select the checkbox for Select an existing file, then press the Browse S3 button and choose the executable you uploaded to S3 during a previous step. Under Path name of file on device, enter "./NewRTOSDemo.exe". This path is where the file that is downloaded during the OTA update will be saved.

    Note: The File type feature is supported in OTA library version v2.0.0external_link or higher.

    Choose S3 file and location on device - click to enlarge

  8. Under IAM role, select the IAM role created for the OTA process. Then press Next to continue.

    Choose IAM role - click to enlarge

  9. Under OTA job configuration, keep the default settings selected for Job run type (snapshot), Job start rollout configuration (constant rate), and Job run timeout configuration (No timeout). Then press the Create job button to finish creating the OTA Update Job.

    OTA Job Configuration - click to enlarge

  10. You can monitor that status of the job by pressing the "View Job" pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as "IN PROGRESS" until the device has successfully rebooted. 

OTA Job Status - click to enlarge

Receive and activate the update

After the job has been created, the device should begin downloading the update. The progress of the download can be monitored by viewing the Visual Studio console. Below is an example of a progress update:

1Current State=[WaitingForFileBlock], Event=[RequestFileBlock], New state=[WaitingForFileBlock]
2Received valid file block: Block index=96, Size=2048
3Number of blocks remaining: 3

The following message will print to the monitor after receiving the final block of the download:

1Received final block of the update.

At this point, the demo will verify the code signing signature of the download. If it's successful, the following message will be printed to the console:

1sig-sha256-ecdsa signature verification passed.

The demo will attempt to activate the new image after successfully downloading the update. For this demo, the update needs to be activated by manually running the downloaded executable. This is indicated by the following message:

1Failed to activate new image: activateNewImage returned error: Manual reset required: OtaPalStatus_t=OtaPalSuccess

The demo will exit with the following message after successfully validating the update.

1OTA agent task stopped. Exiting OTA demo.

If the image was successfully downloaded then it is ready to be activated. Stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in "./NewRTOSDemo.exe" when you created the job, then the new executable will be located in the same directory as the Visual Studio solution. Run this executable. The device will begin to validate the image after the new executable has started. If this process is successful, then the following message will be printed to the console:

1New image validation succeeded in self test mode.

When the job process is complete the state of the job will change from IN PROGRESS to COMPLETED in the IoT console. If the device has successfully passed the self-test process, the status of the job will change to Succeeded.

OTA Job Status - click to enlarge

Functionality

This demo illustrates how to build an application with the OTA library which uses the coreMQTT agent. Utilizing the MQTT agent enables the OTA agent to share the same MQTT broker connection with other tasks.

Demo setup

This example application includes the resources and interface implementations required by the OTA library. See the OTA library design documentationexternal_link for more additional information about these interfaces.

OTA PAL implementation

The OTA PAL interfaceexternal_link is a set of APIs that are used to store, manage, and authenticate downloads. This demo provides an example implementation for the Windows Simulator platform.

Details of the port can be found in the Win32external_link repository on GitHub.

OTA OS Functional interface implementation

The OTA library calls the OS Functional interfaceexternal_link implementations to perform actions that are typically provided by an operating system. This includes managing events, timers, and memory allocation. This demo uses FreeRTOS implementations of these interfaces.

Details of the port can be found in the ota_os_freertos.hexternal_link file on GitHub.

OTA MQTT interface implementation

The OTA MQTT interfaceexternal_link is used by the OTA library to connect to AWS IoT Core and manage notification and request data. The OTA library uses the MQTT protocol to inform AWS IoT Core about job status, receive notifications of pending jobs, and to receive data blocks. The OTA MQTT interface implementation in this example allows multiple tasks to use the same MQTT broker connection.

Details of the port can be found in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

OTA application callback implementation

The OTA library provides a way to call a user defined function after receiving a job. This function is called when the OTA agent has completed an update job and when the device reboots after activating the image and is performing a self-test. Typically if the update job was successful, then the device will want to reset to boot into the new image. If the job was rejected, this callback will typically return without doing anything and wait for another job.

Details of the implementation can be found in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

Memory allocation setup

The OTA agent requires memory to download, decode, and store the job and file. The OTA agent will use the buffers provided by the user unless they are insufficient. In this case, it will dynamically allocate the memory required with the memory abstraction functions defined by the OTA OS Functional interfaceexternal_link implementations.

User configuration

The demo requires some configuration that is specific to the device running it. The details of this can be seen in the earlier demo steps.

Entry point of the demo

The OTA Demo source code can be found in the OtaOverMqttDemoExample.cexternal_link file on GitHub. This function is the primary thread for managing the over-the-air download process. It performs the following steps:

Initialize the OTA agent

In order to use the OTA agent, you'll need to provide interface implementations, buffers, and an app callback function to the "OTA_Init" APIexternal_link before starting the event processing task.

You can see an example of this in the [OtaOverMqttDemoExample.c](https://github.com/FreeRTOS/FreeRTOS/blob/04f0f68f61c7bb029b602931f2fc4a42ebfc47e7/FreeRTOS-Plus/Demo/AWS/Ota_Windows_Simulator/Ota_Over_Mqtt_Demo/DemoTasks/OtaOverMqttDemoExample.c#L1851-L1860external_link rel=) file on GitHub.

OTA callback

Create the OTA agent event processing task

The OTA agent operates by managing a state machine that tracks the current status of the download process. The state machine is influenced by receiving events that are sourced by either internal calls or the main application. The OTA agent requires a loop to be running to receive and process these incoming events before it can start. The demo runs this processing loop in a separate thread that calls the "OTA_EventProcessingTask"external_link API that is defined by the OTA library.

You can see an example of this in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

Start the OTA agent

Once the OTA agent has been configured and the event processing task has been started, the OTA agent can begin the download process. This is done by sending an event with the "OtaAgentEventStart" ID to the OTA agent with the "OTA_SignalEventexternal_link" API.

You can see an example of this in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

Monitor the OTA agent statistics

After the OTA agent has been started, it will continue to operate the download process in the background until it is shutdown. The OTA library provides the "OTA_GetStatistics" APIexternal_link for receiving statistics related to the download progress. These statistics include how many packets have been received, queued, processed, and dropped.

You can see an example of polling and printing these statistics in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

Shutting down the OTA agent

The OTA agent will continue to run in the background until it receives an event to shutdown. The OTA library provides an API for sending this event called "OTA_Shutdown"external_link. The demo calls this API after receiving and activating a job to shutdown the OTA Agent.

You can see an example of this in the OtaOverMqttDemoExample.cexternal_link file on GitHub.

Troubleshooting

The following sections contain information to help you troubleshoot issues with OTA updates.

Topics