NOTE: The OTA library and documentation are part of the
FreeRTOS LTS Roadmap. These libraries are fully functional, but undergoing optimizations or refactoring to improve memory usage, modularity, documentation, demo usability, or test coverage. They are available on
GitHub or part of the
LTS Development Snapshot download
Basic OTA Demo (with Code Signing)
Introduction
Similar to the first demo, this example uses the MQTT protocol to be notified of a pending OTA Update Job and to download the firmware image. This demo builds on top of the first one by using code signing to verify that the firmware image has not been tampered with and comes from a trusted sender.
See the third demo for an example that receives job documents over HTTP with code signing.
Notice: We recommend that you digitally sign your firmware so that the devices that receive and run it can verify that the firmware came from an authorized source and that the firmware hasn’t been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.
Instructions
These instructions will explain the process of:
- Setting up the cloud services to be able to store the firmware image and send it to the device
- Configuring the device and running the OTA client for receiving the update
- Preparing and creating the OTA Update Job for sending the firmware to the device
These steps require some introductory knowledge about over-the-air updates. Information on this topic can be found in the Introduction and Terminology sections. We also recommend that you perform the first demo before attempting to run this demo.
We will be using unique terminology while working with FreeRTOS Windows Simulator:
- “device”: The FreeRTOS Windows Simulator running in the Visual Studio project
- “firmware image”: Executable that is the demo. When working with a physical MCU, this would typically be a binary image.
Setting up the Cloud Services
Get started with the AWS console
This demo application uses the AWS console to create an IoT thing, store a firmware image, and schedule an OTA job. An AWS account is used for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.
Prerequisites for using the OTA Update Manager Service
Create a Thing
An AWS IoT Thing, or simply thing, is the cloud representation of a device. By registering your device as a thing, you are able to store credentials for it, assign permissions to your device, and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.
Follow the steps in creating an IoT thing, private key, and certificate for your device to register an AWS IoT. Take note of your thing name and AWS IoT endpoint, and download the certificate and key associated with your thing.
Your AWS IoT endpoint is displayed as the
Endpoint. It should look like:
<account-specific-prefix>-ats.iot.region.amazonaws.com
Setting up the OTA Client
Source Code Organization
The demo project is called ota_code_signing.sln and can be found on Github in the following directory:
FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing
Configuring the Demo Project
After downloading the source code, open the following project file in Visual Studio:
FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing/ota_code_signing.sln
All of the changes in the following steps are intended to be done on this project.
Configure the AWS IoT Demo Profile
You need to configure credentials for AWS IoT credentials on the client side. The file
aws_iot_demo_profile.h holds the information you need to connect your FreeRTOS project to AWS IoT, including endpoints, credentials, and keys set to the appropriate values. Setup the credentials through the Certificate Configurator by following the steps below:
- In a browser window, open
tools/aws_config_offline/CertificateConfigurator.html.
- Under Thing Name, enter the Thing name you registered on AWS IoT.
- Under AWS IoT Thing Endpoint, enter your AWS IoT endpoint.
- Under Certificate PEM file, choose the
<ID>
-certificate.pem.crt that you downloaded from the AWS IoT console.
- Under Private Key PEM file, choose the
<ID>
-private.pem.key that you downloaded from the AWS IoT console.
- Choose Generate and save
aws_iot_demo_profile.h, and then save the file in
FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/include. This overwrites the existing file in the directory.
Application Version
For a client to accept an OTA update, the version number of the update it’s receiving needs to be higher than the version of the firmware that it’s currently running.
The application version of the device software is set in the “aws_application_version.h” header file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default setting of these macros are respectively 0, 9, and 2. These do not need to be changed for now, but they will be modified in a future step.
Code Sign Verification
Code signature verification can be enabled or disabled using the “configOTA_ENABLE_CODE_SIGNATURE_VERIFICATION” macro that is located in the “aws_ota_agent_config.h” header file. For the purpose of this demo, this has been enabled and does not need to be modified.
OTA Control Protocol
The OTA Control Protocol setting manages what protocol is used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. Only MQTT is supported at this time for control operations.
This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “aws_ota_agent_config.h” header file. The default value for this macro is “OTA_CONTROL_OVER_MQTT” and does not need to be modified for this demo.
OTA Data Protocol
The OTA Data Protocol defines the format that data is transferred over-the-air to and from the device. Currently MQTT and HTTPS are supported for this setting.
This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the “aws_ota_agent_config.h” header file. The default setting for this macro is “OTA_DATA_OVER_MQTT” and does not need to be changed for this demo.
Configure the networking stack
This demo uses the FreeRTOS+ TCP/IP stack, so follow the instructions provided for the TCP/IP starter project. All of these steps should be performed in the OTA demo project, not the TCP/IP starter project referenced in the documentation. Completing the “prerequisites” section should ensure that you:
Verify that the project builds and runs successfully
Before continuing, verify that you are able to build and run the project. You can do this by pressing F5 within the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”. The output should look like:
While the demo is running, it will continue to output the last line in the image. If you are not seeing this output, refer to the troubleshooting guide.
Note: The output will show that the device has received one job, even if you have not sent an OTA job yet. This is expected due to the Client publishing to the same channel it is listening on and can be ignored.
Prepare for creating the OTA Update Job
To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image, it needs to be a higher version than what is running on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow:
- Set the application version number and build the project executable
- Verify that the executable can build and run correctly
- Upload the executable to S3
- Set the application version number to something lower than what was used in step 1
- Build the demo with the lowered version number
- Run the updated demo and let it continue to run while waiting for an OTA Job
Note: An example of a production workflow would be:
- Write functional firmware for the MCU
- Integrate the OTA Client library into the project
- Manually program the board with the initial firmware
- Make changes to the software that are tested locally
- Generate the binary for the new version of the firmware
- Upload the new version to S3 and send it to the MCU with the an OTA Job
Set the application version
To simulate having a “new” firmware image, set the version number to a higher number. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:
#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 2
Build the “new” firmware image
Generate the firmware image by building the project. This can be done pressing “Ctrl+Shift+b” or by navigating to the build tab and pressing build within the Visual Studio project. This will generate the following directory:
FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/ota/ota_code_signing/Debug
Due to the demo running on the FreeRTOS Windows Simulator, the “firmware image” in this case is the executable named “RTOSDemo.exe”.
Verify that the project builds and runs successfully
It’s best practice to verify firmware locally before sending it via an OTA update. See the “Verify that the project builds and runs successfully” section.
Upload the firmware image to the S3 bucket
- Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
- Click on the bucket created in the previous steps.
- Click on the “Upload” button that is under the “Overview” tab.
- Drag and drop “RTOSDemo.exe” to the bucket.
- Click “Upload” to add the executable to the bucket.
Lower the application version
Lower the application version. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:
#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 1
Build and run the OTA client
Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.
Creating the OTA Update Job using the AWS IoT Console
At this point, you should have:
- Created an AWS IoT Thing with the AWS IoT Service
- Setup the S3 bucket and managed permissions for the various services
- Uploaded a “newer” firmware image to the S3 bucket
- Completed the setup required for code signing
- Configured the OTA client running on your device
Create the OTA Update Job
With the OTA Client running and the cloud services setup, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.
- In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.
- Under Create a FreeRTOS OTA update job, choose Create OTA update job.
- You can deploy an OTA update to a single device or a group of devices. Under Select devices to update, choose Select. Choose the Things tab to update a single device. Select the check box next to the IoT thing associated with your device and press Next to continue.
- Under “Select the protocol for firmware image transfer“, choose MQTT
- Under the “Select and sign your firmware image” section, leave the default option of “Sign a new firmware image for me” selected. Under the “Code signing profile” section, press the “Create button”.
- In the “Profile name” section, enter “winsim_codesigning”. In the “Device hardware platform” section, select “Windows Simulator”. Under the “Code signing certificate” section select import and select the certificate and certificate private keys that you generated earlier. If you were using the suggested command, they will be called “ecdsasigner.crt” and “ecdsasigner.key”. After choosing these, select import. Write the path to the “ecdsasigner.crt” certificate that was just imported into the section called “Pathname of code signing certificate on device”. Then press “Create” to make the code signing profile. After doing these steps, it should look like the following image:
- Press select in the “Select your firmware image in S3 or upload it” section and navigate to the executable uploaded during the previous steps. Type in “./NewRTOSDemo.exe” in the “Pathname of firmware image on device” section. This path is where the file downloaded during the OTA update will be saved. Select the IAM role created for the OTA process and then press “Next” to continue.
- Leave the Job Type as the default option (snapshot) and click Create to finish creating the OTA Update Job.
- You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted.
Functionality
The example demonstrates a firmware update over-the-air using OTA job with code signing. The demo creates a single task that connects to the MQTT broker, initializes the OTA Agent, prints OTA statistics, manages reconnects, and restarts if the number of publish failures reaches the limit. It also demonstrates registering an application callback to the OTA Agent and handling OTA complete events. The structure of the demos is as shown below:
void vOTAUpdateDemoTask( void * pvParameters )
{
OTA_State_t eState;
OTA_ConnectionContext_t xOTAConnectionCtx = { 0 };
( void ) pvParameters;
prvInitialiseLibraries();
configPRINTF( ( “OTA demo version %u.%u.%u\r\n”,
xAppFirmwareVersion.u.x.ucMajor,
xAppFirmwareVersion.u.x.ucMinor,
xAppFirmwareVersion.u.x.usBuild ) );
configPRINTF( ( “Creating MQTT Client…\r\n” ) );
for( ; ; )
{
configPRINTF( ( “Connecting to broker…\r\n” ) );
if( _establishMqttConnection( true,
&xMQTTConnection ) == IOT_MQTT_SUCCESS )
{
configPRINTF( ( “Connected to broker.\r\n” ) );
xOTAConnectionCtx.pvControlClient = xMQTTConnection;
xOTAConnectionCtx.pxNetworkInterface = ( void * ) IOT_NETWORK_INTERFACE_FREERTOS;
xOTAConnectionCtx.pvNetworkCredentials = &xNetworkSecurityCredentials;
_retryInterval = otaDemoCONN_RETRY_BASE_INTERVAL_SECONDS;
_networkConnected = true;
if( ( eState = OTA_GetAgentState() ) == eOTA_AgentState_Suspended )
{
OTA_Resume( &xOTAConnectionCtx );
}
if( ( eState = OTA_GetAgentState() ) == eOTA_AgentState_Suspended )
{
OTA_Resume( &xOTAConnectionCtx );
}
OTA_AgentInit( ( void * ) ( &xOTAConnectionCtx ),
( const uint8_t * ) ( awsiotdemoprofileCLIENT_IDENTIFIER ),
App_OTACompleteCallback,
( TickType_t ) ~0 );
while( ( ( eState = OTA_GetAgentState() ) != eOTA_AgentState_Stopped ) && _networkConnected )
{
IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );
configPRINTF( ( “State: %s Received: %u Queued: %u Processed: %u Dropped: %u\r\n”,
pcStateStr[ eState ],
OTA_GetPacketsReceived(),
OTA_GetPacketsQueued(),
OTA_GetPacketsProcessed(),
OTA_GetPacketsDropped() ) );
}
if( _networkConnected == false )
{
if( OTA_Suspend() == kOTA_Err_None )
{
while( ( eState = OTA_GetAgentState() ) != eOTA_AgentState_Suspended )
{
IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );
}
}
}
else
{
IotMqtt_Disconnect( xMQTTConnection, false );
}
}
else
{
configPRINTF( ( “ERROR: Failed to connect to MQTT broker.\r\n” ) );
}
_connectionRetryDelay();
}
}
OTA update demo sample code.
Where _establishMqttConnection is defined as:
static IotMqttError_t _establishMqttConnection( bool awsIotMqttMode,
IotMqttConnection_t * pMqttConnection )
{
IotMqttError_t connectStatus = IOT_MQTT_STATUS_PENDING;
memset( &xConnectInfo, 0, sizeof( xConnectInfo ) );
xConnectInfo.awsIotMqttMode = awsIotMqttMode;
xConnectInfo.cleanSession = true;
xConnectInfo.keepAliveSeconds = otaDemoKEEPALIVE_SECONDS;
xConnectInfo.clientIdentifierLength = ( uint16_t ) strlen( awsiotdemoprofileCLIENT_IDENTIFIER );
xConnectInfo.pClientIdentifier = awsiotdemoprofileCLIENT_IDENTIFIER;
configPRINTF( ( “MQTT demo client identifier is %.*s (length %hu).”,
xConnectInfo.clientIdentifierLength,
xConnectInfo.pClientIdentifier,
xConnectInfo.clientIdentifierLength ) );
connectStatus = IotMqtt_Connect( &xNetworkInfo,
&xConnectInfo,
otaDemoCONN_TIMEOUT_MS,
pMqttConnection );
if( connectStatus != IOT_MQTT_SUCCESS )
{
configPRINTF( ( “MQTT CONNECT returned error %s.”,
IotMqtt_strerror( connectStatus ) ) );
}
return connectStatus;
}
_establishMqttConnection function in OTA sample code.
Initializing the OTA Agent
The OTA Agent is initialized by calling OTA_AgentInit function which initializes the OTA context, registers the callbacks, resets the statistics counters and creates the OTA Agent Task. It returns the state of the OTA Agent , if initialization is successful within xTicksToWait the OTA Agent state is eOTA_AgentState_Ready otherwise it is set as eOTA_AgentState_Stopped.
OTA_State_t OTA_AgentInit( void * pvClient,
const uint8_t * pucThingName,
pxOTACompleteCallback_t xFunc,
TickType_t xTicksToWait )
OTA agent init function signature.
- pvclient is the messaging protocol client context (e.g. an MQTT context).
- pucThingName is a pointer to a C string holding the Thing name.
- xFunc is callback function for when an OTA job is complete.
- xTicksToWait The number of ticks to wait until the OTA Task signals that it is ready.
The console will show the following when the OTA Agent is successfully initialized:
Getting OTA Statistics
The OTA demo/application task can get OTA statistics on the number of messages received, dropped, processed and queued. The statistics are reset when OTA_AgentInit is called again after initialization. The functions that are available to get these statistics are –
- OTA_GetPacketsReceived: Return the number of packets received.
- OTA_GetPacketsProcessed: Return the number of packets processed.
- OTA_GetPacketsQueued: Return the number of packets queued.
- OTA_GetPacketsDropped: Return the number of packets dropped.
The demo outputs these statistics while it’s receiving a download. See the image below for an example.
Suspend on network disconnect and resume after reconnect
If the device disconnects from AWS IoT in the middle of downloading a firmware image, it will pause the download until it reconnects. Below is the callback registered through MQTT to handle such disconnection,
static void prvNetworkDisconnectCallback( void * param,
IotMqttCallbackParam_t * mqttCallbackParams )
{
( void ) param;
switch( mqttCallbackParams->u.disconnectReason )
{
case IOT_MQTT_DISCONNECT_CALLED:
configPRINTF( ( “Mqtt disconnected due to invoking diconnect function.\r\n” ) );
break;
case IOT_MQTT_BAD_PACKET_RECEIVED:
configPRINTF( ( “Mqtt disconnected due to invalid packet received from the network.\r\n” ) );
break;
case IOT_MQTT_KEEP_ALIVE_TIMEOUT:
configPRINTF( ( “Mqtt disconnected due to Keep-alive response not received.\r\n” ) );
break;
default:
configPRINTF( ( “Mqtt disconnected due to unknown reason.” ) );
break;
}
_networkConnected = false;
}
prvNetworkDisconnectCallback function in OTA sample code.
The flag _networkConnected will be set to false when the disconnect callback is invoked. Then OTA_suspend is called to tell the OTA agent to pause the download. Once the device reestablishes the connection to AWS IoT, OTA_resume is called tell OTA agent to resume the download. Below is a snapshot of the device output showing the OTA agent suspends when MQTT connection disconnects, then resumes the same update when the MQTT connection reconnects,
In this demo, the reconnect logic uses an exponential-backoff-and-jitter to prevent the MQTT server getting overwhelmed with connection requests if a fleet of devices get disconnected and then attempt to reconnect at the same time. The reconnection function is shown below.
static void _connectionRetryDelay( void )
{
unsigned int retryIntervalwithJitter = 0;
if( ( _retryInterval * 2 ) >= otaDemoCONN_RETRY_MAX_INTERVAL_SECONDS )
{
_retryInterval = otaDemoCONN_RETRY_MAX_INTERVAL_SECONDS;
}
else
{
_retryInterval *= 2;
}
retryIntervalwithJitter = _retryInterval + ( rand() % _retryInterval );
configPRINTF( ( “Retrying network connection in %d Secs “, retryIntervalwithJitter ) );
IotClock_SleepMs( retryIntervalwithJitter * 1000 );
}
_connectionRetryDelay function in OTA sample code.
OTA Complete Callbacks
The OTA complete callback is called when the OTA job is finished and receives the activate, self-test or failure events during the update process. The OTA agent has either completed the update job or determined that we’re in self test mode. If the image passed is successfully verified, the demo will try to activate the new image. This typically means we resetting the device to run the new firmware. In the case of windows simulator, this means closing the application and running the new executable.
If the update was rejected, the demo will return without doing anything and will wait for another job. If it is reported that we should start test mode, it will perform user defined system checks to make sure the new firmware does the basic things it’s expected to do. For the purposes of this demo, the callback sets the image state as accepted without any verification steps. In general, the accept function will vary depending on your platform.
The callback is defined as :
static void App_OTACompleteCallback( OTA_JobEvent_t eEvent )
{
OTA_Err_t xErr = kOTA_Err_Uninitialized;
if( eEvent == eOTA_JobEvent_Activate )
{
configPRINTF( ( “Received eOTA_JobEvent_Activate callback from OTA Agent.\r\n” ) );
IotMqtt_Disconnect( xMQTTConnection, 0 );
OTA_ActivateNewImage();
configPRINTF( ( “ERROR: New image activation failed.\r\n” ) );
for( ; ; )
{
IotClock_SleepMs( otaDemoTASK_DELAY_SECONDS * 1000 );
}
}
else if( eEvent == eOTA_JobEvent_Fail )
{
configPRINTF( ( “Received eOTA_JobEvent_Fail callback from OTA Agent.\r\n” ) );
}
else if( eEvent == eOTA_JobEvent_StartTest )
{
configPRINTF( ( “Received eOTA_JobEvent_StartTest callback from OTA Agent.\r\n” ) );
xErr = OTA_SetImageState( eOTA_ImageState_Accepted );
if( xErr != kOTA_Err_None )
{
OTA_LOG_L1( ” Error! Failed to set image state as accepted.\r\n” );
}
}
}
OTA complete callback in OTA demo
When the OTA file download is complete the console will output the following logs:
Activating the OTA Image
When the download is complete, stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when creating the job, then the new executable will be located in the same directory as the visual studio solution. For example:
Run that executable, and at the top of the console you should see that the version number has been updated. For example:
If the device has successfully rebooted, the status of the job in the IoT console will say completed.
Troubleshooting
The following sections contain information to help you troubleshoot issues with OTA updates.
Topics
Next Steps
This demo has shown how to securely send OTA Update jobs using the MQTT protocol. For an example of how to receive notifications over MQTT and then download the file with HTTPS, see the third demo. See the API reference and porting guide sections to see how to integrate this library into a FreeRTOS project.
Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.
