OTA Demo (HTTPS)
This example builds on top of the previous example that used MQTT and code signing to securely send a firmware image to the device. While it still uses MQTT for operations like pending job notifications and updating job status, this demo uses HTTPS instead of MQTT to download the file. After the file is downloaded, it verifies the signature of the file before updating the firmware.
Notice: We recommend that you digitally sign your firmware so that the devices that receive and run it can verify that the firmware came from an authorized source and that the firmware hasn’t been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.
InstructionsThese instructions will explain the process of:
- Setting up the cloud services to be able to store the firmware image and send it to the device
- Configuring the device and running the OTA client for receiving the update
- Preparing and creating the OTA Update Job for sending the firmware to the device
- “device”: The FreeRTOS Windows Simulator running in the Visual Studio project
- “firmware image”: Windows executable that is the demo. When working with a physical MCU, this would typically be a firmware binary image.
Setting up the Cloud Services
Get started with the AWS console
This demo application uses the AWS console to create an IoT thing, store a firmware image, and schedule an OTA job. An AWS account is used for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.
Prerequisites for using the OTA Update Manager Service
- S3 is an AWS Service that enables you to store files in the cloud that can be accessed by you or other services. This is used by the OTA Update Manager Service to store the firmware image in an S3 “bucket” before sending it to the device. Create an Amazon S3 Bucket to Store Your Update.
- By default, the OTA Update Manager does not have permission to access the S3 bucket that will contain the firmware image. An OTA Service Role is required to allow the OTA Update Manager Service to read and write to the S3 buckets. Create an OTA Update Service Role.
- An OTA User Policy is required to give your account permissions to interact with the AWS services required for creating an OTA Update. Create an OTA User Policy.
- Create a Code-Signing Certificate.
- Grant Access to Code Signing for AWS IoT.
Create a Thing
An AWS IoT Thing, or simply thing, is the cloud representation of a device. By registering your device as a thing, you are able to store credentials for it, assign permissions to your device, and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.
Follow the steps in creating an IoT thing, private key, and certificate for your device to register an AWS IoT. Take note of your thing name and AWS IoT endpoint, and download the certificate and key associated with your thing.
- Browse to the AWS IoT console.
- In the navigation pane, choose Settings.
Setting up the OTA Client
Source Code Organization
The demo project is called ota_code_signing_http.sln and can be found on Github in the following directory:
Configuring the Demo Project
After downloading the source code, open the following project file in Visual Studio:
All of the changes in the following steps are intended to be done on this project.
Configure the AWS IoT Demo Profile
You need to configure credentials for AWS IoT credentials on the client side. The file
aws_iot_demo_profile.h holds the information you need to connect your FreeRTOS project to AWS IoT, including endpoints, credentials, and keys set to the appropriate values. Setup the credentials through the Certificate Configurator by following the steps below:
- In a browser window, open
- Under Thing Name, enter the Thing name you registered on AWS IoT.
- Under AWS IoT Thing Endpoint, enter your AWS IoT endpoint.
- Under Certificate PEM file, choose the
- Under Private Key PEM file, choose the
- Choose Generate and save
aws_iot_demo_profile.h, and then save the file in
FreeRTOS-Plus\Demo\FreeRTOS_IoT_Libraries\include. This overwrites the existing file in the directory.
For a client to accept an OTA update, the version number of the update it’s receiving needs to be higher than the version of the firmware that it’s currently running.
The application version of the device software is set in the “aws_application_version.h” header file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default setting of these macros are respectively 0, 9, and 2. These do not need to be changed for now, but they will be modified in a future step.Code Sign Verification
Code signature verification can be enabled or disabled using the “configOTA_ENABLE_CODE_SIGNATURE_VERIFICATION” macro that is located in the “aws_ota_agent_config.h” header file. For the purpose of this demo, this has been enabled and does not need to be modified.OTA Control Protocol
The OTA Control Protocol setting manages what protocol is used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. Only MQTT is supported at this time for control operations.
This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “aws_ota_agent_config.h” header file. The default value for this macro is “OTA_CONTROL_OVER_MQTT” and does not need to be modified for this demo.OTA Data Protocol
The OTA Data Protocol defines the format that data is transferred over the air to and from the device. Currently MQTT and HTTP are supported for this setting.
This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the “aws_ota_agent_config.h” header file. This macro is set to “OTA_DATA_OVER_HTTP” and does not need to be changed for this demo.Configure the networking stack
This demo uses the FreeRTOS+ TCP/IP stack, so follow the instructions provided for the TCP/IP starter project. All of these steps should be performed in the OTA demo project, not the TCP/IP starter project referenced in the documentation. Completing the “prerequisites” section should ensure that you:
- Have the pre-requisite components installed (such as WinPCap).
- Optionally set a static or dynamic IP address, gateway address and netmask.
- Optionally set a MAC address.
- Select an Ethernet network interface on your host machine.
Verify that the project builds and runs successfully
Before continuing, verify that you are able to build and run the project. You can do this by pressing F5 within the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”. The output should look like:
While the demo is running, it will continue to output the last line in the image. If you are not seeing this output, refer to the troubleshooting guide.
Note: The output will show that the device has received one job, even if you have not sent an OTA job yet. This is expected due to the Client publishing to the same channel it is listening on and can be ignored.
Prepare for creating the OTA Update JobTo send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image, it needs to be a higher version than what is running on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow:
- Set the application version number and build the project executable
- Verify that the executable can build and run correctly
- Upload the executable to S3
- Set the application version number to something lower than what was used in step 1
- Build the demo with the lowered version number
- Run the updated demo and let it continue to run while waiting for an OTA Job
- Write functional firmware for the MCU
- Integrate the OTA Client library into the project
- Manually program the board with the initial firmware
- Make changes to the software that are tested locally
- Generate the binary for the new version of the firmware
- Upload the new version to S3 and send it to the MCU with the an OTA Job
To simulate having a “new” firmware image, set the version number to a higher number. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:
#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
Build the “new” firmware image
#define APP_VERSION_BUILD 2
Generate the firmware image by building the project. This can be done pressing “Ctrl+Shift+b” or by navigating to the build tab and pressing build within the Visual Studio project. This will generate the following directory:
Due to the demo running on the FreeRTOS Windows Simulator, the “firmware image” in this case is the executable named “RTOSDemo.exe”.Verify that the project builds and runs successfully
It’s best practice to verify firmware locally before sending it via an OTA update. See the “Verify that the project builds and runs successfully” section.Upload the firmware image to the S3 bucket
- Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
- Click on the bucket created in the previous steps.
- Click on the “Upload” button that is under the “Overview” tab.
- Drag and drop “RTOSDemo.exe” to the bucket.
- Click “Upload” to add the executable to the bucket.
Lower the application version. For this demo, update the following macro values that can be found in the “aws_application_version.h” header file to:
#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
Build and run the OTA client
#define APP_VERSION_BUILD 1
Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.
Creating the OTA Update Job using the AWS IoT ConsoleAt this point, you should have:
- Created an AWS IoT Thing with the AWS IoT Service
- Setup the S3 bucket and managed permissions for the various services
- Uploaded a “newer” firmware image to the S3 bucket
- Completed the setup required for code signing
- Configured the OTA client running on your device
Create the OTA Update JobWith the OTA Client running and the cloud services setup, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.
- In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.
- Under Create a FreeRTOS OTA update job, choose Create OTA update job.
- You can deploy an OTA update to a single device or a group of devices. Under Select devices to update, choose Select. Choose the Things tab to update a single device. Select the check box next to the IoT thing associated with your device and press Next to continue.
- Under “Select the protocol for firmware image transfer“, choose HTTP
- Under the “Select and sign your firmware image” section, leave the default option of “Sign a new firmware image for me” selected. Under the “Code signing profile” section, press the “Create button”.
- In the “Profile name” section, enter “winsim_codesigning”. In the “Device hardware platform” section, select “Windows Simulator”. Under the “Code signing certificate” section select import and select the certificate and certificate private keys that you generated earlier. If you were using the suggested command, they will be called “ecdsasigner.crt” and “ecdsasigner.key”. After choosing these, select import. Write the path to the “ecdsasigner.crt” certificate that was just imported into the section called “Pathname of code signing certificate on device”. Then press “Create” to make the code signing profile. After doing these steps, it should look like the following image:
- Press select in the “Select your firmware image in S3 or upload it” section and navigate to the executable uploaded during the previous steps. Type in “./NewRTOSDemo.exe” in the “Pathname of firmware image on device” section. This path is where the the file downloaded during the OTA update will be saved. Select the IAM role created for the OTA process and then press “Next” to continue.
- Leave the Job Type as the default option (snapshot).
- Enter a unique job ID and click Create to finish creating the OTA Update Job.
- You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted.
FunctionalityThe example demonstrates a firmware update over-the-air using OTA job with code signing and HTTP. The demo creates a single task that connects to the MQTT broker, initializes the OTA Agent, prints OTA statistics, manages reconnects, and restarts if the number of publish failures reaches the limit. It also demonstrates registering an application callback to the OTA Agent and handling OTA complete events. The structure of the demos is as shown below:
Where prxCreateNetworkConnection is defined as :
Initializing the OTA Agent
The OTA Agent is initialized by calling OTA_AgentInit function which initializes the OTA context, registers the callbacks, resets the statistics counters and creates the OTA Agent Task. It returns the state of the OTA Agent , if initialization is successful within xTicksToWait the OTA Agent state is eOTA_AgentState_Ready otherwise it is set as eOTA_AgentState_Stopped.
The console will show the following when the OTA Agent is successfully initialized:
Getting OTA StatisticsThe OTA demo/application task can get OTA statistics on the number of messages received, dropped, processed and queued. The statistics are reset when OTA_AgentInit is called again after initialization. The functions that are available to get these statistics are -
- OTA_GetPacketsReceived: Return the number of packets received.
- OTA_GetPacketsProcessed: Return the number of packets processed.
- OTA_GetPacketsQueued: Return the number of packets queued.
- OTA_GetPacketsDropped: Return the number of packets dropped.
The demo outputs these statistics while it’s receiving a download. See the image below for an example.
OTA Complete Callbacks
The OTA complete callback is called when the OTA job is finished and receives the activate, self-test or failure events during the update process. The OTA agent has either completed the update job or determined that we're in self test mode. If the image passed is successfully verified, the demo will try to activate the new image. This typically means we resetting the device to run the new firmware. In the case of windows simulator, this means closing the application and running the new executable.
If the update was rejected, the demo returns without doing anything and will wait for another job. If it reported that we should start test mode, it will perform user defined system checks to make sure the new firmware does the basic things it’s expected to do. For the purposes of this demo, the callback just sets the image state as accepted without any verification steps. In general, the accept function will vary depending on your platform.
The callback is defined as :
When the OTA file download is complete the console will output the following logs:
Activating the OTA Image
When the download is complete, stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when creating the job, then the new executable will be located in the same directory as the visual studio solution. For example:
Run that executable, and at the top of the console you should see that the version number has been updated. For example:
If the device has successfully rebooted, the status of the job in the IoT console will say completed.
TroubleshootingThe following sections contain information to help you troubleshoot issues with OTA updates.
This demo has shown how to securely send OTA Update jobs using the MQTT and HTTPS protocols. See the OTA API reference and porting guide sections to see how to integrate this library into a FreeRTOS project.