Task pool library.

A task pool is an adaptive set of threads that can grow and shrink to execute a user-provided callback through a user-defined job that can be scheduled with a non-blocking call. The design principles are to minimize the memory footprint while allowing non-blocking execution. The adaptive behavior allows serving jobs in a timely manner, without allocating system resources for the entire duration of the application, or causing recurring allocation/deallocation patterns. The user does not have to worry about synchronization and thread management other than within its own application.

Features of this library include:

Non-blocking API functions to schedule immediate and deferred jobs.
Ability to create statically and dynamically allocated jobs.
Scalable performance and footprint. The configuration settings allow this library to be tailored to a system's resources.
Customizable caching for low memory overhead when creating jobs dynamically.

This library uses a user-specified set of threads to process jobs specified by the user as a callback and a context. Overall, the task pool hinges on two main data structures: the task pool Job (IotTaskPoolJob_t) and the task pool itself (IotTaskPool_t). A task pool job carries the information about the user callback and context, one flag to track the status and a link structure for moving the job in and out of the dispatch queue and cache. User can create two types of jobs: static and recyclable. Static jobs are intended for users that know exactly how many jobs they will schedule (e.g. see Defender scenario above) or for embedding in other data structures. Static jobs need no destruction, and creation simply sets the user callback and context. Recyclable jobs are intended for scenario where user cannot know ahead of time how many jobs she will need. Recyclable jobs are dynamically allocated, and can be either destroyed after use or recycled. If jobs are recycled they are maintained in a cache (IotTaskPoolCache_t) owned by the task pool itself, and re-used when user wants to create more recyclable jobs. The task pool cache has a compile time limit, and can be pre-populated with recyclable jobs by simply creating recyclable jobs and recycling them, in an effort to limit memory allocations at run-time. This is handy for scenarios where user is aware of the steady state requirements for his application. User jobs are queued through a non-blocking call and processed asynchronously in the order they are received.

Task pool API functions Provides a set of functions to queue an asynchronous operation on the Dispatch Queue. API functions are non-blocking and return after successfully queuing an operation.
Worker threads in the task pool are woken up when operations arrive in the dispatch queue. These threads remove operations from the dispatch queue in FIFO order and execute the user-provided callback. After executing the user callback, the task pool threads try and execute any remaining jobs in the dispatch queue. The task pool tries and execute a user job as soon as it is received and if there are no threads available it will try and create one, up to the maximum number of allowed threads. The user can specificy the minimum and maximum number of threads allowed when creating the task pool.
The user can try and cancel a job after the task has been scheduled. Cancellation is only allowed before the task enters execution.

Threads are created with Iot_CreateDetachedThread. Because the platform layer may be re-implemented across systems, threads will be allocated for the task pool library on-the-go on some systems, while other systems may use an always-allocated thread pool.