Skip to content
forked from yma67/JAMSched

Pseudo Real-Time Task Scheduler + Coroutine Engine + Tiny RPC on MQTT

License

Notifications You must be signed in to change notification settings

citelab/JAMSched

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

JAMSched Build Status

Real Time Scheduler + Parallel Coroutine Engine for JAMScript C (Device) Side, with support of Remote Procedure Call and separate data-transfer path on Redis

Scheduling

Resources

Threads

  • 1 Kernel Level Thread for Real-Time/Interactive Jobs (jobs with deadline/time constraints)
  • n Kernel Level Thread for Batch Jobs (jobs without time constraints)

Coroutines

  • Capable of supporting shared-stack coroutines and stand-alone stack coroutines, scheduled with no difference
  • Automatic type deduction of C++ functions
  • Each task is allocated to as a coroutine, mapped to 1 kernel level thread
  • More than 1M tasks per GB
  • Switching time: ~30ns on Intel Core i7 10710U

Real Time Tasks

Description

  • Task with hard start/deadline
  • Scheduled on Real-Time Kernel Level Thread
  • All Real-Time tasks runs in a Single Kernel Thread

Scheduling Strategy

  • A static schedule of Real-Time tasks as a finite list of time intervals are used to schedule Hard Real-Time tasks
  • Each time interval has a RT-Task ID, beginning of an interval defines the start of the task with the ID
  • An ID equals to 0 defines a slot for Non-Real-Time tasks
  • Schedule of Real Time Tasks are downloaded for each cycle
  • Length of the cycle is defined by the end time of last interval in the finite list
  • Time Point defined in the interval are time points relative to the start of the cycle
  • Scheduler picks up Real Time task according to the ID associated with interval
  • 2 schedules with different distributions of Non-RealTime slots are provided, picked up according to some algorithm

Performance

Jitter
  • Jitter = (StartTimeOfRTTask - BeginOfTimeInterval), relative to the start of cycle
  • Jitter is within 20us, usually within 10us

Interactive Tasks

Description

  • Task with soft deadline
  • Scheduled on Real-Time Kernel Level Thread, possibly Non-Real-Time Kernel Level Thread if expired

Scheduling Strategy

  • If task has not passed its deadline, schedule it in Earliest Deadline First
  • Otherwise, if the programmer allows this task to degrade to a Batch Task, it will degrade.
  • Otherwise, it would be pushed into a Bounded LIFO Stack, and will be executed if there is no task in EDF queue
  • If the LIFO stack exceeded its capacity, it would pop out (cancel) the eariest-pushed task
  • Just like how we process our emails, assignments, and messages...

Batch Task

Description

  • Task without deadline
  • Scheduled on Non-Real-Time Kernel Level Thread
  • Capable of Parallel Execution

Scheduling Strategy

  • Only Batch Tasks could be executed in worker threads
  • FIFO with Work Steal (between Non-Real-Time Kernel Level Threads)
  • Work Steal is triggered if one Non-Real-Time Kernel Level Thread run out of tasks
  • Load Balancing: Tasks are distributed by the main scheduler to the Non-Real-Time Kernel Level Threads with least amout of tasks

Synchronization Primitives

Purpose

  • Synchronize at user-space, without blocking Kernel-Level thread
  • Blocking Kernel level threads could break the real-time constraints
  • Easier for Asynchronous Programming

Spin Mutex

Condition Variable

Mutex

Semaphore

Future

Timer as Timing Wheel

  • Powered by timeout.c: Tickless Hierarchical Timing Wheel

Precision

  • Ideally 30us, normally 50us

Remote Procedure Call (RPC)

This is a Toy-Level simple ADD-ON of RPC capability to the scheduler

  • Each RPC Invocation is matched to a Batch Task

Fault-torelance of this RPC depends on JAMScript JavaScript-Side

Protocol

  • JSON/Cbor on MQTT

Fault-torelance

  • Heartbeat monitoring
  • Multiple connections
  • Acknowledgement, retry and timeout of connection failure

Note

Please return by a pointer to memory allocated on heap by malloc/calloc to avoid memory leak.

Example

jamc::RIBScheduler ribScheduler(1024 * 256, "tcp://localhost:1883", "app-1", "dev-1");
ribScheduler.RegisterRPCall("DuplicateCString", strdup);
ribScheduler.RegisterRPCall("RPCFunctionJSync", [] (int a, int b) -> int {
    std::cout << "Sync Add of " << a << " + " << b << std::endl;
    return a + b;
});
ribScheduler.RegisterRPCall("RPCFunctionJAsync", [] (int a, int b) -> int {
    std::cout << "Async Subtract of " << a << " - " << b << std::endl;
    return a - b;
});
ribScheduler.RegisterRPCall("ConcatCString", [] (char *dest, const char *src) -> char* {
    printf("please return by a pointer to memory allocated on heap");
    return strdup(strcat(dest, src));
});

Named Local Invocation

Example of Registration

jamc::RIBScheduler ribScheduler(1024 * 256);
ribScheduler.RegisterLocalExecution("TestExec", [] (int a, int b) -> int {
    return a + b;
});

Example of Invocation

auto fu = ribScheduler.CreateLocalNamedInteractiveExecution<int>(
    // Interactive Task Attributes
    {false, 1024}, std::chrono::milliseconds(1000), std::chrono::microseconds(50),
    // Execution Name
    std::string("TestExec"),
    // Execution Parameters
    3, 4
);
// after some lines of code...
int val = fu.get();

Note

Also available for Batch Tasks

Exception

InvalidArgumentException

Although there is std::invalid_argument, to appreciate Prof. Gunter Mussbacher giving me an A- grade in ECSE223 - Model Based Programming, I would like to separately define the class to be InvalidArgumentException, which is the same InvalidArgumentException from ECSE223 course project.

General Workflow

Create an application

  • goto app folder, and make a folder for your application
  • add a CMakeLists.txt for configuration and link your favoriate machine learning libraries here
  • add a main.c/main.cpp, and other files you need
  • make sure your include your folder in app/CMakeLists.txt
  • if you are VSCode Users, goto VSCode Users - Reasons to Use

Create a module

  • goto src folder, and make a folder for your module
  • add a CMakeLists.txt for your library
  • add files you need
  • make sure your include your folder in src/CMakeLists.txt

Testing

CiteLab Coding Style (draft)

  • vscode formatter style: Visual Studio
  • avoid unnecessary memcpy/copy construct of large object
  • avoid unnecessary notify of threads on Condition Variable
  • avoid random comment outs without explanation
  • avoid unhandled memory allocations, like strdup
  • more to come...

Note to VSCode Users

Reasons to use

  • free yourselves from gdb, cmake... commands
  • looks cool

Build

Prereq

  • boost (>= 1.65, but preferred 1.73+)
  • paho-mqtt3a
  • hiredis
  • libevent
  • tcmalloc

Note

  • install CMake Tools, available from https://marketplace.visualstudio.com/items?itemName=ms-vscode.cmake-tools
  • configure, and build all project using the top right buttons
  • supported compiler: Clang 6.0.0, Clang 9.0.0, Clang 11.0.0, GCC 9.3.0, GCC 10.1.0
  • on macos, use cmake -D CMAKE_C_COMPILER=/usr/local/opt/llvm/bin/clang -D CMAKE_CXX_COMPILER=/usr/local/opt/llvm/bin/clang++ ../

Debug

  • Goto Run tab, and choose which app you would like to add/debug
  • To debug testcases, run VSCode task "clang-9 - test-debug"

Debug Your App

  • Add the following "chunk" in launch.json
  • <name>: name of your app folder
  • <exec>: name of your app executable
{
    "name": "clang-9 - <name>",
    "type": "cppdbg",
    "request": "launch",
    "program": "${workspaceFolder}/build/app/<name>/<exec>",
    "args": [],
    "stopAtEntry": false,
    "cwd": "${workspaceFolder}/build/app/<name>",
    "environment": [],
    "externalConsole": false,
    "MIMode": "lldb",
    "setupCommands": [
        {
            "description": "format lldbprint",
            "text": "-enable-pretty-printing",
            "ignoreFailures": true
        }
    ],
    "preLaunchTask": "build project",
    "miDebuggerPath": "/usr/bin/lldb"
}

About

Pseudo Real-Time Task Scheduler + Coroutine Engine + Tiny RPC on MQTT

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 86.0%
  • C 11.6%
  • CMake 2.2%
  • Erlang 0.2%