Milestones 1 & 4: async overhaul & state persistence
This milestone consists of a few sub-tasks:
-
A new client API encoding, based on frames and discrete command chunks -
A new command line interface for ratctl
and map it onto the new client API -
Processing one chunk at a time. Each chunk produces at least one ERIS block worth of data. The chunk size should be chosen based on target chunk size and total message size. For example, a message producing 3 1K blocks doesn't need to be transported in a 8M chunk. Streaming a 10GB movie might warrant the 8M chunks however. - Encrypt the chunk with ChaCha20, the shared key derived from a diffie-hellman between the sender and recipient (public key address from recipient, and private key from keystore for the sender).
- Create a block from the chunk and stream it into the journal
- Slice the message into a set of carrier frames
- Dispatch the frames along the most effective endpoint driver/ route
-
Investigate spawn strategies and default configurations. Following are a few questions: - Can we combine async "bundles" to allow running on truly single threaded (i.e. embedded) platforms?
- Can we "split" an async environment to run across two cores, while sharing data structures between them?
- How do we spawn the worker threads for large message transfers? How large does a message have to be for the request to be handled by a dedicated thread? Do we have a thread pool regardless? How do we handle varying sized of available threads to make sure resources are not underserved?
- Etc... :S
-
Port the existing http stack to a tokio-backed framework (warp?)
Edited by Katharina Fey