irdest-corrupted issueshttps://git.irde.st/we/irdest-corrupted/-/issues2021-08-07T06:44:05Zhttps://git.irde.st/we/irdest-corrupted/-/issues/12Alexandria internals refactor2021-08-07T06:44:05ZKatharina FeyAlexandria internals refactorAs part of the NLnet grant to work on Irdest I allocated a milestone to get Alexandria to the `1.0` finish line. It's an ambitious target, seeing as the database is not yet [5 years old](https://twitter.com/sadisticsystems/status/942334...As part of the NLnet grant to work on Irdest I allocated a milestone to get Alexandria to the `1.0` finish line. It's an ambitious target, seeing as the database is not yet [5 years old](https://twitter.com/sadisticsystems/status/942334089578680320) (:wink:) but it will be important to be able to rely on storage in the future, and hopefully this will also make the `irdest-core/store` module obsolete!
Following is an outline of work that needs to happen. This issue will track development of these features.
## Alexandria: re-scope
As part of qaul.net Alexandria needed to handle many types of payloads. This is no longer the case! As such `blob` support needs to be removed. Furthermore, we will need to support large records, and possibly records where a key-value store approach is not ideal. Following is an outline of the **new alexandria scope**
* Encrypted data & metadata
* Identity concealing. Hide user IDs. Is this zero knowledge?
* Records based. Path based. Two types of records
* Key-value stores
* Table records with shared schemas
* Search tags. Encrypted tag cache. Per-user.
* Encrypted schemas.
* Insert data easily without manual diff creation
* Use diffs for transactions internally.
* Generational (epoch based) garbage collection
* Sync data dynamically.
* Break large records up into chunks that can be streamed from disk.
### Things that are stored
* Records! Chunk based.
* User sessions, with hidden user IDs. How?
* Per-user tag-cache
* Per-user private key store
* Per-user public key store (not encrypted - only user ID is unknown)
### The threat model
Hiding information from an adversarial user is impossible. Alexandria can not hide its own memory from the root user. Data MUST exist in un-encrypted form at some point or another.
Watching alexandria in memory WILL reveal information about data present in the database.
Protect data synced to disk. Confiscated devices, broken disk encryption, stolen files. Don't reveal user identity or information payloads to on-disk attacks. If the cops steal your phone, can they indict you?
## Internal overview
Following is a graph of data available via APIs, internal components and their on-disk counterparts.
![alexandria-overview](/uploads/efd08ca912588d6fba0a0dde8142690a/alexandria-overview.png)
## Outline
Following is an implementation outline
- [x] Investigate whether the `Encrypted` abstraction introduced in alexandria `v0.1` is still relevant
- [ ] Take a closer look at every component that exists in the code currently and map its inputs and outputs
- Where are modules being used?
- Are there unused modules in the library?
- [ ] Build a solid user session management module
- [ ] Allow user registration, login, logout, destruction
- [ ] Sync user sessions & keys to disk
- [ ] Tag & schema cache
- [ ] Encrypt caches with root user key
- [ ] Provide an API for other modules to query the caches
- [ ] Load information from disk that is not present in the cache (how?)
- [ ] Records store
- [ ] Investigate whether to split the store implementation between KV and tables or keep them in the same stare
- [ ] Create an API for manipulating tables & diffing KV records
- [ ] Create transactional deltas for the store
- [ ] Apply deltas atomically
Add and remove items from the above outline as needed.https://git.irde.st/we/irdest-corrupted/-/issues/9Reduce the amount of warnings2021-05-13T15:53:17ZKatharina FeyReduce the amount of warningsCurrently we have a lot of `unused code` warnings around the codebase. Some of them will go away naturally as we connect more pieces of the puzzle together, but others can be eliminated by either
1. Deleting code that is no longer need...Currently we have a lot of `unused code` warnings around the codebase. Some of them will go away naturally as we connect more pieces of the puzzle together, but others can be eliminated by either
1. Deleting code that is no longer needed
2. Making API endpoints `pub` that are currently only `pub(crate)` or similar
3. Gating testing-only API endpoints behind `#[cfg(feature = "testing")]` or similarhttps://git.irde.st/we/irdest-corrupted/-/issues/8Rename the project2021-08-03T09:03:41ZKatharina FeyRename the projectWe have lots of references to the old name in the repository. Most of them have been taken care of (in code anyway), but a lot of documentation still mentions `qaul`.
This is hard to fix automatically, especially because `qaul` and `ir...We have lots of references to the old name in the repository. Most of them have been taken care of (in code anyway), but a lot of documentation still mentions `qaul`.
This is hard to fix automatically, especially because `qaul` and `irdest` don't map perfectly onto each other gramatically. Thus we should do this process by hand. Any MRs to fix spelling, naming, etc are welcome. Please reference this issue in your MRs!https://git.irde.st/we/irdest-corrupted/-/issues/6Example service: chat2021-05-17T21:58:50ZKatharina FeyExample service: chatAs part of testing different routing approaches in Irdest, and to showcase the capabilities of the service API/ model we want to create a simple chat service.
The idea behind this service isn't to create a comprehensive chat application...As part of testing different routing approaches in Irdest, and to showcase the capabilities of the service API/ model we want to create a simple chat service.
The idea behind this service isn't to create a comprehensive chat application, but to create a simple example as to how people can communicate via Irdest, _and_ to create a simple entry point for newcomers to talk to people already using Irdest.
The requirements and development state will be tracked in this issue. Please refer to it in any change set that addresses the chat service.
## Design outline
`irdest-core` provides a service some encrypted storage, namespaced by user. This means that the service itself doesn't have to manage storage, and can do so via the irdest API.
Apart from that there are two main areas that this service needs to focus on: accepting user text messages and mapping them to irdest-core messages, and managing the state of chat rooms.
## Roadmap
- [x] Improve irdest-core storage API (#7)
- [x] Map service API to irdest-sdk (-)
- [ ] Add persistent storage backend to alexandria (#10)
- [ ] Create chat service API & SDK library (-)
- [ ] Room creation/ update/ deletion logic (#11)Katharina FeyKatharina Feyhttps://git.irde.st/we/irdest-corrupted/-/issues/5Binary distributions2021-04-04T15:16:06ZKatharina FeyBinary distributionsThis is a tracking issue for binary client distribution. As of 2021-03-02 there are three clients we may want to publish:
- [irdest-hub](clients/irdest-hub)
- [irpc-client](clients/irpc-client)
- [irdest-gtk](clients/irdest-gtk)
A ful...This is a tracking issue for binary client distribution. As of 2021-03-02 there are three clients we may want to publish:
- [irdest-hub](clients/irdest-hub)
- [irpc-client](clients/irpc-client)
- [irdest-gtk](clients/irdest-gtk)
A full irdest installation always involves `irdest-hub`, with some mechanism to make the daemon run in the background (systemd units, openrc, ...?), and one of the two clients. For most users the GTK client will be preferred.
## Tasks
- [ ] Build fully static Rust binaries (output an `.appimage` for each binary?)
- [ ] Provide release tarballs, including vendored dependencies for distribution packagers
- [ ] Create a small installer for full userspace installations on legacy Linux distributions
- [ ] What's the upgrade story? How does irdest learn of new versions?https://git.irde.st/we/irdest-corrupted/-/issues/4Alexandria API improvement2021-08-07T06:44:05ZKatharina FeyAlexandria API improvementThe `alexandria` API is causing some boilerplate in the `irdest-core` storage code. Some things are missing to improve the situation.
- [ ] Rename `.batch(...)` to `.batch_insert(...)`
- [ ] Add `.batch_update(...)` next to `.update(.....The `alexandria` API is causing some boilerplate in the `irdest-core` storage code. Some things are missing to improve the situation.
- [ ] Rename `.batch(...)` to `.batch_insert(...)`
- [ ] Add `.batch_update(...)` next to `.update(...)`
- [ ] Add `.upsert(...)` and `.batch_upsert(...)` as combinations of the above functions
- [ ] Provide a derive macro for `ToDiff`
- [ ] Create `alexandria-derive` (or `alexandria-macros`?) crate to let `ircore-types` depend on it, without having to depend on the whole database.Katharina FeyKatharina Feyhttps://git.irde.st/we/irdest-corrupted/-/issues/3Network integration tests2021-02-26T19:25:54ZKatharina FeyNetwork integration testsCurrently testing is only done locally, with in-memory channels between virtual components. This is fine for small-scale tests, but doesn't replicate a lot of the issues that can occur between nodes.
Following is an outline of features...Currently testing is only done locally, with in-memory channels between virtual components. This is fine for small-scale tests, but doesn't replicate a lot of the issues that can occur between nodes.
Following is an outline of features that should be supported by the test harness:
- [ ] Create a test network with 2 VMs that are connected to each other
- [ ] `netmod` driver implementations (TCP/ UDP/ See #2)
- [ ] High-level message API functions
- [ ] Create a "chain" network with 3 VMs
- [ ] Basic `netmod` driver test
- [ ] User discovery propagation
- [ ] Message flood benchmark
- [ ] Create a "ring" network with 3 VMs
- [ ] Basic `netmod` driver test
- [ ] User discovery propagation (& de-duplication!)
- [ ] General message flood de-duplicationMilan Milan https://git.irde.st/we/irdest-corrupted/-/issues/1Implement basic Android app UI and proof of concept chat2021-02-26T15:55:06ZKatharina FeyImplement basic Android app UI and proof of concept chatAs part of the 2021 Google Summer of Code we are looking for someone to refactor the existing Android code to be more in-line with modern Android development, and build a set of screens that can later on be expanded to support various fu...As part of the 2021 Google Summer of Code we are looking for someone to refactor the existing Android code to be more in-line with modern Android development, and build a set of screens that can later on be expanded to support various functions provided by qaul services.
The following basic screens are required
* User sign-in, registration, and management (password/ avatar change, etc)
* Adaptable bottom-navigation with four configurable shortcuts to services
* Swipe-able sidebar with additional service functions
The backend logic required by the app is to launch the `qrpc-broker`, and corresponding services (some utilities are available via the `android-support` crate). Java/ Kotlin bindings to various `sdk` crates (`libqaul-sdk`, `qaulchat-sdk`, etc) need to be developed.
If you are interested in working on this project, please get in touch with us!