Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
SMILA/Specifications/Processing Message Resequencer/Full Resequencer
< SMILA | Specifications
not continued
this concept is not developed further ATM for lack of need and b/c it is basically the same as the Smart Resequencer. ideas there mostly apply here too.
this concept is not developed further ATM for lack of need and b/c it is basically the same as the Smart Resequencer. ideas there mostly apply here too.
Full Resequencer (FRS)
Synopsis: The Resequencer will update the processing target in the exact order as the crawler or agent adds PRs to connectivity.
The processing will be like so:
- the router will feed Q1 with PRs.
For the resequencer to know the order, a new meta info needs to be added -- the sequence number (SN). it must be generated by the agent or by the agent controller - the processing piplines are as normal, but:
- w/o the step of calling the processing target
- they add the result to a new queue, Q2
- the Resequencer will listen on Q2 and picks up all PRs
- starting with the first record: feed consecutive chunks of PRs to the processing target
- wait for PRs only a max. amount of time (timeout)
PRO
- no processing target can ask for more and correct result is always possible
- it is possible to add a note into the index for records ending up in the DLQ, ie. record was not indexed due to processing error.
CON
- setup is a more complex
- added overhead due to more steps in the processing chain
- change to agents or agent controller
- overkill, b/c there is no need to resequence all PRs, only those with same ID
- lost PRs will cause the FRS to delay all following PRs up to the timeout