We need a python developer to write two scripts. which we call "subcomponents", for a Message Queue component in one of our systems,
The consumer picks up document from a collection performs the specified command (for now an SQL query) and then marks the document as completed and moves on.
- "Garbage collector"
The garbage collector checks for documents that are stuck in transit.
All the documents have a timestamp when they have been started, and if they are older than the specified timeout setting for a document, it will be put back into the queue.
The scripts should be,
- Configurable separately through a configuration file
- Contain a logging function
- Connect and insert/update data in MariaDB
- Connect and insert/update data in MongoDB
Functions related to the script,
- Easy start, stop and restart
- Exception handling
The work flow for the scripts is really straightforward,
- Connect to MongoDB
- Find and modify documents based...