So I have a Wordpress plugin, that pulls articles from a news website through their own API (I will let you know which news website it is and will also provide the link to their API documentation via private message, so you can look everything up).
There are 3 main issues, that would be needed to be resolved.
1) CORRECT THE ORDER OF PARSED ARTICLES/POSTS
The posts are in the wrong order. Example: 3 posts were posted during the day on the news website, 1 at 2pm, 1 at 3pm and 1 at 4pm. Now when the cronjob runs at 6pm, it will get the post from 4pm first, then the one from 3pm and the 2pm at last. Of course the reverse and therefore original order would be correct.
2) MAKE THE POSTS IMPORT FASTER
The intervall, in which the cronjob runs, is too slow. It runs every 3 minutes now for each of the news departments (will provide link to them via private message, as well) . Because there are hundreds of them and the plugin/cronjob only checks 1 department at a time, it can take up to around 7/8 hours until a department is being checked for new posts again. Ideally, I would like to have new posts imported 5-10 minutes after they were originally published, if not faster. (Hidden math question, so I know you actually read all of this: Please answer what twelve plus nine equals to and put the answer at the beginning of your proposal) I am aware, that this would probably need more server capacity than I am currently using, but maybe there is even a workaround for that. It would be much appreciated, if there would be a solution, which would keep my monthly server costs in an acceptable range.
3) GET ALL(!) POSTS
This is by far most important issue and the reason I put this job in the "Testing"-category: Some posts are just not being parsed. I don't know why, but some posts are just not being parsed. Also, I could not find a pattern (maybe you can find one?). I would say, that like 80% of posts are being parsed from the news website, but the rest is just not. Sometimes all posts on a specific day are imported, then again only 3 out of 5 or so. It is so strange.