I need someone to work American/European/African hours. Ideal candidate will be based in Latin America or West / Southern Africa as these are low cost-of-living reasons.
My company's technical team have three main tasks to perform: spiders, parsers and reports. I'm looking for full time help with reports, which is a web development role, but also backup help on spiders and parsers, which are more python roles.
The team uses Heroku, Amazon S3, Git, PostgreSQL, Python, django, highcharts, CircleCI, Trello, Slack...
- A spider is a piece of code that fetches data from the Internet and stores it in raw format. That's it. A spider does not parse any data and does not render that data in any way.
- The spiders fetch data from the Internet either through a documented API or simulating a user interacting with a website through HTTP requests. For both interacting with APIs and simulating user interaction, we recommend the use of requests.
- A spider runs on a pre-defined schedule: daily, weekly, monthly, depends on how often that data is updated.
- A parser is a piece of code that parses a document that contains data into a format that's ready to be written to our database. That's it. Parsers do not fetch data nor render it to the frontend in any way.
- Parsers use libraries like BeautifulSoup to parse data from a HTML/XML file, or jsonstandard library from a JSON document.
- A report is a piece of software that fetches data from the database and displays it in a way that allows a human being from extracting meaningful insights from it. That's it. For our context, the most important kind of report is a chart. A chart is a piece of software that exposes the many values of an information in a timeframe.
- A report does not fetch data, nor parses any data. It obtains the data in a format ready to be rendered from the database. It only worries about presentational issues, like: should I print this number with a thousand separator? Should I print this chart in red or blue?