As a Web Data Extraction Specialist, you’ll be working closely with the publishing and editorial team to design and create reusable tools to automate data “scraping” from a variety of websites and APIs. You will be putting your skills to good use - helping consumers make informed decisions based on the data you’ve scraped.
You’ll work on data extraction across a number of our niches, including topics such as credit cards, shopping, home loans, travel and telco. Your work will vary day-to-day, but you will have a primary responsibility of automating and managing our data scraping infrastructure.
Who you are:
We are looking for a data specialist with experience in large scale web data scraping, processing and normalisation. You should already have the programming skills required build these tools using libraries you are familiar with. While we prefer tools built in PHP, Python or Node.js, we will not be limiting your choice of language and libraries if you’re able to achieve the same outcome.
To be successful in this role, you need to have a passion for helping consumers make decisions by building large datasets for analysis and comparison. You've researched who we are and are excited by our mission and want to use your skills to support our content.
- Create reusable tools and processes to gather acquire or “scrape” both structured and unstructured data from a variety of data sources
- Work closely with our publishing team to determine the data required and develop processes to normalise the data accordingly
- Automate the processes to enable scheduled data extraction and normalisation tasks
- Develop and maintain database infrastructure to store and manage large datasets
- Construct database queries and export the data into a spreadsheet format for further analysis and processing