We want to have all the health insurance policies scraped from http://privatehealth.gov.au/dynamic/compare.aspx
We think the easiest way to do this is to
- Systematically click through to the table view, select all the policies on a page (e.g. https://www.dropbox.com/s/g6tik3oqo81h431/ss-hic.png
- Then click on the compare policies button down the bottom, and you will be taken to a side by side view (e.g. https://www.dropbox.com/s/sxxintas12rie7m/Compare%20health%20insurance%20policies%20-%20product%20comparison.png
- Then scrape this data into the database. You can see the structure we want (as well as one example policy with all data already placed) here: https://www.dropbox.com/s/ji496higavropq9/private-health-db.xlsx
Please let us know if you have any questions.