SEO Quotient (www.SEOQ.com) is a website that seeks to answer the big question ... why does his website rank higher than mine?
To do this, I need to crawl a website looking for specific information and then bring that back and present in a meaningful way. I would be looking for things like keywords in the title tag, what CMS is being used, how fast or slow is the site, does it pass Google's mobile friendly test, etc.
The front-end of the system was built in WordPress, PHP, Bootstrap, HTML, CSS and the back-end of the system was built in Python, Django, PHP, AngularJS, etc. Some folks who have done work like this in the past thing that PERL is the method to use. I'm sure there are plenty of ways to crawl and get the data but, ultimately, it needs to live in our Python/Django/MySQL back-end. Probably, you will also need to know Git to help contribute to the repository. Knowing how to get data to/from APIs and perhaps how to build APIs...