We need an experienced developer to write code which can do this:
Suppose that we have a database with URLs from the internet.
Dataset : url, crawled, validated, rating
0. If dataset is empty, exit. Else, fetch a entry from db.
1. If entry has attribute "validated:yes" goto 0, else continue.
2. Fetch the site using URL.
3. Process the site's content using pattern recognition(Explained below)
4. If successful, mark "validated:yes, confidence:<<output from ANN>>, else mark "validated:error".
5. Goto 0.
Whenever the parser finds word "free", "trial", "free trial", "basic", "introductory", "$0" or similar terms on the page, that page should be given a good confidence rating.
Choice of programming language is left open.
If anything is unclear, please let me know.