Reddit Data Collection: r/adjusters Complete Thread and Comment Scrape

Posted 4 hours ago

Worldwide

Summary

I need a Python developer to collect complete thread and comment data from Reddit's r/adjusters subreddit for a research project. SCOPE All threads and all comments — including nested replies at every depth — from the trailing 24 months. Nested replies are essential. I need the full conversation structure, not just top-level posts. OUTPUT JSON or CSV — tell me which you prefer and why. Each record should include: thread title, thread URL, author, timestamp, comment text, comment depth, parent ID, upvote count, and any metadata you think adds value. Deleted and removed comments should be handled explicitly rather than silently dropped. DELIVERABLES 1. A paid test run on one week of data so I can validate the format before full collection 2. The complete dataset once the format is approved 3. A short note on your method — library used, rate-limit handling, deduplication approach PLEASE ANSWER THIS IN YOUR PROPOSAL: Reddit's API caps listing retrieval at roughly 1,000 items per sort order. How will you get complete coverage of a 24-month window despite that cap? TERMS Reddit's official API only. Compliant collection — no HTML crawling. If you see a better approach than what I've described, propose it rather than following my spec silently. WORKING STYLE I respond quickly and I'd rather hear about a problem early than receive a clean-looking file that's quietly incomplete. Ask questions.

  • $400.00

    Fixed-price
  • Intermediate
    Experience Level
  • Remote Job
  • One-time project
    Project Type
Skills and Expertise
Mandatory skills
Data Extraction
Data Scraping
Activity on this job
  • Proposals:20 to 50
  • Last viewed by client:3 hours ago
  • Interviewing:
    3
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since Feb 19, 2017
  • USA
    Phoenix4:02 PM
  • $175 total spent
    1 hire, 0 active
  • Legal
    Mid-sized company (10-99 people)

Explore similar jobs on Upwork

Local Lead GenerationHourly‐ Posted 3 weeks ago
Web Scraping
Data Scraping
Data Extraction
Lead Generation
Data Entry
Data Mining
Data Collection
Data Processing
Web Scraping Framework
Web Crawler Framework
Web Scraping Software
Web Scraping Plugin
Web API
Search Tool
Search Engine
Microsoft Word
Data Entry
Administrative Support
Microsoft Excel

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo