Web Scraping Specialist for Municipal Government Data Collection

We are seeking an experienced Web Scraping Specialist to build and maintain robust, automated pipelines for collecting public data and video content from various municipal government websites across the United States. This is a substantial, highly structured project focused on systematically gathering public meeting records, agendas, minutes, and media files, then organizing them into a strict local directory structure. Key Responsibilities & Technical Requirements: Targeted Scraping: Automate the extraction of meeting agendas, minutes, and video recordings from US municipal websites. Format Standardization: Download PDFs directly. If agendas/minutes are only available in HTML format, the scraper must automatically convert them to PDF. Media Downloader: Successfully extract and download large government video files. Strict Folder Architecture: All downloaded assets must be saved to the local file system using the following exact directory structure: downloads/[LOCATION]/[MEETING_TYPE]/[DATE]/[VIDEO and PDF/s] Clean Deliverables: Deliver a clean, production-ready Python project complete with a standard requirements.txt file for easy dependency management. Project Scope & Logistics Scope: Medium project (highly focused, well-defined requirements) Duration: 1 to 3 months Commitment: Contract-based (No plans for full-time hire at the moment) Preferred Skills Advanced Python (BeautifulSoup, Scrapy, Selenium, or Playwright) Experience bypassing bot-detection/JavaScript-heavy municipal portals HTML-to-PDF conversion libraries (e.g., Weasyprint, pdfkit) Handling large file streams/video downloads in Python Clean code practices and structured file management