RClone copy files with big metadata from local Ubuntu with wifi to S3 AWS

Posted last month

Worldwide

Summary

teach me how to use RClone I need to learn RClone how to copy many files with meta data from my local ubuntu computer with wifi connection to internet to S3 bucket AWS https://rclone.org/ https://github.com/rclone/rclone difficulty S3 has limited to 2KB size of metadata for file scope use python to simulate 1000 files in several folders myfolder1, my folder2, etc with random size from 1 MB to 30 MB random metadata size from 1 KB to 15 KB different type like txt png docs objects etc automatically copy them to S3 bucket make sure to have error messages saved to aws cloudwatch when copy error happen or metadata coping issues or hash is not the same (check hash of original file vs copy of S3) millstones 1 millstone use ec2 and python simulate files on ec2 using python use python to copy files to S3 2 millstone use ec2 and python and Rclone simulate files on ec2 using python install RClone on this ec2 use Rclone to copy files to S3 3 millstone use local ubuntu linux with home wifi connection to internet and python and Rclone simulate files on ec2 using python install RClone on this ubuntu use Rclone to copy files to S3 local home ubuntu with wifi : 1) instal python 2) instal aws cli 3) install rclone 4) synch rclone with aws S3 5 ) use python to simulate single files (metadata and data in one file ) with different folders and different size (1 or 30 MB) and different metadata size (1 to 7 KB) 6) copy files to S3 by using rclone but send to AWS cloud watch or error messages when copy if failed since metadata is too big or copy done correctly for each file you provide instructional material how to do step by step docs file with instructions especially screen copy for each step Acceptance criteria i can do it by myself on my aws account and my home ubuntu. as result i will be able to reproduce all by myself I do not provide access to aws or home ubuntu for development 2 days to do

  • $30.00

    Fixed-price
  • Intermediate
    Experience Level
  • Remote Job
  • One-time project
    Project Type
Skills and Expertise
Mandatory skills
Amazon Web Services
Python
Activity on this job
  • Proposals:5 to 10
  • Last viewed by client:3 weeks ago
  • Hires:
    3
  • Interviewing:
    2
  • Invites sent:
    8
  • Unanswered invites:
    5
About the client
Member since Oct 4, 2018
  • Canada
    Toronto6:03 PM
  • $2.6K total spent
    126 hires, 8 active
  • Tech & IT
    Individual client

Explore similar jobs on Upwork

Microsoft Azure
Automated Monitoring
Configuration Management
Infrastructure Management
Google Cloud Platform
Amazon Web Services
Cloud Computing
Cloudflare
Senior DevOps Engineer with InfoSec VaultHourly‐ Posted 4 weeks ago
Azure DevOps Server
DevOps
Kubernetes
HashiCorp Vault

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo