This repository contains a class called StorageType() that allows easy switching between Azure Blob Storage and S3 buckets (or vice versa) using an input string of 's3' or 'blob'.
Assuming you already know how to migrate your files from one cloud to another (if not check rclone), you can use StorageType() to list, upload, download, and remove files and directories within either of the two storage systems.
Or simply use the class that creates the S3 client (S3DirectoryClient) or the blob client (BlobDirectoryClient) individually from storage/s3_storage.py or storage/blob_storage.py, respectively.
- Clone this repo.
git clone https://github.com/xxotto/S3-Blob-switch
cd S3-Blob-switch- Install
requirements.txt.
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
-
Add your own S3 Buckets and Blob Storage keys to
.envfile. -
Review and modify the
main.pyfile and add the methods to your client as desired. -
Run the
main.pycode with your changes and new methods, and if you want to switch storage types, modify the input string in theStorageType()class.
-
Look for functions or methods from the azure-storage or boto3 libraries to avoid unnecessary iteration over all files.
-
Explore alternatives to avoid manual handling of paths.