This document provides instructions on how to download data from Mobito.
For each use case, two examples are provided:
Using AWS CLI
Using Python’s Boto3 library
Installation
Before executing any commands, ensure you have installed your chosen tool or library. Use the links below for installation:
Downloading files
Using the AWS CLI
To authenticate and download data from S3, follow these steps:
Configure AWS CLI: Run the following command in your terminal and enter your access key, secret key, default region, and output format:
aws configure
Download Files: After configuring AWS CLI, use the following command to download all files from the specified S3 path:
aws s3 sync s3://example-data-bucket/folder-1/ /local/path/to/save/files
Replace “example-data-bucket/folder-1/” with the S3 path provided by Mobito in the AWS S3 Bucket Access Details document
Replace "/local/path/to/save/files" with the local directory where you want to save the downloaded files.
This command uses the aws s3 sync to synchronize the specified S3 bucket path with your local directory, ensuring all files are downloaded efficiently.
Using Boto3 for python
Before running the script, ensure Boto3 is installed. If not, install it using:
pip install boto3
Once installed, you can run the following script:
1 import boto3
2 from botocore.exceptions import NoCredentialsError
3
4 def download_files_from_s3(access_key, secret_key, bucket_name, s3_prefix, local_directory):
5 # Create an S3 client
6 s3 = boto3.client(
7 's3',
8 aws_access_key_id=access_key,
9 aws_secret_access_key=secret_key
10 )
11
12 try:
13 # List objects in the specified S3 path
14 response = s3.list_objects_v2(Bucket=bucket_name, Prefix=s3_prefix)
15
16 # Download each file to the local directory
17 for obj in response.get('Contents', []):
18 key = obj['Key']
19 local_file_path = f"{local_directory}/{key.split('/')[-1]}"
20
21 s3.download_file(bucket_name, key, local_file_path)
22 print(f"Downloaded: {key} to {local_file_path}")
23
24 except NoCredentialsError:
25 print("Credentials not available or not valid.")
26 except Exception as e:
27 print(f"An error occurred: {e}")
28
29 if __name__ == "__main__":
30 # Replace with your AWS credentials and other details
31 aws_access_key = "<ACCESS-KEY-ID>"
32 aws_secret_key = "<SECRET-KEY>"
33 bucket_name = "example-data-bucket"
34 s3_prefix = "folder-1"
35 local_directory = "/local/path/to/save/files"
36
37 # Call the function to download files
38 download_files_from_s3(aws_access_key, aws_secret_key, bucket_name, s3_prefix, local_directory)
39