AWS_Get_files_from_S3_bucket.ipynb

Kernel: Python 3

AWS - List objects from S3 bucket

Give Feedback | Bug report

Tags: #aws #cloud #storage #S3bucket #operations #snippet #list #objects

Author: Maxime Jublou

Last update: 2023-11-20 (Created: 2021-09-20)

Description: This notebook retrieves objects from an Amazon Web Services (AWS) S3 bucket, allowing users to easily access their data stored in the cloud.

Input

Import libraries

import naas
try:
    import boto3
except:
    !pip install boto3 getpass4
    import boto3

Setup variables

Mandatory

• aws_access_key_id: This variable is used to store the AWS access key ID.

• aws_secret_access_key: This variable is used to store the AWS secret access key.

• bucket_name: The name of the S3 bucket from which you want to list the files.

# Mandatory
aws_access_key_id = naas.secret.get("AWS_ACCESS_KEY_ID") or "YOUR_AWS_ACCESS_KEY_ID"
aws_secret_access_key = naas.secret.get("AWS_SECRET_ACCESS_KEY") or "YOUR_AWS_SECRET_ACCESS_KEY"
bucket_name = "naas-example"

Model

Connect to AWS

s3_client = boto3.client(
    "s3",
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key
)

List objects in the S3 bucket

response = s3_client.list_objects_v2(Bucket=bucket_name)
print("Results:", len(response['Contents']))

Output

Display pre-signed URL

# Process the response
if 'Contents' in response:
    for file in response['Contents']:
        file_name = file['Key']
        print(file_name)