How to download the files from AWS S3 into local system using java
HOW TO DOWNLOAD THE FILES FROM AWS S3 INTO LOCAL SYSTEM USING JAVA
- First, create the object of AWSCredentials and pass the aws_access_key_id and aws_secret_access_key as parameters.
- Next, create the s3client object for connecting to the aws s3 bucket.
- For creating a connection we can pass the AWSCredentials object as a parameter.
- Next, create the TransferManager object using s3client, TransferManager provides asynchronous management for uploads and downloads between your application.
- Using the s3client.listObjects get all list of object in aws s3 bucket.
- Using Iteratoriterate the al objects. using the S3ObjectSummary object to get all information of the object(file name.lastmodified date).
- Finally, close the connection of TransferManager object otherwise it's running continuously.
- Using the shutdownNow(); the method to close the connection.
import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.ListObjectsRequest;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.amazonaws.services.s3.transfer.TransferManager;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintStream;
import java.util.Iterator;
import java.util.List;
/**
*
* @author Janardhan Randhi
* Date : Feb 4 2019
* Description : This class Downlaod the files from AWS S3 bucket into local system.
*
*/
public class AWSFiledownload
{
public AWSFiledownload() {}
public static void main(String[] args)
throws AmazonClientException, InterruptedException, FileNotFoundException, IOException
{
//create a client connection based on credentials
AWSCredentials credentials = new BasicAWSCredentials("***************key", "*****************pwd");
AmazonS3 s3client = new AmazonS3Client(credentials);
// create bucket - name must be unique for all S3 users also small letter required
String bucketName = "BucketName";
String key = "test_folder/"; //folder name
long startTime = System.currentTimeMillis();
TransferManager xfer_mgr = new TransferManager(s3client);
try
{
ObjectListing objectListing = s3client.listObjects(new ListObjectsRequest()
.withBucketName(bucketName)
.withPrefix(key));
Iterator localIterator = objectListing.getObjectSummaries().iterator();
if (localIterator.hasNext())
{
S3ObjectSummary objectSummary = (S3ObjectSummary)localIterator.next();
System.out.println(objectSummary.getKey());
S3Object object = s3client.getObject(new GetObjectRequest(bucketName, objectSummary.getKey()));
InputStream reader = new BufferedInputStream(object.getObjectContent());
File file = new File("C:\\Users\\Administrator\\Desktop\\AWS_Download\\" + objectSummary.getKey());
OutputStream writer = new BufferedOutputStream(new FileOutputStream(file));
int read = -1;
while ((read = reader.read()) != -1) {
writer.write(read);
}
writer.flush();
writer.close();
reader.close();
s3client.deleteObject(bucketName, objectSummary.getKey());
}
}
catch (AmazonServiceException e) {
System.err.println(e.getMessage());
}
finally
{
xfer_mgr.shutdownNow();
}
System.out.println("All File are downloaded successfully from " + key + " Directory and Deleted in " + key + "Directory");
long endTime = System.currentTimeMillis();
System.out.println("Time taken to download a multiple files : " + (endTime - startTime) + " milliseconds");
}
}
I appreciate several from the Information which has been composed, and especially the remarks posted I will visit once more. Hotspot Shield Elite APK
ReplyDeleteWts the use of transfer manager object here. You are just opening and closing it
ReplyDeleteI don t have the time at the moment to fully read your site but I have bookmarked it and also add your RSS feeds. I will be back in a day or two. thanks for a great site. https://www.deviantart.com/lancekr
ReplyDelete