Upload .wav File to Amazon S3 Bucket

Uploading files to a bucket in Amazon Lightsail using multipart upload

Terminal updated: Jan x, 2022

Multipart upload allows you to upload a single file to your saucepan as a ready of parts. Each part is a contiguous portion of the file'south data. Y'all tin can upload these file parts independently and in any order. If transmission of any role fails, you tin retransmit that part without affecting other parts. After all parts of your file are uploaded, Amazon S3 assembles these parts and creates the object in your bucket in Amazon Lightsail. In general, when your object size reaches 100 MB, y'all should consider using multipart uploads instead of uploading the object in a unmarried operation. For more information about buckets, see Object storage in Amazon Lightsail.

Using multipart upload provides the following advantages:

• Improved throughput - You lot tin upload parts in parallel to amend throughput.

• Quick recovery from any network issues - Smaller part size minimizes the impact of restarting a failed upload due to a network error.

• Upload over time - Yous can upload file parts over time. Subsequently you initiate a multipart upload, you lot take 24 hours to complete the multipart upload.

• Begin an upload before you lot know the final file size - You tin can upload a file every bit you are creating it.

We recommend that you use multipart upload in the following ways:

• If you're uploading big files over a stable loftier-bandwidth network, multipart upload maximizes the apply of your bachelor bandwidth by uploading file parts in parallel for multi-threaded functioning.

• If you're uploading over a spotty network, use multipart upload to increase resiliency to network errors past avoiding upload restarts. When using multipart upload, you lot retry uploads only for the interrupted parts. At that place'southward no need to starting time over or upload the entire file again.

Contents:

• Multipart upload process

• Concurrent multipart upload operations

• Multipart upload retention

• Amazon S3 multipart upload limits

• Split the file to upload

• Initiate a multipart upload using the AWS CLI

• Upload a function using the AWS CLI

• Listing parts of a multipart upload using the AWS CLI

• Create a multipart upload .json file

• Complete a multipart upload using the AWS CLI

• List multipart uploads for a saucepan using the AWS CLI

• Cease a multipart upload using the AWS CLI

Multipart upload process

Multipart upload is a 3-pace process that uses Amazon S3 actions to upload files to your bucket in Lightsail:

1. Yous initiate the multipart upload using the CreateMultipartUpload activity.

2. Yous upload the file parts using the UploadPart activeness.

3. You complete the multipart upload using the CompleteMultipartUpload action.

Notation

You can finish a multipart upload afterwards you've initiated information technology by using the AbortMultipartUpload activeness.

When the multipart upload request completes, Amazon S3 constructs the object from the uploaded parts. And so yous can admission the object in the same way that you would access any other object in your bucket.

You lot can list all of your in-progress multipart uploads or get a list of the parts that you lot have uploaded for a specific multipart upload. Each of these operations is explained in this section.

Multipart upload initiation

When you send a request to initiate a multipart upload, Amazon S3 returns a response with an upload ID. This is a unique identifier for your multipart upload. Y'all must include the upload ID whenever y'all upload parts, list the parts, complete an upload, or stop an upload. If you want to provide any metadata describing the object being uploaded, you must specify the metadata in the request to initiate multipart upload.

Parts upload

When uploading a office, in addition to the upload ID, you lot must specify a function number. Yous can choose any part number betwixt 1 and ten,000. A part number uniquely identifies a part and its position in the object you are uploading. The part number that you cull doesn't need to be in a consecutive sequence (for example, it can exist 1, 5, and 14). If you lot upload a new function using the same part number as a previously uploaded office, the previously uploaded part is overwritten.

Whenever y'all upload a part, Amazon S3 returns an ETag header in its response. For each part upload, you must record the role number and the ETag value. You must include these values in the subsequent request to complete the multipart upload.

Note

All uploaded parts of a multipart upload are stored on your bucket. They consume your bucket's storage space until you lot complete the upload, stop the upload, or the upload times-out. For more than information, see Multipart upload retentiveness after in this guide.

Multipart upload completion

When you complete a multipart upload, Amazon S3 creates an object by concatenating the parts in ascending gild based on the part number. If any object metadata was provided in the initiate multipart upload asking, Amazon S3 associates that metadata with the object. Later on a successful complete request, the parts no longer be.

Your complete multipart upload request must include the upload ID and a list of both office numbers and corresponding ETag values. The Amazon S3 response includes an ETag that uniquely identifies the combined object data. This ETag is not necessarily an MD5 hash of the object information.

You lot can optionally stop the multipart upload. Afterward stopping a multipart upload, you cannot upload whatever part using that upload ID over again. All storage from any part of the canceled multipart upload is so freed. If whatever part uploads were in-progress, they tin can however succeed or fail even afterward y'all terminate. To free all storage consumed by all parts, you lot must finish a multipart upload only after all function uploads accept completed.

Multipart upload listings

Y'all can list the parts of a specific multipart upload or all in-progress multipart uploads. The list parts operation returns the parts information that you have uploaded for a specific multipart upload. For each listing parts request, Amazon S3 returns the parts information for the specified multipart upload, upward to a maximum of 1,000 parts. If at that place are more than i,000 parts in the multipart upload, you lot must send a serial of listing part requests to call up all the parts. Note that the returned list of parts doesn't include parts that are nonetheless in the process of uploading. Using the listing multipart uploads performance, yous tin obtain a list of multipart uploads in progress.

An in-progress multipart upload is an upload that you have initiated, but have non withal completed or stopped. Each request returns at nearly one,000 multipart uploads. If at that place are more than than 1,000 multipart uploads in progress, you lot must send additional requests to retrieve the remaining multipart uploads. But use the returned listing for verification. Exercise non use the result of this listing when sending a complete multipart upload request. Instead, maintain your ain list of the role numbers you specified when uploading parts and the respective ETag values that Amazon S3 returns.

Concurrent multipart upload operations

In a distributed development environment, information technology is possible for your application to initiate several updates on the same object at the same fourth dimension. Your application might initiate several multipart uploads using the aforementioned object key. For each of these uploads, your application can and then upload parts and send a complete upload request to Amazon S3 to create the object. When the buckets accept versioning enabled, completing a multipart upload ever creates a new version. For buckets that don't have versioning enabled, other request might take precedence, such equally requests that are received after a multipart upload is initiated and before it's complete.

Note

It is possible for other requests to accept precedence, such as requests that are received after yous initiate a multipart upload and earlier information technology is complete. For example, another operation might delete a key after you lot initiate a multipart upload with that key, and before the multipart upload is consummate. If this occurs, the complete multipart upload response might signal a successful object creation without you ever seeing the object.

Multipart upload retention

All uploaded parts of a multipart upload are stored on your bucket. They consume your bucket's storage space until you lot complete the upload, finish the upload, or the upload times out. A multipart upload times out, and the multipart upload is deleted, subsequently 24 hours from when information technology was created. When you end a multipart upload, or information technology times out, all uploaded parts are deleted and the storage space they used to consume on your saucepan is freed.

Amazon S3 multipart upload limits

The following table provides multipart upload core specifications.

• Maximum object size: five TB

• Maximum number of parts per upload: x,000

• Part numbers: 1-10,000 (inclusive)

• Role size: 5 MB (minimum) - 5 GB (maximum). There is no size limit on the terminal part of your multipart upload.

• Maximum number of parts returned for a list parts request: 1,000

• Maximum number of multipart uploads returned in a list multipart uploads request: 1,000

Carve up the file to upload

Use the separate command on the Linux or Unix operating system to split up a file into multiple parts that yous then upload to your bucket. At that place are similar gratis-ware applications that yous tin can use on the Windows operating organization to divide a file. After y'all split the file into multiple parts, proceed to the Initiate a multipart upload section of this guide.

Initiate a multipart upload using the AWS CLI

Consummate the following procedure to initiate a multipart upload using the AWS Command Line Interface (AWS CLI). Yous do this by using thecreate-multipart-upload command. For more information, come across create-multipart-upload in the AWS CLI Control Reference.

Note

You must install the AWS CLI and configure it for Lightsail and Amazon S3 earlier continuing with this process. For more information, meet Configuring the AWS Command Line Interface to piece of work with Amazon Lightsail.

1. Open a Command Prompt or Final window.

2. Enter the following command to create a multipart upload for your bucket.

                 aws s3api create-multipart-upload --bucket                                  BucketName                                --key                                  ObjectKey                                --acl saucepan-owner-full-control            

   In the command, replace the following example text with your ain:

   • BucketName  - The proper name of the bucket for which yous desire to create a multipart upload.

   • ObjectKey  - The object key to use for the file that y'all will upload.

   Example:

                 aws s3api create-multipart-upload --bucket                                  Md-EXAMPLE-Bucket                                --key                                  sailbot.mp4                                --acl saucepan-owner-total-control            

   You should encounter a event similar to the following case. The response includes an UploadID, which you must specify in subsequent commands to upload parts, and to complete the multipart upload for this object.

   

   Subsequently yous have the UploadID for your multipart upload, go on to the following Upload a part using the AWS CLI department of this guide and start uploading parts.

Upload a part using the AWS CLI

Consummate the following procedure to upload a office of a multipart upload using the AWS Command Line Interface (AWS CLI). You exercise this by using theupload-role command. For more information, come across upload-function in the AWS CLI Command Reference.

Note

Yous must install the AWS CLI and configure information technology for Lightsail and Amazon S3 earlier continuing with this process. For more information, see Configuring the AWS Command Line Interface to piece of work with Amazon Lightsail.

1. Open up a Command Prompt or Concluding window.

2. Enter the following command to upload a part to your bucket.

                 aws s3api upload-part --saucepan                                  BucketName                                --key                                  ObjectKey                                --function-number                                  Number                                --body                                  FilePart                                --upload-id "                  UploadID                " --acl bucket-owner-full-control            

   In the control, supervene upon the following example text with your own:

   • BucketName  - The name of the saucepan for which you want to create a multipart upload.

   • ObjectKey  - The object central to utilise for the file that y'all will upload.

   • Number - The function number of the part you are uploading. A function number uniquely identifies a office and its position in the object y'all are uploading. Make sure to incrementally increment the --part-number parameter with each function that y'all upload. To do so, number them in the order in which Amazon S3 should gather the object when you complete the multipart upload.

   • FilePart - The part file to upload from your computer.

   • UploadID - The upload ID of the multipart upload that you created earlier in this guide.

   Case:

                 aws s3api upload-part --bucket                                  DOC-EXAMPLE-Bucket                                --fundamental                                  sailbot.mp4                                --part-number one --body                                  sailbot.mp4.001                                --upload-id "                  R4QU.mO.exampleiHWiLOeNw7JtXX7OotRhTLsXXCzF21CZdYlfj5lfjtiMnpzVw2WPj.exampleBTmL_N_.42.DlHYOTsITFsX.tO3XOUTTAHiCxY5VR8jWRGdkVkUG                " --acl saucepan-possessor-total-command            

   You lot should come across a result similar to the following example. Repeat the upload-part command for each part you lot upload. The response for each of your upload part requests will include an ETag value for the part that y'all uploaded. Tape the ETag values for each of the parts that you lot upload. You volition need all of the ETag values to complete the multipart upload, which is covered afterward in this guide.

   

List parts of a multipart upload using the AWS CLI

Complete the following procedure to list parts of a multipart upload using the AWS Command Line Interface (AWS CLI). You do this by using thelist-parts control. For more information, see listing-parts in the AWS CLI Command Reference.

Complete this procedure to get the ETag values for all of the uploaded parts in a multipart upload. You volition demand these values to complete the multipart upload later on in this guide. However, if you recorded all of the ETag values from the response of your part uploads, then you can skip this procedure and continue to the Create a multipart upload .json file section of this guide.

Note

You must install the AWS CLI and configure it for Lightsail and Amazon S3 earlier standing with this procedure. For more information, encounter Configuring the AWS Command Line Interface to work with Amazon Lightsail.

1. Open up a Control Prompt or Terminal window.

2. Enter the following control to list the parts of a multipart upload on your bucket.

                 aws s3api listing-parts --bucket                                  BucketName                                --key                                  ObjectKey                                --upload-id "                  UploadID                "            

   In the control, replace the post-obit case text with your own:

   • BucketName  - The name of the bucket for which y'all want to list the parts of a multipart upload.

   • ObjectKey  - The object fundamental of the multipart upload.

   • UploadID - The upload ID of the multipart upload that you lot created earlier in this guide.

   Example:

                 aws s3api list-parts --bucket                                  DOC-Example-Bucket                                --key                                  sailbot.mp4                                --upload-id "                  R4QU.mO.exampleiHWiLOeNw7JtXX7OotRhTLsXXCzF21CZdYlfj5lfjtiMnpzVw2WPj.exampleBTmL_N_.42.DlHYOTsITFsX.tO3XOUTTAHiCxY5VR8jWRGdkVkUG                "            

   You should see a result like to the following case. The response lists all of the office numbers and ETag values for the parts that you uploaded in the multipart upload. Copy these values to your clipboard, and continue to the Create a multipart upload .json section of this guide.

   

Create a multipart upload .json file

Complete the post-obit procedure to create a multipart upload .json file that defines all of the parts you uploaded and their ETag values. This is required later in this guide to complete the multipart upload.

1. Open a text editor, and paste the response from the list-parts command that yous requested in the previous section of this guide.

   The result should expect like the following instance.

   

2. Reformat the text file equally shown in the following case:

   

3. Save the text file to your computer every bit mpstructure.json, and continue to the Complete a multipart upload using the AWS CLI section of this guide.

Complete a multipart upload using the AWS CLI

Complete the following procedure to complete a multipart upload using the AWS Command Line Interface (AWS CLI). Yous exercise this by using thecomplete-multipart-upload command. For more than information, see complete-multipart-upload in the AWS CLI Control Reference.

Annotation

You must install the AWS CLI and configure it for Lightsail and Amazon S3 before continuing with this process. For more information, run across Configuring the AWS Command Line Interface to work with Amazon Lightsail.

1. Open up a Command Prompt or Terminal window.

2. Enter the following command to upload a function to your saucepan.

                 aws s3api complete-multipart-upload --multipart-upload file://                  JSONFileName                                --saucepan                                  BucketName                                --key                                  ObjectKey                                --upload-id "                  UploadID                " --acl bucket-owner-total-control            

   In the command, replace the following example text with your own:

   • JSONFileName  - The name of the .json file that you created earlier in this guide (for example, mpstructure.json).

   • BucketName  - The proper name of the bucket for which yous desire to complete a multipart upload.

   • ObjectKey  - The object cardinal of the multipart upload.

   • UploadID - The upload ID of the multipart upload that you created before in this guide.

   Case:

                 aws s3api complete-multipart-upload --multipart-upload file://                  mpstructure.json                                --bucket                                  DOC-Example-Saucepan                                --key                                  sailbot.mp4                                --upload-id "                  R4QU.mO.exampleiHWiLOeNw7JtXX7OotRhTLsXXCzF21CZdYlfj5lfjtiMnpzVw2WPj.exampleBTmL_N_.42.DlHYOTsITFsX.tO3XOUTTAHiCxY5VR8jWRGdkVkUG                " --acl saucepan-owner-full-control            

   You should see a response similar to the following example. This confirms that the multipart upload is completed. The object is now assembled and available in the bucket.

   

List multipart uploads for a bucket using the AWS CLI

Complete the following process to list all multipart uploads for a bucket using the AWS Command Line Interface (AWS CLI). You lot do this past using thelist-multipart-uploads command. For more information, see list-multipart-uploads in the AWS CLI Control Reference.

Notation

You lot must install the AWS CLI and configure it for Lightsail and Amazon S3 before continuing with this procedure. For more than information, see Configuring the AWS Command Line Interface to work with Amazon Lightsail.

1. Open a Command Prompt or Terminal window.

2. Enter the post-obit control to upload a part to your bucket.

                 aws s3api list-multipart-uploads --saucepan                                  BucketName                                          

   In the command, replace BucketName  with the name of the bucket for which you desire to list all multipart uploads.

   Example:

                 aws s3api list-multipart-uploads --bucket                                  DOC-EXAMPLE-Saucepan                                          

   Yous should encounter a response similar to the post-obit instance.

   

Stop a multipart upload using the AWS CLI

Complete the post-obit process to stop a multipart upload using the AWS Command Line Interface (AWS CLI). Yous exercise this if yous started a multipart upload but no longer desire to go on it. Y'all do this past using theabort-multipart-upload command. For more information, see abort-multipart-upload in the AWS CLI Command Reference.

Notation

You must install the AWS CLI and configure it for Lightsail and Amazon S3 earlier standing with this procedure. For more information, see Configuring the AWS Command Line Interface to work with Amazon Lightsail.

1. Open up a Control Prompt or Terminal window.

2. Enter the post-obit control to upload a role to your bucket.

                 aws s3api abort-multipart-upload --saucepan                                  BucketName                                --key                                  ObjectKey                                --upload-id "                  UploadID                " --acl bucket-possessor-full-command            

   In the command, replace the following example text with your ain:

   • BucketName  - The name of the saucepan for which you lot desire to stop a multipart upload.

   • ObjectKey  - The object fundamental of the multipart upload.

   • UploadID - The upload ID of the multipart upload that you want to stop.

   Example:

                 aws s3api arrest-multipart-upload --bucket                                  Medico-EXAMPLE-Bucket                                --key                                  sailbot.mp4                                --upload-id "                  R4QU.mO.exampleiHWiLOeNw7JtXX7OotRhTLsXXCzF21CZdYlfj5lfjtiMnpzVw2WPj.exampleBTmL_N_.42.DlHYOTsITFsX.tO3XOUTTAHiCxY5VR8jWRGdkVkUG                " --acl bucket-possessor-full-command            

   This command does not return a response. You can run a listing-multipart-uploads command to confirm that the multipart upload was stopped.

Managing buckets and objects in Lightsail

These are the general steps to manage your Lightsail object storage bucket:

1. Learn about objects and buckets in the Amazon Lightsail object storage service. For more than information, run across Object storage in Amazon Lightsail.

2. Learn about the names that you lot can requite your buckets in Amazon Lightsail. For more data, run across Bucket naming rules in Amazon Lightsail.

3. Go started with the Lightsail object storage service past creating a bucket. For more information, run across Creating buckets in Amazon Lightsail.

4. Acquire about security all-time practices for buckets and the access permissions that you can configure for your bucket. You tin make all objects in your bucket public or private, or yous tin can cull to make private objects public. You can as well grant access to your bucket past creating admission keys, attaching instances to your bucket, and granting admission to other AWS accounts. For more than information, run across Security All-time Practices for Amazon Lightsail object storage and Understanding bucket permissions in Amazon Lightsail.

   After learning about bucket access permissions, come across the following guides to grant access to your bucket:

   • Block public access for buckets in Amazon Lightsail

   • Configuring bucket access permissions in Amazon Lightsail

   • Configuring access permissions for individual objects in a bucket in Amazon Lightsail

   • Creating access keys for a bucket in Amazon Lightsail

   • Configuring resources access for a bucket in Amazon Lightsail

   • Configuring cross-business relationship access for a bucket in Amazon Lightsail

5. Learn how to enable admission logging for your saucepan, and how to use admission logs to inspect the security of your bucket. For more information, see the post-obit guides.

   • Access logging for buckets in the Amazon Lightsail object storage service

   • Access log format for a bucket in the Amazon Lightsail object storage service

   • Enabling access logging for a bucket in the Amazon Lightsail object storage service

   • Using access logs for a bucket in Amazon Lightsail to identify requests

6. Create an IAM policy that grants a user the ability to manage a saucepan in Lightsail. For more data, see IAM policy to manage buckets in Amazon Lightsail.

7. Learn about the way that objects in your bucket are labeled and identified. For more information, run across Understanding object key names in Amazon Lightsail.

8. Acquire how to upload files and manage objects in your buckets. For more than data, meet the following guides.

   • Uploading files to a saucepan in Amazon Lightsail

   • Uploading files to a bucket in Amazon Lightsail using multipart upload

   • Viewing objects in a saucepan in Amazon Lightsail

   • Copying or moving objects in a bucket in Amazon Lightsail

   • Downloading objects from a saucepan in Amazon Lightsail

   • Filtering objects in a saucepan in Amazon Lightsail

   • Tagging objects in a bucket in Amazon Lightsail

   • Deleting objects in a bucket in Amazon Lightsail

9. Enable object versioning to preserve, retrieve, and restore every version of every object stored in your bucket. For more data, see Enabling and suspending object versioning in a saucepan in Amazon Lightsail.

10. After enabling object versioning, you lot can restore previous versions of objects in your bucket. For more than information, see Restoring previous versions of objects in a saucepan in Amazon Lightsail.

11. Monitor the utilization of your bucket. For more than information, see Viewing metrics for your bucket in Amazon Lightsail.

12. Configure an alarm for bucket metrics to be notified when the utilization of your saucepan crosses a threshold. For more information, see Creating bucket metric alarms in Amazon Lightsail.

13. Change the storage plan of your bucket if it'due south running depression on storage and network transfer. For more information, see Irresolute the program of your bucket in Amazon Lightsail.

14. Larn how to connect your bucket to other resources. For more than information, see the following tutorials.

    • Tutorial: Connecting a WordPress instance to an Amazon Lightsail saucepan

    • Tutorial: Using an Amazon Lightsail bucket with a Lightsail content delivery network distribution

15. Delete your saucepan if you're no longer using information technology. For more than information, come across Deleting buckets in Amazon Lightsail.

halltrubmisoace.blogspot.com

Source: https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-uploading-files-to-a-bucket-using-multipart-upload

Share this post