Fangyijie Wang

Fang

0%

AWS S3

Posted on

What is S3 ?

  • Object Based
  • Files <= 5 TB
  • No OS or DB storage
  • Unlimited Storage
  • Universal namespace

What is S3 Object?

  • Key * Object name
  • Value * The data itself (made up of bytes)
  • Version ID
  • Metadata * Data about the data you are storing

S3 Security

  • Buckets are private by default (includes all objects within it)
  • Object ACLs: Make individual objects public
  • Bucket policies: Make entire buckets public
  • HTTP status code: When uploading an object to S3 successfully

S3 Versioning

  • All Versions: All Versions of an object are in S3. (writes and deletes)
  • Backup: a great backup tool
  • Cannot Be Disabled: Once enabled, it cannot be disabled * only suspended
  • Lifecycle Rules: Can be integrated with lifecycle rules
  • Supports MFA: Can support multi-factor authentication

S3 Lifecycle Management

  • Automates moving objects between the different storage tiers
  • Can be used in conjunction with versioning
  • Can be applied to current versions and previous versions

S3 Object Lock and Glacier Vault Lock

  • Object Lock
    • Store objects using a write once, read many (WORM) model
    • Can be on individual objects or applied across the bucket
    • It comes in two modes: governance mode (protect objects against most users unless they have special permissions) & compliance mode (protect objects against any users)
  • Glacier Vault Lock
    • Allows you to deploy and enforce compliance controls for individual S3 Glacier vaults with a vault lock policy. You can specify controls, e.g. WORM, in a vault lock policy and lock the policy from future edits. Once locked, the policy cannot be changed

S3 Encryption

  • Encryption in Transit
    • SSL/TLS
    • HTTPS
  • Encryption at Rest
    • Server-side encryption
    • SSE-S3 (AES 256-bit)
    • SSE-KMS
    • SSE-C
  • Client-Side Encryption
    • Encrypt the files before you upload them to S3
  • Encryption with a Bucket Policy
    • A bucket policy can deny all PUT requests that do not include the x-amz-server-side-encryption parameter in the request header

Optimizing S3 Performance

  • Prefix
    • More prefix and more requests per second.
    • You can achieve a number of requests: 3,500 PUT / COPY / POST / DELETE and 5,500 GET / HEAD per second , per prefix
  • SSE-KMS limits * when using it to encrypt objects in S3, keep in mind the limits
    • Uploading/downloading count toward the KMS quota
    • Region-specific, it’s either 5,500, 10,000 or 30,000 requests per second
    • You cannot request a quota increase for KMS
  • Tips:
    • Use multipart uploads to increase performance when uploading files
    • Files > 100 MB (should use multipart uploads). Files > 5 GB (must use multipart uploads)
    • User S3 byte-range fetches to increase performance when downloading files

S3 Replication

  • Replicate objects from one bucket to another
  • Existing objects are not replicated automatically
  • Delete markers are not replicated by default

S3 Comparison