Updated AWS::S3Express::DirectoryBucket

Amazon S3 introduces support for AWS Dedicated Local Zones.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3express-directorybucket.html #s3express #cloudformation

@michabbb I have used #Snowflake for a year for a data warehouse - I like it very much. I must look into #S3Express 🤔

Modern Database Architecture: #Cloud Storage Evolution 🚀

🔹 #ZeroDisk architecture leverages #AWS #S3 as primary storage, eliminating need for traditional disk management

🔸 #Database systems gain instant scalability and elasticity through separation of compute and storage

🔹 New approach enables three deployment options:
- Direct #S3 writes
- #S3Express as cache layer
- #Raft cluster with S3 backend

🔸 Major platforms adopting this architecture:
#Snowflake, #Clickhouse, #Neon, #TiDB

🔹 Trade-offs balance between latency, cost, and durability metrics

🔸 Benefits include serverless operation, instant failover, and unlimited storage capacity

https://avi.im/blag/2024/zero-disk-architecture/

#DevOps

Updated AWS::S3Express::DirectoryBucket Rule

Specifies lifecycle rules for an Amazon S3 bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-rule.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket LifecycleConfiguration

Container for lifecycle rules. You can add as many as 1000 rules.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-lifecycleconfiguration.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket AbortIncompleteMultipartUpload

Specifies the days since the initiation of an incomplete multipart upload that Amazon S3 will wait before permanently removing all parts of the upload.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-abortincompletemultipartupload.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket

Use the AvailabilityZoneName property to return the code for the Availability Zone where the directory bucket was created.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3express-directorybucket.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket ServerSideEncryptionByDefault

Use the AWS::S3Express::DirectoryBucket ServerSideEncryptionByDefault property to apply the default server-side encryption to new objects in a directory bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-serversideencryptionbydefault.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket ServerSideEncryptionRule

Use the AWS::S3Express::DirectoryBucket ServerSideEncryptionRule resource to specify the default server-side encryption configuration for an S3 directory bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-serversideencryptionrule.html #s3express #cloudformation

Updated AWS::S3Express::DirectoryBucket BucketEncryption

Use the AWS::S3Express::DirectoryBucket BucketEncryption property to specify default encryption for a directory bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3express-directorybucket-bucketencryption.html #s3express #cloudformation

New AWS::S3Express::BucketPolicy

Use the AWS::S3Express::BucketPolicy resource to apply an S3 bucket policy to an S3 directory bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3express-bucketpolicy.html #s3express #cloudformation

New AWS::S3Express::DirectoryBucket

Use the AWS::S3Express::DirectoryBucket resource to create an Amazon S3 directory bucket.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3express-directorybucket.html #s3express #cloudformation

And my PR for S3A #s3express support is up. Test coverage incomplete; some fail…proposed a followup rather than a “fix it here” PR as this has production changes for treewalk code to skip “hallucinated directories” which surface above incomplete uploads. Adds pathCapabilities probes for: v2 SDK support, store is s3 express and dir listings potentially inconsistent.

https://github.com/apache/hadoop/pull/6308

#AwS support #S3Express through the S3A connector! Not their one “legacy” s3 connector
The core of the move to AWS v2 SDK was done by a single AWS employee in the S3 team; we did most of the dev in the open; a hint of “third party support” PRs covered minor nits. The EMR release timing makes me think they left out the last “corner cases in NFQE/Benchmarking”, looks like their region code is as still broken as Hadoop trunk. Really unhappy with that SDK change
#reinvent

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-express-one-zone.html