Jul 14, 2017

You shouldn't use amazon s3 as a unix file system in a productive environment. S3 is not built like this.

From the s3fs docs
Important Limitations Eventual Consistency Due to S3's "eventual consistency" limitations file creation can and will occasionally fail. Even after a successful create subsequent reads can fail for an indeterminate time, even after one or more successful reads. Create and read enough files and you will eventually encounter this failure. This is not a flaw in s3fs and it is not something a FUSE wrapper like s3fs can work around. The retries option does not address this issue. Your application must either tolerate or compensate for these failures, for example by retrying creates or reads. For more details, see Eventual Consistency

Post a Comment

Featured Post

Modern tar recognizes the file format

One command works with any supported compression method. tar xf archive.tar.xz tar xf archive.tar.gz tar xf archive.tar etc. ...