Some fellows are using the Minio Client (mc) as their primary client-side tool to work with S3 cloud storage and filesystems. As you may know, mc works with the AWS v4 signature API and it provides a modern alternative under the Apache 2.0 License to UNIX commands (ls, cat, cp, diff, etc).

In the case you are using mc in the client side and Ceph RGW S3 in the server side, you could be experimenting some issues with AWS4 presigned URLs and the error code '403 Forbidden'.

To resolve this issue you need to set to 'false' a new configuration parameter in the RGW S3 configuration file:

rgw s3 auth aws4 force boto2 compat = false

With this configuration in place, RGW S3 will be able to handle mc and other client-side tools experimenting the same issue properly. This configuration option is already available upstream.

By the way, if you are interested to know the origin of this issue you can have a look in this old boto2 bug.

While computing the signature a buggy boto2 version will craft the host using the port number twice while a proper implementation (mc, etc) uses it once only. The result will be two different outputs to compute the same URL.

Amazon S3 will accept as valid both signatures.

In the case of RGW S3, with the new configuration option set to 'false', RGW S3 will compute a second signature in the case of presigned URLs if the first signature computation does not match. The AWS4 presigned URL will be valid if any of the two signatures match.

Enjoy!

Acknowledgments

My work in Ceph is sponsored by Outscale and has been made possible by Igalia and the invaluable help of the Ceph development team. Thanks Pritha, Matt Benjamin and Yehuda for all your support to go upstream!

Comments

comments powered by Disqus