Access Denied for HEAD Bucket Request - bug?

[simon-fauconnier]

Description:

Hello,

I am encountering an issue with checking bucket existence using the minio-cpp client. While the GetObject and PutObject operations work correctly with the same bucket, the HEAD request to check the bucket's existence returns a 403 Forbidden error. Below, I have provided the details of the issue and a comparison with the AWS SDK, which works as expected.

Steps to Reproduce the Issue:

1. Compile and run the following C++ code using minio-cpp:

#include <miniocpp/client.h>

int main() {
  // Create S3 base URL.
  minio::s3::BaseUrl base_url("s3.eu-central-1.amazonaws.com");

  // Create credential provider.
  minio::creds::StaticProvider provider(
      "ACCESS_KEY", "SECRET_KEY");

  // Create S3 client.
  minio::s3::Client client(base_url, &provider);

  // Enable debugging.
  client.Debug(true);

  // Create bucket exists arguments.
  minio::s3::BucketExistsArgs args;
  args.bucket = "test";

  // Call bucket exists.
  minio::s3::BucketExistsResponse resp = client.BucketExists(args);

  // Handle response.
  if (resp) {
    if (resp.exist) {
      std::cout << "Bucket exists" << std::endl;
    } else {
      std::cout << "Bucket does not exist" << std::endl;
    }
  } else {
    std::cout << "Unable to check bucket existence: "
              << resp.Error().String() << std::endl;
  }

  return 0;
}

Observed Behavior:

When client.Debug(true); is enabled, the following output is observed:

*   Trying 52.219.170.218:80...
* Connected to test.s3.eu-central-1.amazonaws.com (52.219.170.218) port 80 (#0)
> HEAD / HTTP/1.1
Host: test.s3.eu-central-1.amazonaws.com
Accept: */*
Authorization: AWS4-HMAC-SHA256 Credential=ACCESS_KEY/20240830/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=SIGNATURE
User-Agent: MinIO (linux; x86_64) minio-cpp/0.3.0
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20240830T115923Z
 
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< x-amz-bucket-region: eu-central-1
< x-amz-request-id: P2MA0W9ZWTS0074N
< x-amz-id-2: GW/dIZAOzwPGhE7GPLeQ2Vg3/t/DlckD7syRDpqh5PajG66cme8NgtKDgM5z8yviy1/JReg4n1Q=
< Content-Type: application/xml
< Date: Fri, 30 Aug 2024 11:59:23 GMT
< Server: AmazonS3
< 
* Connection #0 to host test.s3.eu-central-1.amazonaws.com left intact

Expected Behavior:

The HEAD request should succeed without returning a 403 Forbidden error.

Comparison with AWS SDK:

The same operation using the AWS SDK succeeds without any errors. Here is the successful HEAD request from the AWS SDK:

> (HeaderOut) HEAD /test HTTP/1.1
> Host: s3.eu-central-1.amazonaws.com
> Accept: */*
> amz-sdk-invocation-id: 1E8FD929-8964-45E1-8003-152888E68AB9
> amz-sdk-request: attempt=1
> authorization: AWS4-HMAC-SHA256 Credential=ACCESS_KEY/20240903/eu-central-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-type;host;x-amz-api-version;x-amz-content-sha256;x-amz-date, Signature=SIGNATURE
> content-type: application/xml
> user-agent: aws-sdk-cpp/1.11.396 ua/2.0 md/aws-crt#0.28.2 os/Linux/5.15.0-119-generic md/arch#x86_64 lang/c++#C++11 md/GCC#11.4.0 api/S3
> x-amz-api-version: 2006-03-01
> x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> x-amz-date: 20240903T135742Z

< (Text) Connection #0 to host s3.eu-central-1.amazonaws.com left intact

< [DEBUG] 2024-09-03 13:57:42.575 CurlHttpClient [140181247796288] Returned http response code 200
< [DEBUG] 2024-09-03 13:57:42.575 CurlHttpClient [140181247796288] Returned content type application/xml
< [DEBUG] 2024-09-03 13:57:42.575 AWSClient [140181247796288] Request returned successful response.

Comparison with Python Client:

The issue does not occur when using the Python client for MinIO. The following Python code correctly checks for bucket existence:

from minio import Minio

client = Minio("s3.eu-central-1.amazonaws.com",
    access_key="ACCESS_KEY",
    secret_key="SECRET_KEY",
)

bucket_name = "test"

found = client.bucket_exists(bucket_name)
if not found:
    print("Bucket does not exist:", bucket_name)
else:
    print("Bucket already exists:", bucket_name)

The Python client successfully checks the existence of the bucket without any 403 errors.

Environment Details:

• minio-cpp version: 0.3.0
• Operating system: Ubuntu 22.04.3 LTS
• Compiler: GCC version 11.4.0

[balamurugana]

@simon-fauconnier Please enable trace_on() in your python code, share the output

[simon-fauconnier]

Here is the python trace :

---------START-HTTP---------
HEAD / HTTP/1.1
Host: test.s3.eu-central-1.amazonaws.com
User-Agent: MinIO (Windows; AMD64) minio-py/7.2.5
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20240903T145228Z
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20240903/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=REDACTED

HTTP/1.1 200
x-amz-id-2: Pfzuic2fVj6EDJ1r27Nj9D+imwRaDcwk5MpbGN5QmaY8knvj5euaaoqPQWwY0VTEQjIfOnyZzTyG9/Ju9Ooir3WmvP3TA7x/FMFfC3j5rUg=
x-amz-request-id: Q6AZTNMB8T6AAYK7
Date: Tue, 03 Sep 2024 14:52:30 GMT
x-amz-bucket-region: eu-central-1
x-amz-access-point-alias: false
Content-Type: application/xml
Server: AmazonS3

----------END-HTTP----------

[simon-fauconnier]

Hi @balamurugana, any news about this issue?

[balamurugana]

@simon-fauconnier I would need to get AWS access to debug the issue.

[simon-fauconnier]

Hi @balamurugana
I have sent the credentials to access the bucket to your email address

[balamurugana]

@simon-fauconnier Please check the PR #173

[simon-fauconnier]

The fix #176 solves the issue

Thanks @balamurugana !