Skip to content

Ceph object store: Fix LocationConstraint error #10772

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 1, 2025

Conversation

abh1sar
Copy link
Collaborator

@abh1sar abh1sar commented Apr 24, 2025

Description

This PR fixes #10044

        AmazonS3 client = AmazonS3ClientBuilder.standard()
                .enablePathStyleAccess()
                .withCredentials(
                        new AWSStaticCredentialsProvider(
                                new BasicAWSCredentials(accessKey, secretKey)))
                .withEndpointConfiguration(
                        new AwsClientBuilder.EndpointConfiguration(url, "auto"))
                .build();

Using auto as the signingRegion in AwsClientBuilder forces the s3 client to infer region from the url and set it as LocationConstraint while sending the request which is not required for ceph and causes the issue.
requestRegion = AwsHostNameUtils.parseRegion(requestEndpoint.getHost(), "s3");
Location constraint was being set if the url starts with "s3." So a possible workaround if using this type of endpoint is to not use such kind of name for the endpoint url. Use some proxy or dns.

Have changed the code to pass null instead of auto which is resulting in LocationConstraint to not be set.


One more change is regarding the error msg which is returned while creating a bucket when the same bucket name is already present but created by another user.
Before:
Failed to create bucket with name: bucket2. AccessDenied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: tx00000610f7487f71ab177-00680a2196-ac4a-default; S3 Extended Request ID: ac4a-default-default; Proxy: null)
After:
Failed to create bucket with name: bucket2. Bucket already exists with the name: bucket2

This was because getBucketAcl doesn't work if the bucket was created by another user as the first user doesn't have Acl level access to the bucket.


Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Copy link

codecov bot commented Apr 24, 2025

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 16.02%. Comparing base (80b5d5c) to head (865ed45).
Report is 86 commits behind head on 4.20.

Files with missing lines Patch % Lines
...ge/datastore/driver/CephObjectStoreDriverImpl.java 0.00% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.20   #10772      +/-   ##
============================================
+ Coverage     16.00%   16.02%   +0.01%     
- Complexity    13104    13129      +25     
============================================
  Files          5651     5652       +1     
  Lines        495862   496022     +160     
  Branches      60043    60065      +22     
============================================
+ Hits          79367    79484     +117     
- Misses       407635   407670      +35     
- Partials       8860     8868       +8     
Flag Coverage Δ
uitests 4.00% <ø> (+<0.01%) ⬆️
unittests 16.86% <0.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@abh1sar abh1sar added this to the 4.20.1 milestone Apr 24, 2025
@abh1sar abh1sar requested a review from wido April 24, 2025 12:24
@abh1sar
Copy link
Collaborator Author

abh1sar commented Apr 25, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13181

@abh1sar
Copy link
Collaborator Author

abh1sar commented Apr 28, 2025

@blueorangutan test

@blueorangutan
Copy link

@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-13167)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 55742 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10772-t13167-kvm-ol8.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_restore_vm_strict_tags_failure Failure 59.52 test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure Failure 59.75 test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure Failure 5.79 test_vm_strict_host_tags.py

@abh1sar
Copy link
Collaborator Author

abh1sar commented Apr 29, 2025

@rohityadavcloud @wido @DaanHoogland test failures are unrelated. Can we merge this?

@wido
Copy link
Contributor

wido commented Apr 29, 2025

@rohityadavcloud @wido @DaanHoogland test failures are unrelated. Can we merge this?

Yes, good to merge from my side

@DaanHoogland
Copy link
Contributor

DaanHoogland commented Apr 29, 2025

good to merge if it has been tested by someone else than the author.

(edit: these failures are happening on all branches and PRs atm)

@rohityadavcloud rohityadavcloud merged commit dfd64b1 into apache:4.20 May 1, 2025
17 of 25 checks passed
@rohityadavcloud rohityadavcloud deleted the ceph-location-constraint branch May 1, 2025 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants