Skip to content

Added AWS Bedrock Provider #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

harsh-791
Copy link

Add AWS Bedrock Provider

Resolves - #2

Summary

This PR adds support for the AWS Bedrock provider to the project, enabling integration with Amazon Bedrock models via the new llm-bedrock component.


Key Changes

  • Introduced a new provider: llm-bedrock
  • Implemented all required traits and interfaces for Bedrock integration
  • Added a test harness and configuration for Bedrock in tests/bedrock-test
  • Updated workspace and build scripts to include the new provider
  • Ensured compatibility with existing project structure and conventions

Testing

  • The provider builds successfully and integrates with the test harness.
  • The test attempts to invoke the AWS Bedrock model using credentials loaded from .env.
  • Note: Actual model invocation requires AWS account access to Bedrock models. If you encounter AccessDeniedException or Unavailable errors, request model access via the AWS Console or AWS Support.

Additional Notes

  • Both test/ and tests/ folders are present by design; see the README for details.
  • All code and configuration changes follow project conventions.

/claim #2

@harsh-791
Copy link
Author

harsh-791 commented May 2, 2025

@vigoo @DEGOES
could you please review the changes and let me know if any changes are required

@vigoo
Copy link
Collaborator

vigoo commented May 5, 2025

A few initial comments:

  • To use the Rust AWS SDK within Golem you have to use https://crates.io/crates/aws-smithy-wasm
  • There is no multithreading in Golem workers. It's still possible to use async Rust using a single thread, you can check the rust:async component template as a starting point
  • There should not be new bedrock specific test cases in the test component, but the existing 6 test cases should work well when linked against bedrock.
  • There should not be hardcoded parameters in the client, everything should be coming from the user's Config and input values.
  • A video of running all six tests with Golem, showing the results, is required for getting a more in depth review

@harsh-791
Copy link
Author

@vigoo Thanks for the feedback! I'm currently facing a few issues and would really appreciate some clarification:

AWS SDK Integration (v0.1.4):

  • I'm having trouble understanding the correct way to use the SDK for Bedrock operations—specifically the service names, operation methods, and how to handle request/response logic.
  • Should I be using a different approach for making HTTP requests in the WASM environment?
  • Are there any specific configuration requirements for the AWS client when running in Golem?

Error Handling & Testing:

  • What's the recommended way to handle AWS errors and map them to Golem’s error codes?
  • Should I be using mocked responses or real credentials for testing?
  • Are there any specific test cases I should focus on beyond the existing six?

Streaming Implementation:

  • For streaming with Bedrock, should I be using invoke_stream or a different method?
  • What's the correct way to handle streaming responses and related errors in WASM?
  • Are there any specific considerations I should keep in mind when implementing streaming in this environment?

Thanks again, and I’d appreciate any guidance on these points!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants