Skip to content

Refactor stream handling to expose event labels #589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ingemar
Copy link

@ingemar ingemar commented May 1, 2025

Description:

This PR refactors how streaming responses are handled to make stream event labels accessible to the user-defined stream handler.

Why This Change?

This update enables access to the event labels in streamed OpenAI API responses. These labels provide important context (e.g., response.created, response.completed) previously unavailable to user-defined stream handlers.

Exposing this metadata makes building more responsive and structured streaming experiences easier, while keeping the API flexible and backward compatible.

Summary of Changes

  • Moved the code from OpenAI::HTTP#to_json_stream into a new class named OpenAI::Stream
  • OpenAI::Stream maintains backward compatibility by inspecting the arity of the proc and calling it with one or two arguments accordingly
  • Replaced all calls to the #to_json_stream method with the new OpenAI::Stream interface
  • Updated all examples and specs to replace unused _bytesize params with _event (since, as far as I understand, the bytesize was never actually passed)

Example Before vs. After

Before:

stream: proc do |chunk, _bytesize|
  puts chunk["delta"]
end

After:

stream: proc do |chunk, event|
  puts "[#{event}] #{chunk["delta"]}"
end

Compatibility

  • Fully backward compatible: if a stream proc only takes one argument, it will behave as before
  • All tests passing

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?

end
end

def to_proc
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to convert to a Proc because Faraday enforces the stream handler to be that.

handle_http_error(chunk: chunk, env: env) if env&.status != 200

parser.feed(chunk) do |event, data|
next if data == DONE
Copy link
Author

@ingemar ingemar May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be changed to check the event instead. Maybe that would be more elegant.
Example:

next if event == "done"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant