Skip to content

[JS] UnhandledRejection in Streaming Structured Output Parsing #2406

Open
@dongyangli1226

Description

@dongyangli1226

Describe the bug
I use Genkit to stream structured output. When the structured output is not a valid JSON, Genkit library will throw an exception from the output getter when parsing it. My system seems to catch the error but Genkit library still raises an unhandledRejection that will crash my application.

Error:
SyntaxError: JSON5: invalid character '{' at 1:2619
at syntaxError (/workspace/node_modules/json5/lib/parse.js:1110:17)
at invalidChar (/workspace/node_modules/json5/lib/parse.js:1055:12)
at Object.afterArrayValue (/workspace/node_modules/json5/lib/parse.js:694:15)
at Object.default (/workspace/node_modules/json5/lib/parse.js:168:37)
at lex (/workspace/node_modules/json5/lib/parse.js:100:42)
at Object.parse (/workspace/node_modules/json5/lib/parse.js:25:17)
at extractJson (/workspace/node_modules/@genkit-ai/ai/lib/extract.js:75:37)
at Message.parseMessage [as parser] (/workspace/node_modules/@genkit-ai/ai/lib/formats/json.js:48:47)
at get output (/workspace/node_modules/@genkit-ai/ai/lib/message.js:65:25)
at get output (/workspace/node_modules/@genkit-ai/ai/lib/generate/response.js:100:24)
at GenerateResponse.assertValidSchema (/workspace/node_modules/@genkit-ai/ai/lib/generate/response.js:77:22)
at generate (/workspace/node_modules/@genkit-ai/ai/lib/generate/action.js:217:45)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async /workspace/node_modules/@genkit-ai/ai/lib/generate/action.js:77:22
at async /workspace/node_modules/@genkit-ai/core/lib/tracing/instrumentation.js:88:24
at async runInNewSpan (/workspace/node_modules/@genkit-ai/core/lib/tracing/instrumentation.js:77:10)

To Reproduce

      const res = await examplePromptThatProducesInvalidJSON.stream(promptInput, {
        use: [simulateConstrainedGeneration()],
      });

      try{
        for await (const chunk of res.stream) { 
              const outputChunk = chunk.output; //this will raise the unhandledRejection
        }
     } catch (error) {
          // The parsing error will be caught however later on Genkit still throws the handled rejection
          console.log(error);
     }

Expected behavior
When JSON parsing error occurs. the error is logged, then an uncaught exception throws that crashes application.

Runtime (please complete the following information):

  • OS: MacOS

** Node version

  • 22.14.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingjs

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions