oas2tson assumes strict camelcase naming -- bug

[jmjf]

TL;DR:

• The commit identified in the "Main issue" section assumes names follow strict camelcase. That assumption is not valid.
• The commit changes oas2tson only. Different behavior in oas2ts helped me identify the cause by introducing API inconsistency.
• There needs to be a solution to the problem that commit solves that doesn't assume strict camelcase and that is applied consistently across commands. I'm not sure what that solution should be but am open to having a conversation about it.
  • oas2ts doesn't seem to have the problem, so maybe that's the pattern to follow for a fix.
• Workaround: Use v1.4.1.

EDIT

The more I look at this, the more I'm convinced the original PR/issue is misusing -p. I believe that idea aligns with comments from @ilteoood and others on #230 that question the idea of generating path specs.

Per documentation, the -p option identifies "properties/definitions in the OpenAPI file to convert." In other words, -p ApiResponse,Pets, not -p paths.

The PR needs to be reverted.

I like the idea @faizplus seems to be pursuing -- generating path specs from OpenAPI to use as schemas in Fastify route schemas. On the other hand, I usually import TSON schemas and use them to define the route schemas so the schemas aren't that much to build. And because I've built a few paths with complex query parameter conditions that required hacking the schemas together (must have A + B || C || A + B + C).

URLs are case sensitive, so /pets/findByStatus is not /pets/findbystatus is not Pets/FindByStatus. The PR assumes URLs are not case sensitive. I agree that's questionable naming, but the specs are the specs.

Background

We follow a naming convention that uses attribute names like 'ipAddress and someOtherIPAddress in objects, where the abbreviation IP is upper case except when it's the first term in a name. In our OpenAPI schemas, we define all attributes as schemas (IPAddress, SomeOtherIPAddress) so we can manage the metadata in one place (e.g., descriptions, types, etc.). This approach lets us write clear descriptions and helpful notes for our API consumers while keeping the spec DRY.

Main issue

Starting with openapi-transformer-toolkit@1.4.3, the oas2tson command generates file does not exist errors in a tmp directory for names that have consecutive uppercase characters. The oas2tson output includes IpAddress.ts and SomeOtherIpAddress.ts and does not include any schemas that $ref: 'path/to/Attributes.yaml#/components/schemas/IPAddress or .../SomeOtherIPAddress.

The error does not happen for oas2ts and the output of oas2ts includes IPAddress.d.ts and SomeOtherIPAddress.d.ts and all types that use those attributes.

The workaround is to use v1.4.1, which works as expected.

I believe this issue is caused by d56e9a8, which changed paths.ts to use lodash-camelcase, which enforces strict camelcase. This change affects oas2tson, but not oas2ts.

Reproduction

In ./example/openapi.yaml, update the ApiResponse schema and add FooBARBaz as follows.

    ApiResponse:
      type: object
      properties:
        code:
          type: integer
          format: int32
        type:
          type: string
        message:
          type: string
        fooBARBaz:
          $ref: '#/components/schemas/FooBARBaz'
      xml:
        name: '##default'
    FooBARBaz:
      type: string
      description: some description

npm run oas2ts: no errors and I get FooBARBaz.d.ts in ./example/output/types.

npm run oas2tson: error shown below (added break before stack so it's easier to find), no ApiResponse.ts or FooBARBaz.ts in output. (That's inconsistent with what I saw at work, but this setup is slightly different. The error is the same except for the file name node of the path.)

{"level":40,"time":1722177165036,"pid":8849,"hostname":"akaishi","err":{"type":"ResolverError","message":"Error opening file \"/tmp/tempjson/FooBARBaz.json\" \nENOENT: no such file or directory, open '/tmp/tempjson/FooBARBaz.json'",

"stack":"JSONParserError: Error opening file \"/tmp/tempjson/FooBARBaz.json\" \nENOENT: no such file or directory, open '/tmp/tempjson/FooBARBaz.json'\n    at Object.read (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/resolvers/file.js:61:19)","code":"ERESOLVER","name":"ResolverError","source":"/tmp/tempjson/FooBARBaz.json","path":null,"ioErrorCode":"ENOENT","raw":{"stack":"JSONParserError: Error opening file \"/tmp/tempjson/FooBARBaz.json\" \nENOENT: no such file or directory, open '/tmp/tempjson/FooBARBaz.json'\n    at Object.read (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/resolvers/file.js:61:19)","code":"ERESOLVER","name":"ResolverError","message":"Error opening file \"/tmp/tempjson/FooBARBaz.json\" \nENOENT: no such file or directory, open '/tmp/tempjson/FooBARBaz.json'","source":"/tmp/tempjson/FooBARBaz.json","path":null,"ioErrorCode":"ENOENT","footprint":"null+/tmp/tempjson/FooBARBaz.json+ERESOLVER+Error opening file \"/tmp/tempjson/FooBARBaz.json\" \nENOENT: no such file or directory, open '/tmp/tempjson/FooBARBaz.json'"}},"msg":"Failed to convert non-object attribute, skipping"}

Names like Foo_Bar_Baz also fail with the same error.

Other

Edit ./src/utils/paths.ts and change the formatFileName function to return the name it receives:

export const formatFileName = (title: string): string => title
  // _upperFirst(_camelCase(title))

npm run oas2tson: gets no errors; output includes ApiResponse.ts and FooBARBaz.ts as expected. This proves this commit's changes are forcing strict camelcase.

I cannot find anything in the OpenAPI spec or JSON Schema spec that requires strict camelcase. Maybe I'm missing something. But if that requirement isn't the specs somewhere, this commit makes an invalid assumption about names.

The commit message says fix: oas2tson throwing error when path -p option is provided. I'm not sure which is wrong but the command specification says, the -p option is properties.

https://github.com/nearform/openapi-transformer-toolkit/blob/7ea28ac7b6425e25ae17d3fadad3b514951118b1/src/commands/oas2tson.ts#L186C1-L188C74

oas2ts has the same specification. The other commands don't have a -p option.

I'm not sure what this option is supposed to do, so cannot make a recommendation on how to fix the problem. But...

Using code patched as described in this section, npm run oas2tson -- -p paths fails, but npm run oas2ts -- -p paths does not fail and handles FooBARBaz and Foo_Bar_Baz correctly. Maybe oas2ts has already solved this problem and both commands should use the same approach.

Secondary issue

EDIT: Ignore this secondary issue. I'm able to run tests successfully now.

I cloned the repo hoping to diagnose the issue and maybe fix it but the sequence

git clone...
cd openapi-transformer-toolkit
npm i
npm run test

gets the following error (added break before stack)

{"level":40,"time":1722175526477,"pid":7265,"hostname":"akaishi","err":{"type":"MissingPointerError","message":"Token \"components\" does not exist.",

"stack":"JSONParserError: Token \"components\" does not exist.\n    at Pointer.resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/pointer.js:103:23)\n    at $Ref.resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/ref.js:81:28)\n    at $Refs._resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/refs.js:158:21)\n    at dereference$Ref (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:160:27)\n    at crawl (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:95:40)\n    at crawl (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:105:44)\n    at dereference (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:45:26)\n    at $RefParser.dereference (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/index.js:182:42)\n    at processJSON (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/src/commands/oas2tson.ts:95:32)\n    at runCommand (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/src/commands/oas2tson.ts:150:5)","code":"EUNMATCHEDRESOLVER","name":"MissingPointerError","source":"/tmp/tempjson/UserUsername.json","path":null,"raw":{"stack":"JSONParserError: Token \"components\" does not exist.\n    at Pointer.resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/pointer.js:103:23)\n    at $Ref.resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/ref.js:81:28)\n    at $Refs._resolve (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/refs.js:158:21)\n    at dereference$Ref (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:160:27)\n    at crawl (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:95:40)\n    at crawl (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:105:44)\n    at dereference (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/dereference.js:45:26)\n    at $RefParser.dereference (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/node_modules/@apidevtools/json-schema-ref-parser/dist/lib/index.js:182:42)\n    at processJSON (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/src/commands/oas2tson.ts:95:32)\n    at runCommand (/home/jmjf/dev/gh-wk/openapi-transformer-toolkit/src/commands/oas2tson.ts:150:5)","code":"EUNMATCHEDRESOLVER","name":"MissingPointerError","message":"Token \"components\" does not exist.","source":"/tmp/tempjson/UserUsername.json","path":null,"footprint":"null+/tmp/tempjson/UserUsername.json+EUNMATCHEDRESOLVER+Token \"components\" does not exist."}},"msg":"Failed to convert non-object attribute, skipping"}

Dunno what's going on here.

[jmjf]

@simoneb Trying oas2json for comparison, I see ./dist/esm/src/cli.js oas2json -i ./example/openapi.yml -o example/output/json -p User failing with message "Failed to convert non-object attribute, skipping".

The example in the docs uses -p paths but describes the option as "properties/definitions in the OpenAPI file to convert."

So, let me ask, what is this option really doing?

runCommand gets 'User,Tag' as propertiesToExport
defintionKeywords is ['User', 'Tag', 'components.schemas'] because the last item is explicitly added, which means it will generate for everything, right?

oas2json writes a paths directory with or without -p paths and that directory does seem to have JSON Schema paths in JSON, but -p seems to make no difference in the output. Incorrect. I think maybe it was hanging around across runs.

For now I'm going to focus on reverting the other PR. I suggest figuring out what -p should do and making it do that should be a separate PR.

[github-actions]

🎉 This issue has been resolved in version 1.5.0 🎉

The release is available on:

• npm package
• GitHub release

Your optic bot 📦🚀