Convert E2E scenario of PetStore and Todo project for smoke tests

[lirenhe]

Clear and concise description of the problem

Description:

We have the following projects to verify the E2E scenarios of TypeSpec.

https://github.com/allenjzhang/typespec-e2e-demo/tree/main/petstore
https://github.com/allenjzhang/typespec-e2e-demo/tree/main/todoApp

As we are not going to maintain that personal repo, we need to convert them into smoke tests and leveraging the existing pipeline to validate the E2E scenarios.
With this, every time there is a new change, we could verify it won't break the existing E2E.

cc @lmazuel @allenjzhang

Checklist

• Follow our Code of Conduct
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[timotheeguerin]

The purpose of spector/cadl-ranch is to define targetted scenarios, A whole spec is not really its goal, this just goes back to autorest test server which was a huge mess to understand the the goal of things.

[weidongxu-microsoft]

The purpose of spector/cadl-ranch is to define targetted scenarios, A whole spec is not really its goal, this just goes back to autorest test server which was a huge mess to understand the the goal of things.

Agree. I remember the agreed approach is to make sure all features (especially the feature that caused problem for emitter when we generate for PetStore and TodoApp) used in the E2E scenario is all covered in http-specs tests (e.g. pageable, multipart with HttpPart, status code range, error response etc.).

@allenjzhang Is it OK for me to list the features used in E2E scenario, and list the corresponding tests in http-specs? When they are on parity, we call this issue done?
Also, one question, what is your opinion on whether we should add tests for TypeSpec.Rest.Resource templates and models (it was used in PetStore)?

[timotheeguerin]

yeah please if there is some cases in petstore that cause problem please list them and add them one by one as individual scenarios

[lmazuel]

Agree, we should not move everything at once, but keep it as single scenarios for Spector.

However, I think there is value to have a set of spec that we all smoke tests emitters on, but this is not for Spector to do that.

[lirenhe]

Thanks all for the feedback. Update the title for the work needed.

[weidongxu-microsoft]

A few things to decide:

1. Do we plan to integrate it into microsoft/typespec CI? Or we put it in another repository with daily build pulling latest emitter (similar to nightly build)?
2. If we had it integrated into microsoft/typespec, what do we plan for the procedure when adding new API into smoke test. As the server side is only a stub and requires handwritten implementation? I am worried about some issue in server could affect CI of client.
3. If we had it in another repository, how do we sync the effort to update (e.g. we may get outdated code in that repo).

It appears OK to integrate it into microsoft/typespec.
We may put the TypeSpec of the smoke test into e.g. a smoke-http-specs lib (tsp only, without mockapi).
Then have client emitter and server emitter generate code based on this (server may need handwritten code to implement, and client may need handwritten code to call the API). And finally decide how to connect a client to a server.

• client part maybe easy, instead of start spector-server, start the server app and run tests as usual
• server part could be harder, it may code a simpler REST API client, or use test from one of the clients

Let me know what you think @allenjzhang