Skip to content

✨ api-server: listing of all solver jobs #7566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Apr 23, 2025

What do these changes do?

This PR exposes in the public API, listing of all solver jobs (list_all_solver_jobs) as dev-feature

image

Controller-layer (public-api):

  • Added a new API endpoint /-/releases/-/jobs to list all solver jobs with pagination (but not shown in OAS since include_in_schema=False)
  • For the sake of clarity, filters to this listing will follow up in a separate PR (NOTE to @mguidon )

Service-layer

  • Introduced a new list_jobs method in the SolverService class to fetch solver jobs with pagination and filtering by resource name prefix
  • New dependency of SolverService with underlying webserver's rpc client
  • Removed dependency injection in the constructors of services and clients and moved into api/dependencies. Depends creates a relation between service-layer and controller, therefore it should not be in the service-layer.

Schema and Validation Updates

  • Renamed the creation_at field to created_at in the ProjectJobRpcGet model and updated corresponding schema examples for consistency.
  • Added a new utility function trim_string_before to enforce string length limits during validation. (packages/models-library/src/models_library/utils/common_validators.py)

Related issue/s

How to test

cd services/api-server
make install-dev
make test-dev-unit

Dev-ops

None

@pcrespov pcrespov added the a:apiserver api-server service label Apr 23, 2025
@pcrespov pcrespov added this to the Pauwel Kwak milestone Apr 23, 2025
@pcrespov pcrespov self-assigned this Apr 23, 2025
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

Attention: Patch coverage is 92.04545% with 7 lines in your changes missing coverage. Please review.

Project coverage is 87.69%. Comparing base (bc131b7) to head (35e758f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #7566       +/-   ##
===========================================
+ Coverage   73.68%   87.69%   +14.01%     
===========================================
  Files        1086     1744      +658     
  Lines       43753    67676    +23923     
  Branches      534     1135      +601     
===========================================
+ Hits        32238    59351    +27113     
+ Misses      11372     8011     -3361     
- Partials      143      314      +171     
Flag Coverage Δ
integrationtests 65.01% <0.00%> (-0.06%) ⬇️
unittests 86.89% <92.04%> (-0.35%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.92% <ø> (∅)
pkg_dask_task_models_library 97.38% <ø> (ø)
pkg_models_library 92.73% <100.00%> (+<0.01%) ⬆️
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.27% <ø> (∅)
pkg_service_integration 69.92% <ø> (ø)
pkg_service_library 72.92% <ø> (∅)
pkg_settings_library 90.90% <ø> (ø)
pkg_simcore_sdk 85.72% <ø> (+0.05%) ⬆️
agent 96.46% <ø> (ø)
api_server 92.77% <93.50%> (∅)
autoscaling 96.08% <ø> (∅)
catalog 92.64% <ø> (∅)
clusters_keeper 99.25% <ø> (∅)
dask_sidecar 91.30% <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director 76.80% <ø> (∅)
director_v2 91.12% <ø> (+12.98%) ⬆️
dynamic_scheduler 97.40% <ø> (∅)
dynamic_sidecar 90.15% <ø> (+1.09%) ⬆️
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.18% <ø> (∅)
storage 87.56% <ø> (∅)
webclient ∅ <ø> (∅)
webserver 86.10% <33.33%> (+26.91%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc131b7...35e758f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov force-pushed the is7399/list-projects-rpc-api-server branch from 0b7e00d to 902c768 Compare April 23, 2025 17:36
@pcrespov pcrespov force-pushed the is7399/list-projects-rpc-api-server branch 4 times, most recently from 747001a to 751fea1 Compare April 28, 2025 15:10
@pcrespov pcrespov changed the title WIP: Is7399/list projects rpc api server ✨ api-server: listing of all solver jobs (as dev-feature) Apr 28, 2025
@pcrespov pcrespov marked this pull request as ready for review April 28, 2025 15:47
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@pcrespov pcrespov added the a:webserver issue related to the webserver service label Apr 28, 2025
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍 Let's go over the permission issue together. Also, in general, please keep in mind that we want to keep things flexible - for example, if we decide to introduce another Gateway in the future, like a CLI osparc tool, we should avoid a situation where different Gateways are talking directly to each other instead of communicating through proper services. Let say we would decide that jobs and projects and programs would be also exposed by the CLI tool..

Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@pcrespov pcrespov changed the title ✨ api-server: listing of all solver jobs (as dev-feature) ✨ api-server: listing of all solver jobs Apr 29, 2025
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@pcrespov pcrespov enabled auto-merge (squash) April 29, 2025 11:40
@pcrespov pcrespov added the 🤖-automerge marks PR as ready to be merged for Mergify label Apr 29, 2025
@pcrespov
Copy link
Member Author

@mergify queue

Copy link
Contributor

mergify bot commented Apr 29, 2025

queue

🛑 The pull request has been removed from the queue default

The following conditions don't match anymore:

  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • any of: [🛡 GitHub branch protection]
        • check-neutral = deploy to dockerhub
        • check-skipped = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-neutral = system-tests
        • check-skipped = system-tests
        • check-success = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-neutral = unit-tests
        • check-skipped = unit-tests
        • check-success = unit-tests
      • any of: [🛡 GitHub branch protection]
        • check-neutral = integration-tests
        • check-skipped = integration-tests
        • check-success = integration-tests

Copy link
Contributor

mergify bot commented Apr 29, 2025

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

@pcrespov pcrespov disabled auto-merge April 29, 2025 16:26
@pcrespov pcrespov merged commit 9d62136 into ITISFoundation:master Apr 29, 2025
1 of 2 checks passed
@pcrespov pcrespov deleted the is7399/list-projects-rpc-api-server branch April 29, 2025 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖-automerge marks PR as ready to be merged for Mergify a:apiserver api-server service a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants