Skip to content

♻️Director v2: remove aiopg usage #7576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Apr 24, 2025

What do these changes do?

This PR aims at removing aiopg from the director-v2 codebase. Production code is replaced by asyncpg which is the new sqlalchemy asyncio standard. Note that tests are still using partially using aiopg due to pytest-simcore fixtures. This will be fixed later on.

Enhancements to database utilities:

  • Added maybe_await helper function to handle both synchronous and asynchronous database results, simplifying code that interacts with mixed database drivers (aiopg and asyncpg). ([[1]](https://github.com/ITISFoundation/osparc-simcore/pull/7576/files#diff-74af73d00e2ffe68b7b9ad870d543fd075a2a51f598fdad6ff1e8f643ad7fc0fR26-R64), [[2]](https://github.com/ITISFoundation/osparc-simcore/pull/7576/files#diff-2fee461956529bc30c6bd5d496dc0975a741178898019592813f97ae11c9fdc1L200-L223))
  • Updated database protocols (DBConnection, ResultProxy) to support both aiopg and asyncpg result methods, facilitating the migration to asyncpg. ([packages/postgres-database/src/simcore_postgres_database/_protocols.pyL5-R49](https://github.com/ITISFoundation/osparc-simcore/pull/7576/files#diff-5cd0106cd1e50cbedf7683d6502d867dc437caee25f5fe0e70579741de998660L5-R49))

Documentation and guidelines:

  • Updated copilot-instructions.md to recommend adding comments to functions only when strictly necessary. ([.github/copilot-instructions.mdR25](https://github.com/ITISFoundation/osparc-simcore/pull/7576/files#diff-227c2c26cb2ee0ce0f46a320fc48fbcbdf21801a57f59161b1d0861e8aad55f5R25))

Related issue/s

How to test

Dev-ops checklist

@sanderegg sanderegg added the t:maintenance Some planned maintenance work label Apr 24, 2025
@sanderegg sanderegg added this to the Pauwel Kwak milestone Apr 24, 2025
@sanderegg sanderegg self-assigned this Apr 24, 2025
@sanderegg sanderegg changed the title ♻️director v2: remove aiopg usage ♻️Director v2: remove aiopg usage Apr 24, 2025
Copy link

codecov bot commented Apr 24, 2025

Codecov Report

Attention: Patch coverage is 89.21569% with 22 lines in your changes missing coverage. Please review.

Project coverage is 90.09%. Comparing base (e653a4a) to head (e52ab00).
Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7576      +/-   ##
==========================================
+ Coverage   87.68%   90.09%   +2.40%     
==========================================
  Files        1774     1502     -272     
  Lines       68369    58609    -9760     
  Branches     1125      569     -556     
==========================================
- Hits        59952    52805    -7147     
+ Misses       8108     5648    -2460     
+ Partials      309      156     -153     
Flag Coverage Δ
integrationtests 65.10% <72.91%> (-0.02%) ⬇️
unittests 88.38% <86.76%> (+1.50%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library 92.72% <ø> (ø)
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.27% <87.36%> (+0.09%) ⬆️
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.72% <ø> (+0.05%) ⬆️
agent 96.46% <ø> (ø)
api_server 91.46% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.64% <ø> (ø)
clusters_keeper 99.25% <ø> (ø)
dask_sidecar 91.30% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.80% <ø> (ø)
director_v2 91.15% <91.66%> (-0.23%) ⬇️
dynamic_scheduler 97.40% <ø> (ø)
dynamic_sidecar 90.14% <ø> (-0.02%) ⬇️
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 88.96% <ø> (-0.17%) ⬇️
storage 87.49% <ø> (-0.08%) ⬇️
webclient ∅ <ø> (∅)
webserver 88.41% <ø> (+2.36%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e653a4a...e52ab00. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sanderegg sanderegg force-pushed the maintenance/director-v2/remove-aiopg branch 6 times, most recently from 718d428 to e9fbce2 Compare April 28, 2025 14:43
@sanderegg sanderegg marked this pull request as ready for review April 28, 2025 15:50
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx so much!

@sanderegg sanderegg force-pushed the maintenance/director-v2/remove-aiopg branch 2 times, most recently from dd99b1c to bca70f4 Compare April 29, 2025 06:16
@sanderegg sanderegg force-pushed the maintenance/director-v2/remove-aiopg branch 2 times, most recently from 8be5011 to 0d4ea55 Compare April 29, 2025 08:33
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💪 🙏

@sanderegg sanderegg force-pushed the maintenance/director-v2/remove-aiopg branch from 0d4ea55 to e52ab00 Compare April 29, 2025 08:37
@sanderegg sanderegg merged commit 2ff5c46 into ITISFoundation:master Apr 29, 2025
61 checks passed
@sanderegg sanderegg deleted the maintenance/director-v2/remove-aiopg branch April 29, 2025 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants