Skip to content

worker: add worker.getHeapStatistics() #57888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Apr 17, 2025

Conversation

mcollina
Copy link
Member

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 15, 2025
@mcollina
Copy link
Member Author

In node_v8.cc, the same is exposed via

node/src/node_v8.cc

Lines 57 to 71 in 2204587

#define HEAP_STATISTICS_PROPERTIES(V) \
V(0, total_heap_size, kTotalHeapSizeIndex) \
V(1, total_heap_size_executable, kTotalHeapSizeExecutableIndex) \
V(2, total_physical_size, kTotalPhysicalSizeIndex) \
V(3, total_available_size, kTotalAvailableSize) \
V(4, used_heap_size, kUsedHeapSizeIndex) \
V(5, heap_size_limit, kHeapSizeLimitIndex) \
V(6, malloced_memory, kMallocedMemoryIndex) \
V(7, peak_malloced_memory, kPeakMallocedMemoryIndex) \
V(8, does_zap_garbage, kDoesZapGarbageIndex) \
V(9, number_of_native_contexts, kNumberOfNativeContextsIndex) \
V(10, number_of_detached_contexts, kNumberOfDetachedContextsIndex) \
V(11, total_global_handles_size, kTotalGlobalHandlesSizeIndex) \
V(12, used_global_handles_size, kUsedGlobalHandlesSizeIndex) \
V(13, external_memory, kExternalMemoryIndex)
. Do you want me to use that approach here?

@mcollina mcollina added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 15, 2025
@mcollina mcollina force-pushed the worker-heap-statistics branch from daafec6 to 85d6960 Compare April 15, 2025 10:36
Adds worker.getHeapStatistics() so that the heap usage of the worker
could be observer from the parent thread.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
@mcollina mcollina force-pushed the worker-heap-statistics branch from c10f5e8 to c73d1b6 Compare April 15, 2025 10:59

v8::HeapStatistics heap_stats;
w->isolate_->GetHeapStatistics(&heap_stats);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use the object constructor that takes an array of keys and array of values, it will be faster

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you got an example I can copy from?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Src/node_v8.cc line 350

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Copy link
Member

@Qard Qard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has. An alternative implementation could be something like v8.getWorkerHeapStatistics(worker) to keep it in that v8-specific segment of the API, if that's something we care about. I would not block on this, but I figured it worth bringing up.

Other than that, agreed that the array form object construction would be better.

Copy link

codecov bot commented Apr 15, 2025

Codecov Report

Attention: Patch coverage is 90.21739% with 9 lines in your changes missing coverage. Please review.

Project coverage is 90.23%. Comparing base (2204587) to head (df57fa3).
Report is 15 commits behind head on main.

Files with missing lines Patch % Lines
src/node_worker.cc 88.88% 5 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #57888      +/-   ##
==========================================
- Coverage   90.24%   90.23%   -0.01%     
==========================================
  Files         630      630              
  Lines      185688   185783      +95     
  Branches    36403    36414      +11     
==========================================
+ Hits       167569   167647      +78     
- Misses      10997    11009      +12     
- Partials     7122     7127       +5     
Files with missing lines Coverage Δ
lib/internal/worker.js 99.82% <100.00%> (+<0.01%) ⬆️
src/node_worker.h 90.90% <ø> (ø)
src/node_worker.cc 84.45% <88.88%> (+0.72%) ⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@legendecas
Copy link
Member

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has.

There is a precedent of Worker.getHeapSnapshot, coresponding to v8.getHeapSnapshot.

@mcollina
Copy link
Member Author

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has.

There is also the event loop utilization monitoring that can be done from the outside, too.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
@mcollina
Copy link
Member Author

@joyeecheung @legendecas PTAL.

Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
mcollina and others added 3 commits April 15, 2025 21:50
Co-authored-by: Yagiz Nizipli <yagiz@nizipli.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025
@mcollina
Copy link
Member Author

@Qard @RaisinTen PTAL

@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

Signed-off-by: Matteo Collina <hello@matteocollina.com>
@mcollina
Copy link
Member Author

Sorry, I need another LGTM due to an unexpected test failure on ARM.

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@mcollina mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 17, 2025
@RaisinTen RaisinTen added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Apr 17, 2025
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 17, 2025
@nodejs-github-bot nodejs-github-bot merged commit 33d8e03 into nodejs:main Apr 17, 2025
63 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 33d8e03

@mcollina mcollina deleted the worker-heap-statistics branch April 17, 2025 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants