Description
New Feature / Enhancement Checklist
Current Limitation
Unless I'm misreading the code, it appears that under the hood the Parse.Object.saveAll
method is making many synchronous requests instead of batching them together into a single atomic Bulk request. I see in the MongoStorageAdapter
that Parse uses updateMany
and even deleteMany
, but I don't see insertMany
anywhere and the only create function appears to be createObject
(singular). The batch
endpoint handler also appears to be running the requests separately (here). Please let me know if I'm missing something that we are actually inserting documents in bulk!
Feature / Enhancement Description
By leveraging Mongo's native insert bulk function, we'd reduce number of connections and speed up large requests.
Example Use Case
I need to insert 1000 new documents but want the inserts to happen as close to simultaneously as possible.
Alternatives / Workarounds
Using the Mongo driver directly.
Activity
parse-github-assistant commentedon Jun 3, 2024
Thanks for opening this issue!
mtrezza commentedon Jun 4, 2024
Haven't looked into this now, but one reason may be what the MongoDB driver returns in its response. Depending on that this may need to be implemented as a new separate Parse method, or can be implemented as an improvement of the existing method.
pocketcolin commentedon Jun 4, 2024
Thanks for the insight, @mtrezza . That definitely makes sense. I started digging into the functionality because we noticed that when running a
saveAll
on a large array of new documents the new documents would appear in Mongo a few at a time as opposed to all at once. Mongo does some batching under the hood and indexes can take a second to update when large amounts of data is inserted at once, but the tests seem to indicate that at least some of the delay is on Parse's end. That said, when I really dug into the Mongo driver in Parse, I definitely got lost trying to follow the batch paths so it's totally possible I missed something and Parse is batching those requests. If you have time to verify one way or another, I might be able to help out with this improvement (maybe a new optional config param on saveAll?)!