Add support for Comet experiment data flush after saving checkpoint into CometLogger

[yaricom]

Description & Motivation

Periodic flushing of Comet experiment data by CometLogger is sometimes necessary to ensure all metrics and data remain up to date in the Comet UI and aligned with saved model checkpoints. Some users accomplish this by creating custom Callbacks, though this approach can be rather cumbersome.

Pitch

It is proposed to enhance the CometLogger implementation to support flushing collected data to Comet after each checkpoint is saved. This would be configurable via the flush_every parameter of the CometLogger.

This improvement allows users to manage Comet experiment flushing using the standard PyTorch Lightning approach by registering appropriate ModelCheckpoint callbacks with the Trainer.

Alternatives

No response

Additional context

No response

cc @lantiga @Borda

[yaricom]

The PR with implementation: #20680

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions - the Lightning Team!