feat: Add runner observer architecture

[HyeockJinKim]

resolves #NNN (BA-MMM)

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue
• Installer updates including:
  • Fixtures for db schema changes
  • New mandatory config options
• Update of end-to-end CLI integration tests in ai.backend.test
• API server-client counterparts (e.g., manager API -> client SDK)
• Test case(s) to:
  • Demonstrate the difference of before/after
  • Demonstrate the flow of abstract/conceptual models with a concrete implementation
• Documentation
  • Contents in the docs directory
  • docstrings in public interfaces and type annotations

[Copilot]

Pull Request Overview

This PR implements a runner observer architecture and updates the relevant modules to support the new design while resolving outdated imports. Key changes include:

• Updating the PurgeImagesReq import paths in agent-related modules from dto.manager.rpc_request to dto.agent.rpc_request.
• Adding new abstract backend types and kernel abstractions to support the runner observer design.
• Introducing BUILD files for new backends (docker, dummy, kubernetes) and updating dependency lists.

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/ai/backend/agent/docker/agent.py	Updates the PurgeImagesReq import to use the agent RPC request module.
src/ai/backend/agent/backends/types.py	Introduces abstract backend types for supporting multiple backend architectures.
src/ai/backend/agent/backends/kernel.py	Adds new kernel abstractions and expands functionality for kernel creation and management.
src/ai/backend/agent/backends/image_registry.py	Provides an abstract interface for agent image registry interactions.
src/ai/backend/agent/backends/[kubernetes,dummy,docker]	Adds BUILD files for newly supported backends.
src/ai/backend/agent/agent.py	Updates the PurgeImagesReq import to use the agent RPC request module.
src/ai/backend/agent/BUILD	Updates dependency configurations to include new runner and observer modules.

Comments suppressed due to low confidence (2)

src/ai/backend/agent/docker/agent.py:54

• The import for PurgeImagesReq has been updated to reference dto.agent.rpc_request instead of dto.manager.rpc_request; please ensure that all dependent modules and usage contexts are adjusted accordingly.

from ai.backend.common.dto.agent.rpc_request import PurgeImagesReq

src/ai/backend/agent/agent.py:80

• The updated import for PurgeImagesReq in this file now points to dto.agent.rpc_request; verify that this change is consistently propagated to all related components to avoid integration issues.

from ai.backend.common.dto.agent.rpc_request import PurgeImagesReq

[github-advanced-security]

devskim found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.