You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1.5 KiB
1.5 KiB
Runtime split Patch B
Patch B builds on Patch A and makes the NEW runtime actually process embedding jobs.
What changes
1. New embedding job scheduler
Adds:
EmbeddingJobSchedulerEmbeddingJobSchedulingConfiguration
Behavior:
- enabled only in
NEWruntime mode - active only when
dip.embedding.jobs.enabled=true - periodically calls:
RepresentationEmbeddingOrchestrator.processNextReadyBatch()
2. Generic import hands off to the new embedding job path
GenericDocumentImportService is updated so that in NEW mode it:
- resolves
dip.embedding.default-document-model - ensures the model is registered in
DOC.doc_embedding_model - creates embedding jobs through:
RepresentationEmbeddingOrchestrator.enqueueRepresentation(...)
It no longer creates legacy-style pending embeddings as the primary handoff for the NEW runtime path.
Notes
- This patch assumes Patch A has already introduced:
RuntimeModeRuntimeModeProperties@ConditionalOnRuntimeMode
- This patch does not yet remove the legacy vectorization runtime. That remains the job of subsequent cutover steps.
Expected runtime behavior in NEW mode
GenericDocumentImportServicepersists new generic representations- selected representations are queued into
DOC.doc_embedding_job - scheduler processes pending jobs
- vectors are persisted through the new embedding subsystem
New config
Example:
dip:
runtime:
mode: NEW
embedding:
enabled: true
jobs:
enabled: true
scheduler-delay-ms: 5000