Compare commits
7 Commits
32e9535bff
...
14a6f8d42e
| Author | SHA1 | Date |
|---|---|---|
|
|
14a6f8d42e | |
|
|
94767bc161 | |
|
|
818009555a | |
|
|
094007d817 | |
|
|
e4e9137af5 | |
|
|
17e2bbedf4 | |
|
|
900bfa5918 |
74
README.md
74
README.md
|
|
@ -542,23 +542,29 @@ order by p.received_at desc;
|
|||
|
||||
## Check acquired events
|
||||
|
||||
`eventhub.acquired_event` was replaced by the normalized `eventhub.event` and `eventhub.event_detail` tables.
|
||||
|
||||
```sql
|
||||
select occurred_at,
|
||||
driver_source_entity_id,
|
||||
driver_card_nation,
|
||||
driver_card_number,
|
||||
vehicle_source_entity_id,
|
||||
vehicle_vin,
|
||||
vehicle_registration_nation,
|
||||
vehicle_registration_number,
|
||||
event_domain,
|
||||
event_type,
|
||||
lifecycle,
|
||||
event_signature_hash,
|
||||
event_details,
|
||||
payload
|
||||
from eventhub.acquired_event
|
||||
order by occurred_at desc;
|
||||
select e.occurred_at,
|
||||
e.driver_source_entity_id,
|
||||
e.driver_card_nation,
|
||||
e.driver_card_number,
|
||||
e.vehicle_source_entity_id,
|
||||
e.vehicle_vin,
|
||||
e.vehicle_registration_nation,
|
||||
e.vehicle_registration_number,
|
||||
e.event_domain,
|
||||
e.event_type,
|
||||
e.lifecycle,
|
||||
e.event_signature_hash,
|
||||
d.detail_type,
|
||||
d.attributes as event_details,
|
||||
e.payload
|
||||
from eventhub.event e
|
||||
left join eventhub.event_detail d
|
||||
on d.event_occurred_at = e.occurred_at
|
||||
and d.event_id = e.id
|
||||
order by e.occurred_at desc;
|
||||
```
|
||||
|
||||
## Next implementation steps
|
||||
|
|
@ -639,7 +645,7 @@ For tachograph JDBC extraction, `sourcePackageId` is the original `FileLog.ID`.
|
|||
N planned data_package rows, one per extraction definition and time chunk
|
||||
```
|
||||
|
||||
The SQL extraction routes are intentionally separated from run planning. They should pick planned extraction packages, execute the corresponding SQL, map rows to `EventHubEventDto`, set `sourcePackageRef` when known, and send them to `direct:eventhub-normalized-input`.
|
||||
When `executeImmediately=true` or a configured plan is started with `triggerMode=EXECUTE`, the concrete extractor executes the planned packages, maps rows to `EventHubEventDto`, sets `sourcePackageRef` when known, persists synchronous JDBC batches through `EventHubIngestionService`, and advances the import cursor only after successful persistence.
|
||||
|
||||
### Initial import
|
||||
|
||||
|
|
@ -705,15 +711,18 @@ This is preferred because newly imported original driver-card/VU packages can co
|
|||
|
||||
### Extraction route contract
|
||||
|
||||
A future concrete SQL extraction route should do this:
|
||||
The concrete JDBC extraction route now does this:
|
||||
|
||||
```text
|
||||
planned data_package
|
||||
-> execute SQL for extraction_code and chunk/import scope
|
||||
-> apply source-package watermark parameters for incremental imports
|
||||
-> map source rows to EventHubEventDto
|
||||
-> populate sourcePackageRef if source package metadata is available
|
||||
-> send to direct:eventhub-normalized-input
|
||||
-> only advance eventhub.import_cursor after successful import
|
||||
-> populate sourcePackageRef when source package metadata is available
|
||||
-> hand off mapped events according to eventhub.tachograph.jdbc-extraction-ingest-mode
|
||||
- SYNC_DIRECT: persist controlled DB_EXTRACT packages through EventHubIngestionService
|
||||
- CAMEL_ROUTE: persist the same controlled batches through direct:eventhub-batch-persist-input
|
||||
-> advance eventhub.import_cursor after the configured extraction handoff completed
|
||||
```
|
||||
|
||||
## Configurable scheduled tachograph imports
|
||||
|
|
@ -739,6 +748,7 @@ eventhub:
|
|||
scheduler-enabled: false
|
||||
scheduler-poll-interval-ms: 60000
|
||||
scheduler-trigger-mode: PLAN_ONLY
|
||||
jdbc-extraction-ingest-mode: SYNC_DIRECT
|
||||
import-plans:
|
||||
- plan-key: kralowetz-tachograph-org-147
|
||||
enabled: false
|
||||
|
|
@ -790,13 +800,13 @@ POST /api/eventhub/acquisition/tachograph/imports/configured-plans/kralowetz-tac
|
|||
|
||||
## Concrete extraction extension point
|
||||
|
||||
The scheduler and import-run service are now implemented, but the generated skeleton still does not know the real tachograph DB SQL. The extension point is:
|
||||
The scheduler, import-run service and first JDBC SQL extractor are implemented. The extension point remains:
|
||||
|
||||
```java
|
||||
TachographExtractionBatchExecutor
|
||||
```
|
||||
|
||||
Replace `NoopTachographExtractionBatchExecutor` with an implementation that:
|
||||
`NoopTachographExtractionBatchExecutor` is used only when `eventhub.tachograph.datasource.jdbc-url` is empty. A custom executor can still replace it and should:
|
||||
|
||||
```text
|
||||
1. receives importRunId, packageId, TachographImportRequest, planItem and time chunk
|
||||
|
|
@ -805,15 +815,27 @@ Replace `NoopTachographExtractionBatchExecutor` with an implementation that:
|
|||
4. applies source-package watermark or source-row watermark for incremental updates
|
||||
5. maps rows to EventHubEventDto
|
||||
6. sets sourcePackageRef when the row can be traced to an original card/VU package
|
||||
7. sends events to direct:eventhub-normalized-input or EventHubIngestionService
|
||||
8. returns TachographExtractionBatchResultDto with cursor watermarks
|
||||
7. persist events through `EventHubIngestionService` or a route with explicit completion tracking
|
||||
8. return `TachographExtractionBatchResultDto` with cursor watermarks
|
||||
```
|
||||
|
||||
The import cursor is advanced only when the executor reports `executed=true`. The default no-op executor returns `executed=false`, so it does not move cursors accidentally.
|
||||
|
||||
## JDBC tachograph extraction
|
||||
|
||||
The first concrete extractor is `JdbcTachographExtractionBatchExecutor`. It is enabled only when `eventhub.tachograph.datasource.jdbc-url` is configured. Without that datasource, the application keeps using `NoopTachographExtractionBatchExecutor`.
|
||||
The first concrete extractor is `JdbcTachographExtractionBatchExecutor`. It is enabled only when `eventhub.tachograph.datasource.jdbc-url` is configured. The default `application.yml` maps this to `${TACHOGRAPH_DB_JDBC_URL:}`, so an empty environment does not create the tachograph datasource and the application keeps using `NoopTachographExtractionBatchExecutor`.
|
||||
|
||||
The JDBC extractor now has a configurable handoff mode via `eventhub.tachograph.jdbc-extraction-ingest-mode`:
|
||||
|
||||
```yaml
|
||||
eventhub:
|
||||
tachograph:
|
||||
jdbc-extraction-ingest-mode: SYNC_DIRECT # or CAMEL_ROUTE
|
||||
```
|
||||
|
||||
`SYNC_DIRECT` is the default and persists controlled `DB_EXTRACT` packages directly through `EventHubIngestionService`. This is the recommended mode for cursor-sensitive scheduled imports, because each extraction batch is written before the package is marked imported and before the cursor is advanced.
|
||||
|
||||
`CAMEL_ROUTE` keeps a Camel-based alternative without returning to the unsafe timeout-driven aggregation handoff: the extractor still builds controlled DB extraction batches, but sends each `EventHubEventBatchDto` through `direct:eventhub-batch-persist-input`. This route invokes the same `EventHubIngestionService` synchronously, so cursor advancement remains deterministic while the persistence handoff still goes through Camel.
|
||||
|
||||
Currently implemented extraction definitions:
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,94 @@
|
|||
# Esper PoC: Tachograph Driver-Card Activity Evaluation
|
||||
|
||||
This PoC intentionally uses only existing imported EventHub source events:
|
||||
|
||||
- provider: `TACHOGRAPH`
|
||||
- source kind: `DRIVER_CARD`
|
||||
- extraction code: `CARD_ACTIVITY`
|
||||
- event domain: `DRIVER_ACTIVITY`
|
||||
- event types: `DRIVE`, `WORK`, `AVAILABILITY`, `BREAK_REST`
|
||||
- lifecycles: `START`, `END`
|
||||
|
||||
It does not introduce canonical-event tables yet. The goal is to prove that Esper can replay one driver and one selected period and produce activity-level and operating-period results.
|
||||
|
||||
## Endpoint
|
||||
|
||||
```http
|
||||
GET /api/eventhub/esper-poc/tachograph/driver-card-activities
|
||||
```
|
||||
|
||||
Query parameters:
|
||||
|
||||
| Parameter | Required | Example | Meaning |
|
||||
|---|---:|---|---|
|
||||
| `tenantKey` | yes | `default` | EventHub tenant key. |
|
||||
| `driverEntityId` | yes | UUID | Existing `eventhub.event.driver_entity_id`. |
|
||||
| `occurredFrom` | yes | `2026-04-01T00:00:00Z` | Requested period start. |
|
||||
| `occurredTo` | yes | `2026-05-01T00:00:00Z` | Requested period end. |
|
||||
| `guardHours` | no | `24` | Extra loading window before/after requested period. Needed for activities crossing midnight/month boundaries and long rests crossing period boundaries. |
|
||||
| `significantDrivingMinutes` | no | `3` | DRIVE intervals longer than this threshold count as significant driving periods. |
|
||||
| `mergeGapSeconds` | no | `60` | Consecutive identical activities are merged if the gap is at most this value. |
|
||||
| `operatingPeriodSplitRestHours` | no | `7` | A `BREAK_REST` activity longer than this threshold splits operating time periods. |
|
||||
|
||||
## Produced levels
|
||||
|
||||
### Level 1: Raw
|
||||
|
||||
`raw` contains the original point events from `eventhub.event`.
|
||||
|
||||
### Level 2: Activities
|
||||
|
||||
Esper consumes the raw point events and produces intervals by pairing:
|
||||
|
||||
```text
|
||||
START + END with same sourceRowId, activity type, driver and card slot
|
||||
```
|
||||
|
||||
The service merges consecutive identical activities in the full guard window first, then clips the merged activities to the requested period. This is important because a long `BREAK_REST` crossing the requested boundary must keep its full guard-window duration for operating-period splitting.
|
||||
|
||||
## Operating time periods
|
||||
|
||||
`operatingTimePeriods` are derived from the merged activity timeline.
|
||||
|
||||
A `BREAK_REST` interval splits operating periods when:
|
||||
|
||||
```text
|
||||
activityType = BREAK_REST
|
||||
and duration > operatingPeriodSplitRestHours
|
||||
```
|
||||
|
||||
The default is 7 hours. With the default, a `BREAK_REST` of exactly 7 hours does not split; it must be longer than 7 hours.
|
||||
|
||||
Each operating period contains:
|
||||
|
||||
- `sequenceNumber`
|
||||
- `startedAt`
|
||||
- `endedAt`
|
||||
- `durationSeconds`
|
||||
- `activities`
|
||||
- `workingOperationTimes`
|
||||
- `drivingTimeInterruptionEvaluation`
|
||||
- optional references to the long rest before/after the period
|
||||
|
||||
Departure and arrival are evaluated per operating period:
|
||||
|
||||
- departure = first significant `DRIVE` interval inside that operating period
|
||||
- arrival = end of the last significant `DRIVE` interval inside that operating period
|
||||
- middle/interruption = gaps between significant `DRIVE` intervals inside the same operating period
|
||||
|
||||
## Result semantics
|
||||
|
||||
- `workResultPerDriver` and `workingOperationTimesPerEmployee` currently use the same PoC summary for the whole requested period.
|
||||
- `workingSeconds = DRIVE + WORK`.
|
||||
- `operationSeconds = DRIVE + WORK + AVAILABILITY`.
|
||||
- `breakRestSeconds` is reported separately.
|
||||
- Top-level `drivingTimeInterruptionEvaluation` evaluates the whole requested period.
|
||||
- Each item in `operatingTimePeriods` has its own `drivingTimeInterruptionEvaluation`.
|
||||
|
||||
## Current limitations
|
||||
|
||||
- Uses the existing source-level `driver_entity_id`, not a canonical employee table.
|
||||
- Reads only tachograph driver-card activity events.
|
||||
- Does not merge VU/card duplication.
|
||||
- Does not persist results; the endpoint returns a PoC calculation response.
|
||||
- Esper is used for interval creation. Summary, clipping, operating-period split, and merged activity report calculation are implemented in Java for auditability and easier future migration to canonical events.
|
||||
17
pom.xml
17
pom.xml
|
|
@ -20,6 +20,7 @@
|
|||
<properties>
|
||||
<java.version>21</java.version>
|
||||
<camel.version>4.18.2</camel.version>
|
||||
<esper.version>9.0.0</esper.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
|
|
@ -88,6 +89,22 @@
|
|||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.espertech</groupId>
|
||||
<artifactId>esper-common</artifactId>
|
||||
<version>${esper.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.espertech</groupId>
|
||||
<artifactId>esper-compiler</artifactId>
|
||||
<version>${esper.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.espertech</groupId>
|
||||
<artifactId>esper-runtime</artifactId>
|
||||
<version>${esper.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,150 @@
|
|||
{
|
||||
"info": {
|
||||
"name": "EventHub Esper PoC",
|
||||
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
|
||||
},
|
||||
"item": [
|
||||
{
|
||||
"name": "Evaluate tachograph driver-card activities",
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"header": [],
|
||||
"url": {
|
||||
"raw": "{{baseUrl}}/api/eventhub/esper-poc/tachograph/driver-card-activities?tenantKey={{tenantKey}}&driverEntityId={{driverEntityId}}&occurredFrom={{occurredFrom}}&occurredTo={{occurredTo}}&guardHours=24&significantDrivingMinutes=3&mergeGapSeconds=60&operatingPeriodSplitRestHours=7",
|
||||
"host": [
|
||||
"{{baseUrl}}"
|
||||
],
|
||||
"path": [
|
||||
"api",
|
||||
"eventhub",
|
||||
"esper-poc",
|
||||
"tachograph",
|
||||
"driver-card-activities"
|
||||
],
|
||||
"query": [
|
||||
{
|
||||
"key": "tenantKey",
|
||||
"value": "{{tenantKey}}"
|
||||
},
|
||||
{
|
||||
"key": "driverEntityId",
|
||||
"value": "{{driverEntityId}}"
|
||||
},
|
||||
{
|
||||
"key": "occurredFrom",
|
||||
"value": "{{occurredFrom}}"
|
||||
},
|
||||
{
|
||||
"key": "occurredTo",
|
||||
"value": "{{occurredTo}}"
|
||||
},
|
||||
{
|
||||
"key": "guardHours",
|
||||
"value": "24"
|
||||
},
|
||||
{
|
||||
"key": "significantDrivingMinutes",
|
||||
"value": "3"
|
||||
},
|
||||
{
|
||||
"key": "mergeGapSeconds",
|
||||
"value": "60"
|
||||
},
|
||||
{
|
||||
"key": "operatingPeriodSplitRestHours",
|
||||
"value": "7"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Evaluate tachograph operating periods",
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"header": [],
|
||||
"url": {
|
||||
"raw": "{{baseUrl}}/api/eventhub/esper-poc/tachograph/operating-period-evaluation?tenantKey={{tenantKey}}&driverId={{driverId}}&occurredFrom={{occurredFrom}}&occurredTo={{occurredTo}}&guardHours=24&operatingSplitIdleHours=7&significantDrivingMinutes=3&mergeGapSeconds=0&gapDetectionToleranceSeconds=0&unknownTreatmentMode=AS_BREAK_REST",
|
||||
"host": [
|
||||
"{{baseUrl}}"
|
||||
],
|
||||
"path": [
|
||||
"api",
|
||||
"eventhub",
|
||||
"esper-poc",
|
||||
"tachograph",
|
||||
"operating-period-evaluation"
|
||||
],
|
||||
"query": [
|
||||
{
|
||||
"key": "tenantKey",
|
||||
"value": "{{tenantKey}}"
|
||||
},
|
||||
{
|
||||
"key": "driverId",
|
||||
"value": "{{driverId}}"
|
||||
},
|
||||
{
|
||||
"key": "occurredFrom",
|
||||
"value": "{{occurredFrom}}"
|
||||
},
|
||||
{
|
||||
"key": "occurredTo",
|
||||
"value": "{{occurredTo}}"
|
||||
},
|
||||
{
|
||||
"key": "guardHours",
|
||||
"value": "24"
|
||||
},
|
||||
{
|
||||
"key": "operatingSplitIdleHours",
|
||||
"value": "7"
|
||||
},
|
||||
{
|
||||
"key": "significantDrivingMinutes",
|
||||
"value": "3"
|
||||
},
|
||||
{
|
||||
"key": "mergeGapSeconds",
|
||||
"value": "0"
|
||||
},
|
||||
{
|
||||
"key": "gapDetectionToleranceSeconds",
|
||||
"value": "0"
|
||||
},
|
||||
{
|
||||
"key": "unknownTreatmentMode",
|
||||
"value": "AS_BREAK_REST"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"variable": [
|
||||
{
|
||||
"key": "baseUrl",
|
||||
"value": "http://localhost:8080"
|
||||
},
|
||||
{
|
||||
"key": "tenantKey",
|
||||
"value": "default"
|
||||
},
|
||||
{
|
||||
"key": "driverEntityId",
|
||||
"value": "00000000-0000-0000-0000-000000000000"
|
||||
},
|
||||
{
|
||||
"key": "driverId",
|
||||
"value": "00000000-0000-0000-0000-000000000000"
|
||||
},
|
||||
{
|
||||
"key": "occurredFrom",
|
||||
"value": "2026-04-01T00:00:00Z"
|
||||
},
|
||||
{
|
||||
"key": "occurredTo",
|
||||
"value": "2026-05-01T00:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -69,6 +69,10 @@ public class EventHubCommonIngestionRoute extends RouteBuilder {
|
|||
.forceCompletionOnStop()
|
||||
.process(batchBuildProcessor)
|
||||
.bean(ingestionService, "ingest");
|
||||
|
||||
from("direct:eventhub-batch-persist-input")
|
||||
.routeId("eventhub-direct-batch-persist-route")
|
||||
.bean(ingestionService, "ingest");
|
||||
}
|
||||
|
||||
private String batchInputUri() {
|
||||
|
|
|
|||
|
|
@ -1,5 +1,8 @@
|
|||
package at.procon.eventhub.config;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.EsperActivityMergeMode;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperUnknownTreatmentMode;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperShiftResolutionMode;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.ArrayList;
|
||||
|
|
@ -26,6 +29,7 @@ public class EventHubProperties {
|
|||
|
||||
private final Batch batch = new Batch();
|
||||
private final Tachograph tachograph = new Tachograph();
|
||||
private final EsperPoc esperPoc = new EsperPoc();
|
||||
private final YellowFox yellowFox = new YellowFox();
|
||||
|
||||
public Batch getBatch() {
|
||||
|
|
@ -36,10 +40,90 @@ public class EventHubProperties {
|
|||
return tachograph;
|
||||
}
|
||||
|
||||
public EsperPoc getEsperPoc() {
|
||||
return esperPoc;
|
||||
}
|
||||
|
||||
public YellowFox getYellowFox() {
|
||||
return yellowFox;
|
||||
}
|
||||
|
||||
public static class EsperPoc {
|
||||
private EsperActivityMergeMode activityMergeMode = EsperActivityMergeMode.JAVA;
|
||||
private EsperShiftResolutionMode shiftResolutionMode = EsperShiftResolutionMode.JAVA;
|
||||
private final OperatingPeriodEvaluation operatingPeriodEvaluation = new OperatingPeriodEvaluation();
|
||||
|
||||
public EsperActivityMergeMode getActivityMergeMode() {
|
||||
return activityMergeMode;
|
||||
}
|
||||
|
||||
public void setActivityMergeMode(EsperActivityMergeMode activityMergeMode) {
|
||||
this.activityMergeMode = activityMergeMode == null ? EsperActivityMergeMode.JAVA : activityMergeMode;
|
||||
}
|
||||
|
||||
public EsperShiftResolutionMode getShiftResolutionMode() {
|
||||
return shiftResolutionMode;
|
||||
}
|
||||
|
||||
public void setShiftResolutionMode(EsperShiftResolutionMode shiftResolutionMode) {
|
||||
this.shiftResolutionMode = shiftResolutionMode == null ? EsperShiftResolutionMode.JAVA : shiftResolutionMode;
|
||||
}
|
||||
|
||||
public OperatingPeriodEvaluation getOperatingPeriodEvaluation() {
|
||||
return operatingPeriodEvaluation;
|
||||
}
|
||||
}
|
||||
|
||||
public static class OperatingPeriodEvaluation {
|
||||
private int operatingSplitIdleHours = 7;
|
||||
private int significantDrivingMinutes = 3;
|
||||
private int mergeGapSeconds = 0;
|
||||
private int gapDetectionToleranceSeconds = 0;
|
||||
private EsperUnknownTreatmentMode unknownTreatmentMode = EsperUnknownTreatmentMode.AS_BREAK_REST;
|
||||
|
||||
public int getOperatingSplitIdleHours() {
|
||||
return operatingSplitIdleHours;
|
||||
}
|
||||
|
||||
public void setOperatingSplitIdleHours(int operatingSplitIdleHours) {
|
||||
this.operatingSplitIdleHours = Math.max(1, operatingSplitIdleHours);
|
||||
}
|
||||
|
||||
public int getSignificantDrivingMinutes() {
|
||||
return significantDrivingMinutes;
|
||||
}
|
||||
|
||||
public void setSignificantDrivingMinutes(int significantDrivingMinutes) {
|
||||
this.significantDrivingMinutes = Math.max(1, significantDrivingMinutes);
|
||||
}
|
||||
|
||||
public int getMergeGapSeconds() {
|
||||
return mergeGapSeconds;
|
||||
}
|
||||
|
||||
public void setMergeGapSeconds(int mergeGapSeconds) {
|
||||
this.mergeGapSeconds = Math.max(0, mergeGapSeconds);
|
||||
}
|
||||
|
||||
public int getGapDetectionToleranceSeconds() {
|
||||
return gapDetectionToleranceSeconds;
|
||||
}
|
||||
|
||||
public void setGapDetectionToleranceSeconds(int gapDetectionToleranceSeconds) {
|
||||
this.gapDetectionToleranceSeconds = Math.max(0, gapDetectionToleranceSeconds);
|
||||
}
|
||||
|
||||
public EsperUnknownTreatmentMode getUnknownTreatmentMode() {
|
||||
return unknownTreatmentMode;
|
||||
}
|
||||
|
||||
public void setUnknownTreatmentMode(EsperUnknownTreatmentMode unknownTreatmentMode) {
|
||||
this.unknownTreatmentMode = unknownTreatmentMode == null
|
||||
? EsperUnknownTreatmentMode.AS_BREAK_REST
|
||||
: unknownTreatmentMode;
|
||||
}
|
||||
}
|
||||
|
||||
public static class Batch {
|
||||
/** Number of events collected before a package is persisted. */
|
||||
private int completionSize = 5000;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,95 @@
|
|||
package at.procon.eventhub.esperpoc.api;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.EsperActivityMergeMode;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperOperatingPeriodRequest;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperOperatingPeriodResultDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperPocRequest;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperPocResultDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperShiftResolutionMode;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperUnknownTreatmentMode;
|
||||
import at.procon.eventhub.esperpoc.service.EsperOperatingPeriodEvaluationService;
|
||||
import at.procon.eventhub.esperpoc.service.EsperPocDriverCardActivityService;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
import org.springframework.format.annotation.DateTimeFormat;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
@RestController
|
||||
@RequestMapping("/api/eventhub/esper-poc")
|
||||
public class EsperPocController {
|
||||
|
||||
private final EsperPocDriverCardActivityService service;
|
||||
private final EsperOperatingPeriodEvaluationService operatingPeriodEvaluationService;
|
||||
|
||||
public EsperPocController(
|
||||
EsperPocDriverCardActivityService service,
|
||||
EsperOperatingPeriodEvaluationService operatingPeriodEvaluationService
|
||||
) {
|
||||
this.service = service;
|
||||
this.operatingPeriodEvaluationService = operatingPeriodEvaluationService;
|
||||
}
|
||||
|
||||
@GetMapping("/tachograph/driver-card-activities")
|
||||
public ResponseEntity<EsperPocResultDto> evaluateDriverCardActivities(
|
||||
@RequestParam String tenantKey,
|
||||
@RequestParam UUID driverEntityId,
|
||||
@RequestParam @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME) OffsetDateTime occurredFrom,
|
||||
@RequestParam @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME) OffsetDateTime occurredTo,
|
||||
@RequestParam(defaultValue = "24") Integer guardHours,
|
||||
@RequestParam(defaultValue = "3") Integer significantDrivingMinutes,
|
||||
@RequestParam(defaultValue = "60") Integer mergeGapSeconds,
|
||||
@RequestParam(defaultValue = "7") Integer operatingPeriodSplitRestHours,
|
||||
@RequestParam(defaultValue = "420") Integer shiftEndMarkPeriodMinutes,
|
||||
@RequestParam(defaultValue = "5") Integer absenceBeginEndMinActivityMinutes,
|
||||
@RequestParam(required = false) EsperActivityMergeMode activityMergeMode,
|
||||
@RequestParam(required = false) EsperShiftResolutionMode shiftResolutionMode
|
||||
) {
|
||||
EsperPocRequest request = new EsperPocRequest(
|
||||
tenantKey,
|
||||
driverEntityId,
|
||||
occurredFrom,
|
||||
occurredTo,
|
||||
guardHours,
|
||||
significantDrivingMinutes,
|
||||
mergeGapSeconds,
|
||||
operatingPeriodSplitRestHours,
|
||||
shiftEndMarkPeriodMinutes,
|
||||
absenceBeginEndMinActivityMinutes,
|
||||
activityMergeMode,
|
||||
shiftResolutionMode
|
||||
);
|
||||
return ResponseEntity.ok(service.evaluate(request));
|
||||
}
|
||||
|
||||
@GetMapping("/tachograph/operating-period-evaluation")
|
||||
public ResponseEntity<EsperOperatingPeriodResultDto> evaluateOperatingPeriods(
|
||||
@RequestParam String tenantKey,
|
||||
@RequestParam UUID driverId,
|
||||
@RequestParam @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME) OffsetDateTime occurredFrom,
|
||||
@RequestParam @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME) OffsetDateTime occurredTo,
|
||||
@RequestParam(defaultValue = "24") Integer guardHours,
|
||||
@RequestParam(required = false) Integer operatingSplitIdleHours,
|
||||
@RequestParam(required = false) Integer significantDrivingMinutes,
|
||||
@RequestParam(required = false) Integer mergeGapSeconds,
|
||||
@RequestParam(required = false) Integer gapDetectionToleranceSeconds,
|
||||
@RequestParam(required = false) EsperUnknownTreatmentMode unknownTreatmentMode
|
||||
) {
|
||||
EsperOperatingPeriodRequest request = new EsperOperatingPeriodRequest(
|
||||
tenantKey,
|
||||
driverId,
|
||||
occurredFrom,
|
||||
occurredTo,
|
||||
guardHours,
|
||||
operatingSplitIdleHours,
|
||||
significantDrivingMinutes,
|
||||
mergeGapSeconds,
|
||||
gapDetectionToleranceSeconds,
|
||||
unknownTreatmentMode
|
||||
);
|
||||
return ResponseEntity.ok(operatingPeriodEvaluationService.evaluate(request));
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record ActivityIntervalDto(
|
||||
UUID driverEntityId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
String sourceRowId,
|
||||
List<String> sourceRowIds,
|
||||
boolean clippedToRequestedPeriod,
|
||||
String level
|
||||
) {
|
||||
public static ActivityIntervalDto raw(
|
||||
UUID driverEntityId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
String sourceRowId
|
||||
) {
|
||||
return new ActivityIntervalDto(
|
||||
driverEntityId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
activityType,
|
||||
cardSlot,
|
||||
cardStatus,
|
||||
drivingStatus,
|
||||
sourceKind,
|
||||
startedAt,
|
||||
endedAt,
|
||||
Duration.between(startedAt, endedAt).getSeconds(),
|
||||
sourceRowId,
|
||||
sourceRowId == null ? List.of() : List.of(sourceRowId),
|
||||
false,
|
||||
"RAW_INTERVAL"
|
||||
);
|
||||
}
|
||||
|
||||
public ActivityIntervalDto withTime(OffsetDateTime newStartedAt, OffsetDateTime newEndedAt, boolean clipped) {
|
||||
return new ActivityIntervalDto(
|
||||
driverEntityId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
activityType,
|
||||
cardSlot,
|
||||
cardStatus,
|
||||
drivingStatus,
|
||||
sourceKind,
|
||||
newStartedAt,
|
||||
newEndedAt,
|
||||
Duration.between(newStartedAt, newEndedAt).getSeconds(),
|
||||
sourceRowId,
|
||||
sourceRowIds,
|
||||
clipped,
|
||||
level
|
||||
);
|
||||
}
|
||||
|
||||
public ActivityIntervalDto asMerged(List<String> mergedSourceRowIds) {
|
||||
return new ActivityIntervalDto(
|
||||
driverEntityId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
activityType,
|
||||
cardSlot,
|
||||
cardStatus,
|
||||
drivingStatus,
|
||||
sourceKind,
|
||||
startedAt,
|
||||
endedAt,
|
||||
durationSeconds,
|
||||
sourceRowId,
|
||||
mergedSourceRowIds,
|
||||
clippedToRequestedPeriod,
|
||||
"MERGED_ACTIVITY"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
public record DriverWorkSummaryDto(
|
||||
UUID driverEntityId,
|
||||
OffsetDateTime periodFrom,
|
||||
OffsetDateTime periodTo,
|
||||
long drivingSeconds,
|
||||
long workSeconds,
|
||||
long availabilitySeconds,
|
||||
long breakRestSeconds,
|
||||
long workingSeconds,
|
||||
long operationSeconds,
|
||||
Map<String, Long> secondsByActivity
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
|
||||
public record DrivingInterruptionDto(
|
||||
OffsetDateTime from,
|
||||
OffsetDateTime to,
|
||||
long durationSeconds,
|
||||
String previousDrivingSourceRowId,
|
||||
String nextDrivingSourceRowId
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
public enum EsperActivityMergeMode {
|
||||
JAVA,
|
||||
ESPER
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperOperatingPeriodRequest(
|
||||
@NotBlank String tenantKey,
|
||||
@NotNull UUID driverId,
|
||||
@NotNull OffsetDateTime occurredFrom,
|
||||
@NotNull OffsetDateTime occurredTo,
|
||||
Integer guardHours,
|
||||
Integer operatingSplitIdleHours,
|
||||
Integer significantDrivingMinutes,
|
||||
Integer mergeGapSeconds,
|
||||
Integer gapDetectionToleranceSeconds,
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode
|
||||
) {
|
||||
public EsperOperatingPeriodRequest {
|
||||
if (occurredFrom != null && occurredTo != null && !occurredFrom.isBefore(occurredTo)) {
|
||||
throw new IllegalArgumentException("occurredFrom must be before occurredTo");
|
||||
}
|
||||
guardHours = guardHours == null ? 24 : Math.max(0, guardHours);
|
||||
operatingSplitIdleHours = operatingSplitIdleHours == null ? null : Math.max(1, operatingSplitIdleHours);
|
||||
significantDrivingMinutes = significantDrivingMinutes == null ? null : Math.max(1, significantDrivingMinutes);
|
||||
mergeGapSeconds = mergeGapSeconds == null ? null : Math.max(0, mergeGapSeconds);
|
||||
gapDetectionToleranceSeconds = gapDetectionToleranceSeconds == null ? null : Math.max(0, gapDetectionToleranceSeconds);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperOperatingPeriodResultDto(
|
||||
String tenantKey,
|
||||
UUID driverId,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo,
|
||||
OffsetDateTime loadedFrom,
|
||||
OffsetDateTime loadedTo,
|
||||
int rawEventCount,
|
||||
int driverCardRawEventCount,
|
||||
int vehicleUnitRawEventCount,
|
||||
int driverCardIntervalCount,
|
||||
int vehicleUnitIntervalCount,
|
||||
int resolvedKnownIntervalCount,
|
||||
int evaluationIntervalCount,
|
||||
int periodizedIntervalCount,
|
||||
int mergedIntervalCount,
|
||||
int nonDrivingIntervalCount,
|
||||
int operatingPeriodCount,
|
||||
int operatingSplitIdleHours,
|
||||
int significantDrivingMinutes,
|
||||
int mergeGapSeconds,
|
||||
int gapDetectionToleranceSeconds,
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode,
|
||||
List<RawActivityEventDto> rawEvents,
|
||||
List<ActivityIntervalDto> resolvedKnownIntervals,
|
||||
List<ActivityIntervalDto> evaluationIntervals,
|
||||
List<OperatingPeriodActivityIntervalDto> periodizedIntervals,
|
||||
List<OperatingPeriodActivityIntervalDto> mergedIntervals,
|
||||
List<NonDrivingIntervalDto> nonDrivingIntervals,
|
||||
List<OperatingPeriodDto> operatingPeriods,
|
||||
List<String> notes
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperPocRequest(
|
||||
@NotBlank String tenantKey,
|
||||
@NotNull UUID driverEntityId,
|
||||
@NotNull OffsetDateTime occurredFrom,
|
||||
@NotNull OffsetDateTime occurredTo,
|
||||
Integer guardHours,
|
||||
Integer significantDrivingMinutes,
|
||||
Integer mergeGapSeconds,
|
||||
Integer operatingPeriodSplitRestHours,
|
||||
Integer shiftEndMarkPeriodMinutes,
|
||||
Integer absenceBeginEndMinActivityMinutes,
|
||||
EsperActivityMergeMode activityMergeMode,
|
||||
EsperShiftResolutionMode shiftResolutionMode
|
||||
) {
|
||||
public EsperPocRequest {
|
||||
if (occurredFrom != null && occurredTo != null && !occurredFrom.isBefore(occurredTo)) {
|
||||
throw new IllegalArgumentException("occurredFrom must be before occurredTo");
|
||||
}
|
||||
guardHours = guardHours == null ? 24 : Math.max(0, guardHours);
|
||||
significantDrivingMinutes = significantDrivingMinutes == null ? 3 : Math.max(1, significantDrivingMinutes);
|
||||
mergeGapSeconds = mergeGapSeconds == null ? 60 : Math.max(0, mergeGapSeconds);
|
||||
operatingPeriodSplitRestHours = operatingPeriodSplitRestHours == null ? 7 : Math.max(1, operatingPeriodSplitRestHours);
|
||||
shiftEndMarkPeriodMinutes = shiftEndMarkPeriodMinutes == null ? 420 : Math.max(1, shiftEndMarkPeriodMinutes);
|
||||
absenceBeginEndMinActivityMinutes = absenceBeginEndMinActivityMinutes == null ? 5 : Math.max(1, absenceBeginEndMinActivityMinutes);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperPocResultDto(
|
||||
String tenantKey,
|
||||
UUID driverEntityId,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo,
|
||||
OffsetDateTime loadedFrom,
|
||||
OffsetDateTime loadedTo,
|
||||
int rawEventCount,
|
||||
int driverCardRawEventCount,
|
||||
int vehicleUnitRawEventCount,
|
||||
int driverCardRawIntervalCount,
|
||||
int vehicleUnitRawIntervalCount,
|
||||
int rawIntervalCount,
|
||||
int mergedActivityCount,
|
||||
int operatingTimePeriodCount,
|
||||
int resolvedWorkShiftCount,
|
||||
int operatingPeriodSplitRestHours,
|
||||
int shiftEndMarkPeriodMinutes,
|
||||
int absenceBeginEndMinActivityMinutes,
|
||||
EsperActivityMergeMode activityMergeMode,
|
||||
EsperShiftResolutionMode shiftResolutionMode,
|
||||
List<RawActivityEventDto> raw,
|
||||
List<ActivityIntervalDto> rawIntervals,
|
||||
List<ActivityIntervalDto> activities,
|
||||
List<OperatingTimePeriodDto> operatingTimePeriods,
|
||||
List<ResolvedWorkShiftDto> workingShifts,
|
||||
DriverWorkSummaryDto workResultPerDriver,
|
||||
DriverWorkSummaryDto workingOperationTimesPerEmployee,
|
||||
ShiftDrivingEvaluationDto drivingTimeInterruptionEvaluation,
|
||||
List<String> notes
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
public enum EsperShiftResolutionMode {
|
||||
JAVA,
|
||||
ESPER
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
public enum EsperUnknownTreatmentMode {
|
||||
AS_BREAK_REST,
|
||||
AS_UNKNOWN_NON_BREAK
|
||||
}
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
|
||||
public record NonDrivingIntervalDto(
|
||||
UUID driverId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String cardSlot,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
long operatingPeriodNo,
|
||||
OffsetDateTime operatingPeriodStartedAt,
|
||||
String closedBy,
|
||||
boolean containsUnknown,
|
||||
long unknownSeconds,
|
||||
boolean clippedToRequestedPeriod
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record OperatingPeriodActivityIntervalDto(
|
||||
UUID driverId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
String sourceRowId,
|
||||
List<String> sourceRowIds,
|
||||
boolean clippedToRequestedPeriod,
|
||||
String level,
|
||||
long operatingPeriodNo,
|
||||
OffsetDateTime operatingPeriodStartedAt,
|
||||
boolean newOperatingPeriod,
|
||||
Long gapSincePreviousActivitySeconds,
|
||||
boolean synthetic
|
||||
) {
|
||||
public static OperatingPeriodActivityIntervalDto periodized(
|
||||
ActivityIntervalDto interval,
|
||||
long operatingPeriodNo,
|
||||
OffsetDateTime operatingPeriodStartedAt,
|
||||
boolean newOperatingPeriod,
|
||||
Long gapSincePreviousActivitySeconds
|
||||
) {
|
||||
return new OperatingPeriodActivityIntervalDto(
|
||||
interval.driverEntityId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.activityType(),
|
||||
interval.cardSlot(),
|
||||
interval.cardStatus(),
|
||||
interval.drivingStatus(),
|
||||
interval.sourceKind(),
|
||||
interval.startedAt(),
|
||||
interval.endedAt(),
|
||||
interval.durationSeconds(),
|
||||
interval.sourceRowId(),
|
||||
interval.sourceRowIds(),
|
||||
interval.clippedToRequestedPeriod(),
|
||||
"PERIODIZED_ACTIVITY",
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
newOperatingPeriod,
|
||||
gapSincePreviousActivitySeconds,
|
||||
"UNKNOWN_GAP".equals(interval.level())
|
||||
);
|
||||
}
|
||||
|
||||
public OperatingPeriodActivityIntervalDto withTime(OffsetDateTime newStartedAt, OffsetDateTime newEndedAt, boolean clipped) {
|
||||
return new OperatingPeriodActivityIntervalDto(
|
||||
driverId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
activityType,
|
||||
cardSlot,
|
||||
cardStatus,
|
||||
drivingStatus,
|
||||
sourceKind,
|
||||
newStartedAt,
|
||||
newEndedAt,
|
||||
Duration.between(newStartedAt, newEndedAt).getSeconds(),
|
||||
sourceRowId,
|
||||
sourceRowIds,
|
||||
clipped,
|
||||
level,
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
newOperatingPeriod,
|
||||
gapSincePreviousActivitySeconds,
|
||||
synthetic
|
||||
);
|
||||
}
|
||||
|
||||
public OperatingPeriodActivityIntervalDto asMerged(OffsetDateTime newEndedAt, List<String> mergedSourceRowIds) {
|
||||
return new OperatingPeriodActivityIntervalDto(
|
||||
driverId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
activityType,
|
||||
cardSlot,
|
||||
cardStatus,
|
||||
drivingStatus,
|
||||
sourceKind,
|
||||
startedAt,
|
||||
newEndedAt,
|
||||
Duration.between(startedAt, newEndedAt).getSeconds(),
|
||||
sourceRowId,
|
||||
mergedSourceRowIds,
|
||||
clippedToRequestedPeriod,
|
||||
"MERGED_ACTIVITY",
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
newOperatingPeriod,
|
||||
gapSincePreviousActivitySeconds,
|
||||
synthetic
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
|
||||
public record OperatingPeriodDto(
|
||||
long operatingPeriodNo,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
String closedBy,
|
||||
List<ActivityIntervalDto> rawActivities,
|
||||
long breakRestSeconds,
|
||||
long drivingSeconds,
|
||||
long workSeconds,
|
||||
long availabilitySeconds,
|
||||
long unknownSeconds,
|
||||
int intervalCount,
|
||||
ShiftDrivingEvaluationDto drivingTimeInterruptionEvaluation,
|
||||
boolean clippedToRequestedPeriod
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
|
||||
public record OperatingTimePeriodDto(
|
||||
int sequenceNumber,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
ActivityIntervalDto splitStartedAfterLongRest,
|
||||
ActivityIntervalDto splitEndedByLongRest,
|
||||
List<ActivityIntervalDto> activities,
|
||||
DriverWorkSummaryDto workingOperationTimes,
|
||||
ShiftDrivingEvaluationDto drivingTimeInterruptionEvaluation
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
|
||||
public record RawActivityEventDto(
|
||||
UUID eventId,
|
||||
OffsetDateTime occurredAt,
|
||||
String sourceRowId,
|
||||
String externalSourceEventId,
|
||||
String sourceKind,
|
||||
String extractionCode,
|
||||
UUID driverEntityId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String lifecycle,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourcePackageId
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record ResolvedWorkShiftDto(
|
||||
int sequenceNumber,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
OffsetDateTime nextShiftStartedAt,
|
||||
long durationSeconds,
|
||||
long dailyRestingTimeSeconds,
|
||||
OffsetDateTime drivingFrom,
|
||||
OffsetDateTime drivingTo,
|
||||
OffsetDateTime absenceFrom,
|
||||
OffsetDateTime absenceTo,
|
||||
List<UUID> vehicleIds,
|
||||
List<UUID> vehicleRegistrationIds,
|
||||
String usedDataKind,
|
||||
String shiftKind,
|
||||
boolean beginTimeDeviationInRange,
|
||||
boolean endTimeDeviationInRange,
|
||||
boolean noDrivingActivity,
|
||||
boolean dailyRestingTimeBeforeShiftOver24Hours,
|
||||
boolean durationOver24Hours,
|
||||
boolean dailyRestingTimeBetween9And11Hours,
|
||||
boolean dailyRestingTimeBetween7And9Hours,
|
||||
int activitiesCount,
|
||||
int drivingCount,
|
||||
int availabilityCount,
|
||||
int workCount,
|
||||
int breakRestCount,
|
||||
int unknownCount,
|
||||
int workingTimeCount,
|
||||
DriverWorkSummaryDto workingOperationTimes,
|
||||
ShiftDrivingEvaluationDto drivingTimeInterruptionEvaluation,
|
||||
List<ActivityIntervalDto> activities
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
package at.procon.eventhub.esperpoc.dto;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
|
||||
public record ShiftDrivingEvaluationDto(
|
||||
int significantDrivingMinutes,
|
||||
OffsetDateTime departureAt,
|
||||
OffsetDateTime arrivalAt,
|
||||
ActivityIntervalDto firstSignificantDrivingPeriod,
|
||||
ActivityIntervalDto lastSignificantDrivingPeriod,
|
||||
List<DrivingInterruptionDto> interruptionsBetweenSignificantDrivingPeriods
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,95 @@
|
|||
package at.procon.eventhub.esperpoc.persistence;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.RawActivityEventDto;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.stereotype.Repository;
|
||||
|
||||
@Repository
|
||||
public class EsperPocActivityRepository {
|
||||
|
||||
private final JdbcTemplate jdbcTemplate;
|
||||
|
||||
public EsperPocActivityRepository(JdbcTemplate jdbcTemplate) {
|
||||
this.jdbcTemplate = jdbcTemplate;
|
||||
}
|
||||
|
||||
public List<RawActivityEventDto> findDriverActivityEvents(
|
||||
String tenantKey,
|
||||
UUID driverEntityId,
|
||||
OffsetDateTime occurredFrom,
|
||||
OffsetDateTime occurredTo
|
||||
) {
|
||||
return jdbcTemplate.query(
|
||||
"""
|
||||
select
|
||||
event.id,
|
||||
event.occurred_at,
|
||||
coalesce(
|
||||
event.payload #>> '{raw,sourceRowId}',
|
||||
regexp_replace(event.external_source_event_id, ':(START|END)$', '')
|
||||
) as source_row_id,
|
||||
event.external_source_event_id,
|
||||
source.source_kind,
|
||||
coalesce(pkg.extraction_code,
|
||||
case
|
||||
when source.source_kind = 'VEHICLE_UNIT' then 'VU_ACTIVITY'
|
||||
else 'CARD_ACTIVITY'
|
||||
end
|
||||
) as extraction_code,
|
||||
event.driver_id,
|
||||
event.vehicle_id,
|
||||
event.vehicle_registration_id,
|
||||
event.event_type,
|
||||
event.lifecycle,
|
||||
detail.attributes ->> 'cardSlot' as card_slot,
|
||||
detail.attributes ->> 'cardStatus' as card_status,
|
||||
detail.attributes ->> 'drivingStatus' as driving_status,
|
||||
event.source_package_id
|
||||
from eventhub.event event
|
||||
join eventhub.event_source source on source.id = event.event_source_id
|
||||
join eventhub.data_package pkg on pkg.id = event.data_package_id
|
||||
left join eventhub.event_detail detail on detail.event_occurred_at = event.occurred_at
|
||||
and detail.event_id = event.id
|
||||
and detail.detail_type = 'DRIVER_ACTIVITY'
|
||||
where pkg.tenant_key = ?
|
||||
and source.provider_key = 'TACHOGRAPH'
|
||||
and (
|
||||
(source.source_kind = 'DRIVER_CARD' and coalesce(pkg.extraction_code, 'CARD_ACTIVITY') = 'CARD_ACTIVITY')
|
||||
or
|
||||
(source.source_kind = 'VEHICLE_UNIT' and coalesce(pkg.extraction_code, 'VU_ACTIVITY') = 'VU_ACTIVITY')
|
||||
)
|
||||
and event.driver_id = ?
|
||||
and event.occurred_at >= ?
|
||||
and event.occurred_at < ?
|
||||
and event.event_domain = 'DRIVER_ACTIVITY'
|
||||
and event.event_type in ('DRIVE', 'WORK', 'AVAILABILITY', 'BREAK_REST')
|
||||
and event.lifecycle in ('START', 'END')
|
||||
order by event.occurred_at, event.lifecycle, event.event_type, event.id
|
||||
""",
|
||||
(rs, rowNum) -> new RawActivityEventDto(
|
||||
(UUID) rs.getObject("id"),
|
||||
rs.getObject("occurred_at", OffsetDateTime.class),
|
||||
rs.getString("source_row_id"),
|
||||
rs.getString("external_source_event_id"),
|
||||
rs.getString("source_kind"),
|
||||
rs.getString("extraction_code"),
|
||||
(UUID) rs.getObject("driver_id"),
|
||||
(UUID) rs.getObject("vehicle_id"),
|
||||
(UUID) rs.getObject("vehicle_registration_id"),
|
||||
rs.getString("event_type"),
|
||||
rs.getString("lifecycle"),
|
||||
rs.getString("card_slot"),
|
||||
rs.getString("card_status"),
|
||||
rs.getString("driving_status"),
|
||||
rs.getString("source_package_id")
|
||||
),
|
||||
tenantKey,
|
||||
driverEntityId,
|
||||
occurredFrom,
|
||||
occurredTo
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperActivityIntervalEvent(
|
||||
UUID driverEntityId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
String sourceRowId,
|
||||
List<String> sourceRowIds,
|
||||
boolean clippedToRequestedPeriod,
|
||||
String level
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import java.time.Duration;
|
||||
import java.time.LocalDate;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
|
||||
final class EsperActivitySemantics {
|
||||
|
||||
private static final Set<String> ACTIVE_ACTIVITY_TYPES = Set.of("DRIVE", "WORK", "AVAILABILITY");
|
||||
|
||||
private EsperActivitySemantics() {
|
||||
}
|
||||
|
||||
static boolean canMerge(ActivityIntervalDto left, ActivityIntervalDto right, Duration tolerance) {
|
||||
boolean sameDriver = Objects.equals(left.driverEntityId(), right.driverEntityId());
|
||||
boolean sameActivity = Objects.equals(left.activityType(), right.activityType());
|
||||
boolean sameSlot = Objects.equals(left.cardSlot(), right.cardSlot());
|
||||
boolean sameCardStatus = Objects.equals(left.cardStatus(), right.cardStatus());
|
||||
boolean sameDrivingStatus = Objects.equals(left.drivingStatus(), right.drivingStatus());
|
||||
boolean sameSourceKind = Objects.equals(left.sourceKind(), right.sourceKind());
|
||||
long gapSeconds = Duration.between(left.endedAt(), right.startedAt()).getSeconds();
|
||||
boolean adjacentOrOverlapping = gapSeconds <= tolerance.getSeconds();
|
||||
return sameDriver && sameActivity && sameSlot && sameCardStatus && sameDrivingStatus && sameSourceKind && adjacentOrOverlapping;
|
||||
}
|
||||
|
||||
static boolean isShiftActivity(ActivityIntervalDto interval) {
|
||||
return isKnownActivity(interval) && ACTIVE_ACTIVITY_TYPES.contains(interval.activityType());
|
||||
}
|
||||
|
||||
static boolean isRestOrUnknown(ActivityIntervalDto interval) {
|
||||
return ("BREAK_REST".equals(interval.activityType()) && isKnownActivity(interval))
|
||||
|| isUnknownActivity(interval);
|
||||
}
|
||||
|
||||
static boolean isKnownActivity(ActivityIntervalDto interval) {
|
||||
return interval.cardStatus() == null
|
||||
|| !"NOT_INSERTED".equals(interval.cardStatus())
|
||||
|| "KNOWN".equals(interval.drivingStatus());
|
||||
}
|
||||
|
||||
static boolean isUnknownActivity(ActivityIntervalDto interval) {
|
||||
if ("UNKNOWN_ACTIVITY".equals(interval.activityType())) {
|
||||
return true;
|
||||
}
|
||||
return "UNKNOWN".equals(interval.drivingStatus())
|
||||
|| ("NOT_INSERTED".equals(interval.cardStatus()) && !"KNOWN".equals(interval.drivingStatus()));
|
||||
}
|
||||
|
||||
static LocalDate recordDate(ActivityIntervalDto interval) {
|
||||
return interval.startedAt().toLocalDate();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
|
||||
record EsperClosedOperatingPeriod(
|
||||
long operatingPeriodNo,
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
long durationSeconds,
|
||||
String closedBy
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,405 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.RawActivityEventDto;
|
||||
import com.espertech.esper.common.client.EPCompiled;
|
||||
import com.espertech.esper.common.client.EventBean;
|
||||
import com.espertech.esper.common.client.configuration.Configuration;
|
||||
import com.espertech.esper.compiler.client.CompilerArguments;
|
||||
import com.espertech.esper.compiler.client.EPCompileException;
|
||||
import com.espertech.esper.compiler.client.EPCompilerProvider;
|
||||
import com.espertech.esper.runtime.client.EPDeployException;
|
||||
import com.espertech.esper.runtime.client.EPDeployment;
|
||||
import com.espertech.esper.runtime.client.EPRuntime;
|
||||
import com.espertech.esper.runtime.client.EPRuntimeProvider;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Comparator;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import java.util.function.Consumer;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class EsperDriverActivityEngine {
|
||||
|
||||
private static final AtomicLong RUNTIME_COUNTER = new AtomicLong();
|
||||
|
||||
private static final String INTERVAL_EPL = """
|
||||
@name('driverCardActivityIntervals')
|
||||
select
|
||||
s.driverEntityId as driverEntityId,
|
||||
s.vehicleId as vehicleId,
|
||||
s.vehicleRegistrationId as vehicleRegistrationId,
|
||||
s.eventType as activityType,
|
||||
s.cardSlot as cardSlot,
|
||||
s.cardStatus as cardStatus,
|
||||
s.drivingStatus as drivingStatus,
|
||||
s.sourceKind as sourceKind,
|
||||
s.occurredAt as startedAt,
|
||||
e.occurredAt as endedAt,
|
||||
s.sourceRowId as sourceRowId
|
||||
from pattern [
|
||||
every s = RawDriverActivityPoint(lifecycle = 'START') ->
|
||||
e = RawDriverActivityPoint(
|
||||
lifecycle = 'END',
|
||||
sourceRowId = s.sourceRowId,
|
||||
eventType = s.eventType,
|
||||
driverEntityId = s.driverEntityId,
|
||||
cardSlot = s.cardSlot
|
||||
)
|
||||
]
|
||||
""";
|
||||
|
||||
private static final String INTERVAL_STREAM_EPL = """
|
||||
@name('driverActivityIntervalStream')
|
||||
select * from DriverActivityInterval
|
||||
""";
|
||||
|
||||
public List<ActivityIntervalDto> buildIntervals(List<RawActivityEventDto> rawEvents) {
|
||||
if (rawEvents == null || rawEvents.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
|
||||
List<ActivityIntervalDto> intervals = new ArrayList<>();
|
||||
executeWithRuntime(
|
||||
configuration -> configuration.getCommon().addEventType("RawDriverActivityPoint", EsperRawDriverActivityPoint.class),
|
||||
INTERVAL_EPL,
|
||||
"driverCardActivityIntervals",
|
||||
newData -> collectIntervals(newData, intervals),
|
||||
runtime -> {
|
||||
List<EsperRawDriverActivityPoint> points = rawEvents.stream()
|
||||
.sorted(Comparator.comparing(RawActivityEventDto::occurredAt).thenComparing(RawActivityEventDto::lifecycle))
|
||||
.map(this::toEsperPoint)
|
||||
.toList();
|
||||
for (EsperRawDriverActivityPoint point : points) {
|
||||
runtime.getEventService().sendEventBean(point, "RawDriverActivityPoint");
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
return intervals.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt).thenComparing(ActivityIntervalDto::endedAt))
|
||||
.toList();
|
||||
}
|
||||
|
||||
public List<ActivityIntervalDto> mergeConsecutiveIdenticalActivities(
|
||||
List<ActivityIntervalDto> intervals,
|
||||
Duration mergeGapTolerance
|
||||
) {
|
||||
List<ActivityIntervalDto> sorted = sortedPositiveIntervals(intervals);
|
||||
if (sorted.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
|
||||
MergedActivityCollector collector = new MergedActivityCollector(mergeGapTolerance);
|
||||
executeIntervalStream(sorted, collector::accept);
|
||||
return collector.finish();
|
||||
}
|
||||
|
||||
public List<EsperResolvedShiftSpan> resolveShiftSpans(
|
||||
List<ActivityIntervalDto> intervals,
|
||||
Duration shiftEndThreshold,
|
||||
Duration contiguousGapTolerance
|
||||
) {
|
||||
List<ActivityIntervalDto> sorted = sortedPositiveIntervals(intervals);
|
||||
if (sorted.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
|
||||
ShiftSpanCollector collector = new ShiftSpanCollector(shiftEndThreshold, contiguousGapTolerance);
|
||||
executeIntervalStream(sorted, collector::accept);
|
||||
return collector.finish();
|
||||
}
|
||||
|
||||
private void executeIntervalStream(List<ActivityIntervalDto> intervals, Consumer<ActivityIntervalDto> consumer) {
|
||||
executeWithRuntime(
|
||||
configuration -> configuration.getCommon().addEventType("DriverActivityInterval", EsperActivityIntervalEvent.class),
|
||||
INTERVAL_STREAM_EPL,
|
||||
"driverActivityIntervalStream",
|
||||
newData -> collectInputIntervals(newData, consumer),
|
||||
runtime -> {
|
||||
for (ActivityIntervalDto interval : intervals) {
|
||||
runtime.getEventService().sendEventBean(toEsperInterval(interval), "DriverActivityInterval");
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
private void executeWithRuntime(
|
||||
Consumer<Configuration> configurationSetup,
|
||||
String epl,
|
||||
String statementName,
|
||||
Consumer<EventBean[]> listener,
|
||||
Consumer<EPRuntime> sender
|
||||
) {
|
||||
EPRuntime runtime = null;
|
||||
try {
|
||||
Configuration configuration = new Configuration();
|
||||
configurationSetup.accept(configuration);
|
||||
String runtimeUri = "eventhub-esper-poc-" + RUNTIME_COUNTER.incrementAndGet();
|
||||
runtime = EPRuntimeProvider.getRuntime(runtimeUri, configuration);
|
||||
|
||||
CompilerArguments arguments = new CompilerArguments(configuration);
|
||||
EPCompiled compiled = EPCompilerProvider.getCompiler().compile(epl, arguments);
|
||||
EPDeployment deployment = runtime.getDeploymentService().deploy(compiled);
|
||||
runtime.getDeploymentService()
|
||||
.getStatement(deployment.getDeploymentId(), statementName)
|
||||
.addListener((newData, oldData, statement, rt) -> listener.accept(newData));
|
||||
|
||||
sender.accept(runtime);
|
||||
} catch (EPCompileException | EPDeployException e) {
|
||||
throw new IllegalStateException("Cannot compile/deploy Esper PoC EPL", e);
|
||||
} finally {
|
||||
if (runtime != null) {
|
||||
runtime.destroy();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void collectIntervals(EventBean[] newData, List<ActivityIntervalDto> intervals) {
|
||||
if (newData == null) {
|
||||
return;
|
||||
}
|
||||
for (EventBean event : newData) {
|
||||
OffsetDateTime startedAt = (OffsetDateTime) event.get("startedAt");
|
||||
OffsetDateTime endedAt = (OffsetDateTime) event.get("endedAt");
|
||||
intervals.add(ActivityIntervalDto.raw(
|
||||
(UUID) event.get("driverEntityId"),
|
||||
(UUID) event.get("vehicleId"),
|
||||
(UUID) event.get("vehicleRegistrationId"),
|
||||
(String) event.get("activityType"),
|
||||
(String) event.get("cardSlot"),
|
||||
(String) event.get("cardStatus"),
|
||||
(String) event.get("drivingStatus"),
|
||||
(String) event.get("sourceKind"),
|
||||
startedAt,
|
||||
endedAt,
|
||||
(String) event.get("sourceRowId")
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
private void collectInputIntervals(EventBean[] newData, Consumer<ActivityIntervalDto> consumer) {
|
||||
if (newData == null) {
|
||||
return;
|
||||
}
|
||||
for (EventBean event : newData) {
|
||||
EsperActivityIntervalEvent interval = (EsperActivityIntervalEvent) event.getUnderlying();
|
||||
consumer.accept(new ActivityIntervalDto(
|
||||
interval.driverEntityId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.activityType(),
|
||||
interval.cardSlot(),
|
||||
interval.cardStatus(),
|
||||
interval.drivingStatus(),
|
||||
interval.sourceKind(),
|
||||
interval.startedAt(),
|
||||
interval.endedAt(),
|
||||
interval.durationSeconds(),
|
||||
interval.sourceRowId(),
|
||||
interval.sourceRowIds(),
|
||||
interval.clippedToRequestedPeriod(),
|
||||
interval.level()
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
private EsperRawDriverActivityPoint toEsperPoint(RawActivityEventDto event) {
|
||||
return new EsperRawDriverActivityPoint(
|
||||
event.eventId(),
|
||||
event.occurredAt(),
|
||||
event.sourceRowId(),
|
||||
event.externalSourceEventId(),
|
||||
event.driverEntityId(),
|
||||
event.vehicleId(),
|
||||
event.vehicleRegistrationId(),
|
||||
event.activityType(),
|
||||
event.lifecycle(),
|
||||
event.cardSlot(),
|
||||
event.cardStatus(),
|
||||
event.drivingStatus(),
|
||||
event.sourceKind()
|
||||
);
|
||||
}
|
||||
|
||||
private EsperActivityIntervalEvent toEsperInterval(ActivityIntervalDto interval) {
|
||||
return new EsperActivityIntervalEvent(
|
||||
interval.driverEntityId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.activityType(),
|
||||
interval.cardSlot(),
|
||||
interval.cardStatus(),
|
||||
interval.drivingStatus(),
|
||||
interval.sourceKind(),
|
||||
interval.startedAt(),
|
||||
interval.endedAt(),
|
||||
interval.durationSeconds(),
|
||||
interval.sourceRowId(),
|
||||
interval.sourceRowIds(),
|
||||
interval.clippedToRequestedPeriod(),
|
||||
interval.level()
|
||||
);
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> sortedPositiveIntervals(List<ActivityIntervalDto> intervals) {
|
||||
if (intervals == null || intervals.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
return intervals.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
private static final class MergedActivityCollector {
|
||||
private final Duration mergeGapTolerance;
|
||||
private final List<ActivityIntervalDto> merged = new ArrayList<>();
|
||||
private ActivityIntervalDto current;
|
||||
private List<String> currentSources = List.of();
|
||||
|
||||
private MergedActivityCollector(Duration mergeGapTolerance) {
|
||||
this.mergeGapTolerance = mergeGapTolerance;
|
||||
}
|
||||
|
||||
private void accept(ActivityIntervalDto next) {
|
||||
if (current == null) {
|
||||
current = next;
|
||||
currentSources = new ArrayList<>(next.sourceRowIds());
|
||||
return;
|
||||
}
|
||||
if (EsperActivitySemantics.canMerge(current, next, mergeGapTolerance)) {
|
||||
currentSources.addAll(next.sourceRowIds());
|
||||
OffsetDateTime newEndedAt = current.endedAt().isAfter(next.endedAt()) ? current.endedAt() : next.endedAt();
|
||||
current = new ActivityIntervalDto(
|
||||
current.driverEntityId(),
|
||||
current.vehicleId() == null ? next.vehicleId() : current.vehicleId(),
|
||||
current.vehicleRegistrationId() == null ? next.vehicleRegistrationId() : current.vehicleRegistrationId(),
|
||||
current.activityType(),
|
||||
current.cardSlot(),
|
||||
current.cardStatus(),
|
||||
current.drivingStatus(),
|
||||
current.sourceKind(),
|
||||
current.startedAt(),
|
||||
newEndedAt,
|
||||
Duration.between(current.startedAt(), newEndedAt).getSeconds(),
|
||||
current.sourceRowId(),
|
||||
List.copyOf(currentSources),
|
||||
current.clippedToRequestedPeriod() || next.clippedToRequestedPeriod(),
|
||||
"MERGED_ACTIVITY"
|
||||
);
|
||||
return;
|
||||
}
|
||||
merged.add(current.asMerged(List.copyOf(currentSources)));
|
||||
current = next;
|
||||
currentSources = new ArrayList<>(next.sourceRowIds());
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> finish() {
|
||||
if (current != null) {
|
||||
merged.add(current.asMerged(List.copyOf(currentSources)));
|
||||
}
|
||||
return List.copyOf(merged);
|
||||
}
|
||||
}
|
||||
|
||||
private static final class ShiftSpanCollector {
|
||||
private final Duration shiftEndThreshold;
|
||||
private final Duration contiguousGapTolerance;
|
||||
private final List<ActivityIntervalDto> seen = new ArrayList<>();
|
||||
private final List<EsperResolvedShiftSpan> spans = new ArrayList<>();
|
||||
private OffsetDateTime shiftBegin;
|
||||
private long accumulatedRestSeconds;
|
||||
private OffsetDateTime prevActivityEnd;
|
||||
private OffsetDateTime pendingShiftEndTriggerAt;
|
||||
|
||||
private ShiftSpanCollector(Duration shiftEndThreshold, Duration contiguousGapTolerance) {
|
||||
this.shiftEndThreshold = shiftEndThreshold;
|
||||
this.contiguousGapTolerance = contiguousGapTolerance;
|
||||
}
|
||||
|
||||
private void accept(ActivityIntervalDto interval) {
|
||||
seen.add(interval);
|
||||
if (shiftBegin == null) {
|
||||
if (EsperActivitySemantics.isShiftActivity(interval)) {
|
||||
shiftBegin = interval.startedAt();
|
||||
}
|
||||
prevActivityEnd = interval.endedAt();
|
||||
return;
|
||||
}
|
||||
if (!shiftBegin.isBefore(interval.startedAt())) {
|
||||
prevActivityEnd = interval.endedAt();
|
||||
return;
|
||||
}
|
||||
|
||||
if (pendingShiftEndTriggerAt != null && EsperActivitySemantics.isShiftActivity(interval)) {
|
||||
appendShiftSpan(shiftBegin, interval.startedAt());
|
||||
shiftBegin = interval.startedAt();
|
||||
pendingShiftEndTriggerAt = null;
|
||||
accumulatedRestSeconds = 0;
|
||||
}
|
||||
|
||||
if (EsperActivitySemantics.isRestOrUnknown(interval)) {
|
||||
long gapSeconds = prevActivityEnd == null ? 0 : Math.max(0, Duration.between(prevActivityEnd, interval.startedAt()).getSeconds());
|
||||
boolean contiguous = prevActivityEnd != null
|
||||
&& Math.abs(Duration.between(prevActivityEnd, interval.startedAt()).getSeconds()) <= contiguousGapTolerance.getSeconds();
|
||||
boolean crossedRecordDate = prevActivityEnd != null
|
||||
&& !interval.startedAt().toLocalDate().equals(prevActivityEnd.toLocalDate());
|
||||
if (contiguous) {
|
||||
accumulatedRestSeconds += interval.durationSeconds();
|
||||
} else if (crossedRecordDate) {
|
||||
accumulatedRestSeconds = interval.durationSeconds() + gapSeconds;
|
||||
} else {
|
||||
accumulatedRestSeconds = interval.durationSeconds();
|
||||
}
|
||||
if (accumulatedRestSeconds >= shiftEndThreshold.getSeconds() && pendingShiftEndTriggerAt == null) {
|
||||
pendingShiftEndTriggerAt = interval.startedAt();
|
||||
}
|
||||
} else if (EsperActivitySemantics.isShiftActivity(interval)) {
|
||||
long gapSeconds = prevActivityEnd == null ? 0 : Math.max(0, Duration.between(prevActivityEnd, interval.startedAt()).getSeconds());
|
||||
boolean crossedRecordDate = prevActivityEnd != null
|
||||
&& !interval.startedAt().toLocalDate().equals(prevActivityEnd.toLocalDate());
|
||||
if (crossedRecordDate && accumulatedRestSeconds + gapSeconds >= shiftEndThreshold.getSeconds()) {
|
||||
appendShiftSpan(shiftBegin, interval.startedAt());
|
||||
shiftBegin = interval.startedAt();
|
||||
}
|
||||
accumulatedRestSeconds = 0;
|
||||
pendingShiftEndTriggerAt = null;
|
||||
}
|
||||
|
||||
prevActivityEnd = interval.endedAt();
|
||||
}
|
||||
|
||||
private List<EsperResolvedShiftSpan> finish() {
|
||||
if (shiftBegin != null) {
|
||||
appendShiftSpan(shiftBegin, null);
|
||||
}
|
||||
return List.copyOf(spans);
|
||||
}
|
||||
|
||||
private void appendShiftSpan(OffsetDateTime currentShiftBegin, OffsetDateTime nextShiftBegin) {
|
||||
if (currentShiftBegin == null) {
|
||||
return;
|
||||
}
|
||||
OffsetDateTime shiftEnd = seen.stream()
|
||||
.filter(interval -> !interval.startedAt().isBefore(currentShiftBegin))
|
||||
.filter(interval -> nextShiftBegin == null || interval.startedAt().isBefore(nextShiftBegin))
|
||||
.filter(EsperActivitySemantics::isShiftActivity)
|
||||
.map(ActivityIntervalDto::endedAt)
|
||||
.max(OffsetDateTime::compareTo)
|
||||
.orElse(null);
|
||||
if (shiftEnd == null || !shiftEnd.isAfter(currentShiftBegin)) {
|
||||
return;
|
||||
}
|
||||
long restSeconds = nextShiftBegin == null ? 0 : Math.max(0, Duration.between(shiftEnd, nextShiftBegin).getSeconds());
|
||||
spans.add(new EsperResolvedShiftSpan(currentShiftBegin, shiftEnd, nextShiftBegin, restSeconds));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,257 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.OperatingPeriodActivityIntervalDto;
|
||||
import com.espertech.esper.common.client.EPCompiled;
|
||||
import com.espertech.esper.common.client.EventBean;
|
||||
import com.espertech.esper.common.client.configuration.Configuration;
|
||||
import com.espertech.esper.compiler.client.CompilerArguments;
|
||||
import com.espertech.esper.compiler.client.EPCompileException;
|
||||
import com.espertech.esper.compiler.client.EPCompilerProvider;
|
||||
import com.espertech.esper.runtime.client.EPDeployException;
|
||||
import com.espertech.esper.runtime.client.EPDeployment;
|
||||
import com.espertech.esper.runtime.client.EPRuntime;
|
||||
import com.espertech.esper.runtime.client.EPRuntimeProvider;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Comparator;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class EsperOperatingPeriodEngine {
|
||||
|
||||
private static final AtomicLong RUNTIME_COUNTER = new AtomicLong();
|
||||
private static final String INPUT_STREAM_EPL = """
|
||||
@name('operatingPeriodIntervalStream')
|
||||
select * from OperatingPeriodIntervalInputEvent
|
||||
""";
|
||||
|
||||
public EsperOperatingPeriodEvaluation evaluate(
|
||||
List<ActivityIntervalDto> intervals,
|
||||
Duration operatingSplitIdleThreshold
|
||||
) {
|
||||
List<ActivityIntervalDto> sorted = sortedPositiveIntervals(intervals);
|
||||
if (sorted.isEmpty()) {
|
||||
return new EsperOperatingPeriodEvaluation(List.of(), List.of());
|
||||
}
|
||||
PeriodizationCollector collector = new PeriodizationCollector(operatingSplitIdleThreshold);
|
||||
executeWithRuntime(
|
||||
configuration -> configuration.getCommon().addEventType("OperatingPeriodIntervalInputEvent", EsperOperatingPeriodIntervalInputEvent.class),
|
||||
INPUT_STREAM_EPL,
|
||||
"operatingPeriodIntervalStream",
|
||||
newData -> collectInputIntervals(newData, collector),
|
||||
runtime -> {
|
||||
for (ActivityIntervalDto interval : sorted) {
|
||||
runtime.getEventService().sendEventBean(toInputEvent(interval), "OperatingPeriodIntervalInputEvent");
|
||||
}
|
||||
}
|
||||
);
|
||||
return collector.finish();
|
||||
}
|
||||
|
||||
private void executeWithRuntime(
|
||||
java.util.function.Consumer<Configuration> configurationSetup,
|
||||
String epl,
|
||||
String statementName,
|
||||
java.util.function.Consumer<EventBean[]> listener,
|
||||
java.util.function.Consumer<EPRuntime> sender
|
||||
) {
|
||||
EPRuntime runtime = null;
|
||||
try {
|
||||
Configuration configuration = new Configuration();
|
||||
configurationSetup.accept(configuration);
|
||||
String runtimeUri = "eventhub-esper-operating-period-" + RUNTIME_COUNTER.incrementAndGet();
|
||||
runtime = EPRuntimeProvider.getRuntime(runtimeUri, configuration);
|
||||
|
||||
CompilerArguments arguments = new CompilerArguments(configuration);
|
||||
EPCompiled compiled = EPCompilerProvider.getCompiler().compile(epl, arguments);
|
||||
EPDeployment deployment = runtime.getDeploymentService().deploy(compiled);
|
||||
runtime.getDeploymentService()
|
||||
.getStatement(deployment.getDeploymentId(), statementName)
|
||||
.addListener((newData, oldData, statement, rt) -> listener.accept(newData));
|
||||
sender.accept(runtime);
|
||||
} catch (EPCompileException | EPDeployException e) {
|
||||
throw new IllegalStateException("Cannot compile/deploy Esper operating-period EPL", e);
|
||||
} finally {
|
||||
if (runtime != null) {
|
||||
runtime.destroy();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void collectInputIntervals(EventBean[] newData, PeriodizationCollector collector) {
|
||||
if (newData == null) {
|
||||
return;
|
||||
}
|
||||
for (EventBean event : newData) {
|
||||
collector.accept((EsperOperatingPeriodIntervalInputEvent) event.getUnderlying());
|
||||
}
|
||||
}
|
||||
|
||||
private EsperOperatingPeriodIntervalInputEvent toInputEvent(ActivityIntervalDto interval) {
|
||||
return new EsperOperatingPeriodIntervalInputEvent(
|
||||
interval.driverEntityId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.activityType(),
|
||||
interval.cardSlot(),
|
||||
interval.cardStatus(),
|
||||
interval.drivingStatus(),
|
||||
interval.sourceKind(),
|
||||
interval.startedAt().toInstant().toEpochMilli(),
|
||||
interval.endedAt().toInstant().toEpochMilli(),
|
||||
interval.durationSeconds() * 1000L,
|
||||
interval.sourceRowId(),
|
||||
interval.sourceRowIds(),
|
||||
interval.clippedToRequestedPeriod(),
|
||||
"UNKNOWN_GAP".equals(interval.level())
|
||||
);
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> sortedPositiveIntervals(List<ActivityIntervalDto> intervals) {
|
||||
if (intervals == null || intervals.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
return intervals.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
public record EsperOperatingPeriodEvaluation(
|
||||
List<OperatingPeriodActivityIntervalDto> periodizedIntervals,
|
||||
List<EsperClosedOperatingPeriod> closedPeriods
|
||||
) {
|
||||
}
|
||||
|
||||
private static final class PeriodizationCollector {
|
||||
private final Duration operatingSplitIdleThreshold;
|
||||
private final List<OperatingPeriodActivityIntervalDto> periodizedIntervals = new ArrayList<>();
|
||||
private final List<EsperClosedOperatingPeriod> closedPeriods = new ArrayList<>();
|
||||
private boolean hasOpenPeriod;
|
||||
private long operatingPeriodNo;
|
||||
private OffsetDateTime operatingPeriodStartedAt;
|
||||
private OffsetDateTime lastKnownActivityEndAt;
|
||||
|
||||
private PeriodizationCollector(Duration operatingSplitIdleThreshold) {
|
||||
this.operatingSplitIdleThreshold = operatingSplitIdleThreshold;
|
||||
}
|
||||
|
||||
private void accept(EsperOperatingPeriodIntervalInputEvent interval) {
|
||||
ActivityIntervalDto dto = new ActivityIntervalDto(
|
||||
interval.driverId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.activityType(),
|
||||
interval.cardSlot(),
|
||||
interval.cardStatus(),
|
||||
interval.drivingStatus(),
|
||||
interval.sourceKind(),
|
||||
OffsetDateTime.ofInstant(java.time.Instant.ofEpochMilli(interval.startTs()), java.time.ZoneOffset.UTC),
|
||||
OffsetDateTime.ofInstant(java.time.Instant.ofEpochMilli(interval.endTs()), java.time.ZoneOffset.UTC),
|
||||
interval.durationMs() / 1000L,
|
||||
interval.sourceRowId(),
|
||||
interval.sourceRowIds(),
|
||||
interval.clippedToRequestedPeriod(),
|
||||
interval.synthetic() ? "UNKNOWN_GAP" : "RAW_INTERVAL"
|
||||
);
|
||||
|
||||
if ("UNKNOWN".equals(dto.activityType())) {
|
||||
if (!hasOpenPeriod) {
|
||||
return;
|
||||
}
|
||||
if (dto.durationSeconds() >= operatingSplitIdleThreshold.getSeconds()) {
|
||||
closeCurrent("UNKNOWN_GAP");
|
||||
return;
|
||||
}
|
||||
periodizedIntervals.add(OperatingPeriodActivityIntervalDto.periodized(
|
||||
dto,
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
false,
|
||||
Math.max(0, Duration.between(lastKnownActivityEndAt, dto.startedAt()).getSeconds())
|
||||
));
|
||||
return;
|
||||
}
|
||||
|
||||
if (!hasOpenPeriod) {
|
||||
operatingPeriodNo = operatingPeriodNo < 1 ? 1 : operatingPeriodNo + 1;
|
||||
hasOpenPeriod = true;
|
||||
operatingPeriodStartedAt = dto.startedAt();
|
||||
lastKnownActivityEndAt = dto.endedAt();
|
||||
periodizedIntervals.add(OperatingPeriodActivityIntervalDto.periodized(
|
||||
dto,
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
true,
|
||||
null
|
||||
));
|
||||
return;
|
||||
}
|
||||
|
||||
long gapSeconds = Math.max(0, Duration.between(lastKnownActivityEndAt, dto.startedAt()).getSeconds());
|
||||
if (gapSeconds >= operatingSplitIdleThreshold.getSeconds()) {
|
||||
closeCurrent("IDLE_GAP");
|
||||
operatingPeriodNo++;
|
||||
hasOpenPeriod = true;
|
||||
operatingPeriodStartedAt = dto.startedAt();
|
||||
lastKnownActivityEndAt = dto.endedAt();
|
||||
periodizedIntervals.add(OperatingPeriodActivityIntervalDto.periodized(
|
||||
dto,
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
true,
|
||||
gapSeconds
|
||||
));
|
||||
return;
|
||||
}
|
||||
|
||||
periodizedIntervals.add(OperatingPeriodActivityIntervalDto.periodized(
|
||||
dto,
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
false,
|
||||
gapSeconds
|
||||
));
|
||||
if (dto.endedAt().isAfter(lastKnownActivityEndAt)) {
|
||||
lastKnownActivityEndAt = dto.endedAt();
|
||||
}
|
||||
}
|
||||
|
||||
private EsperOperatingPeriodEvaluation finish() {
|
||||
if (hasOpenPeriod) {
|
||||
closeCurrent("FLUSH");
|
||||
}
|
||||
return new EsperOperatingPeriodEvaluation(
|
||||
periodizedIntervals.stream()
|
||||
.sorted(Comparator.comparing(OperatingPeriodActivityIntervalDto::startedAt)
|
||||
.thenComparing(OperatingPeriodActivityIntervalDto::endedAt)
|
||||
.thenComparing(OperatingPeriodActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList(),
|
||||
closedPeriods.stream()
|
||||
.sorted(Comparator.comparing(EsperClosedOperatingPeriod::startedAt))
|
||||
.toList()
|
||||
);
|
||||
}
|
||||
|
||||
private void closeCurrent(String closedBy) {
|
||||
if (!hasOpenPeriod || operatingPeriodStartedAt == null || lastKnownActivityEndAt == null) {
|
||||
hasOpenPeriod = false;
|
||||
return;
|
||||
}
|
||||
closedPeriods.add(new EsperClosedOperatingPeriod(
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
lastKnownActivityEndAt,
|
||||
Duration.between(operatingPeriodStartedAt, lastKnownActivityEndAt).getSeconds(),
|
||||
closedBy
|
||||
));
|
||||
hasOpenPeriod = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,807 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.DrivingInterruptionDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperOperatingPeriodRequest;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperOperatingPeriodResultDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperUnknownTreatmentMode;
|
||||
import at.procon.eventhub.esperpoc.dto.NonDrivingIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.OperatingPeriodActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.OperatingPeriodDto;
|
||||
import at.procon.eventhub.esperpoc.dto.RawActivityEventDto;
|
||||
import at.procon.eventhub.esperpoc.dto.ShiftDrivingEvaluationDto;
|
||||
import at.procon.eventhub.esperpoc.persistence.EsperPocActivityRepository;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Comparator;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
public class EsperOperatingPeriodEvaluationService {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(EsperOperatingPeriodEvaluationService.class);
|
||||
|
||||
private final EsperPocActivityRepository activityRepository;
|
||||
private final EsperDriverActivityEngine activityEngine;
|
||||
private final EsperOperatingPeriodEngine operatingPeriodEngine;
|
||||
private final EventHubProperties properties;
|
||||
|
||||
public EsperOperatingPeriodEvaluationService(
|
||||
EsperPocActivityRepository activityRepository,
|
||||
EsperDriverActivityEngine activityEngine,
|
||||
EsperOperatingPeriodEngine operatingPeriodEngine
|
||||
) {
|
||||
this(activityRepository, activityEngine, operatingPeriodEngine, null);
|
||||
}
|
||||
|
||||
@Autowired
|
||||
public EsperOperatingPeriodEvaluationService(
|
||||
EsperPocActivityRepository activityRepository,
|
||||
EsperDriverActivityEngine activityEngine,
|
||||
EsperOperatingPeriodEngine operatingPeriodEngine,
|
||||
EventHubProperties properties
|
||||
) {
|
||||
this.activityRepository = activityRepository;
|
||||
this.activityEngine = activityEngine;
|
||||
this.operatingPeriodEngine = operatingPeriodEngine;
|
||||
this.properties = properties;
|
||||
}
|
||||
|
||||
public EsperOperatingPeriodResultDto evaluate(EsperOperatingPeriodRequest request) {
|
||||
long startedNanos = System.nanoTime();
|
||||
OffsetDateTime requestedFrom = utc(request.occurredFrom());
|
||||
OffsetDateTime requestedTo = utc(request.occurredTo());
|
||||
OffsetDateTime loadedFrom = requestedFrom.minusHours(request.guardHours());
|
||||
OffsetDateTime loadedTo = requestedTo.plusHours(request.guardHours());
|
||||
Duration splitIdleThreshold = Duration.ofHours(resolveOperatingSplitIdleHours(request));
|
||||
Duration significantDrivingThreshold = Duration.ofMinutes(resolveSignificantDrivingMinutes(request));
|
||||
Duration mergeGapTolerance = Duration.ofSeconds(resolveMergeGapSeconds(request));
|
||||
Duration gapDetectionTolerance = Duration.ofSeconds(resolveGapDetectionToleranceSeconds(request));
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode = resolveUnknownTreatmentMode(request);
|
||||
|
||||
long dbStartedNanos = System.nanoTime();
|
||||
List<RawActivityEventDto> rawEvents = activityRepository.findDriverActivityEvents(
|
||||
request.tenantKey(),
|
||||
request.driverId(),
|
||||
loadedFrom,
|
||||
loadedTo
|
||||
);
|
||||
long dbElapsedMs = elapsedMillis(dbStartedNanos);
|
||||
List<RawActivityEventDto> driverCardRawEvents = rawEvents.stream()
|
||||
.filter(event -> "DRIVER_CARD".equals(event.sourceKind()))
|
||||
.toList();
|
||||
List<RawActivityEventDto> vehicleUnitRawEvents = rawEvents.stream()
|
||||
.filter(event -> "VEHICLE_UNIT".equals(event.sourceKind()))
|
||||
.toList();
|
||||
|
||||
long cardIntervalsStartedNanos = System.nanoTime();
|
||||
List<ActivityIntervalDto> driverCardRawIntervals = activityEngine.buildIntervals(driverCardRawEvents);
|
||||
long cardIntervalsElapsedMs = elapsedMillis(cardIntervalsStartedNanos);
|
||||
long vuIntervalsStartedNanos = System.nanoTime();
|
||||
List<ActivityIntervalDto> vehicleUnitRawIntervals = activityEngine.buildIntervals(vehicleUnitRawEvents);
|
||||
long vuIntervalsElapsedMs = elapsedMillis(vuIntervalsStartedNanos);
|
||||
|
||||
long vuGapFillStartedNanos = System.nanoTime();
|
||||
List<ActivityIntervalDto> resolvedKnownLoadedIntervals = resolveVuFillGaps(driverCardRawIntervals, vehicleUnitRawIntervals);
|
||||
long vuGapFillElapsedMs = elapsedMillis(vuGapFillStartedNanos);
|
||||
|
||||
long unknownGapStartedNanos = System.nanoTime();
|
||||
List<ActivityIntervalDto> evaluationLoadedIntervals = synthesizeUnknownGaps(
|
||||
resolvedKnownLoadedIntervals,
|
||||
gapDetectionTolerance
|
||||
);
|
||||
long unknownGapElapsedMs = elapsedMillis(unknownGapStartedNanos);
|
||||
|
||||
long periodizeStartedNanos = System.nanoTime();
|
||||
EsperOperatingPeriodEngine.EsperOperatingPeriodEvaluation evaluation = operatingPeriodEngine.evaluate(
|
||||
evaluationLoadedIntervals,
|
||||
splitIdleThreshold
|
||||
);
|
||||
long periodizeElapsedMs = elapsedMillis(periodizeStartedNanos);
|
||||
|
||||
long mergeStartedNanos = System.nanoTime();
|
||||
List<OperatingPeriodActivityIntervalDto> mergedLoadedIntervals = mergeConsecutiveActivities(
|
||||
evaluation.periodizedIntervals(),
|
||||
mergeGapTolerance
|
||||
);
|
||||
long mergeElapsedMs = elapsedMillis(mergeStartedNanos);
|
||||
|
||||
long nonDrivingStartedNanos = System.nanoTime();
|
||||
List<NonDrivingIntervalDto> nonDrivingLoadedIntervals = buildNonDrivingIntervals(
|
||||
mergedLoadedIntervals,
|
||||
significantDrivingThreshold,
|
||||
unknownTreatmentMode
|
||||
);
|
||||
long nonDrivingElapsedMs = elapsedMillis(nonDrivingStartedNanos);
|
||||
|
||||
List<OperatingPeriodActivityIntervalDto> periodizedIntervals = clipOperatingIntervals(
|
||||
evaluation.periodizedIntervals(),
|
||||
requestedFrom,
|
||||
requestedTo
|
||||
);
|
||||
List<OperatingPeriodActivityIntervalDto> mergedIntervals = clipOperatingIntervals(
|
||||
mergedLoadedIntervals,
|
||||
requestedFrom,
|
||||
requestedTo
|
||||
);
|
||||
List<NonDrivingIntervalDto> nonDrivingIntervals = clipNonDrivingIntervals(
|
||||
nonDrivingLoadedIntervals,
|
||||
requestedFrom,
|
||||
requestedTo
|
||||
);
|
||||
List<OperatingPeriodDto> operatingPeriods = buildOperatingPeriods(
|
||||
evaluation.closedPeriods(),
|
||||
periodizedIntervals,
|
||||
clipKnownActivities(resolvedKnownLoadedIntervals, requestedFrom, requestedTo),
|
||||
requestedFrom,
|
||||
requestedTo,
|
||||
mergeGapTolerance,
|
||||
significantDrivingThreshold
|
||||
);
|
||||
long totalElapsedMs = elapsedMillis(startedNanos);
|
||||
|
||||
log.info("Esper operating-period evaluation tenant={} driverId={} requestedFrom={} requestedTo={} loadedFrom={} loadedTo={} unknownMode={} rawEvents={} cardRawEvents={} vuRawEvents={} cardIntervals={} vuIntervals={} resolvedKnownIntervals={} evaluationIntervals={} periodizedIntervals={} mergedIntervals={} nonDrivingIntervals={} operatingPeriods={} timingsMs={{dbRetrieve={}, cardIntervalEsper={}, vuIntervalEsper={}, vuGapFill={}, synthUnknown={}, periodizeEsper={}, merge={}, nonDriving={}, total={}}}",
|
||||
request.tenantKey(),
|
||||
request.driverId(),
|
||||
requestedFrom,
|
||||
requestedTo,
|
||||
loadedFrom,
|
||||
loadedTo,
|
||||
unknownTreatmentMode,
|
||||
rawEvents.size(),
|
||||
driverCardRawEvents.size(),
|
||||
vehicleUnitRawEvents.size(),
|
||||
driverCardRawIntervals.size(),
|
||||
vehicleUnitRawIntervals.size(),
|
||||
resolvedKnownLoadedIntervals.size(),
|
||||
evaluationLoadedIntervals.size(),
|
||||
periodizedIntervals.size(),
|
||||
mergedIntervals.size(),
|
||||
nonDrivingIntervals.size(),
|
||||
operatingPeriods.size(),
|
||||
dbElapsedMs,
|
||||
cardIntervalsElapsedMs,
|
||||
vuIntervalsElapsedMs,
|
||||
vuGapFillElapsedMs,
|
||||
unknownGapElapsedMs,
|
||||
periodizeElapsedMs,
|
||||
mergeElapsedMs,
|
||||
nonDrivingElapsedMs,
|
||||
totalElapsedMs);
|
||||
|
||||
return new EsperOperatingPeriodResultDto(
|
||||
request.tenantKey(),
|
||||
request.driverId(),
|
||||
requestedFrom,
|
||||
requestedTo,
|
||||
loadedFrom,
|
||||
loadedTo,
|
||||
rawEvents.size(),
|
||||
driverCardRawEvents.size(),
|
||||
vehicleUnitRawEvents.size(),
|
||||
driverCardRawIntervals.size(),
|
||||
vehicleUnitRawIntervals.size(),
|
||||
resolvedKnownLoadedIntervals.size(),
|
||||
evaluationLoadedIntervals.size(),
|
||||
periodizedIntervals.size(),
|
||||
mergedIntervals.size(),
|
||||
nonDrivingIntervals.size(),
|
||||
operatingPeriods.size(),
|
||||
resolveOperatingSplitIdleHours(request),
|
||||
resolveSignificantDrivingMinutes(request),
|
||||
resolveMergeGapSeconds(request),
|
||||
resolveGapDetectionToleranceSeconds(request),
|
||||
unknownTreatmentMode,
|
||||
rawEvents,
|
||||
resolvedKnownLoadedIntervals,
|
||||
evaluationLoadedIntervals,
|
||||
periodizedIntervals,
|
||||
mergedIntervals,
|
||||
nonDrivingIntervals,
|
||||
operatingPeriods,
|
||||
notes(
|
||||
unknownTreatmentMode,
|
||||
resolveOperatingSplitIdleHours(request),
|
||||
resolveSignificantDrivingMinutes(request),
|
||||
resolveGapDetectionToleranceSeconds(request)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
List<ActivityIntervalDto> synthesizeUnknownGaps(
|
||||
List<ActivityIntervalDto> resolvedKnownLoadedIntervals,
|
||||
Duration gapDetectionTolerance
|
||||
) {
|
||||
List<ActivityIntervalDto> allKnown = sortedPositiveIntervals(resolvedKnownLoadedIntervals);
|
||||
List<ActivityIntervalDto> nonRestActivities = allKnown.stream()
|
||||
.filter(interval -> !"BREAK_REST".equals(interval.activityType()))
|
||||
.toList();
|
||||
if (nonRestActivities.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
|
||||
List<ActivityIntervalDto> result = new ArrayList<>();
|
||||
for (int index = 0; index < nonRestActivities.size(); index++) {
|
||||
ActivityIntervalDto current = nonRestActivities.get(index);
|
||||
result.add(current);
|
||||
if (index + 1 >= nonRestActivities.size()) {
|
||||
continue;
|
||||
}
|
||||
ActivityIntervalDto next = nonRestActivities.get(index + 1);
|
||||
if (!next.startedAt().isAfter(current.endedAt())) {
|
||||
continue;
|
||||
}
|
||||
OffsetDateTime gapStart = current.endedAt();
|
||||
OffsetDateTime gapEnd = next.startedAt();
|
||||
List<ActivityIntervalDto> uncoveredGapSegments = subtractCoverage(
|
||||
unknownGapTemplate(current, next, gapStart, gapEnd),
|
||||
allKnown
|
||||
);
|
||||
for (ActivityIntervalDto gap : uncoveredGapSegments) {
|
||||
if (gap.durationSeconds() > gapDetectionTolerance.getSeconds()) {
|
||||
result.add(gap);
|
||||
}
|
||||
}
|
||||
}
|
||||
return result.stream()
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
List<OperatingPeriodActivityIntervalDto> mergeConsecutiveActivities(
|
||||
List<OperatingPeriodActivityIntervalDto> intervals,
|
||||
Duration mergeGapTolerance
|
||||
) {
|
||||
if (intervals == null || intervals.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
List<OperatingPeriodActivityIntervalDto> sorted = intervals.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.sorted(Comparator.comparing(OperatingPeriodActivityIntervalDto::startedAt)
|
||||
.thenComparing(OperatingPeriodActivityIntervalDto::endedAt)
|
||||
.thenComparing(OperatingPeriodActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
List<OperatingPeriodActivityIntervalDto> result = new ArrayList<>();
|
||||
OperatingPeriodActivityIntervalDto current = null;
|
||||
List<String> currentSources = new ArrayList<>();
|
||||
for (OperatingPeriodActivityIntervalDto next : sorted) {
|
||||
if (current == null) {
|
||||
current = next;
|
||||
currentSources = new ArrayList<>(next.sourceRowIds());
|
||||
continue;
|
||||
}
|
||||
if (canMerge(current, next, mergeGapTolerance)) {
|
||||
currentSources.addAll(next.sourceRowIds());
|
||||
current = current.asMerged(max(current.endedAt(), next.endedAt()), List.copyOf(currentSources));
|
||||
} else {
|
||||
result.add(current.asMerged(current.endedAt(), List.copyOf(currentSources)));
|
||||
current = next;
|
||||
currentSources = new ArrayList<>(next.sourceRowIds());
|
||||
}
|
||||
}
|
||||
if (current != null) {
|
||||
result.add(current.asMerged(current.endedAt(), List.copyOf(currentSources)));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
List<NonDrivingIntervalDto> buildNonDrivingIntervals(
|
||||
List<OperatingPeriodActivityIntervalDto> mergedIntervals,
|
||||
Duration significantDrivingThreshold,
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode
|
||||
) {
|
||||
if (mergedIntervals == null || mergedIntervals.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
List<NonDrivingIntervalDto> result = new ArrayList<>();
|
||||
NonDrivingAccumulator open = null;
|
||||
for (OperatingPeriodActivityIntervalDto interval : mergedIntervals) {
|
||||
if (open != null && open.operatingPeriodNo != interval.operatingPeriodNo()) {
|
||||
result.add(open.close("NEW_OPERATING_PERIOD"));
|
||||
open = null;
|
||||
}
|
||||
if (startsOrExtendsNonDriving(interval, unknownTreatmentMode)) {
|
||||
open = open == null ? NonDrivingAccumulator.open(interval) : open.extend(interval);
|
||||
continue;
|
||||
}
|
||||
if ("UNKNOWN".equals(interval.activityType()) && unknownTreatmentMode == EsperUnknownTreatmentMode.AS_UNKNOWN_NON_BREAK) {
|
||||
if (open != null) {
|
||||
result.add(open.close("UNKNOWN_GAP"));
|
||||
open = null;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if ("DRIVE".equals(interval.activityType())) {
|
||||
if (interval.durationSeconds() >= significantDrivingThreshold.getSeconds()) {
|
||||
if (open != null) {
|
||||
result.add(open.close("SIGNIFICANT_DRIVING"));
|
||||
open = null;
|
||||
}
|
||||
} else if (open != null) {
|
||||
open = open.extend(interval);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (open != null) {
|
||||
result.add(open.close("FLUSH"));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private boolean startsOrExtendsNonDriving(
|
||||
OperatingPeriodActivityIntervalDto interval,
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode
|
||||
) {
|
||||
if ("WORK".equals(interval.activityType()) || "AVAILABILITY".equals(interval.activityType())) {
|
||||
return true;
|
||||
}
|
||||
return "UNKNOWN".equals(interval.activityType())
|
||||
&& unknownTreatmentMode == EsperUnknownTreatmentMode.AS_BREAK_REST;
|
||||
}
|
||||
|
||||
private boolean canMerge(
|
||||
OperatingPeriodActivityIntervalDto left,
|
||||
OperatingPeriodActivityIntervalDto right,
|
||||
Duration mergeGapTolerance
|
||||
) {
|
||||
long gapSeconds = Duration.between(left.endedAt(), right.startedAt()).getSeconds();
|
||||
return Objects.equals(left.driverId(), right.driverId())
|
||||
&& left.operatingPeriodNo() == right.operatingPeriodNo()
|
||||
&& Objects.equals(left.activityType(), right.activityType())
|
||||
&& Objects.equals(left.cardSlot(), right.cardSlot())
|
||||
&& Objects.equals(left.cardStatus(), right.cardStatus())
|
||||
&& Objects.equals(left.drivingStatus(), right.drivingStatus())
|
||||
&& Objects.equals(left.sourceKind(), right.sourceKind())
|
||||
&& left.synthetic() == right.synthetic()
|
||||
&& gapSeconds <= mergeGapTolerance.getSeconds();
|
||||
}
|
||||
|
||||
private List<OperatingPeriodDto> buildOperatingPeriods(
|
||||
List<EsperClosedOperatingPeriod> closedPeriods,
|
||||
List<OperatingPeriodActivityIntervalDto> clippedPeriodizedIntervals,
|
||||
List<ActivityIntervalDto> clippedKnownActivities,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo,
|
||||
Duration mergeGapTolerance,
|
||||
Duration significantDrivingThreshold
|
||||
) {
|
||||
Map<Long, List<OperatingPeriodActivityIntervalDto>> intervalsByPeriod = new LinkedHashMap<>();
|
||||
for (OperatingPeriodActivityIntervalDto interval : clippedPeriodizedIntervals) {
|
||||
intervalsByPeriod.computeIfAbsent(interval.operatingPeriodNo(), ignored -> new ArrayList<>()).add(interval);
|
||||
}
|
||||
List<OperatingPeriodDto> result = new ArrayList<>();
|
||||
for (EsperClosedOperatingPeriod closedPeriod : closedPeriods) {
|
||||
if (!closedPeriod.endedAt().isAfter(requestedFrom) || !closedPeriod.startedAt().isBefore(requestedTo)) {
|
||||
continue;
|
||||
}
|
||||
OffsetDateTime start = max(closedPeriod.startedAt(), requestedFrom);
|
||||
OffsetDateTime end = min(closedPeriod.endedAt(), requestedTo);
|
||||
if (!end.isAfter(start)) {
|
||||
continue;
|
||||
}
|
||||
List<OperatingPeriodActivityIntervalDto> intervals = intervalsByPeriod.getOrDefault(closedPeriod.operatingPeriodNo(), List.of());
|
||||
List<ActivityIntervalDto> rawActivities = clipKnownActivitiesToPeriod(clippedKnownActivities, start, end);
|
||||
long breakRestSeconds = rawActivities.stream()
|
||||
.filter(activity -> "BREAK_REST".equals(activity.activityType()))
|
||||
.mapToLong(ActivityIntervalDto::durationSeconds)
|
||||
.sum();
|
||||
ShiftDrivingEvaluationDto drivingEvaluation = evaluateSignificantDrivingWithEsper(
|
||||
rawActivities,
|
||||
mergeGapTolerance,
|
||||
significantDrivingThreshold
|
||||
);
|
||||
long drivingSeconds = sumActivitySeconds(intervals, "DRIVE");
|
||||
long workSeconds = sumActivitySeconds(intervals, "WORK");
|
||||
long availabilitySeconds = sumActivitySeconds(intervals, "AVAILABILITY");
|
||||
long unknownSeconds = sumActivitySeconds(intervals, "UNKNOWN");
|
||||
result.add(new OperatingPeriodDto(
|
||||
closedPeriod.operatingPeriodNo(),
|
||||
start,
|
||||
end,
|
||||
Duration.between(start, end).getSeconds(),
|
||||
closedPeriod.closedBy(),
|
||||
rawActivities,
|
||||
breakRestSeconds,
|
||||
drivingSeconds,
|
||||
workSeconds,
|
||||
availabilitySeconds,
|
||||
unknownSeconds,
|
||||
intervals.size(),
|
||||
drivingEvaluation,
|
||||
!start.equals(closedPeriod.startedAt()) || !end.equals(closedPeriod.endedAt())
|
||||
));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private long sumActivitySeconds(List<OperatingPeriodActivityIntervalDto> intervals, String activityType) {
|
||||
return intervals.stream()
|
||||
.filter(interval -> activityType.equals(interval.activityType()))
|
||||
.mapToLong(OperatingPeriodActivityIntervalDto::durationSeconds)
|
||||
.sum();
|
||||
}
|
||||
|
||||
private List<OperatingPeriodActivityIntervalDto> clipOperatingIntervals(
|
||||
List<OperatingPeriodActivityIntervalDto> intervals,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo
|
||||
) {
|
||||
return intervals.stream()
|
||||
.map(interval -> {
|
||||
OffsetDateTime start = max(interval.startedAt(), requestedFrom);
|
||||
OffsetDateTime end = min(interval.endedAt(), requestedTo);
|
||||
if (!end.isAfter(start)) {
|
||||
return null;
|
||||
}
|
||||
boolean clipped = interval.clippedToRequestedPeriod()
|
||||
|| !start.equals(interval.startedAt())
|
||||
|| !end.equals(interval.endedAt());
|
||||
return interval.withTime(start, end, clipped);
|
||||
})
|
||||
.filter(Objects::nonNull)
|
||||
.toList();
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> clipKnownActivities(
|
||||
List<ActivityIntervalDto> activities,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo
|
||||
) {
|
||||
return clipKnownActivitiesToPeriod(activities, requestedFrom, requestedTo);
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> clipKnownActivitiesToPeriod(
|
||||
List<ActivityIntervalDto> activities,
|
||||
OffsetDateTime periodFrom,
|
||||
OffsetDateTime periodTo
|
||||
) {
|
||||
return activities.stream()
|
||||
.map(activity -> {
|
||||
OffsetDateTime start = max(activity.startedAt(), periodFrom);
|
||||
OffsetDateTime end = min(activity.endedAt(), periodTo);
|
||||
if (!end.isAfter(start)) {
|
||||
return null;
|
||||
}
|
||||
boolean clipped = activity.clippedToRequestedPeriod()
|
||||
|| !start.equals(activity.startedAt())
|
||||
|| !end.equals(activity.endedAt());
|
||||
return activity.withTime(start, end, clipped);
|
||||
})
|
||||
.filter(Objects::nonNull)
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
private List<NonDrivingIntervalDto> clipNonDrivingIntervals(
|
||||
List<NonDrivingIntervalDto> intervals,
|
||||
OffsetDateTime requestedFrom,
|
||||
OffsetDateTime requestedTo
|
||||
) {
|
||||
return intervals.stream()
|
||||
.map(interval -> {
|
||||
OffsetDateTime start = max(interval.startedAt(), requestedFrom);
|
||||
OffsetDateTime end = min(interval.endedAt(), requestedTo);
|
||||
if (!end.isAfter(start)) {
|
||||
return null;
|
||||
}
|
||||
return new NonDrivingIntervalDto(
|
||||
interval.driverId(),
|
||||
interval.vehicleId(),
|
||||
interval.vehicleRegistrationId(),
|
||||
interval.cardSlot(),
|
||||
start,
|
||||
end,
|
||||
Duration.between(start, end).getSeconds(),
|
||||
interval.operatingPeriodNo(),
|
||||
interval.operatingPeriodStartedAt(),
|
||||
interval.closedBy(),
|
||||
interval.containsUnknown(),
|
||||
interval.unknownSeconds(),
|
||||
!start.equals(interval.startedAt()) || !end.equals(interval.endedAt())
|
||||
);
|
||||
})
|
||||
.filter(Objects::nonNull)
|
||||
.toList();
|
||||
}
|
||||
|
||||
private ActivityIntervalDto unknownGapTemplate(
|
||||
ActivityIntervalDto previous,
|
||||
ActivityIntervalDto next,
|
||||
OffsetDateTime gapStart,
|
||||
OffsetDateTime gapEnd
|
||||
) {
|
||||
return new ActivityIntervalDto(
|
||||
previous.driverEntityId(),
|
||||
Objects.equals(previous.vehicleId(), next.vehicleId()) ? previous.vehicleId() : null,
|
||||
Objects.equals(previous.vehicleRegistrationId(), next.vehicleRegistrationId()) ? previous.vehicleRegistrationId() : null,
|
||||
"UNKNOWN",
|
||||
Objects.equals(previous.cardSlot(), next.cardSlot()) ? previous.cardSlot() : null,
|
||||
previous.cardStatus(),
|
||||
"UNKNOWN",
|
||||
"SYNTHETIC_GAP",
|
||||
gapStart,
|
||||
gapEnd,
|
||||
Duration.between(gapStart, gapEnd).getSeconds(),
|
||||
null,
|
||||
List.of(),
|
||||
false,
|
||||
"UNKNOWN_GAP"
|
||||
);
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> resolveVuFillGaps(
|
||||
List<ActivityIntervalDto> driverCardRawIntervals,
|
||||
List<ActivityIntervalDto> vehicleUnitRawIntervals
|
||||
) {
|
||||
List<ActivityIntervalDto> driverCard = sortedPositiveIntervals(driverCardRawIntervals);
|
||||
List<ActivityIntervalDto> vehicleUnit = sortedPositiveIntervals(vehicleUnitRawIntervals);
|
||||
if (driverCard.isEmpty()) {
|
||||
return vehicleUnit;
|
||||
}
|
||||
if (vehicleUnit.isEmpty()) {
|
||||
return driverCard;
|
||||
}
|
||||
|
||||
List<ActivityIntervalDto> resolved = new ArrayList<>(driverCard);
|
||||
for (ActivityIntervalDto vuInterval : vehicleUnit) {
|
||||
resolved.addAll(subtractCoverage(vuInterval, driverCard));
|
||||
}
|
||||
return resolved.stream()
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::sourceKind, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> subtractCoverage(
|
||||
ActivityIntervalDto candidate,
|
||||
List<ActivityIntervalDto> coverage
|
||||
) {
|
||||
List<ActivityIntervalDto> result = new ArrayList<>();
|
||||
OffsetDateTime cursor = candidate.startedAt();
|
||||
for (ActivityIntervalDto covered : coverage) {
|
||||
if (!covered.endedAt().isAfter(cursor)) {
|
||||
continue;
|
||||
}
|
||||
if (!covered.startedAt().isBefore(candidate.endedAt())) {
|
||||
break;
|
||||
}
|
||||
OffsetDateTime overlapStart = max(cursor, covered.startedAt());
|
||||
if (overlapStart.isAfter(cursor)) {
|
||||
result.add(candidate.withTime(cursor, overlapStart, candidate.clippedToRequestedPeriod()));
|
||||
}
|
||||
if (covered.endedAt().isAfter(cursor)) {
|
||||
cursor = max(cursor, covered.endedAt());
|
||||
}
|
||||
if (!candidate.endedAt().isAfter(cursor)) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (candidate.endedAt().isAfter(cursor)) {
|
||||
result.add(candidate.withTime(cursor, candidate.endedAt(), candidate.clippedToRequestedPeriod()));
|
||||
}
|
||||
return result.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.toList();
|
||||
}
|
||||
|
||||
private List<ActivityIntervalDto> sortedPositiveIntervals(List<ActivityIntervalDto> intervals) {
|
||||
if (intervals == null || intervals.isEmpty()) {
|
||||
return List.of();
|
||||
}
|
||||
return intervals.stream()
|
||||
.filter(interval -> interval.endedAt().isAfter(interval.startedAt()))
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt)
|
||||
.thenComparing(ActivityIntervalDto::endedAt)
|
||||
.thenComparing(ActivityIntervalDto::activityType, Comparator.nullsLast(String::compareTo)))
|
||||
.toList();
|
||||
}
|
||||
|
||||
private ShiftDrivingEvaluationDto evaluateSignificantDrivingWithEsper(
|
||||
List<ActivityIntervalDto> rawActivities,
|
||||
Duration mergeGapTolerance,
|
||||
Duration significantDrivingThreshold
|
||||
) {
|
||||
if (rawActivities.isEmpty()) {
|
||||
return new ShiftDrivingEvaluationDto(
|
||||
(int) significantDrivingThreshold.toMinutes(),
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
List.of()
|
||||
);
|
||||
}
|
||||
List<ActivityIntervalDto> mergedActivities = activityEngine.mergeConsecutiveIdenticalActivities(
|
||||
rawActivities,
|
||||
mergeGapTolerance
|
||||
);
|
||||
List<ActivityIntervalDto> significantDrivingPeriods = mergedActivities.stream()
|
||||
.filter(activity -> "DRIVE".equals(activity.activityType()))
|
||||
.filter(activity -> activity.durationSeconds() > significantDrivingThreshold.getSeconds())
|
||||
.sorted(Comparator.comparing(ActivityIntervalDto::startedAt))
|
||||
.toList();
|
||||
if (significantDrivingPeriods.isEmpty()) {
|
||||
return new ShiftDrivingEvaluationDto(
|
||||
(int) significantDrivingThreshold.toMinutes(),
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
List.of()
|
||||
);
|
||||
}
|
||||
List<DrivingInterruptionDto> interruptions = new ArrayList<>();
|
||||
for (int index = 1; index < significantDrivingPeriods.size(); index++) {
|
||||
ActivityIntervalDto previous = significantDrivingPeriods.get(index - 1);
|
||||
ActivityIntervalDto next = significantDrivingPeriods.get(index);
|
||||
if (next.startedAt().isAfter(previous.endedAt())) {
|
||||
interruptions.add(new DrivingInterruptionDto(
|
||||
previous.endedAt(),
|
||||
next.startedAt(),
|
||||
Duration.between(previous.endedAt(), next.startedAt()).getSeconds(),
|
||||
previous.sourceRowId(),
|
||||
next.sourceRowId()
|
||||
));
|
||||
}
|
||||
}
|
||||
ActivityIntervalDto first = significantDrivingPeriods.get(0);
|
||||
ActivityIntervalDto last = significantDrivingPeriods.get(significantDrivingPeriods.size() - 1);
|
||||
return new ShiftDrivingEvaluationDto(
|
||||
(int) significantDrivingThreshold.toMinutes(),
|
||||
first.startedAt(),
|
||||
last.endedAt(),
|
||||
first,
|
||||
last,
|
||||
interruptions
|
||||
);
|
||||
}
|
||||
|
||||
private int resolveOperatingSplitIdleHours(EsperOperatingPeriodRequest request) {
|
||||
if (request.operatingSplitIdleHours() != null) {
|
||||
return request.operatingSplitIdleHours();
|
||||
}
|
||||
return properties == null ? 7 : properties.getEsperPoc().getOperatingPeriodEvaluation().getOperatingSplitIdleHours();
|
||||
}
|
||||
|
||||
private int resolveSignificantDrivingMinutes(EsperOperatingPeriodRequest request) {
|
||||
if (request.significantDrivingMinutes() != null) {
|
||||
return request.significantDrivingMinutes();
|
||||
}
|
||||
return properties == null ? 3 : properties.getEsperPoc().getOperatingPeriodEvaluation().getSignificantDrivingMinutes();
|
||||
}
|
||||
|
||||
private int resolveMergeGapSeconds(EsperOperatingPeriodRequest request) {
|
||||
if (request.mergeGapSeconds() != null) {
|
||||
return request.mergeGapSeconds();
|
||||
}
|
||||
return properties == null ? 0 : properties.getEsperPoc().getOperatingPeriodEvaluation().getMergeGapSeconds();
|
||||
}
|
||||
|
||||
private int resolveGapDetectionToleranceSeconds(EsperOperatingPeriodRequest request) {
|
||||
if (request.gapDetectionToleranceSeconds() != null) {
|
||||
return request.gapDetectionToleranceSeconds();
|
||||
}
|
||||
return properties == null ? 0 : properties.getEsperPoc().getOperatingPeriodEvaluation().getGapDetectionToleranceSeconds();
|
||||
}
|
||||
|
||||
private EsperUnknownTreatmentMode resolveUnknownTreatmentMode(EsperOperatingPeriodRequest request) {
|
||||
if (request.unknownTreatmentMode() != null) {
|
||||
return request.unknownTreatmentMode();
|
||||
}
|
||||
return properties == null
|
||||
? EsperUnknownTreatmentMode.AS_BREAK_REST
|
||||
: properties.getEsperPoc().getOperatingPeriodEvaluation().getUnknownTreatmentMode();
|
||||
}
|
||||
|
||||
private List<String> notes(
|
||||
EsperUnknownTreatmentMode unknownTreatmentMode,
|
||||
int operatingSplitIdleHours,
|
||||
int significantDrivingMinutes,
|
||||
int gapDetectionToleranceSeconds
|
||||
) {
|
||||
return List.of(
|
||||
"This endpoint runs in parallel to the existing working-shift PoC and does not change its semantics.",
|
||||
"BREAK_REST events are ignored for activity evaluation but still prevent synthetic UNKNOWN intervals from being created over covered rest spans.",
|
||||
"Synthetic UNKNOWN intervals are created only for uncovered gaps between non-rest activities.",
|
||||
"UNKNOWN treatment mode is " + unknownTreatmentMode + ".",
|
||||
"Operating periods split after " + operatingSplitIdleHours + " hours of no non-rest activity; significant driving closes non-driving intervals from " + significantDrivingMinutes + " minutes onward.",
|
||||
"Synthetic UNKNOWN gaps are only emitted when uncovered time exceeds " + gapDetectionToleranceSeconds + " seconds."
|
||||
);
|
||||
}
|
||||
|
||||
private OffsetDateTime utc(OffsetDateTime value) {
|
||||
return value == null ? null : value.withOffsetSameInstant(java.time.ZoneOffset.UTC);
|
||||
}
|
||||
|
||||
private OffsetDateTime max(OffsetDateTime left, OffsetDateTime right) {
|
||||
return left.isAfter(right) ? left : right;
|
||||
}
|
||||
|
||||
private OffsetDateTime min(OffsetDateTime left, OffsetDateTime right) {
|
||||
return left.isBefore(right) ? left : right;
|
||||
}
|
||||
|
||||
private long elapsedMillis(long startedNanos) {
|
||||
return Math.round((System.nanoTime() - startedNanos) / 1_000_000.0d);
|
||||
}
|
||||
|
||||
private static final class NonDrivingAccumulator {
|
||||
private final UUID driverId;
|
||||
private UUID vehicleId;
|
||||
private UUID vehicleRegistrationId;
|
||||
private final String cardSlot;
|
||||
private final OffsetDateTime startedAt;
|
||||
private OffsetDateTime endedAt;
|
||||
private final long operatingPeriodNo;
|
||||
private final OffsetDateTime operatingPeriodStartedAt;
|
||||
private boolean containsUnknown;
|
||||
private long unknownSeconds;
|
||||
|
||||
private NonDrivingAccumulator(OperatingPeriodActivityIntervalDto interval) {
|
||||
this.driverId = interval.driverId();
|
||||
this.vehicleId = interval.vehicleId();
|
||||
this.vehicleRegistrationId = interval.vehicleRegistrationId();
|
||||
this.cardSlot = interval.cardSlot();
|
||||
this.startedAt = interval.startedAt();
|
||||
this.endedAt = interval.endedAt();
|
||||
this.operatingPeriodNo = interval.operatingPeriodNo();
|
||||
this.operatingPeriodStartedAt = interval.operatingPeriodStartedAt();
|
||||
this.containsUnknown = "UNKNOWN".equals(interval.activityType());
|
||||
this.unknownSeconds = "UNKNOWN".equals(interval.activityType()) ? interval.durationSeconds() : 0L;
|
||||
}
|
||||
|
||||
private static NonDrivingAccumulator open(OperatingPeriodActivityIntervalDto interval) {
|
||||
return new NonDrivingAccumulator(interval);
|
||||
}
|
||||
|
||||
private NonDrivingAccumulator extend(OperatingPeriodActivityIntervalDto interval) {
|
||||
if (interval.vehicleId() != null) {
|
||||
this.vehicleId = interval.vehicleId();
|
||||
}
|
||||
if (interval.vehicleRegistrationId() != null) {
|
||||
this.vehicleRegistrationId = interval.vehicleRegistrationId();
|
||||
}
|
||||
if (interval.endedAt().isAfter(this.endedAt)) {
|
||||
this.endedAt = interval.endedAt();
|
||||
}
|
||||
if ("UNKNOWN".equals(interval.activityType())) {
|
||||
this.containsUnknown = true;
|
||||
this.unknownSeconds += interval.durationSeconds();
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
private NonDrivingIntervalDto close(String closedBy) {
|
||||
return new NonDrivingIntervalDto(
|
||||
driverId,
|
||||
vehicleId,
|
||||
vehicleRegistrationId,
|
||||
cardSlot,
|
||||
startedAt,
|
||||
endedAt,
|
||||
Duration.between(startedAt, endedAt).getSeconds(),
|
||||
operatingPeriodNo,
|
||||
operatingPeriodStartedAt,
|
||||
closedBy,
|
||||
containsUnknown,
|
||||
unknownSeconds,
|
||||
false
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record EsperOperatingPeriodIntervalInputEvent(
|
||||
UUID driverId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String activityType,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind,
|
||||
long startTs,
|
||||
long endTs,
|
||||
long durationMs,
|
||||
String sourceRowId,
|
||||
List<String> sourceRowIds,
|
||||
boolean clippedToRequestedPeriod,
|
||||
boolean synthetic
|
||||
) {
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,102 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.UUID;
|
||||
|
||||
public final class EsperRawDriverActivityPoint {
|
||||
private final UUID eventId;
|
||||
private final OffsetDateTime occurredAt;
|
||||
private final String sourceRowId;
|
||||
private final String externalSourceEventId;
|
||||
private final UUID driverEntityId;
|
||||
private final UUID vehicleId;
|
||||
private final UUID vehicleRegistrationId;
|
||||
private final String eventType;
|
||||
private final String lifecycle;
|
||||
private final String cardSlot;
|
||||
private final String cardStatus;
|
||||
private final String drivingStatus;
|
||||
private final String sourceKind;
|
||||
|
||||
public EsperRawDriverActivityPoint(
|
||||
UUID eventId,
|
||||
OffsetDateTime occurredAt,
|
||||
String sourceRowId,
|
||||
String externalSourceEventId,
|
||||
UUID driverEntityId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String eventType,
|
||||
String lifecycle,
|
||||
String cardSlot,
|
||||
String cardStatus,
|
||||
String drivingStatus,
|
||||
String sourceKind
|
||||
) {
|
||||
this.eventId = eventId;
|
||||
this.occurredAt = occurredAt;
|
||||
this.sourceRowId = sourceRowId;
|
||||
this.externalSourceEventId = externalSourceEventId;
|
||||
this.driverEntityId = driverEntityId;
|
||||
this.vehicleId = vehicleId;
|
||||
this.vehicleRegistrationId = vehicleRegistrationId;
|
||||
this.eventType = eventType;
|
||||
this.lifecycle = lifecycle;
|
||||
this.cardSlot = cardSlot;
|
||||
this.cardStatus = cardStatus;
|
||||
this.drivingStatus = drivingStatus;
|
||||
this.sourceKind = sourceKind;
|
||||
}
|
||||
|
||||
public UUID getEventId() {
|
||||
return eventId;
|
||||
}
|
||||
|
||||
public OffsetDateTime getOccurredAt() {
|
||||
return occurredAt;
|
||||
}
|
||||
|
||||
public String getSourceRowId() {
|
||||
return sourceRowId;
|
||||
}
|
||||
|
||||
public String getExternalSourceEventId() {
|
||||
return externalSourceEventId;
|
||||
}
|
||||
|
||||
public UUID getDriverEntityId() {
|
||||
return driverEntityId;
|
||||
}
|
||||
|
||||
public UUID getVehicleId() {
|
||||
return vehicleId;
|
||||
}
|
||||
|
||||
public UUID getVehicleRegistrationId() {
|
||||
return vehicleRegistrationId;
|
||||
}
|
||||
|
||||
public String getEventType() {
|
||||
return eventType;
|
||||
}
|
||||
|
||||
public String getLifecycle() {
|
||||
return lifecycle;
|
||||
}
|
||||
|
||||
public String getCardSlot() {
|
||||
return cardSlot;
|
||||
}
|
||||
|
||||
public String getCardStatus() {
|
||||
return cardStatus;
|
||||
}
|
||||
|
||||
public String getDrivingStatus() {
|
||||
return drivingStatus;
|
||||
}
|
||||
|
||||
public String getSourceKind() {
|
||||
return sourceKind;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
|
||||
record EsperResolvedShiftSpan(
|
||||
OffsetDateTime startedAt,
|
||||
OffsetDateTime endedAt,
|
||||
OffsetDateTime nextShiftStartedAt,
|
||||
long dailyRestingTimeSeconds
|
||||
) {
|
||||
long durationSeconds() {
|
||||
return Duration.between(startedAt, endedAt).getSeconds();
|
||||
}
|
||||
}
|
||||
|
|
@ -1,16 +1,12 @@
|
|||
package at.procon.eventhub.importing;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.EventHubPackageRequest;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.dto.ImportRunStatus;
|
||||
import at.procon.eventhub.importing.persistence.ImportCursorRepository;
|
||||
import at.procon.eventhub.importing.persistence.ImportRunRepository;
|
||||
import at.procon.eventhub.persistence.DataPackageRepository.CamelBatchGroupStatus;
|
||||
import at.procon.eventhub.persistence.DataPackageRepository;
|
||||
import at.procon.eventhub.persistence.EventSourceRepository;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
|
|
@ -26,32 +22,23 @@ import org.slf4j.LoggerFactory;
|
|||
*/
|
||||
public abstract class AbstractImportExecutionService<R extends ImportRunRequest, B extends ExtractionBatchResult> {
|
||||
|
||||
private static final Duration ASYNC_INGEST_AWAIT_TIMEOUT = Duration.ofHours(6);
|
||||
private static final Duration ASYNC_INGEST_POLL_INTERVAL = Duration.ofSeconds(2);
|
||||
private static final Duration ASYNC_INGEST_FAILURE_GRACE_PERIOD = Duration.ofSeconds(90);
|
||||
private static final Duration ASYNC_INGEST_STALL_GRACE_PERIOD = Duration.ofSeconds(90);
|
||||
private static final Duration ASYNC_INGEST_PROGRESS_LOG_INTERVAL = Duration.ofSeconds(30);
|
||||
|
||||
private final Logger log = LoggerFactory.getLogger(getClass());
|
||||
|
||||
private final EventSourceRepository eventSourceRepository;
|
||||
private final ImportRunRepository importRunRepository;
|
||||
private final DataPackageRepository dataPackageRepository;
|
||||
private final ImportCursorRepository importCursorRepository;
|
||||
private final EventHubProperties eventHubProperties;
|
||||
|
||||
protected AbstractImportExecutionService(
|
||||
EventSourceRepository eventSourceRepository,
|
||||
ImportRunRepository importRunRepository,
|
||||
DataPackageRepository dataPackageRepository,
|
||||
ImportCursorRepository importCursorRepository,
|
||||
EventHubProperties eventHubProperties
|
||||
ImportCursorRepository importCursorRepository
|
||||
) {
|
||||
this.eventSourceRepository = eventSourceRepository;
|
||||
this.importRunRepository = importRunRepository;
|
||||
this.dataPackageRepository = dataPackageRepository;
|
||||
this.importCursorRepository = importCursorRepository;
|
||||
this.eventHubProperties = eventHubProperties;
|
||||
}
|
||||
|
||||
protected ImportRunResultDto createImportRun(R request, boolean executeImmediately) {
|
||||
|
|
@ -193,7 +180,6 @@ public abstract class AbstractImportExecutionService<R extends ImportRunRequest,
|
|||
"extractedEventTypeCounts", result.eventTypeCounts()
|
||||
));
|
||||
}
|
||||
awaitAsyncIngestCompletion(importRunId, request, plannedPackage, result);
|
||||
results.add(result);
|
||||
dataPackageRepository.markImported(plannedPackage.packageId(), result.eventsInserted());
|
||||
if (result.executed()) {
|
||||
|
|
@ -221,107 +207,6 @@ public abstract class AbstractImportExecutionService<R extends ImportRunRequest,
|
|||
results.stream().filter(ExtractionBatchResult::executed).count());
|
||||
}
|
||||
|
||||
private void awaitAsyncIngestCompletion(UUID importRunId, R request, PlannedPackage plannedPackage, B result) {
|
||||
int expectedCamelBatches = expectedCamelBatchCount(result.eventsInserted());
|
||||
if (expectedCamelBatches <= 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
EventHubPackageRequest packageInfo = extractionAggregatePackageInfo(
|
||||
importRunId,
|
||||
request,
|
||||
eventSourceForItem(request.eventSource(), plannedPackage.planItem()),
|
||||
plannedPackage.planItem(),
|
||||
plannedPackage.chunk()
|
||||
);
|
||||
String aggregatePackageKey = aggregatePackageKey(packageInfo);
|
||||
Instant deadline = Instant.now().plus(ASYNC_INGEST_AWAIT_TIMEOUT);
|
||||
Instant nextProgressLogAt = Instant.now().plus(ASYNC_INGEST_PROGRESS_LOG_INTERVAL);
|
||||
Instant failedStateObservedAt = null;
|
||||
Instant lastStateChangeAt = Instant.now();
|
||||
CamelBatchGroupStatus previousState = null;
|
||||
|
||||
while (Instant.now().isBefore(deadline)) {
|
||||
CamelBatchGroupStatus state = dataPackageRepository.findCamelBatchGroupStatus(
|
||||
plannedPackage.eventSourceId(),
|
||||
request.tenantKey(),
|
||||
aggregatePackageKey
|
||||
);
|
||||
if (state.totalCount() >= expectedCamelBatches && state.successCount() >= expectedCamelBatches) {
|
||||
return;
|
||||
}
|
||||
|
||||
boolean stateChanged = previousState == null || !previousState.equals(state);
|
||||
if (stateChanged) {
|
||||
lastStateChangeAt = Instant.now();
|
||||
}
|
||||
if (state.failedCount() > 0 && stateChanged) {
|
||||
failedStateObservedAt = Instant.now();
|
||||
} else if (state.failedCount() == 0) {
|
||||
failedStateObservedAt = null;
|
||||
}
|
||||
|
||||
if (state.failedCount() > 0
|
||||
&& failedStateObservedAt != null
|
||||
&& Instant.now().isAfter(failedStateObservedAt.plus(ASYNC_INGEST_FAILURE_GRACE_PERIOD))) {
|
||||
throw new IllegalStateException(
|
||||
"Async EventHub ingest failed for importRunId=" + importRunId
|
||||
+ " aggregatePackageKey=" + aggregatePackageKey
|
||||
+ " expectedCamelBatches=" + expectedCamelBatches
|
||||
+ " observedCamelBatches=" + state.totalCount()
|
||||
+ " failedCamelBatches=" + state.failedCount()
|
||||
+ " failedMessage=" + state.failedMessage()
|
||||
);
|
||||
}
|
||||
|
||||
if (state.totalCount() < expectedCamelBatches
|
||||
&& state.importingCount() == 0
|
||||
&& state.failedCount() == 0
|
||||
&& Instant.now().isAfter(lastStateChangeAt.plus(ASYNC_INGEST_STALL_GRACE_PERIOD))) {
|
||||
throw new IllegalStateException(
|
||||
"Async EventHub ingest stalled for importRunId=" + importRunId
|
||||
+ " aggregatePackageKey=" + aggregatePackageKey
|
||||
+ " expectedCamelBatches=" + expectedCamelBatches
|
||||
+ " observedCamelBatches=" + state.totalCount()
|
||||
+ " successfulCamelBatches=" + state.successCount()
|
||||
+ " importingCamelBatches=" + state.importingCount()
|
||||
+ " failedCamelBatches=" + state.failedCount()
|
||||
);
|
||||
}
|
||||
|
||||
if (Instant.now().isAfter(nextProgressLogAt)) {
|
||||
log.info("Waiting for async EventHub ingest provider={} importRunId={} extractionPackageId={} aggregatePackageKey={} expectedCamelBatches={} observedCamelBatches={} successfulCamelBatches={} failedCamelBatches={} importingCamelBatches={}",
|
||||
providerPackagePrefix(),
|
||||
importRunId,
|
||||
plannedPackage.packageId(),
|
||||
aggregatePackageKey,
|
||||
expectedCamelBatches,
|
||||
state.totalCount(),
|
||||
state.successCount(),
|
||||
state.failedCount(),
|
||||
state.importingCount());
|
||||
nextProgressLogAt = Instant.now().plus(ASYNC_INGEST_PROGRESS_LOG_INTERVAL);
|
||||
}
|
||||
|
||||
previousState = state;
|
||||
sleepQuietly(ASYNC_INGEST_POLL_INTERVAL);
|
||||
}
|
||||
|
||||
CamelBatchGroupStatus finalState = dataPackageRepository.findCamelBatchGroupStatus(
|
||||
plannedPackage.eventSourceId(),
|
||||
request.tenantKey(),
|
||||
aggregatePackageKey
|
||||
);
|
||||
throw new IllegalStateException(
|
||||
"Timed out waiting for async EventHub ingest for importRunId=" + importRunId
|
||||
+ " aggregatePackageKey=" + aggregatePackageKey
|
||||
+ " expectedCamelBatches=" + expectedCamelBatches
|
||||
+ " observedCamelBatches=" + finalState.totalCount()
|
||||
+ " successfulCamelBatches=" + finalState.successCount()
|
||||
+ " failedCamelBatches=" + finalState.failedCount()
|
||||
);
|
||||
}
|
||||
|
||||
private EventHubPackageRequest packageRequestFor(
|
||||
R request,
|
||||
EventSourceDto itemEventSource,
|
||||
|
|
@ -339,43 +224,6 @@ public abstract class AbstractImportExecutionService<R extends ImportRunRequest,
|
|||
);
|
||||
}
|
||||
|
||||
private EventHubPackageRequest extractionAggregatePackageInfo(
|
||||
UUID importRunId,
|
||||
R request,
|
||||
EventSourceDto itemEventSource,
|
||||
ImportPlanItemDto item,
|
||||
ImportTimeChunkDto chunk
|
||||
) {
|
||||
return new EventHubPackageRequest(
|
||||
request.tenantKey(),
|
||||
itemEventSource,
|
||||
request.sourceGroup(),
|
||||
chunkScope(request.importScope(), chunk),
|
||||
item.eventFamily().name(),
|
||||
chunk.occurredFrom() == null ? null : chunk.occurredFrom().toLocalDate(),
|
||||
providerPackagePrefix() + ":" + item.sourceKind() + ":" + item.extractionCode()
|
||||
+ ":RUN-" + importRunId + ":CHUNK-" + chunk.sequence()
|
||||
);
|
||||
}
|
||||
|
||||
private int expectedCamelBatchCount(int extractedEventCount) {
|
||||
if (extractedEventCount <= 0) {
|
||||
return 0;
|
||||
}
|
||||
int completionSize = Math.max(1, eventHubProperties.getBatch().getCompletionSize());
|
||||
return ((extractedEventCount - 1) / completionSize) + 1;
|
||||
}
|
||||
|
||||
private String aggregatePackageKey(EventHubPackageRequest packageInfo) {
|
||||
return packageInfo.tenantKey()
|
||||
+ ":" + packageInfo.eventSource().stableKey()
|
||||
+ ":" + (packageInfo.sourceGroup() == null ? "NO_GROUP" : packageInfo.sourceGroup().stableKey())
|
||||
+ ":" + (packageInfo.importScope() == null ? "NO_SCOPE" : packageInfo.importScope().stableKey())
|
||||
+ ":" + packageInfo.eventFamily()
|
||||
+ ":" + (packageInfo.businessDate() == null ? "NO_DATE" : packageInfo.businessDate())
|
||||
+ ":" + packageInfo.externalPackageId();
|
||||
}
|
||||
|
||||
private at.procon.eventhub.dto.ImportScopeDto chunkScope(at.procon.eventhub.dto.ImportScopeDto scope, ImportTimeChunkDto chunk) {
|
||||
if (scope == null) {
|
||||
return at.procon.eventhub.dto.ImportScopeDto.tenantAll(chunk.occurredFrom(), chunk.occurredTo());
|
||||
|
|
@ -389,15 +237,6 @@ public abstract class AbstractImportExecutionService<R extends ImportRunRequest,
|
|||
);
|
||||
}
|
||||
|
||||
private void sleepQuietly(Duration duration) {
|
||||
try {
|
||||
Thread.sleep(duration.toMillis());
|
||||
} catch (InterruptedException ex) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw new IllegalStateException("Interrupted while waiting for async EventHub ingest", ex);
|
||||
}
|
||||
}
|
||||
|
||||
private record PlannedPackage(UUID packageId, int eventSourceId, ImportPlanItemDto planItem, ImportTimeChunkDto chunk) {
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,12 @@
|
|||
package at.procon.eventhub.importing.extraction;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.DataPackageType;
|
||||
import at.procon.eventhub.dto.EventHubEventBatchDto;
|
||||
import at.procon.eventhub.dto.EventHubEventDto;
|
||||
import at.procon.eventhub.dto.EventHubPackageRequest;
|
||||
import at.procon.eventhub.dto.EventHubPackageResult;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.dto.ImportCursorStateDto;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
|
|
@ -10,11 +15,14 @@ import at.procon.eventhub.importing.ImportPlanItemDto;
|
|||
import at.procon.eventhub.importing.ImportRunRequest;
|
||||
import at.procon.eventhub.importing.ImportTimeChunkDto;
|
||||
import at.procon.eventhub.importing.persistence.ImportCursorRepository;
|
||||
import at.procon.eventhub.service.EventHubIngestionService;
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
|
|
@ -33,18 +41,24 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
private static final int EVENT_EXTRACTION_PROGRESS_LOG_INTERVAL = 5000;
|
||||
|
||||
private final NamedParameterJdbcTemplate jdbcTemplate;
|
||||
private final EventHubIngestionService ingestionService;
|
||||
private final ProducerTemplate producerTemplate;
|
||||
private final EventHubProperties eventHubProperties;
|
||||
private final ResourceLoader resourceLoader;
|
||||
private final ImportCursorRepository importCursorRepository;
|
||||
|
||||
protected AbstractJdbcExtractionBatchExecutor(
|
||||
NamedParameterJdbcTemplate jdbcTemplate,
|
||||
EventHubIngestionService ingestionService,
|
||||
ProducerTemplate producerTemplate,
|
||||
EventHubProperties eventHubProperties,
|
||||
ResourceLoader resourceLoader,
|
||||
ImportCursorRepository importCursorRepository
|
||||
) {
|
||||
this.jdbcTemplate = jdbcTemplate;
|
||||
this.ingestionService = ingestionService;
|
||||
this.producerTemplate = producerTemplate;
|
||||
this.eventHubProperties = eventHubProperties;
|
||||
this.resourceLoader = resourceLoader;
|
||||
this.importCursorRepository = importCursorRepository;
|
||||
}
|
||||
|
|
@ -87,17 +101,22 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
Map<String, Object> params = parameters(request, chunkScope, cursor);
|
||||
String sql = loadSql(definition.sqlResource());
|
||||
ExtractedEventStats stats = new ExtractedEventStats();
|
||||
log.info("Reading EventHub events provider={} tenant={} importRunId={} packageId={} extractionCode={} sourceKind={} chunk={} occurredFrom={} occurredTo={}",
|
||||
List<EventHubEventDto> pendingEvents = new ArrayList<>(jdbcPersistBatchSize());
|
||||
log.info("Reading EventHub events provider={} tenant={} importRunId={} packageId={} extractionCode={} sourceKind={} chunk={} occurredFrom={} occurredTo={} ingestMode={}",
|
||||
providerPackagePrefix(), request.tenantKey(), importRunId, packageId, planItem.extractionCode(), planItem.sourceKind(),
|
||||
chunk.sequence(), chunk.occurredFrom(), chunk.occurredTo());
|
||||
chunk.sequence(), chunk.occurredFrom(), chunk.occurredTo(), jdbcExtractionIngestMode());
|
||||
jdbcTemplate.query(sql, params, rs -> {
|
||||
EventHubEventDto event = definition.rowMapper().map(rs, stats.eventsMapped(), context);
|
||||
producerTemplate.sendBody(normalizedInputUri(), event);
|
||||
pendingEvents.add(event);
|
||||
stats.accept(event);
|
||||
if (pendingEvents.size() >= jdbcPersistBatchSize()) {
|
||||
flushPersistBatch(request, importRunId, packageId, planItem, chunk, packageInfo, pendingEvents, stats);
|
||||
}
|
||||
if (stats.eventsMapped() % EVENT_EXTRACTION_PROGRESS_LOG_INTERVAL == 0) {
|
||||
logEventExtractionProgress(request, importRunId, packageId, planItem, chunk, stats);
|
||||
}
|
||||
});
|
||||
flushPersistBatch(request, importRunId, packageId, planItem, chunk, packageInfo, pendingEvents, stats);
|
||||
logEventExtractionFinished(request, importRunId, packageId, planItem, chunk, stats);
|
||||
|
||||
return resultFor(packageId, planItem, chunk, cursor, stats);
|
||||
|
|
@ -119,8 +138,111 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
return "SOURCE";
|
||||
}
|
||||
|
||||
protected String normalizedInputUri() {
|
||||
return "direct:eventhub-normalized-input";
|
||||
protected int jdbcPersistBatchSize() {
|
||||
return Math.max(1, eventHubProperties.getBatch().getCompletionSize());
|
||||
}
|
||||
|
||||
protected EventHubProperties.JdbcExtractionIngestMode jdbcExtractionIngestMode() {
|
||||
return eventHubProperties.getTachograph().getJdbcExtractionIngestMode();
|
||||
}
|
||||
|
||||
protected int eventsInsertedOrSubmitted(ExtractedEventStats stats) {
|
||||
return stats.eventsInserted();
|
||||
}
|
||||
|
||||
protected String camelBatchPersistUri() {
|
||||
return "direct:eventhub-batch-persist-input";
|
||||
}
|
||||
|
||||
private void flushPersistBatch(
|
||||
R request,
|
||||
UUID importRunId,
|
||||
UUID extractionPackageId,
|
||||
ImportPlanItemDto planItem,
|
||||
ImportTimeChunkDto chunk,
|
||||
EventHubPackageRequest packageInfo,
|
||||
List<EventHubEventDto> pendingEvents,
|
||||
ExtractedEventStats stats
|
||||
) {
|
||||
if (pendingEvents.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
int batchNo = stats.nextPersistBatchNo();
|
||||
List<EventHubEventDto> eventsToPersist = List.copyOf(pendingEvents);
|
||||
pendingEvents.clear();
|
||||
|
||||
EventHubEventBatchDto batch = new EventHubEventBatchDto(
|
||||
packageInfo.externalPackageId() + ":JDBC-" + batchNo,
|
||||
packageInfo,
|
||||
DataPackageType.DB_EXTRACT,
|
||||
occurredFrom(eventsToPersist, chunk),
|
||||
occurredTo(eventsToPersist, chunk),
|
||||
eventsToPersist,
|
||||
persistBatchMetadata(request, importRunId, extractionPackageId, planItem, chunk, batchNo, eventsToPersist)
|
||||
);
|
||||
EventHubPackageResult result = persistBatch(batch);
|
||||
stats.acceptPersistResult(result);
|
||||
log.info("Persisted EventHub extraction batch provider={} tenant={} importRunId={} extractionPackageId={} extractionCode={} sourceKind={} chunk={} batchNo={} received={} inserted={}",
|
||||
providerPackagePrefix(), request.tenantKey(), importRunId, extractionPackageId, planItem.extractionCode(), planItem.sourceKind(),
|
||||
chunk.sequence(), batchNo, result.receivedCount(), result.insertedCount());
|
||||
}
|
||||
|
||||
private EventHubPackageResult persistBatch(EventHubEventBatchDto batch) {
|
||||
if (jdbcExtractionIngestMode() == EventHubProperties.JdbcExtractionIngestMode.CAMEL_ROUTE) {
|
||||
return producerTemplate.requestBody(camelBatchPersistUri(), batch, EventHubPackageResult.class);
|
||||
}
|
||||
return ingestionService.ingest(batch);
|
||||
}
|
||||
|
||||
private Map<String, Object> persistBatchMetadata(
|
||||
R request,
|
||||
UUID importRunId,
|
||||
UUID extractionPackageId,
|
||||
ImportPlanItemDto planItem,
|
||||
ImportTimeChunkDto chunk,
|
||||
int batchNo,
|
||||
List<EventHubEventDto> events
|
||||
) {
|
||||
Map<String, Object> metadata = new LinkedHashMap<>();
|
||||
metadata.put("ingestMode", jdbcExtractionIngestMode().name());
|
||||
metadata.put("importRunId", importRunId.toString());
|
||||
metadata.put("extractionPackageId", extractionPackageId.toString());
|
||||
metadata.put("tenantKey", request.tenantKey());
|
||||
metadata.put("mode", request.mode().name());
|
||||
metadata.put("acquisitionStrategy", request.acquisitionStrategy().name());
|
||||
metadata.put("eventFamily", planItem.eventFamily().name());
|
||||
metadata.put("sourceKind", planItem.sourceKind());
|
||||
metadata.put("extractionCode", planItem.extractionCode());
|
||||
metadata.put("entityAxis", planItem.entityAxis());
|
||||
metadata.put("chunkSequence", chunk.sequence());
|
||||
metadata.put("chunkOccurredFrom", chunk.occurredFrom() == null ? null : chunk.occurredFrom().toString());
|
||||
metadata.put("chunkOccurredTo", chunk.occurredTo() == null ? null : chunk.occurredTo().toString());
|
||||
metadata.put("batchNo", batchNo);
|
||||
metadata.put("receivedEventCount", events.size());
|
||||
metadata.put("sourcePackageRefPolicy", "Original source package is preserved per acquired event.");
|
||||
return metadata;
|
||||
}
|
||||
|
||||
private OffsetDateTime occurredFrom(List<EventHubEventDto> events, ImportTimeChunkDto chunk) {
|
||||
if (chunk.occurredFrom() != null) {
|
||||
return chunk.occurredFrom();
|
||||
}
|
||||
return events.stream()
|
||||
.map(EventHubEventDto::occurredAt)
|
||||
.filter(value -> value != null)
|
||||
.min(OffsetDateTime::compareTo)
|
||||
.orElse(null);
|
||||
}
|
||||
|
||||
private OffsetDateTime occurredTo(List<EventHubEventDto> events, ImportTimeChunkDto chunk) {
|
||||
if (chunk.occurredTo() != null) {
|
||||
return chunk.occurredTo();
|
||||
}
|
||||
return events.stream()
|
||||
.map(EventHubEventDto::occurredAt)
|
||||
.filter(value -> value != null)
|
||||
.max(OffsetDateTime::compareTo)
|
||||
.orElse(null);
|
||||
}
|
||||
|
||||
private void logEventExtractionProgress(
|
||||
|
|
@ -159,6 +281,7 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
params.put("occurredTo", scope == null ? null : scope.occurredTo());
|
||||
params.put("organisationId", organisationId);
|
||||
params.put("includeChildren", scope != null && scope.includeChildren());
|
||||
params.put("sourcePackageWatermarkEnabled", request.acquisitionStrategy() == AcquisitionStrategy.SOURCE_PACKAGE_WATERMARK ? 1 : 0);
|
||||
params.put("lastSourcePackageImportedAt", cursor == null ? null : cursor.lastSourcePackageImportedAt());
|
||||
params.put("lastSourcePackageId", cursor == null ? null : cursor.lastSourcePackageId());
|
||||
params.put("lastSourcePackageIdNumeric", parseLong(cursor == null ? null : cursor.lastSourcePackageId()));
|
||||
|
|
@ -277,10 +400,28 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
return maxSourcePackageId;
|
||||
}
|
||||
|
||||
private int eventsInserted;
|
||||
private int persistBatchNo;
|
||||
|
||||
public int eventsInserted() {
|
||||
return eventsInserted;
|
||||
}
|
||||
|
||||
public Map<String, Integer> eventTypeCounts() {
|
||||
return eventTypeCounts;
|
||||
}
|
||||
|
||||
private int nextPersistBatchNo() {
|
||||
persistBatchNo++;
|
||||
return persistBatchNo;
|
||||
}
|
||||
|
||||
private void acceptPersistResult(EventHubPackageResult result) {
|
||||
if (result != null) {
|
||||
eventsInserted += result.insertedCount();
|
||||
}
|
||||
}
|
||||
|
||||
private void accept(EventHubEventDto event) {
|
||||
eventsMapped++;
|
||||
eventTypeCounts.merge(eventTypeKey(event), 1, Integer::sum);
|
||||
|
|
@ -290,10 +431,17 @@ public abstract class AbstractJdbcExtractionBatchExecutor<R extends ImportRunReq
|
|||
|
||||
OffsetDateTime importedAt = event.sourcePackageRef().importedIntoSourceAt();
|
||||
String sourcePackageId = event.sourcePackageRef().sourcePackageId();
|
||||
if (importedAt != null
|
||||
&& (lastSourcePackageImportedAt == null || importedAt.compareTo(lastSourcePackageImportedAt) > 0)) {
|
||||
lastSourcePackageImportedAt = importedAt;
|
||||
lastSourcePackageIdByImportedAt = sourcePackageId;
|
||||
if (importedAt != null) {
|
||||
if (lastSourcePackageImportedAt == null || importedAt.compareTo(lastSourcePackageImportedAt) > 0) {
|
||||
lastSourcePackageImportedAt = importedAt;
|
||||
lastSourcePackageIdByImportedAt = sourcePackageId;
|
||||
} else if (importedAt.compareTo(lastSourcePackageImportedAt) == 0
|
||||
&& sourcePackageId != null
|
||||
&& !sourcePackageId.isBlank()
|
||||
&& (lastSourcePackageIdByImportedAt == null
|
||||
|| compareSourcePackageId(sourcePackageId, lastSourcePackageIdByImportedAt) > 0)) {
|
||||
lastSourcePackageIdByImportedAt = sourcePackageId;
|
||||
}
|
||||
}
|
||||
if (sourcePackageId != null
|
||||
&& !sourcePackageId.isBlank()
|
||||
|
|
|
|||
|
|
@ -48,6 +48,13 @@ public class DataPackageRepository {
|
|||
SourceGroupRefDto sourceGroup = packageInfo == null ? null : packageInfo.sourceGroup();
|
||||
ImportScopeDto importScope = packageInfo == null ? null : packageInfo.importScope();
|
||||
SourceGroupRefDto rootOrg = importScope == null ? null : importScope.rootSourceOrganisation();
|
||||
UUID importRunId = metadataUuid(metadata, "importRunId");
|
||||
String extractionCode = metadataString(metadata, "extractionCode");
|
||||
String extractionSourceKind = metadataString(metadata, "sourceKind");
|
||||
String entityAxis = metadataString(metadata, "entityAxis");
|
||||
Integer batchNo = metadataInteger(metadata, "batchNo");
|
||||
OffsetDateTime chunkFrom = metadataOffsetDateTime(metadata, "chunkOccurredFrom");
|
||||
OffsetDateTime chunkTo = metadataOffsetDateTime(metadata, "chunkOccurredTo");
|
||||
|
||||
return jdbcTemplate.query(
|
||||
con -> {
|
||||
|
|
@ -64,9 +71,16 @@ public class DataPackageRepository {
|
|||
received_at, event_count, metadata
|
||||
) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, now(), ?, ?::jsonb)
|
||||
on conflict (tenant_key, event_source_id, package_key) do update
|
||||
set status = excluded.status,
|
||||
set import_run_id = excluded.import_run_id,
|
||||
status = excluded.status,
|
||||
occurred_from = excluded.occurred_from,
|
||||
occurred_to = excluded.occurred_to,
|
||||
extraction_code = excluded.extraction_code,
|
||||
extraction_source_kind = excluded.extraction_source_kind,
|
||||
entity_axis = excluded.entity_axis,
|
||||
batch_no = excluded.batch_no,
|
||||
chunk_from = excluded.chunk_from,
|
||||
chunk_to = excluded.chunk_to,
|
||||
event_count = excluded.event_count,
|
||||
metadata = excluded.metadata,
|
||||
error_message = null,
|
||||
|
|
@ -75,7 +89,7 @@ public class DataPackageRepository {
|
|||
""");
|
||||
ps.setObject(1, id);
|
||||
ps.setInt(2, eventSourceId);
|
||||
ps.setObject(3, null);
|
||||
ps.setObject(3, importRunId);
|
||||
ps.setString(4, packageInfo == null ? "default" : packageInfo.tenantKey());
|
||||
ps.setString(5, packageKey);
|
||||
ps.setString(6, packageType.name());
|
||||
|
|
@ -94,12 +108,12 @@ public class DataPackageRepository {
|
|||
ps.setString(19, packageInfo == null ? null : packageInfo.eventFamily());
|
||||
ps.setObject(20, packageInfo == null ? null : packageInfo.businessDate());
|
||||
ps.setString(21, packageInfo == null ? packageKey : packageInfo.externalPackageId());
|
||||
ps.setString(22, null);
|
||||
ps.setString(23, null);
|
||||
ps.setString(24, null);
|
||||
ps.setObject(25, null);
|
||||
ps.setObject(26, null);
|
||||
ps.setObject(27, null);
|
||||
ps.setString(22, extractionCode);
|
||||
ps.setString(23, extractionSourceKind);
|
||||
ps.setString(24, entityAxis);
|
||||
ps.setObject(25, batchNo);
|
||||
ps.setObject(26, chunkFrom);
|
||||
ps.setObject(27, chunkTo);
|
||||
ps.setString(28, null);
|
||||
ps.setString(29, null);
|
||||
ps.setString(30, null);
|
||||
|
|
@ -343,6 +357,61 @@ public class DataPackageRepository {
|
|||
);
|
||||
}
|
||||
|
||||
private UUID metadataUuid(Map<String, Object> metadata, String key) {
|
||||
String value = metadataString(metadata, key);
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return UUID.fromString(value);
|
||||
} catch (IllegalArgumentException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private String metadataString(Map<String, Object> metadata, String key) {
|
||||
if (metadata == null) {
|
||||
return null;
|
||||
}
|
||||
Object value = metadata.get(key);
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String text = value.toString().trim();
|
||||
return text.isEmpty() ? null : text;
|
||||
}
|
||||
|
||||
private Integer metadataInteger(Map<String, Object> metadata, String key) {
|
||||
String value = metadataString(metadata, key);
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return Integer.parseInt(value);
|
||||
} catch (NumberFormatException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private OffsetDateTime metadataOffsetDateTime(Map<String, Object> metadata, String key) {
|
||||
Object value = metadata == null ? null : metadata.get(key);
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
if (value instanceof OffsetDateTime offsetDateTime) {
|
||||
return offsetDateTime;
|
||||
}
|
||||
String text = value.toString().trim();
|
||||
if (text.isEmpty()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return OffsetDateTime.parse(text);
|
||||
} catch (RuntimeException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private String toJson(Map<String, Object> value) {
|
||||
try {
|
||||
return objectMapper.writeValueAsString(normalizeMetadataMap(value));
|
||||
|
|
|
|||
|
|
@ -0,0 +1,372 @@
|
|||
package at.procon.eventhub.persistence;
|
||||
|
||||
import at.procon.eventhub.dto.DriverCardRefDto;
|
||||
import at.procon.eventhub.dto.DriverRefDto;
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.stereotype.Repository;
|
||||
import org.springframework.transaction.annotation.Transactional;
|
||||
|
||||
@Repository
|
||||
public class DriverIdentityRepository {
|
||||
|
||||
private final JdbcTemplate jdbcTemplate;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public DriverIdentityRepository(JdbcTemplate jdbcTemplate, ObjectMapper objectMapper) {
|
||||
this.jdbcTemplate = jdbcTemplate;
|
||||
this.objectMapper = objectMapper;
|
||||
}
|
||||
|
||||
public UUID resolveOrCreateDriverId(
|
||||
String tenantKey,
|
||||
int eventSourceId,
|
||||
DriverRefDto driverRef
|
||||
) {
|
||||
if (driverRef == null || !driverRef.hasAnyReference()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
String normalizedTenantKey = normalizeRequired(tenantKey, "tenantKey");
|
||||
String sourceDriverEntityId = normalizeNullable(driverRef.sourceEntityId());
|
||||
DriverCardRefDto driverCard = driverRef.driverCard();
|
||||
String cardNation = driverCard == null ? null : normalizeNullable(driverCard.nation());
|
||||
String cardNumber = driverCard == null ? null : normalizeNullable(driverCard.number());
|
||||
|
||||
UUID driverId = resolveDriverId(normalizedTenantKey, eventSourceId, sourceDriverEntityId, cardNation, cardNumber);
|
||||
if (driverId == null) {
|
||||
Map<String, Object> payload = new LinkedHashMap<>();
|
||||
put(payload, "source", "event");
|
||||
put(payload, "source_driver_entity_id", sourceDriverEntityId);
|
||||
put(payload, "card_nation", cardNation);
|
||||
put(payload, "card_number", cardNumber);
|
||||
driverId = createDriver(
|
||||
normalizedTenantKey,
|
||||
eventSourceId,
|
||||
sourceDriverEntityId,
|
||||
cardNation,
|
||||
cardNumber,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
payload
|
||||
);
|
||||
}
|
||||
|
||||
touchDriver(driverId, sourceDriverEntityId, cardNation, cardNumber);
|
||||
return driverId;
|
||||
}
|
||||
|
||||
@Transactional
|
||||
public int reconcileFromMasterData(String tenantKey, int eventSourceId) {
|
||||
String normalizedTenantKey = normalizeRequired(tenantKey, "tenantKey");
|
||||
int updates = reconcileDriversFromMasterData(normalizedTenantKey, eventSourceId);
|
||||
updates += projectDriverCardsFromMasterData(normalizedTenantKey, eventSourceId);
|
||||
return updates;
|
||||
}
|
||||
|
||||
private int reconcileDriversFromMasterData(String tenantKey, int eventSourceId) {
|
||||
Long count = jdbcTemplate.queryForObject(
|
||||
compatibleSourcesCte() + """
|
||||
, master_drivers as (
|
||||
select distinct on (nullif(trim(source_entity_id), ''))
|
||||
event_source_id,
|
||||
nullif(trim(source_entity_id), '') as source_driver_entity_id,
|
||||
nullif(trim(payload ->> 'first_names'), '') as first_names,
|
||||
coalesce(
|
||||
nullif(trim(payload ->> 'last_name'), ''),
|
||||
nullif(trim(payload ->> 'surname'), '')
|
||||
) as last_name,
|
||||
cast(nullif(trim(payload ->> 'birth_date'), '') as date) as birth_date,
|
||||
source_updated_at,
|
||||
payload
|
||||
from eventhub.source_master_entity
|
||||
where tenant_key = ?
|
||||
and event_source_id in (select id from compatible_sources)
|
||||
and entity_type = 'DRIVER'
|
||||
and nullif(trim(source_entity_id), '') is not null
|
||||
and source_entity_id not like 'DRIVER_CARD:%'
|
||||
order by nullif(trim(source_entity_id), ''), updated_at desc
|
||||
),
|
||||
updated_by_source as (
|
||||
update eventhub.driver driver
|
||||
set first_names = coalesce(master.first_names, driver.first_names),
|
||||
last_name = coalesce(master.last_name, driver.last_name),
|
||||
birth_date = coalesce(master.birth_date, driver.birth_date),
|
||||
source_updated_at = master.source_updated_at,
|
||||
payload = driver.payload || master.payload,
|
||||
updated_at = now()
|
||||
from master_drivers master
|
||||
where driver.tenant_key = ?
|
||||
and driver.event_source_id in (select id from compatible_sources)
|
||||
and driver.source_driver_entity_id = master.source_driver_entity_id
|
||||
returning driver.id
|
||||
),
|
||||
inserted as (
|
||||
insert into eventhub.driver(
|
||||
id, tenant_key, event_source_id, source_driver_entity_id,
|
||||
first_names, last_name, birth_date, source_updated_at, payload, updated_at
|
||||
)
|
||||
select gen_random_uuid(),
|
||||
?,
|
||||
master.event_source_id,
|
||||
master.source_driver_entity_id,
|
||||
master.first_names,
|
||||
master.last_name,
|
||||
master.birth_date,
|
||||
master.source_updated_at,
|
||||
master.payload,
|
||||
now()
|
||||
from master_drivers master
|
||||
where not exists (
|
||||
select 1
|
||||
from eventhub.driver existing
|
||||
where existing.tenant_key = ?
|
||||
and existing.event_source_id in (select id from compatible_sources)
|
||||
and existing.source_driver_entity_id = master.source_driver_entity_id
|
||||
)
|
||||
returning id
|
||||
)
|
||||
select (select count(*) from updated_by_source)
|
||||
+ (select count(*) from inserted)
|
||||
""",
|
||||
Long.class,
|
||||
eventSourceId,
|
||||
tenantKey,
|
||||
tenantKey,
|
||||
tenantKey,
|
||||
tenantKey
|
||||
);
|
||||
return count == null ? 0 : Math.toIntExact(count);
|
||||
}
|
||||
|
||||
private int projectDriverCardsFromMasterData(String tenantKey, int eventSourceId) {
|
||||
Long count = jdbcTemplate.queryForObject(
|
||||
compatibleSourcesCte() + """
|
||||
, driver_card_projection as (
|
||||
select distinct on (rel.to_source_entity_id)
|
||||
rel.to_source_entity_id as source_driver_entity_id,
|
||||
nullif(trim(card.payload ->> 'card_nation'), '') as card_nation,
|
||||
nullif(trim(card.payload ->> 'card_number'), '') as card_number,
|
||||
rel.source_updated_at
|
||||
from eventhub.source_master_relation rel
|
||||
join eventhub.source_master_entity card
|
||||
on card.tenant_key = rel.tenant_key
|
||||
and card.event_source_id = rel.event_source_id
|
||||
and card.entity_type = 'DRIVER_CARD'
|
||||
and card.source_entity_id = rel.from_source_entity_id
|
||||
where rel.tenant_key = ?
|
||||
and rel.event_source_id in (select id from compatible_sources)
|
||||
and rel.relation_type = 'DRIVER_CARD_DRIVER'
|
||||
and rel.from_entity_type = 'DRIVER_CARD'
|
||||
and rel.to_entity_type = 'DRIVER'
|
||||
order by rel.to_source_entity_id,
|
||||
rel.valid_to desc nulls last,
|
||||
rel.valid_from desc nulls last,
|
||||
rel.updated_at desc
|
||||
),
|
||||
updated_by_source as (
|
||||
update eventhub.driver driver
|
||||
set card_nation = coalesce(driver.card_nation, projection.card_nation),
|
||||
card_number = coalesce(driver.card_number, projection.card_number),
|
||||
source_updated_at = coalesce(projection.source_updated_at, driver.source_updated_at),
|
||||
updated_at = now()
|
||||
from driver_card_projection projection
|
||||
where driver.tenant_key = ?
|
||||
and driver.event_source_id in (select id from compatible_sources)
|
||||
and driver.source_driver_entity_id = projection.source_driver_entity_id
|
||||
and (
|
||||
(driver.card_nation is null and projection.card_nation is not null)
|
||||
or (driver.card_number is null and projection.card_number is not null)
|
||||
)
|
||||
returning driver.id
|
||||
)
|
||||
select count(*)
|
||||
from updated_by_source
|
||||
""",
|
||||
Long.class,
|
||||
eventSourceId,
|
||||
tenantKey,
|
||||
tenantKey
|
||||
);
|
||||
return count == null ? 0 : Math.toIntExact(count);
|
||||
}
|
||||
|
||||
private UUID resolveDriverId(
|
||||
String tenantKey,
|
||||
int eventSourceId,
|
||||
String sourceDriverEntityId,
|
||||
String cardNation,
|
||||
String cardNumber
|
||||
) {
|
||||
UUID driverId = findBySourceDriverEntityId(tenantKey, eventSourceId, sourceDriverEntityId);
|
||||
if (driverId == null) {
|
||||
driverId = findByCard(tenantKey, eventSourceId, cardNation, cardNumber);
|
||||
}
|
||||
return driverId;
|
||||
}
|
||||
|
||||
private UUID findBySourceDriverEntityId(String tenantKey, int eventSourceId, String sourceDriverEntityId) {
|
||||
if (sourceDriverEntityId == null) {
|
||||
return null;
|
||||
}
|
||||
return jdbcTemplate.query(
|
||||
compatibleSourcesCte() + """
|
||||
select d.id
|
||||
from eventhub.driver d
|
||||
where d.tenant_key = ?
|
||||
and d.event_source_id in (select id from compatible_sources)
|
||||
and d.source_driver_entity_id = ?
|
||||
order by d.updated_at desc
|
||||
limit 1
|
||||
""",
|
||||
rs -> rs.next() ? (UUID) rs.getObject("id") : null,
|
||||
eventSourceId,
|
||||
tenantKey,
|
||||
sourceDriverEntityId
|
||||
);
|
||||
}
|
||||
|
||||
private UUID findByCard(String tenantKey, int eventSourceId, String cardNation, String cardNumber) {
|
||||
if (cardNation == null || cardNumber == null) {
|
||||
return null;
|
||||
}
|
||||
return jdbcTemplate.query(
|
||||
compatibleSourcesCte() + """
|
||||
select d.id
|
||||
from eventhub.driver d
|
||||
where d.tenant_key = ?
|
||||
and d.event_source_id in (select id from compatible_sources)
|
||||
and d.card_nation = ?
|
||||
and d.card_number = ?
|
||||
order by d.updated_at desc
|
||||
limit 1
|
||||
""",
|
||||
rs -> rs.next() ? (UUID) rs.getObject("id") : null,
|
||||
eventSourceId,
|
||||
tenantKey,
|
||||
cardNation,
|
||||
cardNumber
|
||||
);
|
||||
}
|
||||
|
||||
private UUID createDriver(
|
||||
String tenantKey,
|
||||
int eventSourceId,
|
||||
String sourceDriverEntityId,
|
||||
String cardNation,
|
||||
String cardNumber,
|
||||
String firstNames,
|
||||
String lastName,
|
||||
OffsetDateTime sourceUpdatedAt,
|
||||
Map<String, Object> payload
|
||||
) {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
jdbcTemplate.update(
|
||||
"""
|
||||
insert into eventhub.driver(
|
||||
id, tenant_key, event_source_id, source_driver_entity_id,
|
||||
card_nation, card_number, first_names, last_name,
|
||||
source_updated_at, payload, updated_at
|
||||
) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?::jsonb, now())
|
||||
""",
|
||||
driverId,
|
||||
tenantKey,
|
||||
eventSourceId,
|
||||
sourceDriverEntityId,
|
||||
cardNation,
|
||||
cardNumber,
|
||||
firstNames,
|
||||
lastName,
|
||||
sourceUpdatedAt,
|
||||
toJson(payload)
|
||||
);
|
||||
return driverId;
|
||||
}
|
||||
|
||||
private void touchDriver(
|
||||
UUID driverId,
|
||||
String sourceDriverEntityId,
|
||||
String cardNation,
|
||||
String cardNumber
|
||||
) {
|
||||
if (sourceDriverEntityId == null && cardNation == null && cardNumber == null) {
|
||||
return;
|
||||
}
|
||||
jdbcTemplate.update(
|
||||
"""
|
||||
update eventhub.driver
|
||||
set source_driver_entity_id = coalesce(source_driver_entity_id, cast(? as text)),
|
||||
card_nation = coalesce(card_nation, cast(? as text)),
|
||||
card_number = coalesce(card_number, cast(? as text)),
|
||||
updated_at = now()
|
||||
where id = ?
|
||||
and (
|
||||
(source_driver_entity_id is null and cast(? as text) is not null)
|
||||
or (card_nation is null and cast(? as text) is not null)
|
||||
or (card_number is null and cast(? as text) is not null)
|
||||
)
|
||||
""",
|
||||
sourceDriverEntityId,
|
||||
cardNation,
|
||||
cardNumber,
|
||||
driverId,
|
||||
sourceDriverEntityId,
|
||||
cardNation,
|
||||
cardNumber
|
||||
);
|
||||
}
|
||||
|
||||
private String compatibleSourcesCte() {
|
||||
return """
|
||||
with source_context as (
|
||||
select tenant_key, provider_key, source_instance_key, coalesce(tenant_provider_setting_key, '') as tenant_provider_setting_key
|
||||
from eventhub.event_source
|
||||
where id = ?
|
||||
),
|
||||
compatible_sources as (
|
||||
select es.id
|
||||
from eventhub.event_source es
|
||||
join source_context ctx on ctx.tenant_key = es.tenant_key
|
||||
and ctx.provider_key = es.provider_key
|
||||
and ctx.source_instance_key = es.source_instance_key
|
||||
and ctx.tenant_provider_setting_key = coalesce(es.tenant_provider_setting_key, '')
|
||||
)
|
||||
""";
|
||||
}
|
||||
|
||||
private String toJson(Map<String, Object> value) {
|
||||
try {
|
||||
return objectMapper.writeValueAsString(value == null ? Map.of() : value);
|
||||
} catch (JsonProcessingException e) {
|
||||
throw new IllegalArgumentException("Cannot serialize driver identity payload", e);
|
||||
}
|
||||
}
|
||||
|
||||
private void put(Map<String, Object> payload, String key, Object value) {
|
||||
if (value != null) {
|
||||
payload.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
private String normalizeRequired(String value, String fieldName) {
|
||||
String normalized = normalizeNullable(value);
|
||||
if (normalized == null) {
|
||||
throw new IllegalArgumentException(fieldName + " must not be blank");
|
||||
}
|
||||
return normalized;
|
||||
}
|
||||
|
||||
private String normalizeNullable(String value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String trimmed = value.trim();
|
||||
return trimmed.isEmpty() ? null : trimmed;
|
||||
}
|
||||
}
|
||||
|
|
@ -32,6 +32,7 @@ public class EventRepository {
|
|||
private final ObjectMapper objectMapper;
|
||||
private final EventAcquisitionRecordKeyService recordKeyService;
|
||||
private final SourceMasterDataRepository sourceMasterDataRepository;
|
||||
private final DriverIdentityRepository driverIdentityRepository;
|
||||
private final VehicleIdentityRepository vehicleIdentityRepository;
|
||||
|
||||
public EventRepository(
|
||||
|
|
@ -39,12 +40,14 @@ public class EventRepository {
|
|||
ObjectMapper objectMapper,
|
||||
EventAcquisitionRecordKeyService recordKeyService,
|
||||
SourceMasterDataRepository sourceMasterDataRepository,
|
||||
DriverIdentityRepository driverIdentityRepository,
|
||||
VehicleIdentityRepository vehicleIdentityRepository
|
||||
) {
|
||||
this.jdbcTemplate = jdbcTemplate;
|
||||
this.objectMapper = objectMapper;
|
||||
this.recordKeyService = recordKeyService;
|
||||
this.sourceMasterDataRepository = sourceMasterDataRepository;
|
||||
this.driverIdentityRepository = driverIdentityRepository;
|
||||
this.vehicleIdentityRepository = vehicleIdentityRepository;
|
||||
}
|
||||
|
||||
|
|
@ -89,7 +92,7 @@ public class EventRepository {
|
|||
packageId,
|
||||
eventSourceId,
|
||||
event.externalSourceEventId(),
|
||||
refs.driverEntityId(),
|
||||
refs.driverId(),
|
||||
refs.vehicleId(),
|
||||
refs.vehicleRegistrationId(),
|
||||
sourcePackageId,
|
||||
|
|
@ -121,7 +124,7 @@ public class EventRepository {
|
|||
data_package_id uuid not null,
|
||||
event_source_id integer not null,
|
||||
external_source_event_id text not null,
|
||||
driver_entity_id uuid,
|
||||
driver_id uuid,
|
||||
vehicle_id uuid,
|
||||
vehicle_registration_id uuid,
|
||||
source_package_id text,
|
||||
|
|
@ -149,7 +152,7 @@ public class EventRepository {
|
|||
"""
|
||||
insert into eventhub_event_import_stage(
|
||||
row_no, source_record_key_hash, requested_event_id, data_package_id, event_source_id,
|
||||
external_source_event_id, driver_entity_id, vehicle_id, vehicle_registration_id,
|
||||
external_source_event_id, driver_id, vehicle_id, vehicle_registration_id,
|
||||
source_package_id, source_package_entity_id, occurred_at, received_partner_at, received_hub_at,
|
||||
event_domain, event_type, lifecycle, odometer_m, longitude, latitude,
|
||||
payload, manual_entry, event_signature_hash
|
||||
|
|
@ -165,7 +168,7 @@ public class EventRepository {
|
|||
ps.setObject(4, row.packageId());
|
||||
ps.setInt(5, row.eventSourceId());
|
||||
ps.setString(6, row.externalSourceEventId());
|
||||
ps.setObject(7, row.driverEntityId());
|
||||
ps.setObject(7, row.driverId());
|
||||
ps.setObject(8, row.vehicleId());
|
||||
ps.setObject(9, row.vehicleRegistrationId());
|
||||
ps.setString(10, row.sourcePackageId());
|
||||
|
|
@ -233,7 +236,7 @@ public class EventRepository {
|
|||
insert into eventhub.event(
|
||||
id, event_source_id, data_package_id,
|
||||
external_source_event_id,
|
||||
driver_entity_id, vehicle_id, vehicle_registration_id,
|
||||
driver_id, vehicle_id, vehicle_registration_id,
|
||||
source_package_id, source_package_entity_id,
|
||||
occurred_at, received_partner_at, received_hub_at,
|
||||
event_domain, event_type, lifecycle,
|
||||
|
|
@ -244,7 +247,7 @@ public class EventRepository {
|
|||
select
|
||||
source_record.event_id, stage.event_source_id, stage.data_package_id,
|
||||
stage.external_source_event_id,
|
||||
stage.driver_entity_id, stage.vehicle_id, stage.vehicle_registration_id,
|
||||
stage.driver_id, stage.vehicle_id, stage.vehicle_registration_id,
|
||||
stage.source_package_id, stage.source_package_entity_id,
|
||||
source_record.event_occurred_at, stage.received_partner_at, stage.received_hub_at,
|
||||
stage.event_domain, stage.event_type, stage.lifecycle,
|
||||
|
|
@ -357,13 +360,13 @@ public class EventRepository {
|
|||
Map<String, UUID> entityIdCache,
|
||||
Map<String, List<VehicleRefCacheEntry>> vehicleRefCache
|
||||
) {
|
||||
UUID driverEntityId = resolveDriverEntityId(tenantKey, eventSourceId, event, entityIdCache);
|
||||
UUID driverId = resolveDriverId(tenantKey, eventSourceId, event, entityIdCache);
|
||||
ResolvedVehicleReference vehicleRef = resolveVehicleReference(tenantKey, eventSourceId, event, vehicleRefCache);
|
||||
UUID sourcePackageEntityId = resolveSourcePackageEntityId(tenantKey, eventSourceId, event, entityIdCache);
|
||||
return new ResolvedEntityRefs(driverEntityId, vehicleRef.vehicleId(), vehicleRef.vehicleRegistrationId(), sourcePackageEntityId);
|
||||
return new ResolvedEntityRefs(driverId, vehicleRef.vehicleId(), vehicleRef.vehicleRegistrationId(), sourcePackageEntityId);
|
||||
}
|
||||
|
||||
private UUID resolveDriverEntityId(
|
||||
private UUID resolveDriverId(
|
||||
String tenantKey,
|
||||
int eventSourceId,
|
||||
EventHubEventDto event,
|
||||
|
|
@ -373,32 +376,16 @@ public class EventRepository {
|
|||
if (driverRef == null || !driverRef.hasAnyReference()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
DriverCardRefDto card = driverRef.driverCard();
|
||||
String cardKey = card == null || !card.hasValue() ? null : card.stableKey();
|
||||
String sourceEntityId = normalizeNullable(driverRef.sourceEntityId());
|
||||
if (sourceEntityId == null && cardKey != null) {
|
||||
sourceEntityId = "DRIVER_CARD:" + cardKey;
|
||||
String cacheKey = "DRIVER|" + driverRef.stableKey();
|
||||
UUID cached = entityIdCache.get(cacheKey);
|
||||
if (cached != null) {
|
||||
return cached;
|
||||
}
|
||||
if (sourceEntityId == null) {
|
||||
return null;
|
||||
UUID resolved = driverIdentityRepository.resolveOrCreateDriverId(tenantKey, eventSourceId, driverRef);
|
||||
if (resolved != null) {
|
||||
entityIdCache.put(cacheKey, resolved);
|
||||
}
|
||||
|
||||
Map<String, Object> payload = new LinkedHashMap<>();
|
||||
put(payload, "source_entity_id", driverRef.sourceEntityId());
|
||||
put(payload, "driver_card_nation", card == null ? null : card.nation());
|
||||
put(payload, "driver_card_number", card == null ? null : card.number());
|
||||
return resolveEntityId(
|
||||
tenantKey,
|
||||
eventSourceId,
|
||||
"DRIVER",
|
||||
sourceEntityId,
|
||||
cardKey,
|
||||
sourceEntityId,
|
||||
null,
|
||||
payload,
|
||||
entityIdCache
|
||||
);
|
||||
return resolved;
|
||||
}
|
||||
|
||||
private ResolvedVehicleReference resolveVehicleReference(
|
||||
|
|
@ -599,7 +586,7 @@ public class EventRepository {
|
|||
}
|
||||
|
||||
private record ResolvedEntityRefs(
|
||||
UUID driverEntityId,
|
||||
UUID driverId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
UUID sourcePackageEntityId
|
||||
|
|
@ -618,7 +605,7 @@ public class EventRepository {
|
|||
UUID packageId,
|
||||
int eventSourceId,
|
||||
String externalSourceEventId,
|
||||
UUID driverEntityId,
|
||||
UUID driverId,
|
||||
UUID vehicleId,
|
||||
UUID vehicleRegistrationId,
|
||||
String sourcePackageId,
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
|
|||
import org.springframework.jdbc.datasource.DriverManagerDataSource;
|
||||
|
||||
@Configuration
|
||||
@ConditionalOnExpression("'${eventhub.tachograph.datasource.jdbc-url:}' != ''")
|
||||
@ConditionalOnExpression("T(org.springframework.util.StringUtils).hasText('${eventhub.tachograph.datasource.jdbc-url:}')")
|
||||
public class TachographDataSourceConfig {
|
||||
|
||||
private static final String SQL_SERVER_DRIVER_CLASS = "com.microsoft.sqlserver.jdbc.SQLServerDriver";
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
package at.procon.eventhub.tachograph.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.dto.ImportCursorStateDto;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
|
|
@ -8,6 +9,7 @@ import at.procon.eventhub.importing.ImportTimeChunkDto;
|
|||
import at.procon.eventhub.importing.extraction.AbstractJdbcExtractionBatchExecutor;
|
||||
import at.procon.eventhub.importing.extraction.ExtractionDefinition;
|
||||
import at.procon.eventhub.importing.persistence.ImportCursorRepository;
|
||||
import at.procon.eventhub.service.EventHubIngestionService;
|
||||
import at.procon.eventhub.tachograph.dto.TachographExtractionBatchResultDto;
|
||||
import at.procon.eventhub.tachograph.dto.TachographImportRequest;
|
||||
import java.time.LocalDateTime;
|
||||
|
|
@ -26,7 +28,7 @@ import org.springframework.stereotype.Service;
|
|||
|
||||
@Service
|
||||
@ConditionalOnBean(name = "tachographNamedParameterJdbcTemplate")
|
||||
@ConditionalOnExpression("'${eventhub.tachograph.datasource.jdbc-url:}' != ''")
|
||||
@ConditionalOnExpression("T(org.springframework.util.StringUtils).hasText('${eventhub.tachograph.datasource.jdbc-url:}')")
|
||||
public class JdbcTachographExtractionBatchExecutor
|
||||
extends AbstractJdbcExtractionBatchExecutor<TachographImportRequest, TachographExtractionBatchResultDto>
|
||||
implements TachographExtractionBatchExecutor {
|
||||
|
|
@ -35,12 +37,14 @@ public class JdbcTachographExtractionBatchExecutor
|
|||
|
||||
public JdbcTachographExtractionBatchExecutor(
|
||||
@Qualifier("tachographNamedParameterJdbcTemplate") NamedParameterJdbcTemplate tachographJdbcTemplate,
|
||||
EventHubIngestionService ingestionService,
|
||||
ProducerTemplate producerTemplate,
|
||||
EventHubProperties eventHubProperties,
|
||||
ResourceLoader resourceLoader,
|
||||
TachographExtractionDefinitionRegistry definitionRegistry,
|
||||
ImportCursorRepository importCursorRepository
|
||||
) {
|
||||
super(tachographJdbcTemplate, producerTemplate, resourceLoader, importCursorRepository);
|
||||
super(tachographJdbcTemplate, ingestionService, producerTemplate, eventHubProperties, resourceLoader, importCursorRepository);
|
||||
this.definitionRegistry = definitionRegistry;
|
||||
}
|
||||
|
||||
|
|
@ -78,7 +82,7 @@ public class JdbcTachographExtractionBatchExecutor
|
|||
planItem.sourceKind(),
|
||||
stats.eventsMapped(),
|
||||
stats.eventsMapped(),
|
||||
stats.eventsMapped(),
|
||||
eventsInsertedOrSubmitted(stats),
|
||||
0,
|
||||
true,
|
||||
lastSourcePackageImportedAt(stats, cursor),
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ import org.springframework.stereotype.Service;
|
|||
*/
|
||||
@Service
|
||||
@ConditionalOnMissingBean(TachographExtractionBatchExecutor.class)
|
||||
@ConditionalOnExpression("'${eventhub.tachograph.datasource.jdbc-url:}' == ''")
|
||||
@ConditionalOnExpression("!T(org.springframework.util.StringUtils).hasText('${eventhub.tachograph.datasource.jdbc-url:}')")
|
||||
public class NoopTachographExtractionBatchExecutor
|
||||
extends AbstractNoopExtractionBatchExecutor<TachographImportRequest, TachographExtractionBatchResultDto>
|
||||
implements TachographExtractionBatchExecutor {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
package at.procon.eventhub.tachograph.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.importing.AbstractImportExecutionService;
|
||||
import at.procon.eventhub.importing.ImportPlanDto;
|
||||
|
|
@ -33,11 +32,10 @@ public class TachographImportExecutionService
|
|||
ImportRunRepository importRunRepository,
|
||||
DataPackageRepository dataPackageRepository,
|
||||
ImportCursorRepository importCursorRepository,
|
||||
EventHubProperties eventHubProperties,
|
||||
TachographMasterDataRefreshService masterDataRefreshService,
|
||||
TachographExtractionBatchExecutor extractionBatchExecutor
|
||||
) {
|
||||
super(eventSourceRepository, importRunRepository, dataPackageRepository, importCursorRepository, eventHubProperties);
|
||||
super(eventSourceRepository, importRunRepository, dataPackageRepository, importCursorRepository);
|
||||
this.planService = planService;
|
||||
this.masterDataRefreshService = masterDataRefreshService;
|
||||
this.extractionBatchExecutor = extractionBatchExecutor;
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import at.procon.eventhub.importing.masterdata.SourceMasterRelationUpsert;
|
|||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.persistence.EventSourceRepository;
|
||||
import at.procon.eventhub.persistence.SourceMasterDataRepository;
|
||||
import at.procon.eventhub.persistence.DriverIdentityRepository;
|
||||
import at.procon.eventhub.persistence.VehicleIdentityRepository;
|
||||
import at.procon.eventhub.tachograph.dto.TachographImportRequest;
|
||||
import java.io.IOException;
|
||||
|
|
@ -52,6 +53,7 @@ public class TachographMasterDataRefreshService {
|
|||
private final ObjectProvider<NamedParameterJdbcTemplate> tachographJdbcTemplateProvider;
|
||||
private final SourceMasterDataRepository sourceMasterDataRepository;
|
||||
private final EventSourceRepository eventSourceRepository;
|
||||
private final DriverIdentityRepository driverIdentityRepository;
|
||||
private final VehicleIdentityRepository vehicleIdentityRepository;
|
||||
private final ResourceLoader resourceLoader;
|
||||
|
||||
|
|
@ -59,12 +61,14 @@ public class TachographMasterDataRefreshService {
|
|||
@Qualifier("tachographNamedParameterJdbcTemplate") ObjectProvider<NamedParameterJdbcTemplate> tachographJdbcTemplateProvider,
|
||||
SourceMasterDataRepository sourceMasterDataRepository,
|
||||
EventSourceRepository eventSourceRepository,
|
||||
DriverIdentityRepository driverIdentityRepository,
|
||||
VehicleIdentityRepository vehicleIdentityRepository,
|
||||
ResourceLoader resourceLoader
|
||||
) {
|
||||
this.tachographJdbcTemplateProvider = tachographJdbcTemplateProvider;
|
||||
this.sourceMasterDataRepository = sourceMasterDataRepository;
|
||||
this.eventSourceRepository = eventSourceRepository;
|
||||
this.driverIdentityRepository = driverIdentityRepository;
|
||||
this.vehicleIdentityRepository = vehicleIdentityRepository;
|
||||
this.resourceLoader = resourceLoader;
|
||||
}
|
||||
|
|
@ -98,13 +102,17 @@ public class TachographMasterDataRefreshService {
|
|||
|
||||
int relationCount = streamRelations(tachographJdbcTemplate, tenantKey, eventSourceId, RELATIONS_SQL_RESOURCE, loadSql(RELATIONS_SQL_RESOURCE));
|
||||
|
||||
log.info("Reconciling tachograph driver identities from source master data tenant={} source={}",
|
||||
tenantKey, masterDataSource.stableKey());
|
||||
int reconciledDrivers = driverIdentityRepository.reconcileFromMasterData(tenantKey, eventSourceId);
|
||||
|
||||
log.info("Reconciling tachograph vehicle identities from source master data tenant={} source={}",
|
||||
tenantKey, masterDataSource.stableKey());
|
||||
int reconciledVehicles = vehicleIdentityRepository.reconcileFromMasterData(tenantKey, eventSourceId);
|
||||
|
||||
MasterDataRefreshResult result = new MasterDataRefreshResult(entities, relationCount);
|
||||
log.info("Refreshed tachograph source master data tenant={} source={} entities={} relations={} reconciledVehicles={}",
|
||||
tenantKey, masterDataSource.stableKey(), result.entitiesUpserted(), result.relationsUpserted(), reconciledVehicles);
|
||||
log.info("Refreshed tachograph source master data tenant={} source={} entities={} relations={} reconciledDrivers={} reconciledVehicles={}",
|
||||
tenantKey, masterDataSource.stableKey(), result.entitiesUpserted(), result.relationsUpserted(), reconciledDrivers, reconciledVehicles);
|
||||
return result;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,93 @@
|
|||
package at.procon.eventhub.yellowfox.api;
|
||||
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.ImportMode;
|
||||
import at.procon.eventhub.dto.SchedulerTriggerMode;
|
||||
import at.procon.eventhub.yellowfox.dto.ConfiguredYellowFoxD8ImportPlanDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRunResultDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportTriggerResultDto;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8ConfiguredImportPlanService;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8ImportExecutionService;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8ImportPlanService;
|
||||
import jakarta.validation.Valid;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import org.apache.camel.ProducerTemplate;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PathVariable;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
@RestController
|
||||
@RequestMapping("/api/eventhub/acquisition/yellowfox/d8")
|
||||
public class YellowFoxD8ImportController {
|
||||
|
||||
private final ProducerTemplate producerTemplate;
|
||||
private final YellowFoxD8ImportPlanService importPlanService;
|
||||
private final YellowFoxD8ConfiguredImportPlanService configuredImportPlanService;
|
||||
private final YellowFoxD8ImportExecutionService importExecutionService;
|
||||
|
||||
public YellowFoxD8ImportController(
|
||||
ProducerTemplate producerTemplate,
|
||||
YellowFoxD8ImportPlanService importPlanService,
|
||||
YellowFoxD8ConfiguredImportPlanService configuredImportPlanService,
|
||||
YellowFoxD8ImportExecutionService importExecutionService
|
||||
) {
|
||||
this.producerTemplate = producerTemplate;
|
||||
this.importPlanService = importPlanService;
|
||||
this.configuredImportPlanService = configuredImportPlanService;
|
||||
this.importExecutionService = importExecutionService;
|
||||
}
|
||||
|
||||
@PostMapping("/imports/plan")
|
||||
public ResponseEntity<?> planYellowFoxD8Import(@Valid @RequestBody YellowFoxD8ImportRequest request) {
|
||||
return ResponseEntity.ok(importPlanService.createPlan(request));
|
||||
}
|
||||
|
||||
@PostMapping("/imports/start")
|
||||
public ResponseEntity<YellowFoxD8ImportRunResultDto> startYellowFoxD8Import(
|
||||
@Valid @RequestBody YellowFoxD8ImportRequest request,
|
||||
@RequestParam(defaultValue = "false") boolean execute
|
||||
) {
|
||||
YellowFoxD8ImportRunResultDto result = execute
|
||||
? importExecutionService.startAndExecuteImport(request)
|
||||
: producerTemplate.requestBody("direct:yellowfox-d8-import-start", request, YellowFoxD8ImportRunResultDto.class);
|
||||
return ResponseEntity.accepted().body(result);
|
||||
}
|
||||
|
||||
@GetMapping("/imports/configured-plans")
|
||||
public ResponseEntity<List<ConfiguredYellowFoxD8ImportPlanDto>> listConfiguredYellowFoxPlans() {
|
||||
return ResponseEntity.ok(configuredImportPlanService.listPlans());
|
||||
}
|
||||
|
||||
@GetMapping("/imports/configured-plans/{planKey}")
|
||||
public ResponseEntity<ConfiguredYellowFoxD8ImportPlanDto> getConfiguredYellowFoxPlan(@PathVariable String planKey) {
|
||||
return ResponseEntity.ok(configuredImportPlanService.getPlan(planKey));
|
||||
}
|
||||
|
||||
@PostMapping("/imports/configured-plans/{planKey}/start")
|
||||
public ResponseEntity<YellowFoxD8ImportTriggerResultDto> startConfiguredYellowFoxPlan(
|
||||
@PathVariable String planKey,
|
||||
@RequestParam(required = false) ImportMode mode,
|
||||
@RequestParam(required = false) AcquisitionStrategy strategy,
|
||||
@RequestParam(defaultValue = "PLAN_ONLY") SchedulerTriggerMode triggerMode
|
||||
) {
|
||||
YellowFoxD8ImportRequest request = configuredImportPlanService.createRequest(planKey, mode, strategy);
|
||||
YellowFoxD8ImportRunResultDto result = triggerMode == SchedulerTriggerMode.EXECUTE
|
||||
? importExecutionService.startAndExecuteImport(request)
|
||||
: importExecutionService.startImport(request);
|
||||
return ResponseEntity.accepted().body(new YellowFoxD8ImportTriggerResultDto(
|
||||
planKey,
|
||||
request.mode(),
|
||||
request.acquisitionStrategy(),
|
||||
triggerMode,
|
||||
OffsetDateTime.now(),
|
||||
result
|
||||
));
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,11 @@
|
|||
package at.procon.eventhub.yellowfox.camel;
|
||||
|
||||
import at.procon.eventhub.dto.EventHubEventDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8BookingEventMapper;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8IgnitionTransitionDetector;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import org.apache.camel.builder.RouteBuilder;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
|
|
@ -8,17 +13,34 @@ import org.springframework.stereotype.Component;
|
|||
public class YellowFoxD8BookingInputRoute extends RouteBuilder {
|
||||
|
||||
private final YellowFoxD8BookingEventMapper mapper;
|
||||
private final YellowFoxD8IgnitionTransitionDetector ignitionTransitionDetector;
|
||||
|
||||
public YellowFoxD8BookingInputRoute(YellowFoxD8BookingEventMapper mapper) {
|
||||
public YellowFoxD8BookingInputRoute(
|
||||
YellowFoxD8BookingEventMapper mapper,
|
||||
YellowFoxD8IgnitionTransitionDetector ignitionTransitionDetector
|
||||
) {
|
||||
this.mapper = mapper;
|
||||
this.ignitionTransitionDetector = ignitionTransitionDetector;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void configure() {
|
||||
from("direct:yellowfox-d8-booking-input")
|
||||
.routeId("yellowfox-d8-booking-input-route")
|
||||
.process(exchange -> {
|
||||
List<YellowFoxD8BookingDto> bookings = exchange.getMessage().getBody(List.class);
|
||||
YellowFoxD8IgnitionTransitionDetector.Session ignitionSession = ignitionTransitionDetector.newSession(false);
|
||||
List<EventHubEventDto> events = new ArrayList<>();
|
||||
for (YellowFoxD8BookingDto booking : bookings) {
|
||||
events.add(mapper.map(booking));
|
||||
EventHubEventDto ignitionEvent = ignitionSession.detect(booking);
|
||||
if (ignitionEvent != null) {
|
||||
events.add(ignitionEvent);
|
||||
}
|
||||
}
|
||||
exchange.getMessage().setBody(events);
|
||||
})
|
||||
.split(body())
|
||||
.bean(mapper, "map")
|
||||
.to("direct:eventhub-normalized-input")
|
||||
.end();
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
package at.procon.eventhub.yellowfox.camel;
|
||||
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8ImportExecutionService;
|
||||
import org.apache.camel.builder.RouteBuilder;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class YellowFoxD8ImportRequestRoute extends RouteBuilder {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(YellowFoxD8ImportRequestRoute.class);
|
||||
|
||||
private final YellowFoxD8ImportExecutionService executionService;
|
||||
|
||||
public YellowFoxD8ImportRequestRoute(YellowFoxD8ImportExecutionService executionService) {
|
||||
this.executionService = executionService;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void configure() {
|
||||
from("direct:yellowfox-d8-import-start")
|
||||
.routeId("yellowfox-d8-import-start-route")
|
||||
.process(exchange -> {
|
||||
YellowFoxD8ImportRequest request = exchange.getMessage().getBody(YellowFoxD8ImportRequest.class);
|
||||
var result = executionService.startImport(request);
|
||||
log.info("Prepared YellowFox D8 import run importRunId={} plannedPackages={} status={}",
|
||||
result.importRunId(), result.plannedPackageCount(), result.status());
|
||||
exchange.getMessage().setBody(result);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
package at.procon.eventhub.yellowfox.config;
|
||||
|
||||
import javax.sql.DataSource;
|
||||
import org.springframework.beans.factory.annotation.Qualifier;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
|
||||
import org.springframework.jdbc.datasource.DriverManagerDataSource;
|
||||
|
||||
@Configuration
|
||||
@ConditionalOnExpression("T(org.springframework.util.StringUtils).hasText('${eventhub.yellow-fox.datasource.jdbc-url:}')")
|
||||
public class YellowFoxDataSourceConfig {
|
||||
|
||||
@Bean
|
||||
@ConfigurationProperties(prefix = "eventhub.yellow-fox.datasource")
|
||||
public YellowFoxDataSourceProperties yellowFoxDataSourceProperties() {
|
||||
return new YellowFoxDataSourceProperties();
|
||||
}
|
||||
|
||||
@Bean(defaultCandidate = false)
|
||||
public DataSource yellowFoxDataSource(YellowFoxDataSourceProperties config) {
|
||||
DriverManagerDataSource dataSource = new DriverManagerDataSource();
|
||||
dataSource.setUrl(validateJdbcUrl(config));
|
||||
dataSource.setUsername(config.getUsername());
|
||||
dataSource.setPassword(config.getPassword());
|
||||
String driverClassName = trimToNull(config.getDriverClassName());
|
||||
if (driverClassName != null) {
|
||||
dataSource.setDriverClassName(driverClassName);
|
||||
}
|
||||
return dataSource;
|
||||
}
|
||||
|
||||
@Bean
|
||||
public NamedParameterJdbcTemplate yellowFoxNamedParameterJdbcTemplate(
|
||||
@Qualifier("yellowFoxDataSource") DataSource yellowFoxDataSource
|
||||
) {
|
||||
return new NamedParameterJdbcTemplate(yellowFoxDataSource);
|
||||
}
|
||||
|
||||
private String validateJdbcUrl(YellowFoxDataSourceProperties config) {
|
||||
String jdbcUrl = trimToNull(config.getJdbcUrl());
|
||||
if (jdbcUrl == null) {
|
||||
throw new IllegalStateException("eventhub.yellow-fox.datasource.jdbc-url must not be empty");
|
||||
}
|
||||
return jdbcUrl;
|
||||
}
|
||||
|
||||
private String trimToNull(String value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String trimmedValue = value.trim();
|
||||
return trimmedValue.isEmpty() ? null : trimmedValue;
|
||||
}
|
||||
|
||||
public static class YellowFoxDataSourceProperties {
|
||||
private String jdbcUrl;
|
||||
private String username;
|
||||
private String password;
|
||||
private String driverClassName;
|
||||
|
||||
public String getJdbcUrl() {
|
||||
return jdbcUrl;
|
||||
}
|
||||
|
||||
public void setJdbcUrl(String jdbcUrl) {
|
||||
this.jdbcUrl = jdbcUrl;
|
||||
}
|
||||
|
||||
public String getUsername() {
|
||||
return username;
|
||||
}
|
||||
|
||||
public void setUsername(String username) {
|
||||
this.username = username;
|
||||
}
|
||||
|
||||
public String getPassword() {
|
||||
return password;
|
||||
}
|
||||
|
||||
public void setPassword(String password) {
|
||||
this.password = password;
|
||||
}
|
||||
|
||||
public String getDriverClassName() {
|
||||
return driverClassName;
|
||||
}
|
||||
|
||||
public void setDriverClassName(String driverClassName) {
|
||||
this.driverClassName = driverClassName;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
package at.procon.eventhub.yellowfox.dto;
|
||||
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.EventFamily;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.dto.ImportMode;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
import at.procon.eventhub.dto.SourceGroupRefDto;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.Set;
|
||||
|
||||
public record ConfiguredYellowFoxD8ImportPlanDto(
|
||||
String planKey,
|
||||
boolean enabled,
|
||||
String cron,
|
||||
String tenantKey,
|
||||
EventSourceDto eventSource,
|
||||
SourceGroupRefDto sourceGroup,
|
||||
ImportScopeDto importScope,
|
||||
Set<EventFamily> eventFamilies,
|
||||
ImportMode initialMode,
|
||||
ImportMode scheduledMode,
|
||||
AcquisitionStrategy initialStrategy,
|
||||
AcquisitionStrategy scheduledStrategy,
|
||||
boolean refreshMasterDataFirst,
|
||||
OffsetDateTime initialOccurredFrom,
|
||||
OffsetDateTime initialOccurredTo,
|
||||
boolean runInitialOnStartup
|
||||
) {
|
||||
}
|
||||
|
|
@ -11,8 +11,10 @@ public record YellowFoxD8BookingDto(
|
|||
String sourceInstanceKey,
|
||||
String tenantProviderSettingKey,
|
||||
String externalFleetKey,
|
||||
String externalFleetName,
|
||||
String eventId,
|
||||
String key,
|
||||
Integer ignition,
|
||||
Integer eventType,
|
||||
Integer state,
|
||||
OffsetDateTime occurredAt,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
package at.procon.eventhub.yellowfox.dto;
|
||||
|
||||
import at.procon.eventhub.importing.ExtractionBatchResult;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
public record YellowFoxD8ExtractionBatchResultDto(
|
||||
UUID packageId,
|
||||
String extractionCode,
|
||||
String sourceKind,
|
||||
int sourceRowsRead,
|
||||
int eventsMapped,
|
||||
int eventsInserted,
|
||||
int alreadyImported,
|
||||
boolean executed,
|
||||
OffsetDateTime lastSourcePackageImportedAt,
|
||||
String lastSourcePackageId,
|
||||
OffsetDateTime lastSourceRowUpdatedAt,
|
||||
OffsetDateTime lastOccurredTo,
|
||||
Map<String, Integer> eventTypeCounts
|
||||
) implements ExtractionBatchResult {
|
||||
|
||||
public YellowFoxD8ExtractionBatchResultDto {
|
||||
eventTypeCounts = eventTypeCounts == null ? Map.of() : Map.copyOf(eventTypeCounts);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
package at.procon.eventhub.yellowfox.dto;
|
||||
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.EventFamily;
|
||||
import at.procon.eventhub.dto.EventSourceDto;
|
||||
import at.procon.eventhub.dto.ImportMode;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
import at.procon.eventhub.dto.SourceGroupRefDto;
|
||||
import at.procon.eventhub.importing.ImportRunRequest;
|
||||
import jakarta.validation.Valid;
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.NotNull;
|
||||
import java.util.EnumSet;
|
||||
import java.util.Set;
|
||||
|
||||
public record YellowFoxD8ImportRequest(
|
||||
@NotBlank String tenantKey,
|
||||
@Valid @NotNull EventSourceDto eventSource,
|
||||
@Valid SourceGroupRefDto sourceGroup,
|
||||
@Valid @NotNull ImportScopeDto importScope,
|
||||
Set<EventFamily> eventFamilies,
|
||||
ImportMode mode,
|
||||
boolean refreshMasterDataFirst,
|
||||
AcquisitionStrategy acquisitionStrategy
|
||||
) implements ImportRunRequest {
|
||||
public YellowFoxD8ImportRequest {
|
||||
tenantKey = tenantKey == null ? null : tenantKey.trim();
|
||||
if (eventSource == null) {
|
||||
eventSource = new EventSourceDto("YELLOWFOX", "TELEMATICS_PLATFORM", "YELLOWFOX_D8", null, null, null);
|
||||
}
|
||||
if (importScope == null) {
|
||||
importScope = ImportScopeDto.tenantAll(null, null);
|
||||
}
|
||||
if (eventFamilies == null || eventFamilies.isEmpty()) {
|
||||
eventFamilies = EnumSet.of(EventFamily.DRIVER_ACTIVITY, EventFamily.DRIVER_CARD);
|
||||
} else {
|
||||
eventFamilies = EnumSet.copyOf(eventFamilies);
|
||||
}
|
||||
if (mode == null) {
|
||||
mode = ImportMode.INITIAL_BACKFILL;
|
||||
}
|
||||
if (acquisitionStrategy == null || acquisitionStrategy == AcquisitionStrategy.SOURCE_PACKAGE_WATERMARK) {
|
||||
acquisitionStrategy = mode == ImportMode.INCREMENTAL_UPDATE
|
||||
? AcquisitionStrategy.SOURCE_ROW_WATERMARK
|
||||
: AcquisitionStrategy.OCCURRED_AT_WINDOW_WITH_OVERLAP;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
package at.procon.eventhub.yellowfox.dto;
|
||||
|
||||
import at.procon.eventhub.dto.ImportRunStatus;
|
||||
import at.procon.eventhub.importing.ImportPlanDto;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
public record YellowFoxD8ImportRunResultDto(
|
||||
UUID importRunId,
|
||||
ImportRunStatus status,
|
||||
int plannedPackageCount,
|
||||
ImportPlanDto plan,
|
||||
List<UUID> plannedPackageIds
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
package at.procon.eventhub.yellowfox.dto;
|
||||
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.ImportMode;
|
||||
import at.procon.eventhub.dto.SchedulerTriggerMode;
|
||||
import java.time.OffsetDateTime;
|
||||
|
||||
public record YellowFoxD8ImportTriggerResultDto(
|
||||
String planKey,
|
||||
ImportMode mode,
|
||||
AcquisitionStrategy acquisitionStrategy,
|
||||
SchedulerTriggerMode triggerMode,
|
||||
OffsetDateTime triggeredAt,
|
||||
YellowFoxD8ImportRunResultDto result
|
||||
) {
|
||||
}
|
||||
|
|
@ -0,0 +1,238 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.EventHubEventDto;
|
||||
import at.procon.eventhub.dto.EventType;
|
||||
import at.procon.eventhub.dto.ImportCursorStateDto;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
import at.procon.eventhub.dto.SourceGroupType;
|
||||
import at.procon.eventhub.importing.ImportPlanItemDto;
|
||||
import at.procon.eventhub.importing.ImportTimeChunkDto;
|
||||
import at.procon.eventhub.importing.persistence.ImportCursorRepository;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ExtractionBatchResultDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import org.apache.camel.ProducerTemplate;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.beans.factory.annotation.Qualifier;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
|
||||
import org.springframework.core.io.Resource;
|
||||
import org.springframework.core.io.ResourceLoader;
|
||||
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.util.StreamUtils;
|
||||
|
||||
@Service
|
||||
@ConditionalOnExpression("T(org.springframework.util.StringUtils).hasText('${eventhub.yellow-fox.datasource.jdbc-url:}')")
|
||||
public class JdbcYellowFoxD8BookingExtractionBatchExecutor implements YellowFoxD8ExtractionBatchExecutor {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(JdbcYellowFoxD8BookingExtractionBatchExecutor.class);
|
||||
private static final String EXTRACTION_CODE = "YELLOWFOX_D8_BOOKING";
|
||||
private static final int PROGRESS_LOG_INTERVAL = 5000;
|
||||
|
||||
private final NamedParameterJdbcTemplate jdbcTemplate;
|
||||
private final ProducerTemplate producerTemplate;
|
||||
private final ResourceLoader resourceLoader;
|
||||
private final ImportCursorRepository importCursorRepository;
|
||||
private final EventHubProperties properties;
|
||||
private final YellowFoxD8BookingRowMapper rowMapper;
|
||||
private final YellowFoxD8BookingEventMapper eventMapper;
|
||||
private final YellowFoxD8IgnitionTransitionDetector ignitionTransitionDetector;
|
||||
|
||||
public JdbcYellowFoxD8BookingExtractionBatchExecutor(
|
||||
@Qualifier("yellowFoxNamedParameterJdbcTemplate") NamedParameterJdbcTemplate jdbcTemplate,
|
||||
ProducerTemplate producerTemplate,
|
||||
ResourceLoader resourceLoader,
|
||||
ImportCursorRepository importCursorRepository,
|
||||
EventHubProperties properties,
|
||||
YellowFoxD8BookingRowMapper rowMapper,
|
||||
YellowFoxD8BookingEventMapper eventMapper,
|
||||
YellowFoxD8IgnitionTransitionDetector ignitionTransitionDetector
|
||||
) {
|
||||
this.jdbcTemplate = jdbcTemplate;
|
||||
this.producerTemplate = producerTemplate;
|
||||
this.resourceLoader = resourceLoader;
|
||||
this.importCursorRepository = importCursorRepository;
|
||||
this.properties = properties;
|
||||
this.rowMapper = rowMapper;
|
||||
this.eventMapper = eventMapper;
|
||||
this.ignitionTransitionDetector = ignitionTransitionDetector;
|
||||
}
|
||||
|
||||
@Override
|
||||
public YellowFoxD8ExtractionBatchResultDto execute(
|
||||
UUID importRunId,
|
||||
UUID packageId,
|
||||
int eventSourceId,
|
||||
YellowFoxD8ImportRequest request,
|
||||
ImportPlanItemDto planItem,
|
||||
ImportTimeChunkDto chunk
|
||||
) {
|
||||
if (!EXTRACTION_CODE.equals(planItem.extractionCode())) {
|
||||
throw new IllegalArgumentException("Unsupported YellowFox extraction code " + planItem.extractionCode());
|
||||
}
|
||||
|
||||
ImportScopeDto chunkScope = chunkScope(request.importScope(), chunk);
|
||||
ImportCursorStateDto cursor = findCursor(eventSourceId, request, planItem);
|
||||
Map<String, Object> params = parameters(request, chunkScope, cursor);
|
||||
Stats stats = new Stats();
|
||||
YellowFoxD8IgnitionTransitionDetector.Session ignitionSession = ignitionTransitionDetector
|
||||
.newSession(properties.getYellowFox().isEmitInitialIgnitionSnapshot());
|
||||
|
||||
log.info("Reading YellowFox D8 bookings tenant={} importRunId={} packageId={} chunk={} occurredFrom={} occurredTo={} fleetId={} strategy={}",
|
||||
request.tenantKey(), importRunId, packageId, chunk.sequence(), chunk.occurredFrom(), chunk.occurredTo(), params.get("fleetId"), request.acquisitionStrategy());
|
||||
|
||||
jdbcTemplate.query(loadSql(), params, rs -> {
|
||||
stats.sourceRowsRead++;
|
||||
YellowFoxD8BookingDto booking = rowMapper.map(
|
||||
rs,
|
||||
request.tenantKey(),
|
||||
request.eventSource().sourceInstanceKey(),
|
||||
request.eventSource().tenantProviderSettingKey()
|
||||
);
|
||||
stats.acceptSourceRow(booking);
|
||||
if (!hasEventReference(booking)) {
|
||||
stats.skippedRows++;
|
||||
return;
|
||||
}
|
||||
EventHubEventDto primaryEvent = eventMapper.map(booking);
|
||||
send(primaryEvent, stats);
|
||||
EventHubEventDto ignitionEvent = ignitionSession.detect(booking);
|
||||
if (ignitionEvent != null) {
|
||||
send(ignitionEvent, stats);
|
||||
}
|
||||
if (stats.sourceRowsRead % PROGRESS_LOG_INTERVAL == 0) {
|
||||
log.info("YellowFox D8 extraction progress tenant={} importRunId={} packageId={} rows={} events={} byType={}",
|
||||
request.tenantKey(), importRunId, packageId, stats.sourceRowsRead, stats.eventsSent, stats.eventTypeCounts);
|
||||
}
|
||||
});
|
||||
|
||||
log.info("Finished YellowFox D8 extraction tenant={} importRunId={} packageId={} rows={} events={} skippedRows={} byType={}",
|
||||
request.tenantKey(), importRunId, packageId, stats.sourceRowsRead, stats.eventsSent, stats.skippedRows, stats.eventTypeCounts);
|
||||
|
||||
return new YellowFoxD8ExtractionBatchResultDto(
|
||||
packageId,
|
||||
planItem.extractionCode(),
|
||||
planItem.sourceKind(),
|
||||
stats.sourceRowsRead,
|
||||
stats.eventsSent,
|
||||
stats.eventsSent,
|
||||
stats.skippedRows,
|
||||
true,
|
||||
null,
|
||||
stats.lastEventId,
|
||||
stats.lastOccurredAt,
|
||||
stats.lastOccurredAt,
|
||||
stats.eventTypeCounts
|
||||
);
|
||||
}
|
||||
|
||||
private ImportCursorStateDto findCursor(int eventSourceId, YellowFoxD8ImportRequest request, ImportPlanItemDto planItem) {
|
||||
String scopeHash = request.importScope() == null ? "NO_SCOPE" : request.importScope().stableKey();
|
||||
return importCursorRepository.findCursor(
|
||||
request.tenantKey(),
|
||||
eventSourceId,
|
||||
scopeHash,
|
||||
planItem.eventFamily(),
|
||||
planItem.sourceKind(),
|
||||
request.acquisitionStrategy()
|
||||
);
|
||||
}
|
||||
|
||||
private Map<String, Object> parameters(YellowFoxD8ImportRequest request, ImportScopeDto scope, ImportCursorStateDto cursor) {
|
||||
Map<String, Object> params = new HashMap<>();
|
||||
params.put("occurredFrom", scope == null ? null : scope.occurredFrom());
|
||||
params.put("occurredTo", scope == null ? null : scope.occurredTo());
|
||||
params.put("fleetId", fleetId(request));
|
||||
if (request.acquisitionStrategy() == AcquisitionStrategy.SOURCE_ROW_WATERMARK && cursor != null) {
|
||||
OffsetDateTime lastOccurredTo = cursor.lastOccurredTo();
|
||||
String lastSourceRowId = cursor.lastSourcePackageId();
|
||||
if (lastOccurredTo != null && properties.getYellowFox().getOccurredAtOverlap() != null
|
||||
&& !properties.getYellowFox().getOccurredAtOverlap().isZero()) {
|
||||
lastOccurredTo = lastOccurredTo.minus(properties.getYellowFox().getOccurredAtOverlap());
|
||||
lastSourceRowId = null;
|
||||
}
|
||||
params.put("lastOccurredTo", lastOccurredTo);
|
||||
params.put("lastSourceRowId", lastSourceRowId);
|
||||
} else {
|
||||
params.put("lastOccurredTo", null);
|
||||
params.put("lastSourceRowId", null);
|
||||
}
|
||||
return params;
|
||||
}
|
||||
|
||||
private Integer fleetId(YellowFoxD8ImportRequest request) {
|
||||
if (request.sourceGroup() == null || request.sourceGroup().type() != SourceGroupType.FLEET) {
|
||||
return null;
|
||||
}
|
||||
String value = request.sourceGroup().sourceEntityId() == null
|
||||
? request.sourceGroup().code()
|
||||
: request.sourceGroup().sourceEntityId();
|
||||
if (value == null || value.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return Integer.parseInt(value.trim());
|
||||
} catch (NumberFormatException ex) {
|
||||
throw new IllegalArgumentException("YellowFox D8 fleet sourceEntityId/code must be numeric for JDBC import: " + value, ex);
|
||||
}
|
||||
}
|
||||
|
||||
private ImportScopeDto chunkScope(ImportScopeDto scope, ImportTimeChunkDto chunk) {
|
||||
if (scope == null) {
|
||||
return ImportScopeDto.tenantAll(chunk.occurredFrom(), chunk.occurredTo());
|
||||
}
|
||||
return new ImportScopeDto(scope.type(), scope.rootSourceOrganisation(), scope.includeChildren(), chunk.occurredFrom(), chunk.occurredTo());
|
||||
}
|
||||
|
||||
private boolean hasEventReference(YellowFoxD8BookingDto booking) {
|
||||
return (booking.driverRef() != null && booking.driverRef().hasAnyReference())
|
||||
|| (booking.vehicleRef() != null && booking.vehicleRef().hasAnyReference());
|
||||
}
|
||||
|
||||
private void send(EventHubEventDto event, Stats stats) {
|
||||
producerTemplate.sendBody("direct:eventhub-normalized-input", event);
|
||||
stats.acceptEvent(event);
|
||||
}
|
||||
|
||||
private String loadSql() {
|
||||
Resource resource = resourceLoader.getResource("classpath:sql/yellowfox/d8-bookings.sql");
|
||||
try (var in = resource.getInputStream()) {
|
||||
return StreamUtils.copyToString(in, StandardCharsets.UTF_8);
|
||||
} catch (IOException ex) {
|
||||
throw new IllegalStateException("Failed to load YellowFox D8 extraction SQL", ex);
|
||||
}
|
||||
}
|
||||
|
||||
private static class Stats {
|
||||
private int sourceRowsRead;
|
||||
private int eventsSent;
|
||||
private int skippedRows;
|
||||
private OffsetDateTime lastOccurredAt;
|
||||
private String lastEventId;
|
||||
private final Map<String, Integer> eventTypeCounts = new LinkedHashMap<>();
|
||||
|
||||
private void acceptSourceRow(YellowFoxD8BookingDto booking) {
|
||||
if (booking.occurredAt() != null) {
|
||||
lastOccurredAt = booking.occurredAt();
|
||||
lastEventId = booking.eventId();
|
||||
}
|
||||
}
|
||||
|
||||
private void acceptEvent(EventHubEventDto event) {
|
||||
eventsSent++;
|
||||
EventType type = event.eventType();
|
||||
String key = event.eventDomain().name() + "/" + (type == null ? "UNKNOWN" : type.name());
|
||||
eventTypeCounts.merge(key, 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.importing.ImportPlanItemDto;
|
||||
import at.procon.eventhub.importing.ImportTimeChunkDto;
|
||||
import at.procon.eventhub.importing.extraction.AbstractNoopExtractionBatchExecutor;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ExtractionBatchResultDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import java.util.UUID;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
@ConditionalOnExpression("!T(org.springframework.util.StringUtils).hasText('${eventhub.yellow-fox.datasource.jdbc-url:}')")
|
||||
public class NoopYellowFoxD8ExtractionBatchExecutor
|
||||
extends AbstractNoopExtractionBatchExecutor<YellowFoxD8ImportRequest, YellowFoxD8ExtractionBatchResultDto>
|
||||
implements YellowFoxD8ExtractionBatchExecutor {
|
||||
|
||||
@Override
|
||||
protected YellowFoxD8ExtractionBatchResultDto emptyResult(
|
||||
UUID packageId,
|
||||
ImportPlanItemDto planItem,
|
||||
ImportTimeChunkDto chunk
|
||||
) {
|
||||
return new YellowFoxD8ExtractionBatchResultDto(
|
||||
packageId,
|
||||
planItem.extractionCode(),
|
||||
planItem.sourceKind(),
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
false,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
java.util.Map.of()
|
||||
);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String providerName() {
|
||||
return "yellowfox-d8";
|
||||
}
|
||||
}
|
||||
|
|
@ -2,7 +2,8 @@ package at.procon.eventhub.yellowfox.service;
|
|||
|
||||
import at.procon.eventhub.dto.CardSlot;
|
||||
import at.procon.eventhub.dto.CardStatus;
|
||||
import at.procon.eventhub.dto.DrivingStatus;
|
||||
import at.procon.eventhub.dto.DriverRefDto;
|
||||
import at.procon.eventhub.dto.EventDetailsDto;
|
||||
import at.procon.eventhub.dto.EventDomain;
|
||||
import at.procon.eventhub.dto.EventHubEventDto;
|
||||
import at.procon.eventhub.dto.EventHubPackageRequest;
|
||||
|
|
@ -13,6 +14,7 @@ import at.procon.eventhub.dto.GeoPointDto;
|
|||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
import at.procon.eventhub.dto.SourceGroupRefDto;
|
||||
import at.procon.eventhub.dto.SourceGroupType;
|
||||
import at.procon.eventhub.dto.VehicleRefDto;
|
||||
import at.procon.eventhub.service.EventDetailsFactory;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import java.time.LocalDate;
|
||||
|
|
@ -25,6 +27,8 @@ import org.springframework.stereotype.Component;
|
|||
@Component
|
||||
public class YellowFoxD8BookingEventMapper {
|
||||
|
||||
public static final String DETAIL_SOURCE = "YELLOWFOX_D8";
|
||||
|
||||
private final EventDetailsFactory detailsFactory;
|
||||
|
||||
public YellowFoxD8BookingEventMapper(EventDetailsFactory detailsFactory) {
|
||||
|
|
@ -33,30 +37,42 @@ public class YellowFoxD8BookingEventMapper {
|
|||
|
||||
public EventHubEventDto map(YellowFoxD8BookingDto source) {
|
||||
NormalizedEvent normalized = normalize(source.eventType(), source.state());
|
||||
Map<String, Object> payload = new LinkedHashMap<>();
|
||||
if (source.payload() != null) {
|
||||
payload.putAll(source.payload());
|
||||
}
|
||||
payload.put("provider", "YELLOWFOX");
|
||||
payload.put("sourceKind", "TELEMATICS_PLATFORM");
|
||||
payload.put("yellowFoxEventType", source.eventType());
|
||||
payload.put("yellowFoxState", source.state());
|
||||
payload.put("yellowFoxKey", source.key());
|
||||
return event(source, sourceEventId(source, "D8_BOOKING"), normalized, detailsFor(source, normalized), false);
|
||||
}
|
||||
|
||||
EventSourceDto eventSource = new EventSourceDto(
|
||||
"YELLOWFOX",
|
||||
"TELEMATICS_PLATFORM",
|
||||
"YELLOWFOX_D8",
|
||||
source.sourceInstanceKey(),
|
||||
source.tenantProviderSettingKey(),
|
||||
source.externalFleetKey()
|
||||
public EventHubEventDto mapIgnitionTransition(YellowFoxD8BookingDto source, Integer previousIgnitionState) {
|
||||
if (source.ignition() == null) {
|
||||
return null;
|
||||
}
|
||||
boolean ignitionOn = source.ignition() == 1;
|
||||
NormalizedEvent normalized = new NormalizedEvent(
|
||||
EventDomain.IGNITION,
|
||||
ignitionOn ? EventType.IGNITION_ON : EventType.IGNITION_OFF,
|
||||
ignitionOn ? EventLifecycle.ON : EventLifecycle.OFF,
|
||||
null,
|
||||
null
|
||||
);
|
||||
Map<String, Object> attributes = baseAttributes(source, normalized);
|
||||
attributes.put("previousIgnitionState", previousIgnitionState);
|
||||
attributes.put("previousIgnitionMeaning", ignitionMeaning(previousIgnitionState));
|
||||
attributes.put("derivedFrom", "YELLOWFOX_D8_BOOKING");
|
||||
attributes.put("sourceEventId", sourceEventId(source, "D8_BOOKING"));
|
||||
EventDetailsDto details = new EventDetailsDto("IGNITION", detailsFactory.payloadFromMap(attributes));
|
||||
return event(source, sourceEventId(source, "D8_IGNITION") + ":" + (ignitionOn ? "ON" : "OFF"), normalized, details, true);
|
||||
}
|
||||
|
||||
private EventHubEventDto event(
|
||||
YellowFoxD8BookingDto source,
|
||||
String externalSourceEventId,
|
||||
NormalizedEvent normalized,
|
||||
EventDetailsDto details,
|
||||
boolean derived
|
||||
) {
|
||||
EventSourceDto eventSource = eventSource(source);
|
||||
LocalDate businessDate = source.occurredAt().toLocalDate();
|
||||
var occurredFrom = businessDate.atStartOfDay(source.occurredAt().getOffset()).toOffsetDateTime();
|
||||
var occurredTo = businessDate.plusDays(1).atStartOfDay(source.occurredAt().getOffset()).toOffsetDateTime();
|
||||
SourceGroupRefDto sourceGroup = source.externalFleetKey() == null || source.externalFleetKey().isBlank()
|
||||
? null
|
||||
: new SourceGroupRefDto(SourceGroupType.FLEET, source.externalFleetKey(), source.externalFleetKey(), null);
|
||||
SourceGroupRefDto sourceGroup = sourceGroup(source);
|
||||
EventHubPackageRequest packageInfo = new EventHubPackageRequest(
|
||||
tenantOrDefault(source.tenantKey()),
|
||||
eventSource,
|
||||
|
|
@ -69,8 +85,8 @@ public class YellowFoxD8BookingEventMapper {
|
|||
|
||||
return new EventHubEventDto(
|
||||
UUID.randomUUID(),
|
||||
source.eventId(),
|
||||
source.driverRef(),
|
||||
externalSourceEventId,
|
||||
derived && normalized.domain() == EventDomain.IGNITION ? null : source.driverRef(),
|
||||
source.vehicleRef(),
|
||||
source.occurredAt(),
|
||||
source.receivedPartnerAt(),
|
||||
|
|
@ -79,60 +95,170 @@ public class YellowFoxD8BookingEventMapper {
|
|||
normalized.type(),
|
||||
normalized.lifecycle(),
|
||||
source.odometerM(),
|
||||
new GeoPointDto(source.latitude(), source.longitude()),
|
||||
detailsFor(normalized),
|
||||
point(source),
|
||||
details,
|
||||
null,
|
||||
detailsFactory.payloadFromMap(payload),
|
||||
detailsFactory.payloadFromMap(payload(source, derived)),
|
||||
false,
|
||||
packageInfo
|
||||
);
|
||||
}
|
||||
|
||||
public EventSourceDto eventSource(YellowFoxD8BookingDto source) {
|
||||
return new EventSourceDto(
|
||||
"YELLOWFOX",
|
||||
"TELEMATICS_PLATFORM",
|
||||
"YELLOWFOX_D8",
|
||||
source.sourceInstanceKey(),
|
||||
source.tenantProviderSettingKey(),
|
||||
source.externalFleetKey()
|
||||
);
|
||||
}
|
||||
|
||||
private SourceGroupRefDto sourceGroup(YellowFoxD8BookingDto source) {
|
||||
if (source.externalFleetKey() == null || source.externalFleetKey().isBlank()) {
|
||||
return null;
|
||||
}
|
||||
return new SourceGroupRefDto(
|
||||
SourceGroupType.FLEET,
|
||||
source.externalFleetKey(),
|
||||
source.externalFleetKey(),
|
||||
source.externalFleetName() == null ? source.externalFleetKey() : source.externalFleetName()
|
||||
);
|
||||
}
|
||||
|
||||
private GeoPointDto point(YellowFoxD8BookingDto source) {
|
||||
if (source.latitude() == null && source.longitude() == null) {
|
||||
return null;
|
||||
}
|
||||
return new GeoPointDto(source.latitude(), source.longitude());
|
||||
}
|
||||
|
||||
private String tenantOrDefault(String value) {
|
||||
return value == null || value.isBlank() ? "default" : value.trim();
|
||||
}
|
||||
|
||||
private at.procon.eventhub.dto.EventDetailsDto detailsFor(NormalizedEvent normalized) {
|
||||
if (normalized.domain() == EventDomain.DRIVER_ACTIVITY) {
|
||||
return detailsFactory.driverActivity(CardSlot.DRIVER, CardStatus.INSERTED, DrivingStatus.UNKNOWN);
|
||||
private String sourceEventId(YellowFoxD8BookingDto source, String prefix) {
|
||||
String eventId = source.eventId() == null || source.eventId().isBlank() ? "UNKNOWN_EVENT" : source.eventId().trim();
|
||||
return prefix + ":" + eventId + ":" + source.occurredAt();
|
||||
}
|
||||
|
||||
private Map<String, Object> payload(YellowFoxD8BookingDto source, boolean derived) {
|
||||
Map<String, Object> payload = new LinkedHashMap<>();
|
||||
if (source.payload() != null) {
|
||||
payload.putAll(source.payload());
|
||||
}
|
||||
if (normalized.domain() == EventDomain.DRIVER_CARD) {
|
||||
CardStatus status = normalized.lifecycle() == EventLifecycle.INSERT ? CardStatus.INSERTED : CardStatus.NOT_INSERTED;
|
||||
return detailsFactory.driverCard(CardSlot.DRIVER, status);
|
||||
}
|
||||
return null;
|
||||
payload.put("provider", "YELLOWFOX");
|
||||
payload.put("sourceKind", "TELEMATICS_PLATFORM");
|
||||
payload.put("sourceKey", "YELLOWFOX_D8");
|
||||
payload.put("derived", derived);
|
||||
payload.put("yellowFoxEventType", source.eventType());
|
||||
payload.put("yellowFoxState", source.state());
|
||||
payload.put("yellowFoxIgnition", source.ignition());
|
||||
payload.put("yellowFoxKey", source.key());
|
||||
return payload;
|
||||
}
|
||||
|
||||
private EventDetailsDto detailsFor(YellowFoxD8BookingDto source, NormalizedEvent normalized) {
|
||||
String type = switch (normalized.domain()) {
|
||||
case DRIVER_ACTIVITY -> "DRIVER_ACTIVITY";
|
||||
case DRIVER_CARD -> "DRIVER_CARD";
|
||||
default -> "YELLOWFOX_D8_BOOKING";
|
||||
};
|
||||
return new EventDetailsDto(type, detailsFactory.payloadFromMap(baseAttributes(source, normalized)));
|
||||
}
|
||||
|
||||
private Map<String, Object> baseAttributes(YellowFoxD8BookingDto source, NormalizedEvent normalized) {
|
||||
Map<String, Object> attributes = new LinkedHashMap<>();
|
||||
attributes.put("source", DETAIL_SOURCE);
|
||||
attributes.put("yellowFoxEventType", source.eventType());
|
||||
attributes.put("yellowFoxState", source.state());
|
||||
attributes.put("yellowFoxStateMeaning", stateMeaning(source.eventType(), source.state()));
|
||||
attributes.put("yellowFoxKey", source.key());
|
||||
attributes.put("ignitionState", source.ignition());
|
||||
attributes.put("ignitionMeaning", ignitionMeaning(source.ignition()));
|
||||
attributes.put("odometer", source.odometerM());
|
||||
attributes.put("slot", normalized.slot() == null ? null : normalized.slot().name());
|
||||
attributes.put("slotNumber", normalized.slotNumber());
|
||||
attributes.put("cardSlot", normalized.slot() == null ? null : normalized.slot().name());
|
||||
attributes.put("eventDomain", normalized.domain().name());
|
||||
attributes.put("eventType", normalized.type().name());
|
||||
attributes.put("lifecycle", normalized.lifecycle().name());
|
||||
removeNullValues(attributes);
|
||||
return attributes;
|
||||
}
|
||||
|
||||
private void removeNullValues(Map<String, Object> attributes) {
|
||||
attributes.entrySet().removeIf(entry -> entry.getValue() == null);
|
||||
}
|
||||
|
||||
private NormalizedEvent normalize(Integer eventType, Integer state) {
|
||||
if (eventType == null || state == null) {
|
||||
return new NormalizedEvent(EventDomain.TELEMATICS_DATA, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT);
|
||||
return new NormalizedEvent(EventDomain.TELEMATICS_DATA, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT, null, null);
|
||||
}
|
||||
|
||||
return switch (eventType) {
|
||||
case 0, 1 -> normalizeCardActivity(state);
|
||||
case 2, 3 -> normalizeDriverActivity(state);
|
||||
default -> new NormalizedEvent(EventDomain.TELEMATICS_DATA, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT);
|
||||
case 0 -> normalizeCardActivity(CardSlot.DRIVER, 1, state);
|
||||
case 1 -> normalizeCardActivity(CardSlot.CO_DRIVER, 2, state);
|
||||
case 2 -> normalizeDriverActivity(CardSlot.DRIVER, 1, state);
|
||||
case 3 -> normalizeDriverActivity(CardSlot.CO_DRIVER, 2, state);
|
||||
default -> new NormalizedEvent(EventDomain.TELEMATICS_DATA, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT, null, null);
|
||||
};
|
||||
}
|
||||
|
||||
private NormalizedEvent normalizeCardActivity(Integer state) {
|
||||
private NormalizedEvent normalizeCardActivity(CardSlot slot, int slotNumber, Integer state) {
|
||||
return switch (state) {
|
||||
case 0 -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.CARD_WITHDRAWN, EventLifecycle.WITHDRAW);
|
||||
case 1 -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.CARD_INSERTED, EventLifecycle.INSERT);
|
||||
default -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT);
|
||||
case 0 -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.CARD_WITHDRAWN, EventLifecycle.WITHDRAW, slot, slotNumber);
|
||||
case 1 -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.CARD_INSERTED, EventLifecycle.INSERT, slot, slotNumber);
|
||||
default -> new NormalizedEvent(EventDomain.DRIVER_CARD, EventType.UNKNOWN_EVENT, EventLifecycle.SNAPSHOT, slot, slotNumber);
|
||||
};
|
||||
}
|
||||
|
||||
private NormalizedEvent normalizeDriverActivity(Integer state) {
|
||||
private NormalizedEvent normalizeDriverActivity(CardSlot slot, int slotNumber, Integer state) {
|
||||
return switch (state) {
|
||||
case 0 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.BREAK_REST, EventLifecycle.SNAPSHOT);
|
||||
case 1 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.AVAILABILITY, EventLifecycle.SNAPSHOT);
|
||||
case 2 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.WORK, EventLifecycle.SNAPSHOT);
|
||||
case 3 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.DRIVE, EventLifecycle.SNAPSHOT);
|
||||
default -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.UNKNOWN_ACTIVITY, EventLifecycle.SNAPSHOT);
|
||||
case 0 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.BREAK_REST, EventLifecycle.START, slot, slotNumber);
|
||||
case 1 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.AVAILABILITY, EventLifecycle.START, slot, slotNumber);
|
||||
case 2 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.WORK, EventLifecycle.START, slot, slotNumber);
|
||||
case 3 -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.DRIVE, EventLifecycle.START, slot, slotNumber);
|
||||
default -> new NormalizedEvent(EventDomain.DRIVER_ACTIVITY, EventType.UNKNOWN_ACTIVITY, EventLifecycle.SNAPSHOT, slot, slotNumber);
|
||||
};
|
||||
}
|
||||
|
||||
private record NormalizedEvent(EventDomain domain, EventType type, EventLifecycle lifecycle) {
|
||||
private String stateMeaning(Integer eventType, Integer state) {
|
||||
if (eventType == null || state == null) {
|
||||
return null;
|
||||
}
|
||||
if (eventType == 0 || eventType == 1) {
|
||||
return switch (state) {
|
||||
case 0 -> "CARD_REMOVED";
|
||||
case 1 -> "CARD_INSERTED";
|
||||
default -> "UNKNOWN_CARD_STATE";
|
||||
};
|
||||
}
|
||||
if (eventType == 2 || eventType == 3) {
|
||||
return switch (state) {
|
||||
case 0 -> "PAUSE_OR_REST";
|
||||
case 1 -> "STANDBY_TIME";
|
||||
case 2 -> "WORK_TIME";
|
||||
case 3 -> "STEERING_TIME";
|
||||
default -> "UNKNOWN_ACTIVITY_STATE";
|
||||
};
|
||||
}
|
||||
return "UNKNOWN_EVENT_TYPE";
|
||||
}
|
||||
|
||||
private String ignitionMeaning(Integer ignition) {
|
||||
if (ignition == null) {
|
||||
return null;
|
||||
}
|
||||
return ignition == 1 ? "ON" : ignition == 0 ? "OFF" : "UNKNOWN";
|
||||
}
|
||||
|
||||
private record NormalizedEvent(
|
||||
EventDomain domain,
|
||||
EventType type,
|
||||
EventLifecycle lifecycle,
|
||||
CardSlot slot,
|
||||
Integer slotNumber
|
||||
) {
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,117 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.dto.DriverCardRefDto;
|
||||
import at.procon.eventhub.dto.DriverRefDto;
|
||||
import at.procon.eventhub.dto.VehicleRefDto;
|
||||
import at.procon.eventhub.dto.VehicleRegistrationRefDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import java.math.BigDecimal;
|
||||
import java.sql.ResultSet;
|
||||
import java.sql.SQLException;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class YellowFoxD8BookingRowMapper {
|
||||
|
||||
private static final TypeReference<Map<String, Object>> MAP_TYPE = new TypeReference<>() {
|
||||
};
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public YellowFoxD8BookingRowMapper(ObjectMapper objectMapper) {
|
||||
this.objectMapper = objectMapper;
|
||||
}
|
||||
|
||||
public YellowFoxD8BookingDto map(ResultSet rs, String tenantKey, String sourceInstanceKey, String tenantProviderSettingKey) throws SQLException {
|
||||
String eventId = rs.getString("eventid");
|
||||
OffsetDateTime occurredAt = rs.getObject("utc", OffsetDateTime.class);
|
||||
Integer vehicleId = getInteger(rs, "vehicle_id");
|
||||
Integer driverId = getInteger(rs, "driver_id");
|
||||
String vehicleVrn = rs.getString("vehicle_vrn");
|
||||
String vehicleVin = rs.getString("vehicle_vin");
|
||||
String driverCard = rs.getString("driver_card_number");
|
||||
Integer fleetId = getInteger(rs, "fleet_id");
|
||||
String fleetName = rs.getString("fleet_name");
|
||||
Integer odometer = getInteger(rs, "odometer");
|
||||
|
||||
Map<String, Object> payload = payload(rs.getString("payload"));
|
||||
put(payload, "yellowFoxEventId", eventId);
|
||||
put(payload, "yellowFoxOdometerRaw", odometer);
|
||||
put(payload, "vehicleVrn", vehicleVrn);
|
||||
put(payload, "vehicleVin", vehicleVin);
|
||||
put(payload, "driverCardNumber", driverCard);
|
||||
put(payload, "driverFirstName", rs.getString("driver_firstname"));
|
||||
put(payload, "driverLastName", rs.getString("driver_lastname"));
|
||||
put(payload, "fleetId", fleetId);
|
||||
put(payload, "fleetName", fleetName);
|
||||
put(payload, "telematicProviderId", getInteger(rs, "telematic_provider_id"));
|
||||
put(payload, "telematicProviderName", rs.getString("telematic_provider_name"));
|
||||
|
||||
DriverRefDto driverRef = driverId == null && isBlank(driverCard)
|
||||
? null
|
||||
: new DriverRefDto(driverId == null ? null : driverId.toString(), new DriverCardRefDto(null, driverCard));
|
||||
VehicleRefDto vehicleRef = vehicleId == null && isBlank(vehicleVin) && isBlank(vehicleVrn)
|
||||
? null
|
||||
: new VehicleRefDto(
|
||||
vehicleId == null ? null : vehicleId.toString(),
|
||||
vehicleVin,
|
||||
vehicleId == null ? null : vehicleId.toString(),
|
||||
new VehicleRegistrationRefDto(null, vehicleVrn)
|
||||
);
|
||||
|
||||
return new YellowFoxD8BookingDto(
|
||||
tenantKey,
|
||||
sourceInstanceKey,
|
||||
tenantProviderSettingKey,
|
||||
fleetId == null ? null : fleetId.toString(),
|
||||
fleetName,
|
||||
eventId,
|
||||
rs.getString("key"),
|
||||
getInteger(rs, "ignition"),
|
||||
getInteger(rs, "eventtype"),
|
||||
getInteger(rs, "state"),
|
||||
occurredAt,
|
||||
null,
|
||||
driverRef,
|
||||
vehicleRef,
|
||||
odometer == null ? null : odometer.longValue() * 1000L,
|
||||
rs.getBigDecimal("latitude"),
|
||||
rs.getBigDecimal("longitude"),
|
||||
payload
|
||||
);
|
||||
}
|
||||
|
||||
private Integer getInteger(ResultSet rs, String column) throws SQLException {
|
||||
int value = rs.getInt(column);
|
||||
return rs.wasNull() ? null : value;
|
||||
}
|
||||
|
||||
private Map<String, Object> payload(String json) {
|
||||
if (json == null || json.isBlank()) {
|
||||
return new LinkedHashMap<>();
|
||||
}
|
||||
try {
|
||||
return new LinkedHashMap<>(objectMapper.readValue(json, MAP_TYPE));
|
||||
} catch (Exception ex) {
|
||||
Map<String, Object> fallback = new LinkedHashMap<>();
|
||||
fallback.put("rawPayload", json);
|
||||
fallback.put("payloadParseError", ex.getMessage());
|
||||
return fallback;
|
||||
}
|
||||
}
|
||||
|
||||
private void put(Map<String, Object> target, String key, Object value) {
|
||||
if (value != null) {
|
||||
target.put(key, value instanceof BigDecimal bd ? bd.stripTrailingZeros().toPlainString() : value);
|
||||
}
|
||||
}
|
||||
|
||||
private boolean isBlank(String value) {
|
||||
return value == null || value.isBlank();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.ImportMode;
|
||||
import at.procon.eventhub.dto.ImportScopeDto;
|
||||
import at.procon.eventhub.importing.AbstractConfiguredImportPlanService;
|
||||
import at.procon.eventhub.importing.ConfiguredImportPlanDto;
|
||||
import at.procon.eventhub.yellowfox.dto.ConfiguredYellowFoxD8ImportPlanDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
public class YellowFoxD8ConfiguredImportPlanService
|
||||
extends AbstractConfiguredImportPlanService<YellowFoxD8ImportRequest, ConfiguredYellowFoxD8ImportPlanDto> {
|
||||
|
||||
public YellowFoxD8ConfiguredImportPlanService(EventHubProperties properties) {
|
||||
super(() -> properties.getYellowFox().getImportPlans(), "yellowfox-d8");
|
||||
}
|
||||
|
||||
@Override
|
||||
protected YellowFoxD8ImportRequest buildRequest(
|
||||
EventHubProperties.ConfiguredImportPlan plan,
|
||||
ImportMode mode,
|
||||
AcquisitionStrategy strategy,
|
||||
ImportScopeDto scope
|
||||
) {
|
||||
return new YellowFoxD8ImportRequest(
|
||||
plan.getTenantKey(),
|
||||
plan.getEventSource(),
|
||||
plan.getSourceGroup(),
|
||||
scope,
|
||||
plan.getEventFamilies(),
|
||||
mode,
|
||||
false,
|
||||
strategy
|
||||
);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected ConfiguredYellowFoxD8ImportPlanDto toDto(EventHubProperties.ConfiguredImportPlan plan) {
|
||||
ConfiguredImportPlanDto dto = genericDto(plan);
|
||||
return new ConfiguredYellowFoxD8ImportPlanDto(
|
||||
dto.planKey(),
|
||||
dto.enabled(),
|
||||
dto.cron(),
|
||||
dto.tenantKey(),
|
||||
dto.eventSource(),
|
||||
dto.sourceGroup(),
|
||||
dto.importScope(),
|
||||
dto.eventFamilies(),
|
||||
dto.initialMode(),
|
||||
dto.scheduledMode(),
|
||||
dto.initialStrategy(),
|
||||
dto.scheduledStrategy(),
|
||||
false,
|
||||
dto.initialOccurredFrom(),
|
||||
dto.initialOccurredTo(),
|
||||
dto.runInitialOnStartup()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.importing.ImportPlanItemDto;
|
||||
import at.procon.eventhub.importing.ImportTimeChunkDto;
|
||||
import at.procon.eventhub.importing.extraction.ExtractionBatchExecutor;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ExtractionBatchResultDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import java.util.UUID;
|
||||
|
||||
public interface YellowFoxD8ExtractionBatchExecutor
|
||||
extends ExtractionBatchExecutor<YellowFoxD8ImportRequest, YellowFoxD8ExtractionBatchResultDto> {
|
||||
|
||||
@Override
|
||||
YellowFoxD8ExtractionBatchResultDto execute(
|
||||
UUID importRunId,
|
||||
UUID packageId,
|
||||
int eventSourceId,
|
||||
YellowFoxD8ImportRequest request,
|
||||
ImportPlanItemDto planItem,
|
||||
ImportTimeChunkDto chunk
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.dto.EventHubEventDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
@Component
|
||||
public class YellowFoxD8IgnitionTransitionDetector {
|
||||
|
||||
private final YellowFoxD8BookingEventMapper mapper;
|
||||
|
||||
public YellowFoxD8IgnitionTransitionDetector(YellowFoxD8BookingEventMapper mapper) {
|
||||
this.mapper = mapper;
|
||||
}
|
||||
|
||||
public Session newSession(boolean emitInitialSnapshot) {
|
||||
return new Session(mapper, emitInitialSnapshot);
|
||||
}
|
||||
|
||||
public static class Session {
|
||||
private final YellowFoxD8BookingEventMapper mapper;
|
||||
private final boolean emitInitialSnapshot;
|
||||
private final Map<String, Integer> lastIgnitionByVehicle = new HashMap<>();
|
||||
|
||||
private Session(YellowFoxD8BookingEventMapper mapper, boolean emitInitialSnapshot) {
|
||||
this.mapper = mapper;
|
||||
this.emitInitialSnapshot = emitInitialSnapshot;
|
||||
}
|
||||
|
||||
public EventHubEventDto detect(YellowFoxD8BookingDto booking) {
|
||||
if (booking == null || booking.ignition() == null || booking.vehicleRef() == null || !booking.vehicleRef().hasAnyReference()) {
|
||||
return null;
|
||||
}
|
||||
String vehicleKey = booking.vehicleRef().stableKey();
|
||||
Integer previous = lastIgnitionByVehicle.put(vehicleKey, booking.ignition());
|
||||
if (previous == null) {
|
||||
return emitInitialSnapshot ? mapper.mapIgnitionTransition(booking, null) : null;
|
||||
}
|
||||
if (!previous.equals(booking.ignition())) {
|
||||
return mapper.mapIgnitionTransition(booking, previous);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.AcquisitionStrategy;
|
||||
import at.procon.eventhub.dto.EventFamily;
|
||||
import at.procon.eventhub.importing.ImportChunkPlanner;
|
||||
import at.procon.eventhub.importing.ImportPlanDto;
|
||||
import at.procon.eventhub.importing.ImportPlanItemDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import java.util.List;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
public class YellowFoxD8ImportPlanService {
|
||||
|
||||
public static final String SOURCE_KIND = "TELEMATICS_PLATFORM";
|
||||
public static final String EXTRACTION_CODE = "YELLOWFOX_D8_BOOKING";
|
||||
|
||||
private final EventHubProperties properties;
|
||||
private final ImportChunkPlanner chunkPlanner;
|
||||
|
||||
public YellowFoxD8ImportPlanService(EventHubProperties properties, ImportChunkPlanner chunkPlanner) {
|
||||
this.properties = properties;
|
||||
this.chunkPlanner = chunkPlanner;
|
||||
}
|
||||
|
||||
public ImportPlanDto createPlan(YellowFoxD8ImportRequest request) {
|
||||
return new ImportPlanDto(
|
||||
request.tenantKey(),
|
||||
request.mode(),
|
||||
request.acquisitionStrategy(),
|
||||
request.refreshMasterDataFirst(),
|
||||
request.importScope(),
|
||||
request.sourceGroup(),
|
||||
request.eventSource(),
|
||||
chunkPlanner.chunksFor(request, properties.getYellowFox().getDefaultChunkDays()),
|
||||
List.of(item(request.acquisitionStrategy()))
|
||||
);
|
||||
}
|
||||
|
||||
private ImportPlanItemDto item(AcquisitionStrategy strategy) {
|
||||
return new ImportPlanItemDto(
|
||||
EventFamily.DRIVER_ACTIVITY,
|
||||
SOURCE_KIND,
|
||||
EXTRACTION_CODE,
|
||||
List.of("data.d8_booking", "data.vehicle", "data.driver", "data.fleet", "data.telematic_provider"),
|
||||
"VEHICLE",
|
||||
"YellowFox D8 booking card/activity events with derived ignition transitions",
|
||||
strategy
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
package at.procon.eventhub.yellowfox.service;
|
||||
|
||||
import at.procon.eventhub.config.EventHubProperties;
|
||||
import at.procon.eventhub.dto.SchedulerTriggerMode;
|
||||
import at.procon.eventhub.importing.AbstractImportScheduler;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8ImportRequest;
|
||||
import java.util.List;
|
||||
import org.springframework.boot.context.event.ApplicationReadyEvent;
|
||||
import org.springframework.context.event.EventListener;
|
||||
import org.springframework.scheduling.annotation.Scheduled;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
@Service
|
||||
public class YellowFoxD8ImportScheduler extends AbstractImportScheduler<YellowFoxD8ImportRequest> {
|
||||
|
||||
private final EventHubProperties properties;
|
||||
private final YellowFoxD8ConfiguredImportPlanService configuredPlanService;
|
||||
private final YellowFoxD8ImportExecutionService executionService;
|
||||
|
||||
public YellowFoxD8ImportScheduler(
|
||||
EventHubProperties properties,
|
||||
YellowFoxD8ConfiguredImportPlanService configuredPlanService,
|
||||
YellowFoxD8ImportExecutionService executionService
|
||||
) {
|
||||
this.properties = properties;
|
||||
this.configuredPlanService = configuredPlanService;
|
||||
this.executionService = executionService;
|
||||
}
|
||||
|
||||
@EventListener(ApplicationReadyEvent.class)
|
||||
public void triggerInitialPlansOnStartup() {
|
||||
super.triggerInitialPlansOnStartup();
|
||||
}
|
||||
|
||||
@Scheduled(fixedDelayString = "${eventhub.yellow-fox.scheduler-poll-interval-ms:60000}")
|
||||
public void pollConfiguredPlans() {
|
||||
super.pollConfiguredPlans();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected boolean schedulerEnabled() {
|
||||
return properties.getYellowFox().isSchedulerEnabled();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected List<EventHubProperties.ConfiguredImportPlan> configuredPlans() {
|
||||
return properties.getYellowFox().getImportPlans();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected SchedulerTriggerMode schedulerTriggerMode() {
|
||||
return properties.getYellowFox().getSchedulerTriggerMode();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected YellowFoxD8ImportRequest createInitialRequest(EventHubProperties.ConfiguredImportPlan plan) {
|
||||
return configuredPlanService.createInitialRequest(plan);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected YellowFoxD8ImportRequest createScheduledRequest(EventHubProperties.ConfiguredImportPlan plan) {
|
||||
return configuredPlanService.createScheduledRequest(plan);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void startImport(YellowFoxD8ImportRequest request) {
|
||||
executionService.startImport(request);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void startAndExecuteImport(YellowFoxD8ImportRequest request) {
|
||||
executionService.startAndExecuteImport(request);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String providerName() {
|
||||
return "yellowfox-d8";
|
||||
}
|
||||
}
|
||||
|
|
@ -2,17 +2,17 @@ spring:
|
|||
application:
|
||||
name: eventhub-ingestion-service
|
||||
datasource:
|
||||
url: jdbc:postgresql://localhost:5432/eventhub
|
||||
username: postgres
|
||||
password: P54!pcd#Wi
|
||||
url: jdbc:postgresql://${DB_HOST:localhost}:${DB_PORT:5432}/${DB_NAME:eventhub}
|
||||
username: ${DB_USER:}
|
||||
password: ${DB_PASSWORD:}
|
||||
hikari:
|
||||
maximum-pool-size: 16
|
||||
minimum-idle: 4
|
||||
connection-timeout: 30000
|
||||
validation-timeout: 5000
|
||||
idle-timeout: 60000
|
||||
keepalive-time: 30000
|
||||
max-lifetime: 90000
|
||||
idle-timeout: 300000
|
||||
keepalive-time: 120000
|
||||
max-lifetime: 540000
|
||||
flyway:
|
||||
enabled: true
|
||||
default-schema: eventhub
|
||||
|
|
@ -54,27 +54,33 @@ eventhub:
|
|||
default-chunk-days: 1
|
||||
occurred-at-overlap: 7d
|
||||
|
||||
# Configure this block to enable JdbcTachographExtractionBatchExecutor.
|
||||
# Set TACHOGRAPH_DB_JDBC_URL to enable JdbcTachographExtractionBatchExecutor.
|
||||
datasource:
|
||||
jdbc-url: jdbc:sqlserver://db.bytebar.eu:22996;databaseName=ByteBarDriverSettlement;trustServerCertificate=true
|
||||
username: ReadOnly
|
||||
password: p2=race!
|
||||
jdbc-url: ${TACHOGRAPH_DB_JDBC_URL:}
|
||||
username: ${TACHOGRAPH_DB_USER:}
|
||||
password: ${TACHOGRAPH_DB_PASSWORD:}
|
||||
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
|
||||
|
||||
# Enables the scheduler that regularly triggers configured tachograph import plans.
|
||||
# Default is safe: no scheduled import starts unless explicitly enabled.
|
||||
scheduler-enabled: true
|
||||
scheduler-poll-interval-ms: 60000
|
||||
scheduler-poll-interval-ms: 3600000
|
||||
|
||||
# PLAN_ONLY creates import_run + planned extraction packages.
|
||||
# EXECUTE also invokes the configured TachographExtractionBatchExecutor.
|
||||
scheduler-trigger-mode: EXECUTE
|
||||
|
||||
# JDBC extraction handoff mode:
|
||||
# SYNC_DIRECT = persist controlled JDBC batches directly, cursor-safe default.
|
||||
# CAMEL_ROUTE = persist the same controlled batches through direct:eventhub-batch-persist-input.
|
||||
jdbc-extraction-ingest-mode: ${TACHOGRAPH_JDBC_EXTRACTION_INGEST_MODE:SYNC_DIRECT}
|
||||
|
||||
# Example plan. Keep disabled until the tachograph datasource/extractor is wired.
|
||||
import-plans:
|
||||
- plan-key: kralowetz-tachograph-org-147
|
||||
- plan-key: tachograph-org-14708
|
||||
enabled: true
|
||||
cron: "0 15 * * * *" # hourly at minute 15
|
||||
tenant-key: kralowetz
|
||||
tenant-key: Procon
|
||||
event-source:
|
||||
provider-key: TACHOGRAPH
|
||||
source-kind: MIXED
|
||||
|
|
@ -83,16 +89,16 @@ eventhub:
|
|||
tenant-provider-setting-key: ByteBar-DriverSettlement
|
||||
source-group:
|
||||
type: ORGANISATION
|
||||
source-entity-id: "147"
|
||||
code: "147"
|
||||
name: Kralowetz root organisation
|
||||
source-entity-id: "14708"
|
||||
code: "14708"
|
||||
name: Zeller root organisation
|
||||
import-scope:
|
||||
type: SOURCE_ORGANISATION_SUBTREE
|
||||
root-source-organisation:
|
||||
type: ORGANISATION
|
||||
source-entity-id: "147"
|
||||
code: "147"
|
||||
name: Kralowetz root organisation
|
||||
source-entity-id: "14708"
|
||||
code: "14708"
|
||||
name: Zeller root organisation
|
||||
include-children: true
|
||||
occurred-from: null
|
||||
occurred-to: null
|
||||
|
|
@ -109,7 +115,56 @@ eventhub:
|
|||
scheduled-mode: INCREMENTAL_UPDATE
|
||||
initial-strategy: OCCURRED_AT_WINDOW_WITH_OVERLAP
|
||||
scheduled-strategy: SOURCE_PACKAGE_WATERMARK
|
||||
refresh-master-data-first: false
|
||||
refresh-master-data-first: true
|
||||
initial-occurred-from: "2026-01-21T00:00:00+01:00"
|
||||
initial-occurred-to: "2026-01-25T00:00:00+01:00"
|
||||
initial-occurred-to:
|
||||
run-initial-on-startup: true
|
||||
|
||||
esper-poc:
|
||||
activity-merge-mode: JAVA
|
||||
shift-resolution-mode: JAVA
|
||||
operating-period-evaluation:
|
||||
operating-split-idle-hours: 7
|
||||
significant-driving-minutes: 3
|
||||
merge-gap-seconds: 0
|
||||
gap-detection-tolerance-seconds: 0
|
||||
unknown-treatment-mode: AS_BREAK_REST
|
||||
|
||||
yellow-fox:
|
||||
default-chunk-days: 1
|
||||
occurred-at-overlap: 2h
|
||||
emit-initial-ignition-snapshot: false
|
||||
|
||||
datasource:
|
||||
jdbc-url: ${YELLOWFOX_DB_JDBC_URL:}
|
||||
username: ${YELLOWFOX_DB_USERNAME:}
|
||||
password: ${YELLOWFOX_DB_PASSWORD:}
|
||||
driver-class-name: org.postgresql.Driver
|
||||
|
||||
scheduler-enabled: false
|
||||
scheduler-poll-interval-ms: 60000
|
||||
scheduler-trigger-mode: PLAN_ONLY
|
||||
|
||||
import-plans:
|
||||
- plan-key: yellowfox-d8-default
|
||||
enabled: false
|
||||
cron: "0 */5 * * * *"
|
||||
tenant-key: default
|
||||
event-source:
|
||||
provider-key: YELLOWFOX
|
||||
source-kind: TELEMATICS_PLATFORM
|
||||
source-key: YELLOWFOX_D8
|
||||
source-instance-key: logistics-db-prod
|
||||
tenant-provider-setting-key: yellowfox-main
|
||||
import-scope:
|
||||
type: TENANT_ALL
|
||||
include-children: false
|
||||
event-families:
|
||||
- DRIVER_ACTIVITY
|
||||
- DRIVER_CARD
|
||||
initial-mode: INITIAL_BACKFILL
|
||||
scheduled-mode: INCREMENTAL_UPDATE
|
||||
initial-strategy: OCCURRED_AT_WINDOW_WITH_OVERLAP
|
||||
scheduled-strategy: SOURCE_ROW_WATERMARK
|
||||
refresh-master-data-first: false
|
||||
run-initial-on-startup: false
|
||||
|
|
|
|||
|
|
@ -0,0 +1,15 @@
|
|||
create index if not exists idx_event_detail_yellowfox_slot
|
||||
on eventhub.event_detail(detail_type, (attributes ->> 'slot'), event_occurred_at)
|
||||
where detail_type in ('DRIVER_ACTIVITY', 'DRIVER_CARD');
|
||||
|
||||
create index if not exists idx_event_detail_yellowfox_eventtype_state
|
||||
on eventhub.event_detail(
|
||||
(attributes ->> 'yellowFoxEventType'),
|
||||
(attributes ->> 'yellowFoxState'),
|
||||
event_occurred_at
|
||||
)
|
||||
where attributes ? 'yellowFoxEventType';
|
||||
|
||||
create index if not exists idx_event_detail_yellowfox_ignition
|
||||
on eventhub.event_detail(detail_type, (attributes ->> 'ignitionState'), event_occurred_at)
|
||||
where attributes ? 'ignitionState';
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
create table if not exists eventhub.driver (
|
||||
id uuid primary key,
|
||||
tenant_key text not null,
|
||||
event_source_id integer not null references eventhub.event_source(id),
|
||||
source_driver_entity_id text,
|
||||
card_nation text,
|
||||
card_number text,
|
||||
first_names text,
|
||||
last_name text,
|
||||
birth_date date,
|
||||
source_updated_at timestamptz,
|
||||
payload jsonb not null default '{}'::jsonb,
|
||||
created_at timestamptz not null default now(),
|
||||
updated_at timestamptz not null default now(),
|
||||
constraint chk_driver_card_nation_when_number
|
||||
check (card_number is null or card_nation is not null)
|
||||
);
|
||||
|
||||
insert into eventhub.driver(
|
||||
id, tenant_key, event_source_id, source_driver_entity_id,
|
||||
card_nation, card_number, first_names, last_name, birth_date,
|
||||
source_updated_at, payload, created_at, updated_at
|
||||
)
|
||||
select gen_random_uuid(),
|
||||
sme.tenant_key,
|
||||
sme.event_source_id,
|
||||
case
|
||||
when sme.source_entity_id like 'DRIVER_CARD:%' then null
|
||||
else sme.source_entity_id
|
||||
end as source_driver_entity_id,
|
||||
coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_nation'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_nation'), ''),
|
||||
nullif(split_part(case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else null end, ':', 1), '')
|
||||
) as card_nation,
|
||||
coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_number'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_number'), ''),
|
||||
nullif(substring(case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else null end from position(':' in case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else '' end) + 1), '')
|
||||
) as card_number,
|
||||
coalesce(
|
||||
nullif(trim(sme.payload ->> 'first_names'), ''),
|
||||
nullif(trim(sme.payload ->> 'firstnames'), '')
|
||||
) as first_names,
|
||||
coalesce(
|
||||
nullif(trim(sme.payload ->> 'last_name'), ''),
|
||||
nullif(trim(sme.payload ->> 'surname'), '')
|
||||
) as last_name,
|
||||
cast(nullif(trim(sme.payload ->> 'birth_date'), '') as date) as birth_date,
|
||||
sme.source_updated_at,
|
||||
sme.payload,
|
||||
sme.created_at,
|
||||
sme.updated_at
|
||||
from eventhub.source_master_entity sme
|
||||
where sme.entity_type = 'DRIVER'
|
||||
and not exists (
|
||||
select 1
|
||||
from eventhub.driver existing
|
||||
where existing.tenant_key = sme.tenant_key
|
||||
and existing.event_source_id = sme.event_source_id
|
||||
and existing.source_driver_entity_id is not distinct from case
|
||||
when sme.source_entity_id like 'DRIVER_CARD:%' then null
|
||||
else sme.source_entity_id
|
||||
end
|
||||
and existing.card_nation is not distinct from coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_nation'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_nation'), ''),
|
||||
nullif(split_part(case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else null end, ':', 1), '')
|
||||
)
|
||||
and existing.card_number is not distinct from coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_number'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_number'), ''),
|
||||
nullif(substring(case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else null end from position(':' in case when sme.source_entity_id like 'DRIVER_CARD:%' then substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) else '' end) + 1), '')
|
||||
)
|
||||
);
|
||||
|
||||
alter table eventhub.event
|
||||
add column if not exists driver_id uuid references eventhub.driver(id);
|
||||
|
||||
with source_driver_map as (
|
||||
select distinct on (sme.id)
|
||||
sme.id as source_master_entity_id,
|
||||
driver.id as driver_id
|
||||
from eventhub.source_master_entity sme
|
||||
join eventhub.driver driver
|
||||
on driver.tenant_key = sme.tenant_key
|
||||
and driver.event_source_id = sme.event_source_id
|
||||
and (
|
||||
(
|
||||
sme.source_entity_id not like 'DRIVER_CARD:%'
|
||||
and driver.source_driver_entity_id = sme.source_entity_id
|
||||
) or (
|
||||
sme.source_entity_id like 'DRIVER_CARD:%'
|
||||
and driver.card_nation is not distinct from coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_nation'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_nation'), ''),
|
||||
nullif(split_part(substring(sme.source_entity_id from length('DRIVER_CARD:') + 1), ':', 1), '')
|
||||
)
|
||||
and driver.card_number is not distinct from coalesce(
|
||||
nullif(trim(sme.payload ->> 'card_number'), ''),
|
||||
nullif(trim(sme.payload ->> 'driver_card_number'), ''),
|
||||
nullif(substring(substring(sme.source_entity_id from length('DRIVER_CARD:') + 1) from position(':' in substring(sme.source_entity_id from length('DRIVER_CARD:') + 1)) + 1), '')
|
||||
)
|
||||
)
|
||||
)
|
||||
where sme.entity_type = 'DRIVER'
|
||||
order by sme.id, driver.updated_at desc
|
||||
)
|
||||
update eventhub.event event
|
||||
set driver_id = map.driver_id
|
||||
from source_driver_map map
|
||||
where event.driver_entity_id = map.source_master_entity_id
|
||||
and event.driver_id is null;
|
||||
|
||||
create index if not exists idx_driver_source_entity
|
||||
on eventhub.driver(tenant_key, event_source_id, source_driver_entity_id)
|
||||
where source_driver_entity_id is not null;
|
||||
|
||||
create index if not exists idx_driver_card
|
||||
on eventhub.driver(tenant_key, event_source_id, card_nation, card_number)
|
||||
where card_number is not null;
|
||||
|
||||
create index if not exists idx_event_driver_time
|
||||
on eventhub.event(driver_id, occurred_at desc)
|
||||
where driver_id is not null;
|
||||
|
||||
alter table eventhub.event
|
||||
drop constraint if exists chk_event_driver_or_vehicle_ref;
|
||||
|
||||
alter table eventhub.event
|
||||
add constraint chk_event_driver_or_vehicle_ref
|
||||
check (
|
||||
driver_id is not null
|
||||
or driver_entity_id is not null
|
||||
or vehicle_id is not null
|
||||
or vehicle_registration_id is not null
|
||||
);
|
||||
|
|
@ -0,0 +1,239 @@
|
|||
/*
|
||||
* Repairs and normalizes tachograph driver aggregates after introducing eventhub.driver.
|
||||
*
|
||||
* What it does:
|
||||
* 1. Ensures tachograph DRIVER master-data payload carries last_name while keeping source_master_entity.display_name unchanged.
|
||||
* 2. Upserts eventhub.driver rows from MASTER_DATA DRIVER entities.
|
||||
* 3. Projects card nation/number onto eventhub.driver from DRIVER_CARD_DRIVER relations.
|
||||
* 4. Remaps event.driver_id from provisional card-only drivers to proper source-driver aggregates when possible.
|
||||
* 5. Deletes now-unreferenced provisional tachograph driver rows with no source_driver_entity_id.
|
||||
*
|
||||
* Assumptions:
|
||||
* - Tachograph master-data source is provider_key=TACHOGRAPH, source_kind=MASTER_DATA, source_key=TACHOGRAPH_MASTER_DATA.
|
||||
* - eventhub.driver and event.driver_id already exist.
|
||||
*/
|
||||
|
||||
-- 1) Keep display_name, but ensure DRIVER payload has last_name.
|
||||
with master_sources as (
|
||||
select es.id, es.tenant_key
|
||||
from eventhub.event_source es
|
||||
where es.provider_key = 'TACHOGRAPH'
|
||||
and es.source_kind = 'MASTER_DATA'
|
||||
and es.source_key = 'TACHOGRAPH_MASTER_DATA'
|
||||
),
|
||||
updated_master_payload as (
|
||||
update eventhub.source_master_entity sme
|
||||
set payload = jsonb_strip_nulls(
|
||||
sme.payload
|
||||
|| jsonb_build_object(
|
||||
'first_names', coalesce(sme.payload ->> 'first_names', sme.payload ->> 'firstnames'),
|
||||
'last_name', coalesce(sme.payload ->> 'last_name', sme.payload ->> 'surname')
|
||||
)
|
||||
),
|
||||
updated_at = now()
|
||||
from master_sources ms
|
||||
where sme.tenant_key = ms.tenant_key
|
||||
and sme.event_source_id = ms.id
|
||||
and sme.entity_type = 'DRIVER'
|
||||
returning sme.id
|
||||
)
|
||||
select count(*) as updated_master_payload
|
||||
from updated_master_payload;
|
||||
|
||||
-- 2) Upsert driver aggregates from tachograph master data.
|
||||
with master_sources as (
|
||||
select es.id,
|
||||
es.tenant_key,
|
||||
es.source_instance_key,
|
||||
coalesce(es.tenant_provider_setting_key, '') as tenant_provider_setting_key
|
||||
from eventhub.event_source es
|
||||
where es.provider_key = 'TACHOGRAPH'
|
||||
and es.source_kind = 'MASTER_DATA'
|
||||
and es.source_key = 'TACHOGRAPH_MASTER_DATA'
|
||||
),
|
||||
master_drivers as (
|
||||
select ms.id as master_event_source_id,
|
||||
ms.tenant_key,
|
||||
ms.source_instance_key,
|
||||
ms.tenant_provider_setting_key,
|
||||
d.source_entity_id as source_driver_entity_id,
|
||||
coalesce(nullif(trim(d.payload ->> 'first_names'), ''), nullif(trim(d.payload ->> 'firstnames'), '')) as first_names,
|
||||
coalesce(nullif(trim(d.payload ->> 'last_name'), ''), nullif(trim(d.payload ->> 'surname'), '')) as last_name,
|
||||
cast(nullif(trim(d.payload ->> 'birth_date'), '') as date) as birth_date,
|
||||
d.source_updated_at,
|
||||
d.payload
|
||||
from master_sources ms
|
||||
join eventhub.source_master_entity d
|
||||
on d.tenant_key = ms.tenant_key
|
||||
and d.event_source_id = ms.id
|
||||
and d.entity_type = 'DRIVER'
|
||||
and d.source_entity_id not like 'DRIVER_CARD:%'
|
||||
),
|
||||
compatible_targets as (
|
||||
select md.*,
|
||||
es.id as target_event_source_id
|
||||
from master_drivers md
|
||||
join eventhub.event_source es
|
||||
on es.tenant_key = md.tenant_key
|
||||
and es.provider_key = 'TACHOGRAPH'
|
||||
and es.source_instance_key = md.source_instance_key
|
||||
and coalesce(es.tenant_provider_setting_key, '') = md.tenant_provider_setting_key
|
||||
),
|
||||
updated_drivers as (
|
||||
update eventhub.driver driver
|
||||
set first_names = coalesce(ct.first_names, driver.first_names),
|
||||
last_name = coalesce(ct.last_name, driver.last_name),
|
||||
birth_date = coalesce(ct.birth_date, driver.birth_date),
|
||||
source_updated_at = ct.source_updated_at,
|
||||
payload = driver.payload || ct.payload,
|
||||
updated_at = now()
|
||||
from compatible_targets ct
|
||||
where driver.tenant_key = ct.tenant_key
|
||||
and driver.event_source_id = ct.target_event_source_id
|
||||
and driver.source_driver_entity_id = ct.source_driver_entity_id
|
||||
returning driver.id
|
||||
),
|
||||
inserted_drivers as (
|
||||
insert into eventhub.driver(
|
||||
id, tenant_key, event_source_id, source_driver_entity_id,
|
||||
first_names, last_name, birth_date, source_updated_at, payload, updated_at
|
||||
)
|
||||
select gen_random_uuid(),
|
||||
ct.tenant_key,
|
||||
ct.target_event_source_id,
|
||||
ct.source_driver_entity_id,
|
||||
ct.first_names,
|
||||
ct.last_name,
|
||||
ct.birth_date,
|
||||
ct.source_updated_at,
|
||||
ct.payload,
|
||||
now()
|
||||
from compatible_targets ct
|
||||
where not exists (
|
||||
select 1
|
||||
from eventhub.driver existing
|
||||
where existing.tenant_key = ct.tenant_key
|
||||
and existing.event_source_id = ct.target_event_source_id
|
||||
and existing.source_driver_entity_id = ct.source_driver_entity_id
|
||||
)
|
||||
returning id
|
||||
)
|
||||
select (select count(*) from updated_drivers) as updated_drivers,
|
||||
(select count(*) from inserted_drivers) as inserted_drivers;
|
||||
|
||||
-- 3) Project driver-card identifiers from master-data relations.
|
||||
with master_sources as (
|
||||
select es.id,
|
||||
es.tenant_key,
|
||||
es.source_instance_key,
|
||||
coalesce(es.tenant_provider_setting_key, '') as tenant_provider_setting_key
|
||||
from eventhub.event_source es
|
||||
where es.provider_key = 'TACHOGRAPH'
|
||||
and es.source_kind = 'MASTER_DATA'
|
||||
and es.source_key = 'TACHOGRAPH_MASTER_DATA'
|
||||
),
|
||||
card_projection as (
|
||||
select distinct on (ms.tenant_key, ms.source_instance_key, ms.tenant_provider_setting_key, rel.to_source_entity_id)
|
||||
ms.tenant_key,
|
||||
ms.source_instance_key,
|
||||
ms.tenant_provider_setting_key,
|
||||
rel.to_source_entity_id as source_driver_entity_id,
|
||||
nullif(trim(card.payload ->> 'card_nation'), '') as card_nation,
|
||||
nullif(trim(card.payload ->> 'card_number'), '') as card_number,
|
||||
rel.source_updated_at
|
||||
from master_sources ms
|
||||
join eventhub.source_master_relation rel
|
||||
on rel.tenant_key = ms.tenant_key
|
||||
and rel.event_source_id = ms.id
|
||||
and rel.relation_type = 'DRIVER_CARD_DRIVER'
|
||||
and rel.from_entity_type = 'DRIVER_CARD'
|
||||
and rel.to_entity_type = 'DRIVER'
|
||||
join eventhub.source_master_entity card
|
||||
on card.tenant_key = ms.tenant_key
|
||||
and card.event_source_id = ms.id
|
||||
and card.entity_type = 'DRIVER_CARD'
|
||||
and card.source_entity_id = rel.from_source_entity_id
|
||||
order by ms.tenant_key,
|
||||
ms.source_instance_key,
|
||||
ms.tenant_provider_setting_key,
|
||||
rel.to_source_entity_id,
|
||||
rel.valid_to desc nulls last,
|
||||
rel.valid_from desc nulls last,
|
||||
rel.updated_at desc
|
||||
),
|
||||
updated_driver_cards as (
|
||||
update eventhub.driver driver
|
||||
set card_nation = coalesce(driver.card_nation, projection.card_nation),
|
||||
card_number = coalesce(driver.card_number, projection.card_number),
|
||||
source_updated_at = coalesce(projection.source_updated_at, driver.source_updated_at),
|
||||
updated_at = now()
|
||||
from card_projection projection
|
||||
join eventhub.event_source es
|
||||
on es.id = driver.event_source_id
|
||||
where driver.tenant_key = projection.tenant_key
|
||||
and es.provider_key = 'TACHOGRAPH'
|
||||
and es.source_instance_key = projection.source_instance_key
|
||||
and coalesce(es.tenant_provider_setting_key, '') = projection.tenant_provider_setting_key
|
||||
and driver.source_driver_entity_id = projection.source_driver_entity_id
|
||||
and (
|
||||
(driver.card_nation is null and projection.card_nation is not null)
|
||||
or (driver.card_number is null and projection.card_number is not null)
|
||||
)
|
||||
returning driver.id
|
||||
)
|
||||
select count(*) as updated_driver_cards
|
||||
from updated_driver_cards;
|
||||
|
||||
-- 4) Remap events from provisional card-only drivers to proper source-driver aggregates.
|
||||
with provisional_to_real as (
|
||||
select provisional.id as provisional_driver_id,
|
||||
real.id as real_driver_id
|
||||
from eventhub.driver provisional
|
||||
join eventhub.event_source provisional_source
|
||||
on provisional_source.id = provisional.event_source_id
|
||||
and provisional_source.provider_key = 'TACHOGRAPH'
|
||||
join eventhub.driver real
|
||||
on real.tenant_key = provisional.tenant_key
|
||||
and real.source_driver_entity_id is not null
|
||||
and real.card_nation = provisional.card_nation
|
||||
and real.card_number = provisional.card_number
|
||||
join eventhub.event_source real_source
|
||||
on real_source.id = real.event_source_id
|
||||
and real_source.provider_key = provisional_source.provider_key
|
||||
and real_source.tenant_key = provisional_source.tenant_key
|
||||
and real_source.source_instance_key = provisional_source.source_instance_key
|
||||
and coalesce(real_source.tenant_provider_setting_key, '') = coalesce(provisional_source.tenant_provider_setting_key, '')
|
||||
where provisional.source_driver_entity_id is null
|
||||
and provisional.card_nation is not null
|
||||
and provisional.card_number is not null
|
||||
and provisional.id <> real.id
|
||||
),
|
||||
updated_events as (
|
||||
update eventhub.event e
|
||||
set driver_id = map.real_driver_id
|
||||
from provisional_to_real map
|
||||
where e.driver_id = map.provisional_driver_id
|
||||
and e.driver_id <> map.real_driver_id
|
||||
returning e.id
|
||||
)
|
||||
select count(*) as remapped_events
|
||||
from updated_events;
|
||||
|
||||
-- 5) Delete now-unreferenced provisional tachograph driver rows.
|
||||
with deleted_drivers as (
|
||||
delete from eventhub.driver driver
|
||||
using eventhub.event_source es
|
||||
where es.id = driver.event_source_id
|
||||
and es.provider_key = 'TACHOGRAPH'
|
||||
and driver.source_driver_entity_id is null
|
||||
and driver.card_nation is not null
|
||||
and driver.card_number is not null
|
||||
and not exists (
|
||||
select 1
|
||||
from eventhub.event e
|
||||
where e.driver_id = driver.id
|
||||
)
|
||||
returning driver.id
|
||||
)
|
||||
select count(*) as deleted_provisional_drivers
|
||||
from deleted_drivers;
|
||||
|
|
@ -98,6 +98,8 @@ Base as (
|
|||
left join dbo.Vehicle v on v.ID = cvu.ID_Vehicle
|
||||
left join dbo.Nation vn on vn.ID = v.ID_Nation
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_ACTIVITY:', base.ID, ':', evt.lifecycle) as external_source_event_id,
|
||||
|
|
@ -134,6 +136,7 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -148,3 +151,20 @@ where (:occurredFrom is null or evt.occurred_at >= :occurredFrom)
|
|||
/*
|
||||
* Organisation filter: driver membership in GetOrganisationTree(null, :organisationId, 0, null).
|
||||
*/
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -65,6 +65,8 @@ Base as (
|
|||
)
|
||||
)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_BORDER_CROSSING:', base.ID) as external_source_event_id,
|
||||
|
|
@ -89,8 +91,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -62,6 +62,8 @@ Base as (
|
|||
)
|
||||
)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_LOAD_UNLOAD:', base.ID) as external_source_event_id,
|
||||
|
|
@ -94,8 +96,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -66,6 +66,8 @@ Base as (
|
|||
)
|
||||
)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_PLACE:', base.ID) as external_source_event_id,
|
||||
|
|
@ -93,8 +95,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -61,6 +61,8 @@ Base as (
|
|||
)
|
||||
)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_POSITION:', base.ID) as external_source_event_id,
|
||||
|
|
@ -83,8 +85,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -59,6 +59,8 @@ Base as (
|
|||
)
|
||||
)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_SPECIFIC_CONDITION:', base.ID) as external_source_event_id,
|
||||
|
|
@ -81,8 +83,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -80,6 +80,8 @@ Base as (
|
|||
and (:occurredFrom is null or evt.occurred_at >= :occurredFrom)
|
||||
and (:occurredTo is null or evt.occurred_at < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:CARD_VEHICLES_USED:', base.ID, ':', base.lifecycle) as external_source_event_id,
|
||||
|
|
@ -107,8 +109,26 @@ select
|
|||
|
||||
'DRIVER_CARD' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
base.source_package_imported_at
|
||||
from Base base
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -71,6 +71,8 @@ Base as (
|
|||
and (:occurredFrom is null or evt.occurred_at >= :occurredFrom)
|
||||
and (:occurredTo is null or evt.occurred_at < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:IW_CYCLE:', base.ID, ':', base.lifecycle) as external_source_event_id,
|
||||
|
|
@ -98,6 +100,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -126,3 +129,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ select
|
|||
d.LastUpdate as source_updated_at,
|
||||
d.ID as driver_id,
|
||||
d.Surname as surname,
|
||||
d.Surname as last_name,
|
||||
d.Firstnames as first_names,
|
||||
d.Birthdate as birth_date,
|
||||
d.BirthPlace as birth_place,
|
||||
|
|
|
|||
|
|
@ -64,6 +64,8 @@ Events as (
|
|||
'END' as lifecycle
|
||||
from Base base
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
concat(cast(events.ID as varchar(128)), ':', events.lifecycle) as source_row_id,
|
||||
concat('TACHOGRAPH:SPEEDING_EVENTS:', events.ID, ':', events.lifecycle) as external_source_event_id,
|
||||
|
|
@ -86,6 +88,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(events.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
events.source_package_id_raw as source_package_sort_id,
|
||||
cast(events.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
events.source_package_period_from,
|
||||
events.source_package_period_to,
|
||||
|
|
@ -116,3 +119,20 @@ where (:occurredFrom is null or events.occurred_at >= :occurredFrom)
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -124,6 +124,8 @@ Base as (
|
|||
left join dbo.Card c on c.ID = iw.ID_Card
|
||||
left join dbo.Nation cn on cn.ID = c.ID_Nation
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_ACTIVITY:', base.ID, ':', evt.lifecycle) as external_source_event_id,
|
||||
|
|
@ -160,6 +162,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -174,3 +177,20 @@ where (:occurredFrom is null or evt.occurred_at >= :occurredFrom)
|
|||
/*
|
||||
* Organisation filter: vehicle membership in GetOrganisationTree(null, :organisationId, 0, null).
|
||||
*/
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -46,6 +46,8 @@ Base as (
|
|||
where (:occurredFrom is null or border.Timestamp >= :occurredFrom)
|
||||
and (:occurredTo is null or border.Timestamp < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_BORDER_CROSSING:', base.ID) as external_source_event_id,
|
||||
|
|
@ -70,6 +72,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -98,3 +101,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -43,6 +43,8 @@ Base as (
|
|||
where (:occurredFrom is null or lu.Timestamp >= :occurredFrom)
|
||||
and (:occurredTo is null or lu.Timestamp < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_LOAD_UNLOAD:', base.ID) as external_source_event_id,
|
||||
|
|
@ -75,6 +77,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -103,3 +106,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -44,6 +44,8 @@ Base as (
|
|||
where (:occurredFrom is null or place.EntryTime >= :occurredFrom)
|
||||
and (:occurredTo is null or place.EntryTime < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_PLACE:', base.ID) as external_source_event_id,
|
||||
|
|
@ -71,6 +73,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -99,3 +102,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -42,6 +42,8 @@ Base as (
|
|||
where (:occurredFrom is null or pos.Timestamp >= :occurredFrom)
|
||||
and (:occurredTo is null or pos.Timestamp < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_POSITION:', base.ID) as external_source_event_id,
|
||||
|
|
@ -64,6 +66,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -92,3 +95,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -32,6 +32,8 @@ Base as (
|
|||
where (:occurredFrom is null or cond.EntryTime >= :occurredFrom)
|
||||
and (:occurredTo is null or cond.EntryTime < :occurredTo)
|
||||
)
|
||||
,
|
||||
Extracted as (
|
||||
select
|
||||
cast(base.ID as varchar(128)) as source_row_id,
|
||||
concat('TACHOGRAPH:VU_SPECIFIC_CONDITION:', base.ID) as external_source_event_id,
|
||||
|
|
@ -54,6 +56,7 @@ select
|
|||
|
||||
'VEHICLE_UNIT' as source_package_kind,
|
||||
cast(base.source_package_id_raw as varchar(128)) as source_package_id,
|
||||
base.source_package_id_raw as source_package_sort_id,
|
||||
cast(base.source_package_entity_id_raw as varchar(128)) as source_package_entity_id,
|
||||
base.source_package_period_from,
|
||||
base.source_package_period_to,
|
||||
|
|
@ -82,3 +85,20 @@ where (
|
|||
and rel.GILT_BIS is null
|
||||
)
|
||||
)
|
||||
)
|
||||
select *
|
||||
from Extracted extracted
|
||||
where (
|
||||
:sourcePackageWatermarkEnabled = 0
|
||||
or :lastSourcePackageImportedAt is null
|
||||
or extracted.source_package_imported_at is null
|
||||
or extracted.source_package_imported_at > :lastSourcePackageImportedAt
|
||||
or (
|
||||
extracted.source_package_imported_at = :lastSourcePackageImportedAt
|
||||
and (
|
||||
:lastSourcePackageIdNumeric is null
|
||||
or extracted.source_package_sort_id is null
|
||||
or extracted.source_package_sort_id > :lastSourcePackageIdNumeric
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
# YellowFox D8 SQL
|
||||
|
||||
Recommended source-side index for incremental import:
|
||||
|
||||
```sql
|
||||
create index if not exists d8_booking_utc_eventid_idx
|
||||
on data.d8_booking (utc asc, eventid asc);
|
||||
```
|
||||
|
||||
The import uses SOURCE_ROW_WATERMARK with `(utc, eventid)`.
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
select
|
||||
b.eventid,
|
||||
b.utc,
|
||||
b.vehicle_id,
|
||||
b.driver_id,
|
||||
b.key,
|
||||
b.ignition,
|
||||
b.eventtype,
|
||||
b.state,
|
||||
b.odometer,
|
||||
st_y(b.geom) as latitude,
|
||||
st_x(b.geom) as longitude,
|
||||
b.payload,
|
||||
|
||||
v.vrn as vehicle_vrn,
|
||||
v.vin as vehicle_vin,
|
||||
v.fleet_id as vehicle_fleet_id,
|
||||
|
||||
d.firstname as driver_firstname,
|
||||
d.name as driver_lastname,
|
||||
d.drivers_card as driver_card_number,
|
||||
d.fleet_id as driver_fleet_id,
|
||||
|
||||
f.id as fleet_id,
|
||||
f.name as fleet_name,
|
||||
|
||||
tp.id as telematic_provider_id,
|
||||
tp.name as telematic_provider_name
|
||||
from data.d8_booking b
|
||||
left join data.vehicle v
|
||||
on v.id = b.vehicle_id
|
||||
left join data.driver d
|
||||
on d.id = b.driver_id
|
||||
left join data.fleet f
|
||||
on f.id = coalesce(v.fleet_id, d.fleet_id)
|
||||
left join data.telematic_provider tp
|
||||
on tp.id = v.telematic_provider_id
|
||||
where (:occurredFrom is null or b.utc >= :occurredFrom)
|
||||
and (:occurredTo is null or b.utc < :occurredTo)
|
||||
and (:fleetId is null or f.id = :fleetId)
|
||||
and (
|
||||
:lastOccurredTo is null
|
||||
or b.utc > :lastOccurredTo
|
||||
or (
|
||||
b.utc = :lastOccurredTo
|
||||
and (:lastSourceRowId is null or b.eventid > :lastSourceRowId)
|
||||
)
|
||||
)
|
||||
order by b.utc asc, b.eventid asc;
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
package at.procon.eventhub;
|
||||
|
||||
import at.procon.eventhub.dto.CardSlot;
|
||||
import at.procon.eventhub.dto.DriverCardRefDto;
|
||||
import at.procon.eventhub.dto.DriverRefDto;
|
||||
import at.procon.eventhub.dto.EventDomain;
|
||||
|
|
@ -7,8 +8,8 @@ import at.procon.eventhub.dto.EventLifecycle;
|
|||
import at.procon.eventhub.dto.EventType;
|
||||
import at.procon.eventhub.dto.VehicleRefDto;
|
||||
import at.procon.eventhub.dto.VehicleRegistrationRefDto;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import at.procon.eventhub.service.EventDetailsFactory;
|
||||
import at.procon.eventhub.yellowfox.dto.YellowFoxD8BookingDto;
|
||||
import at.procon.eventhub.yellowfox.service.YellowFoxD8BookingEventMapper;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import java.time.OffsetDateTime;
|
||||
|
|
@ -23,16 +24,67 @@ class YellowFoxD8BookingEventMapperTest {
|
|||
);
|
||||
|
||||
@Test
|
||||
void mapsYellowFoxDrivingStateAsSnapshotWithEventSourceAndDetails() {
|
||||
YellowFoxD8BookingDto source = new YellowFoxD8BookingDto(
|
||||
void mapsYellowFoxDriverSlotActivityAsStartWithSlotDetails() {
|
||||
var source = booking(2, 3, 1);
|
||||
|
||||
var result = mapper.map(source);
|
||||
|
||||
assertThat(result.eventDomain()).isEqualTo(EventDomain.DRIVER_ACTIVITY);
|
||||
assertThat(result.eventType()).isEqualTo(EventType.DRIVE);
|
||||
assertThat(result.lifecycle()).isEqualTo(EventLifecycle.START);
|
||||
assertThat(result.externalSourceEventId()).startsWith("D8_BOOKING:event-1:");
|
||||
assertThat(result.eventDetails().type()).isEqualTo("DRIVER_ACTIVITY");
|
||||
assertThat(result.eventDetails().attributes().get("slot").asText()).isEqualTo(CardSlot.DRIVER.name());
|
||||
assertThat(result.eventDetails().attributes().get("slotNumber").asInt()).isEqualTo(1);
|
||||
assertThat(result.eventDetails().attributes().get("yellowFoxStateMeaning").asText()).isEqualTo("STEERING_TIME");
|
||||
assertThat(result.eventDetails().attributes().get("ignitionState").asInt()).isEqualTo(1);
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsYellowFoxCoDriverSlotActivityAsStartWithSlotDetails() {
|
||||
var result = mapper.map(booking(3, 2, 0));
|
||||
|
||||
assertThat(result.eventDomain()).isEqualTo(EventDomain.DRIVER_ACTIVITY);
|
||||
assertThat(result.eventType()).isEqualTo(EventType.WORK);
|
||||
assertThat(result.lifecycle()).isEqualTo(EventLifecycle.START);
|
||||
assertThat(result.eventDetails().attributes().get("slot").asText()).isEqualTo(CardSlot.CO_DRIVER.name());
|
||||
assertThat(result.eventDetails().attributes().get("slotNumber").asInt()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsCardInsertedUsingSameTachographDomainTypeAndLifecycle() {
|
||||
var result = mapper.map(booking(0, 1, 1));
|
||||
|
||||
assertThat(result.eventDomain()).isEqualTo(EventDomain.DRIVER_CARD);
|
||||
assertThat(result.eventType()).isEqualTo(EventType.CARD_INSERTED);
|
||||
assertThat(result.lifecycle()).isEqualTo(EventLifecycle.INSERT);
|
||||
assertThat(result.eventDetails().attributes().get("slot").asText()).isEqualTo(CardSlot.DRIVER.name());
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsIgnitionTransitionOnlyWhenDetectorRequestsIt() {
|
||||
var result = mapper.mapIgnitionTransition(booking(2, 3, 1), 0);
|
||||
|
||||
assertThat(result.eventDomain()).isEqualTo(EventDomain.IGNITION);
|
||||
assertThat(result.eventType()).isEqualTo(EventType.IGNITION_ON);
|
||||
assertThat(result.lifecycle()).isEqualTo(EventLifecycle.ON);
|
||||
assertThat(result.externalSourceEventId()).contains("D8_IGNITION:event-1:");
|
||||
assertThat(result.eventDetails().attributes().get("previousIgnitionState").asInt()).isEqualTo(0);
|
||||
assertThat(result.eventDetails().attributes().get("ignitionMeaning").asText()).isEqualTo("ON");
|
||||
}
|
||||
|
||||
private YellowFoxD8BookingDto booking(int eventType, int state, int ignition) {
|
||||
return new YellowFoxD8BookingDto(
|
||||
"tenant-1",
|
||||
"yellowfox-tenant-7",
|
||||
"tenant-yellowfox-7",
|
||||
"7",
|
||||
"Fleet 7",
|
||||
"event-1",
|
||||
"key-1",
|
||||
2,
|
||||
3,
|
||||
ignition,
|
||||
eventType,
|
||||
state,
|
||||
OffsetDateTime.parse("2026-04-29T08:15:00+02:00"),
|
||||
null,
|
||||
new DriverRefDto("driver-source-100", new DriverCardRefDto("AT", "D123456789")),
|
||||
|
|
@ -42,20 +94,5 @@ class YellowFoxD8BookingEventMapperTest {
|
|||
null,
|
||||
null
|
||||
);
|
||||
|
||||
var result = mapper.map(source);
|
||||
|
||||
assertThat(result.eventDomain()).isEqualTo(EventDomain.DRIVER_ACTIVITY);
|
||||
assertThat(result.eventType()).isEqualTo(EventType.DRIVE);
|
||||
assertThat(result.lifecycle()).isEqualTo(EventLifecycle.SNAPSHOT);
|
||||
assertThat(result.externalSourceEventId()).isEqualTo("event-1");
|
||||
assertThat(result.packageInfo().tenantKey()).isEqualTo("tenant-1");
|
||||
assertThat(result.packageInfo().eventSource().providerKey()).isEqualTo("YELLOWFOX");
|
||||
assertThat(result.packageInfo().eventSource().sourceKey()).isEqualTo("YELLOWFOX_D8");
|
||||
assertThat(result.driverRef().driverCard().nation()).isEqualTo("AT");
|
||||
assertThat(result.vehicleRef().vehicleRegistration().nation()).isEqualTo("AT");
|
||||
assertThat(result.eventDetails().type()).isEqualTo("DRIVER_ACTIVITY");
|
||||
assertThat(result.payload().get("yellowFoxEventType").asInt()).isEqualTo(2);
|
||||
assertThat(result.payload().get("yellowFoxState").asInt()).isEqualTo(3);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,61 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.RawActivityEventDto;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class EsperDriverActivityEngineTest {
|
||||
|
||||
private final EsperDriverActivityEngine engine = new EsperDriverActivityEngine();
|
||||
|
||||
@Test
|
||||
void buildsActivityIntervalsFromStartAndEndEvents() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
UUID vehicleId = UUID.randomUUID();
|
||||
List<RawActivityEventDto> raw = List.of(
|
||||
raw("100", "START", "DRIVE", "2026-04-30T23:55:00Z", driverId, vehicleId),
|
||||
raw("100", "END", "DRIVE", "2026-05-01T00:10:00Z", driverId, vehicleId),
|
||||
raw("101", "START", "WORK", "2026-05-01T00:10:00Z", driverId, vehicleId),
|
||||
raw("101", "END", "WORK", "2026-05-01T00:40:00Z", driverId, vehicleId)
|
||||
);
|
||||
|
||||
var intervals = engine.buildIntervals(raw);
|
||||
|
||||
assertThat(intervals).hasSize(2);
|
||||
assertThat(intervals.get(0).activityType()).isEqualTo("DRIVE");
|
||||
assertThat(intervals.get(0).durationSeconds()).isEqualTo(15 * 60L);
|
||||
assertThat(intervals.get(1).activityType()).isEqualTo("WORK");
|
||||
assertThat(intervals.get(1).durationSeconds()).isEqualTo(30 * 60L);
|
||||
}
|
||||
|
||||
private RawActivityEventDto raw(
|
||||
String rowId,
|
||||
String lifecycle,
|
||||
String activity,
|
||||
String occurredAt,
|
||||
UUID driverId,
|
||||
UUID vehicleId
|
||||
) {
|
||||
return new RawActivityEventDto(
|
||||
UUID.randomUUID(),
|
||||
OffsetDateTime.parse(occurredAt),
|
||||
rowId,
|
||||
"TACHOGRAPH:CARD_ACTIVITY:" + rowId + ":" + lifecycle,
|
||||
"DRIVER_CARD",
|
||||
"CARD_ACTIVITY",
|
||||
driverId,
|
||||
vehicleId,
|
||||
null,
|
||||
activity,
|
||||
lifecycle,
|
||||
"DRIVER",
|
||||
"INSERTED",
|
||||
"SINGLE",
|
||||
"package-1"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,154 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperUnknownTreatmentMode;
|
||||
import at.procon.eventhub.esperpoc.dto.NonDrivingIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.OperatingPeriodActivityIntervalDto;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class EsperOperatingPeriodEvaluationServiceTest {
|
||||
|
||||
private final EsperOperatingPeriodEngine operatingPeriodEngine = new EsperOperatingPeriodEngine();
|
||||
private final EsperOperatingPeriodEvaluationService service = new EsperOperatingPeriodEvaluationService(
|
||||
null,
|
||||
new EsperDriverActivityEngine(),
|
||||
operatingPeriodEngine
|
||||
);
|
||||
|
||||
@Test
|
||||
void synthesizeUnknownGapsSkipsCoveredBreakRestAndCreatesOnlyUncoveredUnknown() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
List<ActivityIntervalDto> intervals = List.of(
|
||||
activity(driverId, "WORK", "2026-04-01T08:00:00Z", "2026-04-01T09:00:00Z", "w1", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T09:00:00Z", "2026-04-01T10:00:00Z", "r1", "DRIVER_CARD"),
|
||||
activity(driverId, "AVAILABILITY", "2026-04-01T10:00:00Z", "2026-04-01T11:00:00Z", "a1", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T12:00:00Z", "2026-04-01T13:00:00Z", "w2", "DRIVER_CARD")
|
||||
);
|
||||
|
||||
List<ActivityIntervalDto> evaluationIntervals = service.synthesizeUnknownGaps(intervals, Duration.ZERO);
|
||||
|
||||
assertThat(evaluationIntervals).extracting(ActivityIntervalDto::activityType)
|
||||
.containsExactly("WORK", "AVAILABILITY", "UNKNOWN", "WORK");
|
||||
assertThat(evaluationIntervals.get(2).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T11:00:00Z"));
|
||||
assertThat(evaluationIntervals.get(2).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T12:00:00Z"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void periodizationSplitsAfterLongUnknownGapAndShortDrivingDoesNotCloseNonDriving() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
List<ActivityIntervalDto> evaluationIntervals = List.of(
|
||||
activity(driverId, "WORK", "2026-04-01T08:00:00Z", "2026-04-01T09:00:00Z", "w1", "DRIVER_CARD"),
|
||||
activity(driverId, "AVAILABILITY", "2026-04-01T10:00:00Z", "2026-04-01T11:00:00Z", "a1", "DRIVER_CARD"),
|
||||
activity(driverId, "DRIVE", "2026-04-01T11:00:00Z", "2026-04-01T11:02:00Z", "d1", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T11:02:00Z", "2026-04-01T12:00:00Z", "w2", "DRIVER_CARD"),
|
||||
unknown(driverId, "2026-04-01T12:00:00Z", "2026-04-01T20:00:00Z"),
|
||||
activity(driverId, "DRIVE", "2026-04-01T20:00:00Z", "2026-04-01T20:30:00Z", "d2", "DRIVER_CARD")
|
||||
);
|
||||
|
||||
EsperOperatingPeriodEngine.EsperOperatingPeriodEvaluation evaluation = operatingPeriodEngine.evaluate(
|
||||
evaluationIntervals,
|
||||
Duration.ofHours(7)
|
||||
);
|
||||
|
||||
assertThat(evaluation.periodizedIntervals()).extracting(OperatingPeriodActivityIntervalDto::activityType)
|
||||
.containsExactly("WORK", "AVAILABILITY", "DRIVE", "WORK", "DRIVE");
|
||||
assertThat(evaluation.closedPeriods()).hasSize(2);
|
||||
assertThat(evaluation.closedPeriods().get(0).closedBy()).isEqualTo("UNKNOWN_GAP");
|
||||
assertThat(evaluation.closedPeriods().get(1).closedBy()).isEqualTo("FLUSH");
|
||||
|
||||
List<OperatingPeriodActivityIntervalDto> mergedIntervals = service.mergeConsecutiveActivities(
|
||||
evaluation.periodizedIntervals(),
|
||||
Duration.ZERO
|
||||
);
|
||||
List<NonDrivingIntervalDto> nonDrivingIntervals = service.buildNonDrivingIntervals(
|
||||
mergedIntervals,
|
||||
Duration.ofMinutes(3),
|
||||
EsperUnknownTreatmentMode.AS_BREAK_REST
|
||||
);
|
||||
|
||||
assertThat(nonDrivingIntervals).hasSize(1);
|
||||
assertThat(nonDrivingIntervals.get(0).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T08:00:00Z"));
|
||||
assertThat(nonDrivingIntervals.get(0).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T12:00:00Z"));
|
||||
assertThat(nonDrivingIntervals.get(0).closedBy()).isEqualTo("NEW_OPERATING_PERIOD");
|
||||
}
|
||||
|
||||
@Test
|
||||
void unknownCanCloseNonDrivingWhenConfiguredAsNonBreak() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
List<OperatingPeriodActivityIntervalDto> mergedIntervals = List.of(
|
||||
periodized(activity(driverId, "WORK", "2026-04-01T08:00:00Z", "2026-04-01T09:00:00Z", "w1", "DRIVER_CARD"), 1),
|
||||
periodized(unknown(driverId, "2026-04-01T09:00:00Z", "2026-04-01T09:30:00Z"), 1),
|
||||
periodized(activity(driverId, "AVAILABILITY", "2026-04-01T09:30:00Z", "2026-04-01T10:00:00Z", "a1", "DRIVER_CARD"), 1)
|
||||
);
|
||||
|
||||
List<NonDrivingIntervalDto> nonDrivingIntervals = service.buildNonDrivingIntervals(
|
||||
mergedIntervals,
|
||||
Duration.ofMinutes(3),
|
||||
EsperUnknownTreatmentMode.AS_UNKNOWN_NON_BREAK
|
||||
);
|
||||
|
||||
assertThat(nonDrivingIntervals).hasSize(2);
|
||||
assertThat(nonDrivingIntervals.get(0).closedBy()).isEqualTo("UNKNOWN_GAP");
|
||||
assertThat(nonDrivingIntervals.get(0).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T09:00:00Z"));
|
||||
assertThat(nonDrivingIntervals.get(1).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T09:30:00Z"));
|
||||
}
|
||||
|
||||
private ActivityIntervalDto activity(
|
||||
UUID driverId,
|
||||
String activity,
|
||||
String from,
|
||||
String to,
|
||||
String sourceRowId,
|
||||
String sourceKind
|
||||
) {
|
||||
return ActivityIntervalDto.raw(
|
||||
driverId,
|
||||
null,
|
||||
null,
|
||||
activity,
|
||||
"DRIVER",
|
||||
"INSERTED",
|
||||
"KNOWN",
|
||||
sourceKind,
|
||||
OffsetDateTime.parse(from),
|
||||
OffsetDateTime.parse(to),
|
||||
sourceRowId
|
||||
);
|
||||
}
|
||||
|
||||
private ActivityIntervalDto unknown(UUID driverId, String from, String to) {
|
||||
return new ActivityIntervalDto(
|
||||
driverId,
|
||||
null,
|
||||
null,
|
||||
"UNKNOWN",
|
||||
"DRIVER",
|
||||
null,
|
||||
"UNKNOWN",
|
||||
"SYNTHETIC_GAP",
|
||||
OffsetDateTime.parse(from),
|
||||
OffsetDateTime.parse(to),
|
||||
Duration.between(OffsetDateTime.parse(from), OffsetDateTime.parse(to)).getSeconds(),
|
||||
null,
|
||||
List.of(),
|
||||
false,
|
||||
"UNKNOWN_GAP"
|
||||
);
|
||||
}
|
||||
|
||||
private OperatingPeriodActivityIntervalDto periodized(ActivityIntervalDto interval, long operatingPeriodNo) {
|
||||
return OperatingPeriodActivityIntervalDto.periodized(
|
||||
interval,
|
||||
operatingPeriodNo,
|
||||
interval.startedAt(),
|
||||
false,
|
||||
0L
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,237 @@
|
|||
package at.procon.eventhub.esperpoc.service;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import at.procon.eventhub.esperpoc.dto.ActivityIntervalDto;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperActivityMergeMode;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperPocRequest;
|
||||
import at.procon.eventhub.esperpoc.dto.EsperShiftResolutionMode;
|
||||
import java.time.Duration;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class EsperPocDriverCardActivityServiceTest {
|
||||
|
||||
private final EsperDriverActivityEngine engine = new EsperDriverActivityEngine();
|
||||
private final EsperPocDriverCardActivityService service = new EsperPocDriverCardActivityService(null, engine);
|
||||
|
||||
@Test
|
||||
void mergesIdenticalActivitiesAcrossUtcMidnight() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
ActivityIntervalDto beforeMidnight = ActivityIntervalDto.raw(
|
||||
driverId,
|
||||
null,
|
||||
null,
|
||||
"BREAK_REST",
|
||||
"DRIVER",
|
||||
"INSERTED",
|
||||
"SINGLE",
|
||||
"DRIVER_CARD",
|
||||
OffsetDateTime.parse("2026-04-30T23:50:00Z"),
|
||||
OffsetDateTime.parse("2026-05-01T00:00:00Z"),
|
||||
"1"
|
||||
);
|
||||
ActivityIntervalDto afterMidnight = ActivityIntervalDto.raw(
|
||||
driverId,
|
||||
null,
|
||||
null,
|
||||
"BREAK_REST",
|
||||
"DRIVER",
|
||||
"INSERTED",
|
||||
"SINGLE",
|
||||
"DRIVER_CARD",
|
||||
OffsetDateTime.parse("2026-05-01T00:00:00Z"),
|
||||
OffsetDateTime.parse("2026-05-01T00:20:00Z"),
|
||||
"2"
|
||||
);
|
||||
|
||||
var merged = service.mergeConsecutiveIdenticalActivities(
|
||||
List.of(beforeMidnight, afterMidnight),
|
||||
Duration.ZERO
|
||||
);
|
||||
|
||||
assertThat(merged).hasSize(1);
|
||||
assertThat(merged.get(0).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-30T23:50:00Z"));
|
||||
assertThat(merged.get(0).endedAt()).isEqualTo(OffsetDateTime.parse("2026-05-01T00:20:00Z"));
|
||||
assertThat(merged.get(0).durationSeconds()).isEqualTo(30 * 60L);
|
||||
assertThat(merged.get(0).sourceRowIds()).containsExactly("1", "2");
|
||||
}
|
||||
|
||||
@Test
|
||||
void splitsOperatingPeriodsByBreakRestLongerThanConfiguredHoursAndEvaluatesDepartureArrivalPerPeriod() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
EsperPocRequest request = request(driverId, null, null);
|
||||
|
||||
List<ActivityIntervalDto> activities = List.of(
|
||||
activity(driverId, "DRIVE", "2026-04-01T06:00:00Z", "2026-04-01T08:00:00Z", "d1", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T08:00:00Z", "2026-04-01T09:00:00Z", "w1", "DRIVER_CARD"),
|
||||
activity(driverId, "DRIVE", "2026-04-01T09:30:00Z", "2026-04-01T11:00:00Z", "d2", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T20:00:00Z", "2026-04-02T04:30:00Z", "r1", "DRIVER_CARD"),
|
||||
activity(driverId, "DRIVE", "2026-04-02T05:00:00Z", "2026-04-02T07:00:00Z", "d3", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-02T10:00:00Z", "2026-04-02T10:30:00Z", "r2", "DRIVER_CARD"),
|
||||
activity(driverId, "DRIVE", "2026-04-02T10:30:00Z", "2026-04-02T12:00:00Z", "d4", "DRIVER_CARD")
|
||||
);
|
||||
|
||||
var periods = service.buildOperatingTimePeriods(
|
||||
request,
|
||||
request.occurredFrom(),
|
||||
request.occurredTo(),
|
||||
activities
|
||||
);
|
||||
|
||||
assertThat(periods).hasSize(2);
|
||||
assertThat(periods.get(0).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T06:00:00Z"));
|
||||
assertThat(periods.get(0).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T11:00:00Z"));
|
||||
assertThat(periods.get(0).drivingTimeInterruptionEvaluation().departureAt())
|
||||
.isEqualTo(OffsetDateTime.parse("2026-04-01T06:00:00Z"));
|
||||
assertThat(periods.get(0).drivingTimeInterruptionEvaluation().arrivalAt())
|
||||
.isEqualTo(OffsetDateTime.parse("2026-04-01T11:00:00Z"));
|
||||
assertThat(periods.get(0).drivingTimeInterruptionEvaluation().interruptionsBetweenSignificantDrivingPeriods())
|
||||
.hasSize(1);
|
||||
|
||||
assertThat(periods.get(1).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-02T05:00:00Z"));
|
||||
assertThat(periods.get(1).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-02T12:00:00Z"));
|
||||
assertThat(periods.get(1).drivingTimeInterruptionEvaluation().departureAt())
|
||||
.isEqualTo(OffsetDateTime.parse("2026-04-02T05:00:00Z"));
|
||||
assertThat(periods.get(1).drivingTimeInterruptionEvaluation().arrivalAt())
|
||||
.isEqualTo(OffsetDateTime.parse("2026-04-02T12:00:00Z"));
|
||||
assertThat(periods.get(1).workingOperationTimes().breakRestSeconds()).isEqualTo(30 * 60L);
|
||||
}
|
||||
|
||||
@Test
|
||||
void usesVuIntervalsOnlyForUncoveredDriverCardGaps() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
List<ActivityIntervalDto> resolved = service.resolveVuFillGaps(
|
||||
List.of(
|
||||
activity(driverId, "DRIVE", "2026-04-01T08:00:00Z", "2026-04-01T10:00:00Z", "c1", "DRIVER_CARD")
|
||||
),
|
||||
List.of(
|
||||
activity(driverId, "DRIVE", "2026-04-01T09:00:00Z", "2026-04-01T11:00:00Z", "v1", "VEHICLE_UNIT")
|
||||
)
|
||||
);
|
||||
|
||||
assertThat(resolved).hasSize(2);
|
||||
assertThat(resolved.get(0).sourceKind()).isEqualTo("DRIVER_CARD");
|
||||
assertThat(resolved.get(1).sourceKind()).isEqualTo("VEHICLE_UNIT");
|
||||
assertThat(resolved.get(1).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T10:00:00Z"));
|
||||
assertThat(resolved.get(1).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T11:00:00Z"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void resolvesTwoWorkingShiftsAcrossLongRestGap() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
EsperPocRequest request = request(driverId, null, null);
|
||||
|
||||
var shifts = service.buildResolvedWorkShifts(
|
||||
request,
|
||||
request.occurredFrom(),
|
||||
request.occurredTo(),
|
||||
List.of(
|
||||
activity(driverId, "DRIVE", "2026-04-01T06:00:00Z", "2026-04-01T10:00:00Z", "d1", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T10:00:00Z", "2026-04-01T17:30:00Z", "r1", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T17:30:00Z", "2026-04-01T19:00:00Z", "w1", "VEHICLE_UNIT")
|
||||
)
|
||||
);
|
||||
|
||||
assertThat(shifts).hasSize(2);
|
||||
assertThat(shifts.get(0).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T06:00:00Z"));
|
||||
assertThat(shifts.get(0).endedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T10:00:00Z"));
|
||||
assertThat(shifts.get(0).dailyRestingTimeSeconds()).isEqualTo(7L * 60L * 60L + 30L * 60L);
|
||||
assertThat(shifts.get(1).startedAt()).isEqualTo(OffsetDateTime.parse("2026-04-01T17:30:00Z"));
|
||||
assertThat(shifts.get(1).usedDataKind()).isEqualTo("VEHICLE_UNIT");
|
||||
}
|
||||
|
||||
@Test
|
||||
void esperMergeModeMatchesJavaMergeMode() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
List<ActivityIntervalDto> activities = List.of(
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T08:00:00Z", "2026-04-01T09:00:00Z", "r1", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T09:00:00Z", "2026-04-01T10:00:00Z", "r2", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T10:05:00Z", "2026-04-01T11:00:00Z", "w1", "DRIVER_CARD")
|
||||
);
|
||||
|
||||
List<ActivityIntervalDto> javaMerged = service.mergeConsecutiveIdenticalActivities(
|
||||
activities,
|
||||
Duration.ofSeconds(60)
|
||||
);
|
||||
List<ActivityIntervalDto> esperMerged = engine.mergeConsecutiveIdenticalActivities(
|
||||
activities,
|
||||
Duration.ofSeconds(60)
|
||||
);
|
||||
|
||||
assertThat(esperMerged).usingRecursiveComparison().isEqualTo(javaMerged);
|
||||
}
|
||||
|
||||
@Test
|
||||
void esperShiftResolutionModeMatchesJavaShiftResolutionMode() {
|
||||
UUID driverId = UUID.randomUUID();
|
||||
EsperPocRequest javaRequest = request(driverId, EsperActivityMergeMode.JAVA, EsperShiftResolutionMode.JAVA);
|
||||
EsperPocRequest esperRequest = request(driverId, EsperActivityMergeMode.JAVA, EsperShiftResolutionMode.ESPER);
|
||||
List<ActivityIntervalDto> intervals = List.of(
|
||||
activity(driverId, "DRIVE", "2026-04-01T06:00:00Z", "2026-04-01T10:00:00Z", "d1", "DRIVER_CARD"),
|
||||
activity(driverId, "BREAK_REST", "2026-04-01T10:00:00Z", "2026-04-01T17:30:00Z", "r1", "DRIVER_CARD"),
|
||||
activity(driverId, "WORK", "2026-04-01T17:30:00Z", "2026-04-01T19:00:00Z", "w1", "VEHICLE_UNIT")
|
||||
);
|
||||
|
||||
var javaShifts = service.buildResolvedWorkShifts(
|
||||
javaRequest,
|
||||
javaRequest.occurredFrom(),
|
||||
javaRequest.occurredTo(),
|
||||
intervals
|
||||
);
|
||||
var esperShifts = service.buildResolvedWorkShifts(
|
||||
esperRequest,
|
||||
esperRequest.occurredFrom(),
|
||||
esperRequest.occurredTo(),
|
||||
intervals
|
||||
);
|
||||
|
||||
assertThat(esperShifts).usingRecursiveComparison().isEqualTo(javaShifts);
|
||||
}
|
||||
|
||||
private EsperPocRequest request(
|
||||
UUID driverId,
|
||||
EsperActivityMergeMode activityMergeMode,
|
||||
EsperShiftResolutionMode shiftResolutionMode
|
||||
) {
|
||||
return new EsperPocRequest(
|
||||
"default",
|
||||
driverId,
|
||||
OffsetDateTime.parse("2026-04-01T00:00:00Z"),
|
||||
OffsetDateTime.parse("2026-04-03T00:00:00Z"),
|
||||
24,
|
||||
3,
|
||||
60,
|
||||
7,
|
||||
420,
|
||||
5,
|
||||
activityMergeMode,
|
||||
shiftResolutionMode
|
||||
);
|
||||
}
|
||||
|
||||
private ActivityIntervalDto activity(
|
||||
UUID driverId,
|
||||
String activity,
|
||||
String from,
|
||||
String to,
|
||||
String sourceRowId,
|
||||
String sourceKind
|
||||
) {
|
||||
return ActivityIntervalDto.raw(
|
||||
driverId,
|
||||
null,
|
||||
null,
|
||||
activity,
|
||||
"DRIVER",
|
||||
"INSERTED",
|
||||
"KNOWN",
|
||||
sourceKind,
|
||||
OffsetDateTime.parse(from),
|
||||
OffsetDateTime.parse(to),
|
||||
sourceRowId
|
||||
).asMerged(List.of(sourceRowId));
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue