Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions .github/skills/update-protobuf/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
name: update-protobuf
description: >-
Update the protobuf generated Python files from the latest
microsoft/durabletask-protobuf definitions. Use when the proto definitions
need to be refreshed or regenerated.
---

# Update Protobuf Definitions

This skill regenerates the Python protobuf files in `durabletask/internal/`
from the latest proto source at
<https://github.com/microsoft/durabletask-protobuf>.

## Prerequisites

- Python 3.11 must be available on the system. Verify with `py -3.11 --version`
(Windows) or `python3.11 --version` (Linux/macOS). If it is not installed,
stop and ask the user to install it — do **not** use a different version.
- On Linux/macOS, [`jq`](https://jqlang.github.io/jq/) must be installed.
- Internet access is required to download the proto file and query the GitHub
API.

## Steps

### 1. Set up the `.proto_venv` environment (skip if it already exists)

Check whether `.proto_venv/` already exists at the repo root and whether
`grpcio-tools==1.65.4` is installed in it:

```bash
# Windows
.proto_venv\Scripts\pip.exe list 2>$null | Select-String grpcio-tools

# Bash
.proto_venv/bin/pip list 2>/dev/null | grep grpcio-tools
```

If the venv **does not exist** or `grpcio-tools` is missing, create/recreate it:

```bash
# Windows
py -3.11 -m venv .proto_venv
.proto_venv\Scripts\python.exe -m pip install grpcio-tools==1.65.4

# Bash
python3.11 -m venv .proto_venv
.proto_venv/bin/python -m pip install grpcio-tools==1.65.4
```

> [!NOTE]
> Do **not** delete `.proto_venv` after use. It is reused across runs to avoid
> reinstalling `grpcio-tools` each time. The directory is already in
> `.gitignore`.

### 2. Download the latest proto file

Download from the `main` branch of `microsoft/durabletask-protobuf`:

```bash
# Windows (PowerShell)
Invoke-WebRequest `
-Uri "https://raw.githubusercontent.com/microsoft/durabletask-protobuf/refs/heads/main/protos/orchestrator_service.proto" `
-OutFile "durabletask/internal/orchestrator_service.proto"

# Bash
curl -o durabletask/internal/orchestrator_service.proto \
https://raw.githubusercontent.com/microsoft/durabletask-protobuf/refs/heads/main/protos/orchestrator_service.proto
```

### 3. Regenerate the Python files

Run `grpc_tools.protoc` from the **repo root**:

```bash
# Windows
.proto_venv\Scripts\python.exe -m grpc_tools.protoc --proto_path=. --python_out=. --pyi_out=. --grpc_python_out=. ./durabletask/internal/orchestrator_service.proto

# Bash
.proto_venv/bin/python -m grpc_tools.protoc --proto_path=. --python_out=. --pyi_out=. --grpc_python_out=. ./durabletask/internal/orchestrator_service.proto
```

This produces three files in `durabletask/internal/`:

- `orchestrator_service_pb2.py`
- `orchestrator_service_pb2_grpc.py`
- `orchestrator_service_pb2.pyi`

### 4. Update `PROTO_SOURCE_COMMIT_HASH`

Query the GitHub API for the latest commit that touched the proto file and
**overwrite** `durabletask/internal/PROTO_SOURCE_COMMIT_HASH` with that single
hash:

```bash
# Windows (PowerShell)
$response = Invoke-RestMethod `
-Uri "https://api.github.com/repos/microsoft/durabletask-protobuf/commits?path=protos/orchestrator_service.proto&sha=main&per_page=1"
$response[0].sha | Out-File -FilePath "durabletask/internal/PROTO_SOURCE_COMMIT_HASH" -NoNewline -Encoding ascii

# Bash
curl -s -H "Accept: application/vnd.github.v3+json" \
"https://api.github.com/repos/microsoft/durabletask-protobuf/commits?path=protos/orchestrator_service.proto&sha=main&per_page=1" \
| jq -jr '.[0].sha' > durabletask/internal/PROTO_SOURCE_COMMIT_HASH
```

The file should contain exactly one commit hash with no trailing newline.

### 5. Clean up the downloaded proto file

Delete the `.proto` source file — only the generated Python files are kept in
the repo:

```bash
# Windows
Remove-Item durabletask/internal/orchestrator_service.proto

# Bash
rm durabletask/internal/orchestrator_service.proto
```

### 6. Verify

Confirm the generated modules import successfully using the project's main
venv:

```bash
python -c "from durabletask.internal import orchestrator_service_pb2; print('pb2 OK'); from durabletask.internal import orchestrator_service_pb2_grpc; print('pb2_grpc OK')"
```

## Generated files

| File | Description |
|---|---|
| `durabletask/internal/orchestrator_service_pb2.py` | Message classes |
| `durabletask/internal/orchestrator_service_pb2_grpc.py` | gRPC stubs |
| `durabletask/internal/orchestrator_service_pb2.pyi` | Type stubs |
| `durabletask/internal/PROTO_SOURCE_COMMIT_HASH` | Source commit hash |
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ celerybeat.pid
# Environments
.env
.venv
.proto_venv
env/
venv/
ENV/
Expand Down
4 changes: 1 addition & 3 deletions durabletask/internal/PROTO_SOURCE_COMMIT_HASH
Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
443b333f4f65a438dc9eb4f090560d232afec4b7
fd9369c6a03d6af4e95285e432b7c4e943c06970
026329c53fe6363985655857b9ca848ec7238bd2
e42905306605b7be6efb789dc489d27e27e22f71
308 changes: 159 additions & 149 deletions durabletask/internal/orchestrator_service_pb2.py

Large diffs are not rendered by default.

50 changes: 46 additions & 4 deletions durabletask/internal/orchestrator_service_pb2.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -593,8 +593,14 @@ class SendEntityMessageAction(_message.Message):
entityUnlockSent: EntityUnlockSentEvent
def __init__(self, entityOperationSignaled: _Optional[_Union[EntityOperationSignaledEvent, _Mapping]] = ..., entityOperationCalled: _Optional[_Union[EntityOperationCalledEvent, _Mapping]] = ..., entityLockRequested: _Optional[_Union[EntityLockRequestedEvent, _Mapping]] = ..., entityUnlockSent: _Optional[_Union[EntityUnlockSentEvent, _Mapping]] = ...) -> None: ...

class RewindOrchestrationAction(_message.Message):
__slots__ = ("newHistory",)
NEWHISTORY_FIELD_NUMBER: _ClassVar[int]
newHistory: _containers.RepeatedCompositeFieldContainer[HistoryEvent]
def __init__(self, newHistory: _Optional[_Iterable[_Union[HistoryEvent, _Mapping]]] = ...) -> None: ...

class OrchestratorAction(_message.Message):
__slots__ = ("id", "scheduleTask", "createSubOrchestration", "createTimer", "sendEvent", "completeOrchestration", "terminateOrchestration", "sendEntityMessage")
__slots__ = ("id", "scheduleTask", "createSubOrchestration", "createTimer", "sendEvent", "completeOrchestration", "terminateOrchestration", "sendEntityMessage", "rewindOrchestration")
ID_FIELD_NUMBER: _ClassVar[int]
SCHEDULETASK_FIELD_NUMBER: _ClassVar[int]
CREATESUBORCHESTRATION_FIELD_NUMBER: _ClassVar[int]
Expand All @@ -603,6 +609,7 @@ class OrchestratorAction(_message.Message):
COMPLETEORCHESTRATION_FIELD_NUMBER: _ClassVar[int]
TERMINATEORCHESTRATION_FIELD_NUMBER: _ClassVar[int]
SENDENTITYMESSAGE_FIELD_NUMBER: _ClassVar[int]
REWINDORCHESTRATION_FIELD_NUMBER: _ClassVar[int]
id: int
scheduleTask: ScheduleTaskAction
createSubOrchestration: CreateSubOrchestrationAction
Expand All @@ -611,7 +618,8 @@ class OrchestratorAction(_message.Message):
completeOrchestration: CompleteOrchestrationAction
terminateOrchestration: TerminateOrchestrationAction
sendEntityMessage: SendEntityMessageAction
def __init__(self, id: _Optional[int] = ..., scheduleTask: _Optional[_Union[ScheduleTaskAction, _Mapping]] = ..., createSubOrchestration: _Optional[_Union[CreateSubOrchestrationAction, _Mapping]] = ..., createTimer: _Optional[_Union[CreateTimerAction, _Mapping]] = ..., sendEvent: _Optional[_Union[SendEventAction, _Mapping]] = ..., completeOrchestration: _Optional[_Union[CompleteOrchestrationAction, _Mapping]] = ..., terminateOrchestration: _Optional[_Union[TerminateOrchestrationAction, _Mapping]] = ..., sendEntityMessage: _Optional[_Union[SendEntityMessageAction, _Mapping]] = ...) -> None: ...
rewindOrchestration: RewindOrchestrationAction
def __init__(self, id: _Optional[int] = ..., scheduleTask: _Optional[_Union[ScheduleTaskAction, _Mapping]] = ..., createSubOrchestration: _Optional[_Union[CreateSubOrchestrationAction, _Mapping]] = ..., createTimer: _Optional[_Union[CreateTimerAction, _Mapping]] = ..., sendEvent: _Optional[_Union[SendEventAction, _Mapping]] = ..., completeOrchestration: _Optional[_Union[CompleteOrchestrationAction, _Mapping]] = ..., terminateOrchestration: _Optional[_Union[TerminateOrchestrationAction, _Mapping]] = ..., sendEntityMessage: _Optional[_Union[SendEntityMessageAction, _Mapping]] = ..., rewindOrchestration: _Optional[_Union[RewindOrchestrationAction, _Mapping]] = ...) -> None: ...

class OrchestrationTraceContext(_message.Message):
__slots__ = ("spanID", "spanStartTime")
Expand Down Expand Up @@ -1250,16 +1258,50 @@ class SkipGracefulOrchestrationTerminationsResponse(_message.Message):
def __init__(self, unterminatedInstanceIds: _Optional[_Iterable[str]] = ...) -> None: ...

class GetWorkItemsRequest(_message.Message):
__slots__ = ("maxConcurrentOrchestrationWorkItems", "maxConcurrentActivityWorkItems", "maxConcurrentEntityWorkItems", "capabilities")
__slots__ = ("maxConcurrentOrchestrationWorkItems", "maxConcurrentActivityWorkItems", "maxConcurrentEntityWorkItems", "capabilities", "workItemFilters")
MAXCONCURRENTORCHESTRATIONWORKITEMS_FIELD_NUMBER: _ClassVar[int]
MAXCONCURRENTACTIVITYWORKITEMS_FIELD_NUMBER: _ClassVar[int]
MAXCONCURRENTENTITYWORKITEMS_FIELD_NUMBER: _ClassVar[int]
CAPABILITIES_FIELD_NUMBER: _ClassVar[int]
WORKITEMFILTERS_FIELD_NUMBER: _ClassVar[int]
maxConcurrentOrchestrationWorkItems: int
maxConcurrentActivityWorkItems: int
maxConcurrentEntityWorkItems: int
capabilities: _containers.RepeatedScalarFieldContainer[WorkerCapability]
def __init__(self, maxConcurrentOrchestrationWorkItems: _Optional[int] = ..., maxConcurrentActivityWorkItems: _Optional[int] = ..., maxConcurrentEntityWorkItems: _Optional[int] = ..., capabilities: _Optional[_Iterable[_Union[WorkerCapability, str]]] = ...) -> None: ...
workItemFilters: WorkItemFilters
def __init__(self, maxConcurrentOrchestrationWorkItems: _Optional[int] = ..., maxConcurrentActivityWorkItems: _Optional[int] = ..., maxConcurrentEntityWorkItems: _Optional[int] = ..., capabilities: _Optional[_Iterable[_Union[WorkerCapability, str]]] = ..., workItemFilters: _Optional[_Union[WorkItemFilters, _Mapping]] = ...) -> None: ...

class WorkItemFilters(_message.Message):
__slots__ = ("orchestrations", "activities", "entities")
ORCHESTRATIONS_FIELD_NUMBER: _ClassVar[int]
ACTIVITIES_FIELD_NUMBER: _ClassVar[int]
ENTITIES_FIELD_NUMBER: _ClassVar[int]
orchestrations: _containers.RepeatedCompositeFieldContainer[OrchestrationFilter]
activities: _containers.RepeatedCompositeFieldContainer[ActivityFilter]
entities: _containers.RepeatedCompositeFieldContainer[EntityFilter]
def __init__(self, orchestrations: _Optional[_Iterable[_Union[OrchestrationFilter, _Mapping]]] = ..., activities: _Optional[_Iterable[_Union[ActivityFilter, _Mapping]]] = ..., entities: _Optional[_Iterable[_Union[EntityFilter, _Mapping]]] = ...) -> None: ...

class OrchestrationFilter(_message.Message):
__slots__ = ("name", "versions")
NAME_FIELD_NUMBER: _ClassVar[int]
VERSIONS_FIELD_NUMBER: _ClassVar[int]
name: str
versions: _containers.RepeatedScalarFieldContainer[str]
def __init__(self, name: _Optional[str] = ..., versions: _Optional[_Iterable[str]] = ...) -> None: ...

class ActivityFilter(_message.Message):
__slots__ = ("name", "versions")
NAME_FIELD_NUMBER: _ClassVar[int]
VERSIONS_FIELD_NUMBER: _ClassVar[int]
name: str
versions: _containers.RepeatedScalarFieldContainer[str]
def __init__(self, name: _Optional[str] = ..., versions: _Optional[_Iterable[str]] = ...) -> None: ...

class EntityFilter(_message.Message):
__slots__ = ("name",)
NAME_FIELD_NUMBER: _ClassVar[int]
name: str
def __init__(self, name: _Optional[str] = ...) -> None: ...

class WorkItem(_message.Message):
__slots__ = ("orchestratorRequest", "activityRequest", "entityRequest", "healthPing", "entityRequestV2", "completionToken")
Expand Down
Loading