Skip to content

Commit bbd92be

Browse files
ctruedenctrueden
committed
Add notes on Java<->Python structural mapping
1 parent 940a4f2 commit bbd92be

File tree

1 file changed

+216
-0
lines changed

1 file changed

+216
-0
lines changed

‎notes/python-package-structure.md‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# Python Package Structure Plan
2+
3+
This document defines the target Python package structure for appose-python and how the Java API should align with it.
4+
5+
## Design Principles
6+
7+
1. **Plugin Architecture**: Use Python entry points (not ServiceLoader) for extensibility
8+
2. **Monolithic Subsystem Files**: Group related implementations together (`builder.py`, `scheme.py`, `syntax.py`)
9+
3. **Singular Naming**: Use singular names for consistency (`platform.py`, not `platforms.py`)
10+
4. **Protocol-based Interfaces**: Use `typing.Protocol` for duck-typed interfaces, not ABC
11+
5. **Forward Compatibility**: Structure anticipates Java subsystems not yet ported to Python
12+
13+
## Target Python Package Structure
14+
15+
```
16+
appose/
17+
__init__.py # Definition of appose.__all__ via subpackage imports
18+
appose.py # Appose class for toplevel builder access
19+
environment.py # Environment
20+
service.py # Service, Task, TaskStatus, RequestType, ResponseType, TaskEvent
21+
shm.py # NDArray, SharedMemory, DType, Shape, Order
22+
23+
# Subsystems (all implementations + utility class in each)
24+
scheme.py # PixiTomlScheme, EnvironmentYmlScheme, PyProjectTomlScheme,
25+
# RequirementsTxtScheme, Schemes, Scheme protocol
26+
syntax.py # PythonSyntax, GroovySyntax, Syntaxes, ScriptSyntax protocol
27+
28+
# Utilities
29+
platform.py # is_windows(), is_linux(), is_mac(), etc.
30+
proxy.py # create() and proxy utilities
31+
filepath.py # find_exe() and path utilities
32+
process.py # Process utilities
33+
types.py # JSON utilities, typedefs (e.g. Args)
34+
35+
# Workers (language-specific)
36+
python_worker.py # Python worker implementation
37+
groovy_worker.py # Groovy worker implementation (if needed)
38+
39+
# Builder subsystem (each implemention in a separate file)
40+
builder/__init__.py # Builder protocol, BaseBuilder, DynamicBuilder,
41+
# SimpleBuilder, Builders, BuilderFactory protocol
42+
builder/mamba.py # MambaBuilder, MambaBuilderFactory
43+
builder/pixi.py # PixiBuilder, PixiBuilderFactory
44+
builder/uv.py # UvBuilder, UvBuilderFactory
45+
46+
# Tool subsystem (each implemention in a separate file)
47+
tool/__init__.py # Tool protocol
48+
tool/mamba.py # Mamba
49+
tool/pixi.py # Pixi
50+
tool/uv.py # Uv
51+
```
52+
53+
## Plugin System via Entry Points
54+
55+
Extensions can register plugins via `pyproject.toml`:
56+
57+
```python
58+
# In pyproject.toml
59+
entry_points={
60+
'appose.builders': [
61+
'pixi = appose.builder:PixiBuilderFactory',
62+
'mamba = appose.builder:MambaBuilderFactory',
63+
'uv = appose.builder:UvBuilderFactory',
64+
],
65+
'appose.schemes': [
66+
'pixi-toml = appose.scheme:PixiTomlScheme',
67+
'environment-yml = appose.scheme:EnvironmentYmlScheme',
68+
'pyproject-toml = appose.scheme:PyProjectTomlScheme',
69+
'requirements-txt = appose.scheme:RequirementsTxtScheme',
70+
],
71+
'appose.syntaxes': [
72+
'python = appose.syntax:PythonSyntax',
73+
'groovy = appose.syntax:GroovySyntax',
74+
],
75+
}
76+
```
77+
78+
Third-party packages can extend Appose by providing their own entry points.
79+
80+
## Interface Design: Protocol vs ABC
81+
82+
Use `typing.Protocol` for interfaces to support structural subtyping:
83+
84+
```python
85+
from typing import Protocol
86+
87+
class Builder(Protocol):
88+
"""Builder interface for creating environments."""
89+
def build(self) -> Environment: ...
90+
def name(self) -> str: ...
91+
def rebuild(self) -> Environment: ...
92+
93+
class BuilderFactory(Protocol):
94+
"""Factory for creating builders."""
95+
def create_builder(self) -> Builder: ...
96+
def supports_scheme(self, scheme: str) -> bool: ...
97+
def name(self) -> str: ...
98+
```
99+
100+
**Why Protocol?**
101+
- Structural subtyping (duck typing with type hints)
102+
- No explicit inheritance required
103+
- Third-party plugins don't need to import base classes
104+
- More Pythonic
105+
106+
## Import Style
107+
108+
```python
109+
# Recommended imports
110+
from appose import builder, scheme, syntax
111+
from appose import platform, proxy, path
112+
113+
# Usage examples
114+
if platform.is_windows():
115+
print("Running on Windows")
116+
117+
proxy.create(service, var, api)
118+
builder.Builders.find_by_name("pixi")
119+
scheme.Schemes.from_content(content_string)
120+
```
121+
122+
**Why singular naming?**
123+
- Consistency across all modules
124+
- Matches Python stdlib (`json`, `pathlib`, `subprocess`)
125+
- `appose.builder.PixiBuilder` reads naturally
126+
- Avoids confusion between "instances" vs "subsystem"
127+
128+
## API Stub Structure
129+
130+
The Java API dump should produce files matching this structure:
131+
132+
```
133+
api/appose/
134+
__init__.pyi # Appose class (factory methods)
135+
environment.pyi # Environment, Builder protocol
136+
service.pyi # Service, Task, TaskStatus, RequestType, ResponseType, TaskEvent
137+
types.pyi # NDArray, SharedMemory, DType, Shape, Order, Args
138+
builder.pyi # All builder classes + Builders + BuilderFactory protocol
139+
scheme.pyi # All scheme classes + Schemes + Scheme protocol
140+
syntax.pyi # All syntax classes + Syntaxes + ScriptSyntax protocol
141+
platform.pyi # Platform utilities
142+
proxy.pyi # Proxy utilities
143+
path.pyi # Path utilities
144+
process.pyi # Process utilities
145+
python_worker.pyi # Python worker
146+
groovy_worker.pyi # Groovy worker (if applicable)
147+
```
148+
149+
## Java to Python Mapping Rules
150+
151+
### Package to File Mapping
152+
153+
| Java Package/Class | Python Module |
154+
|-------------------------------------|-----------------------|
155+
| `org.apposed.appose.Appose` | `appose.py` |
156+
| `org.apposed.appose.Environment` | `environment.py` |
157+
| `org.apposed.appose.RequestType` | `service.py` |
158+
| `org.apposed.appose.ResponseType` | `service.py` |
159+
| `org.apposed.appose.Service*` | `service.py` |
160+
| `org.apposed.appose.Task*` | `service.py` |
161+
| `org.apposed.appose.scheme.*` | `scheme.py` |
162+
| `org.apposed.appose.syntax.*` | `syntax.py` |
163+
| `org.apposed.appose.NDArray*` | `shm.py` |
164+
| `org.apposed.appose.SharedMemory` | `shm.py` |
165+
| `org.apposed.appose.Builder*` | `builder/__init__.py` |
166+
| `org.apposed.appose.builder.Mamba*` | `builder/mamba.py` |
167+
| `org.apposed.appose.builder.Pixi*` | `builder/pixi.py` |
168+
| `org.apposed.appose.builder.Uv*` | `builder/uv.py` |
169+
| `org.apposed.appose.util.FilePaths` | `filepath.py` |
170+
| `org.apposed.appose.util.Platforms` | `platform.py` |
171+
| `org.apposed.appose.util.Processes` | `process.py` |
172+
| `org.apposed.appose.util.Proxies` | `proxy.py` |
173+
| `org.apposed.appose.util.Types` | `types.py` |
174+
| `org.apposed.appose.GroovyWorker` | N/A |
175+
176+
### Special Cases
177+
178+
**Inner Classes**: Extract and place in same file as parent
179+
- `Service.Task` → `service.py` (as `class Task`)
180+
- `NDArray.DType` → `shm.py` (as `class DType`)
181+
- `NDArray.Shape` → `shm.py` (as `class Shape`)
182+
183+
**AutoCloseable**: Add `__enter__` and `__exit__` methods automatically
184+
- `Service(AutoCloseable)` → adds context manager protocol
185+
- `SharedMemory(AutoCloseable)` → adds context manager protocol
186+
- `NDArray(AutoCloseable)` → adds context manager protocol
187+
188+
**Enums**: Keep as classes with enum values
189+
- `TaskStatus` → class with `INITIAL`, `QUEUED`, etc. as class attributes
190+
191+
## Rationale for Key Decisions
192+
193+
### Why singular names?
194+
- **Consistency**: All modules use singular form
195+
- **stdlib alignment**: Matches Python standard library conventions
196+
- **Natural reading**: `appose.builder.PixiBuilder` flows well
197+
- **Clarity**: Avoids ambiguity about module purpose
198+
199+
### Why Protocol over ABC?
200+
- **Duck typing**: More Pythonic, supports structural subtyping
201+
- **No inheritance required**: Third-party plugins more flexible
202+
- **Type safety**: Still provides full type checking benefits
203+
- **Plugin friendly**: Extensions don't depend on base classes
204+
205+
### Why entry points?
206+
- **Python standard**: De facto plugin mechanism in Python ecosystem
207+
- **Tool support**: pip, setuptools, poetry all support it
208+
- **Ecosystem adoption**: Used by pytest, Flask, Sphinx, etc.
209+
- **Dynamic discovery**: No need to manually register plugins
210+
211+
## References
212+
213+
- [PEP 561 - Distributing and Packaging Type Information](https://peps.python.org/pep-0561/)
214+
- [PEP 544 - Protocols: Structural subtyping](https://peps.python.org/pep-0544/)
215+
- [Entry Points Specification](https://packaging.python.org/en/latest/specifications/entry-points/)
216+
- [Python Packaging User Guide](https://packaging.python.org/)

0 commit comments

Comments
 (0)