Lazy package loading system and event-based discovery coordination

pyvider 2026-03-23

11 files · 490+ · 86-

3 files changed

lazy_import.py New sys.meta_path hook (LazyPackageFinder + LazyPackageLoader) that intercepts imports of listed heavy packages (mkdocs, material, babel, pygments, etc.) and loads them from a compressed zip archive instead of disk
build-lazy-packages.py Build script that extracts the lazy package set from site-packages into a separate compressed archive, separating non-critical packages from the main bundle
__init__.py Installs the lazy loader via install_lazy_loader() at import time; also sets PLUGIN_SERVER_TRANSPORTS=tcp on Windows before any pyvider imports

LazyPackageFinder intercepts sys.meta_path imports for a defined set of heavy optional packages. LazyPackageLoader extracts and executes the package from the archive, avoiding upfront extraction of ~100MB of doc-tool packages (mkdocs, material, babel, pygments, pymarkdown, etc.) that are not needed for RPC server startup. instantiate behavioral
1 file
- lazy_import.py
build-lazy-packages.py separates lazy packages out of site-packages into a standalone zip archive for use with the lazy loader. Intended to reduce initial PSPF extraction time for cold-start provider runs. instantiate behavioral
1 file
- build-lazy-packages.py

5 files changed

provide_command.py Creates an asyncio.Event (discovery_ready_event) that is set after _discover_components_once() completes; registers the event in the hub under the singleton key '_discovery_ready_event'
handler.py Replaces the 0.1s asyncio.sleep() fallback with asyncio.wait_for(discovery_event.wait(), timeout=300.0) — proper event-based wait instead of polling
get_provider_schema.py Added wait on _discovery_ready_event before collecting provider schemas, ensuring GetProviderSchema doesn't run before discovery completes
apply_resource_change.py Fixed indentation errors introduced in prior lazy init commits
plan_resource_change.py Fixed indentation errors introduced in prior lazy init commits

The discovery ready event is registered in the hub at server startup and set once discover_and_signal() completes. ProviderHandler. _ensure_provider_ready() now awaits the event with a 5-minute timeout instead of sleeping 100ms and retrying once — this correctly blocks any RPC handler until discovery is done. decouple behavioral
2 files
- provide_command.py
- handler.py
GetProviderSchema handler now fetches the discovery event from the hub and awaits it before collecting schemas, preventing the schema response from being built against a partially-discovered component registry. harden behavioral
1 file
- get_provider_schema.py

2 files changed

discovery.py Added timing instrumentation to discover_all() and _discover_package() — logs elapsed_seconds, total_elapsed_ms, component counts on completion; logs slow modules (>500ms) at debug level
main.py Gracefully handles UnicodeEncodeError when printing decorative characters to Windows consoles with default code page encoding

discover_all() now measures wall-clock time and logs elapsed_seconds and component counts on completion. _discover_package() logs slow modules (>500ms) to help identify bottlenecks. The parallel discovery attempt was introduced and then reverted to sequential after causing issues. qualify internal
1 file
- discovery.py
Interactive mode output wrapped in try/except UnicodeEncodeError so decorative box-drawing characters don’t crash on Windows terminals with default code page encoding. harden internal
1 file
- main.py

1 file changed

.gitattributes New file enforcing eol=lf for all text files; marks binary files (.bin, .exe, .psp, .zip) explicitly

Added .gitattributes with text=auto eol=lf for all text files and explicit binary markers for known binary extensions. Prevents CRLF line ending drift, especially on Windows developer machines. baseline errata
1 file
- .gitattributes