add entrypoint build modes feature (#31)

Author: thrau

README.md

@@ -39,6 +39,10 @@ If an error occurs at any state of the lifecycle, the `PluginManager` informs th
 
 ### Discovering entrypoints
 
+Plux supports two modes for building entry points: **build-hooks mode** (default) and **manual mode**.
+
+#### Build-hooks mode (default)
+
 To build a source distribution and a wheel of your code with your plugins as entrypoints, simply run `python setup.py plugins sdist bdist_wheel`.
 If you don't have a `setup.py`, you can use the plux build frontend and run `python -m plux entrypoints`.
 
@@ -49,6 +53,42 @@ When a setuptools command is used to create the distribution (e.g., `python setu
 The `plux.json` file becomes a part of the distribution, s.t., the plugins do not have to be discovered every time your distribution is installed elsewhere.
 Discovering at build time also works when using `python -m build`, since it calls registered setuptools scripts.
 
+#### Manual mode
+
+Manual mode is useful for isolated build environments where dependencies cannot be installed, or when build hooks are not suitable for your build process.
+
+To enable manual mode, add the following to your `pyproject.toml`:
+
+```toml
+[tool.plux]
+entrypoint_build_mode = "manual"
+```
+
+In manual mode, plux does not use build hooks. Instead, you manually generate entry points by running:
+
+```bash
+python -m plux entrypoints
+```
+
+This creates a `plux.ini` file in your working directory with the discovered plugins. You can then include this file in your distribution by configuring your `pyproject.toml`:
+
+```toml
+[project]
+dynamic = ["entry-points"]
+
+[tool.setuptools.package-data]
+"*" = ["plux.ini"]
+
+[tool.setuptools.dynamic]
+entry-points = {file = ["plux.ini"]}
+```
+
+You can also manually control the output format and location:
+
+```bash
+python -m plux discover --format ini --output plux.ini
+```
+
 
 Examples
 --------
@@ -201,9 +241,35 @@ build-backend = "setuptools.build_meta"
 Additional configuration
 ------------------------
 
-You can pass additional configuration to Plux, either via the command line or your project `pyproject.toml`. 
+You can pass additional configuration to Plux, either via the command line or your project `pyproject.toml`.
 
-### Excluding Python packages during discovery
+### Configuration options
+
+The following options can be configured in the `[tool.plux]` section of your `pyproject.toml`:
+
+```toml
+[tool.plux]
+# The build mode for entry points: "build-hooks" (default) or "manual"
+entrypoint_build_mode = "manual"
+
+# The file path to scan for plugins (optional)
+path = "mysrc"
+
+# Python packages to exclude during discovery (optional)
+exclude = ["**/database/alembic*"]
+```
+
+#### `entrypoint_build_mode`
+
+Controls how plux generates entry points:
+- `build-hooks` (default): Plux automatically hooks into the build process to generate entry points
+- `manual`: You manually control when and how entry points are generated (see [Manual mode](#manual-mode))
+
+#### `path`
+
+Specifies the file path to scan for plugins. By default, plux scans the entire project.
+
+#### `exclude`
 
 When [discovering entrypoints](#discovering-entrypoints), Plux will try importing your code to discover Plugins.
 Some parts of your codebase might have side effects, or raise errors when imported outside a specific context like some database
@@ -212,14 +278,7 @@ migration scripts.
 You can ignore those Python packages by specifying the `--exclude` flag to the entrypoints discovery commands (`python -m plux entrypoints` or `python setup.py plugins`).
 The option takes a list of comma-separated values that can be paths or package names.
 
-You can also specify those values in the `tool.plux` section of your  `pyproject.toml`:
-
-```toml
-# ...
-
-[tool.plux]
-exclude = ["**/database/alembic*"]
-```
+You can also specify those values in the `tool.plux` section of your `pyproject.toml` as shown above.
 
 Install
 -------

plux/build/config.py

@@ -3,17 +3,75 @@
 future to support ``tox.ini``, ``setup.cfg``, etc.
 """
 import dataclasses
+import enum
 import os
+import sys
 from importlib.util import find_spec
-import typing as t
+
+
+class EntrypointBuildMode(enum.Enum):
+    """
+    The build mode for entrypoints. When ``build-hook`` is used, plux hooks into the build backend's build process using
+    various extension points. Currently, we only support setuptools, where plux generates a plugin index and adds it to
+    the .egg-info directory, which is later used to generate entry points.
+
+    The alternative is ``manual``, where build hooks are disabled and the user is responsible for generating and
+    referencing entry points.
+    """
+    MANUAL = "manual"
+    BUILD_HOOK = "build-hook"
 
 
 @dataclasses.dataclass
 class PluxConfiguration:
     """
     Configuration object with sane default values.
     """
-    exclude: t.List[str] = dataclasses.field(default_factory=list)
+
+    path: str = "."
+    """The path to scan for plugins."""
+
+    exclude: list[str] = dataclasses.field(default_factory=list)
+    """A list of paths to exclude from scanning."""
+
+    entrypoint_build_mode: EntrypointBuildMode = EntrypointBuildMode.BUILD_HOOK
+    """The point in the build process path plugins should be discovered and entrypoints generated."""
+
+    entrypoint_static_file: str = "plux.ini"
+    """The name of the entrypoint ini file if entrypoint_build_mode is set to MANUAL."""
+
+    def merge(
+        self,
+        path: str = None,
+        exclude: list[str] = None,
+        entrypoint_build_mode: EntrypointBuildMode = None,
+        entrypoint_static_file: str = None,
+    ) -> "PluxConfiguration":
+        """
+        Merges or overwrites the given values into the current configuration and returns a new configuration object.
+        If the passed values are None, they are not changed.
+        """
+        return PluxConfiguration(
+            path=path if path is not None else self.path,
+            exclude=list(set((exclude if exclude is not None else []) + self.exclude)),
+            entrypoint_build_mode=entrypoint_build_mode if entrypoint_build_mode is not None else self.entrypoint_build_mode,
+            entrypoint_static_file=entrypoint_static_file if entrypoint_static_file is not None else self.entrypoint_static_file,
+        )
+
+
+def read_plux_config_from_workdir(workdir: str = None) -> PluxConfiguration:
+    """
+    Reads the plux configuration from the specified workdir. Currently, it only checks for a ``pyproject.toml`` to
+    parse. If no pyproject.toml is found, a default configuration is returned.
+
+    :param workdir: The workdir which defaults to the current working directory.
+    :return: A plux configuration object
+    """
+    try:
+        pyproject_file = os.path.join(workdir or os.getcwd(), "pyproject.toml")
+        return parse_pyproject_toml(pyproject_file)
+    except FileNotFoundError:
+        return PluxConfiguration()
 
 
 def parse_pyproject_toml(path: str | os.PathLike[str]) -> PluxConfiguration:
@@ -22,7 +80,7 @@ def parse_pyproject_toml(path: str | os.PathLike[str]) -> PluxConfiguration:
     object from the found values. Uses tomli or tomllib to parse the file.
 
     :param path: Path to the pyproject.toml file.
-    :return: A ``Configuration`` object containing the parsed values.
+    :return: A plux configuration object containing the parsed values.
     :raises FileNotFoundError: If the file does not exist.
     """
     if find_spec("tomllib"):
@@ -32,13 +90,28 @@ def parse_pyproject_toml(path: str | os.PathLike[str]) -> PluxConfiguration:
     else:
         raise ImportError("Could not find a TOML parser. Please install either tomllib or tomli.")
 
+    # read the file
     if not os.path.exists(path):
         raise FileNotFoundError(f"No pyproject.toml found at {path}")
-
     with open(path, "rb") as file:
         pyproject_config = load_toml(file)
 
+    # find the [tool.plux] section
     tool_table = pyproject_config.get("tool", {})
     tool_config = tool_table.get("plux", {})
 
-    return PluxConfiguration(**tool_config)
+    # filter out all keys that are not available in the config object
+    kwargs = {}
+    for key, value in tool_config.items():
+        if key not in PluxConfiguration.__annotations__:
+            print(f"Warning: ignoring unknown key {key} in [tool.plux] section of {path}", file=sys.stderr)
+            continue
+
+        kwargs[key] = value
+
+    # parse entrypoint_build_mode enum
+    if mode := kwargs.get("entrypoint_build_mode"):
+        # will raise a ValueError exception if the mode is invalid
+        kwargs["entrypoint_build_mode"] = EntrypointBuildMode(mode)
+
+    return PluxConfiguration(**kwargs)

plux/build/discovery.py

@@ -1,12 +1,19 @@
 """
-Buildtool independent utils to discover plugins from the codebase.
+Buildtool independent utils to discover plugins from the codebase, and write index files.
 """
+import configparser
 import inspect
+import json
 import logging
-from types import ModuleType
+import sys
 import typing as t
+from types import ModuleType
 
 from plux import PluginFinder, PluginSpecResolver, PluginSpec
+from plux.core.entrypoint import discover_entry_points, EntryPointDict
+
+if t.TYPE_CHECKING:
+    from _typeshed import SupportsWrite
 
 LOG = logging.getLogger(__name__)
 
@@ -41,3 +48,74 @@ def find_plugins(self) -> t.List[PluginSpec]:
                         pass
 
         return plugins
+
+
+class PluginIndexBuilder:
+    """
+    Builds an index file containing all discovered plugins. The index file can be written to stdout, or to a file.
+    The writer supports two formats: json and ini.
+    """
+
+    def __init__(
+        self,
+        plugin_finder: PluginFinder,
+        output_format: t.Literal["json", "ini"] = "json",
+    ):
+        self.plugin_finder = plugin_finder
+        self.output_format = output_format
+
+    def write(self, fp: "SupportsWrite[str]" = sys.stdout) -> EntryPointDict:
+        """
+        Discover entry points using the configured ``PluginFinder``, and write the entry points into a file.
+
+        :param fp: The file-like object to write to.
+        :return: The discovered entry points that were written into the file.
+        """
+        ep = discover_entry_points(self.plugin_finder)
+
+        # sort entrypoints alphabetically in each group first
+        for group in ep:
+            ep[group].sort()
+
+        if self.output_format == "json":
+            json.dump(ep, fp, sort_keys=True, indent=2)
+        elif self.output_format == "ini":
+            cfg = configparser.ConfigParser()
+            cfg.read_dict(self.convert_to_nested_entry_point_dict(ep))
+            cfg.write(fp)
+        else:
+            raise ValueError(f"unknown plugin index output format {self.output_format}")
+
+        return ep
+
+    @staticmethod
+    def convert_to_nested_entry_point_dict(ep: EntryPointDict) -> dict[str, dict[str, str]]:
+        """
+        Converts and ``EntryPointDict`` to a nested dict, where the keys are the section names and values are
+        dictionaries. Each dictionary maps entry point names to their values. It also sorts the output alphabetically.
+
+        Example:
+            Input EntryPointDict:
+            {
+                'console_scripts': ['app=module:main', 'tool=module:cli'],
+                'plux.plugins': ['plugin1=pkg.module:Plugin1']
+            }
+
+            Output nested dict:
+            {
+                'console_scripts': {
+                    'app': 'module:main',
+                    'tool': 'module:cli'
+                },
+                'plux.plugins': {
+                    'plugin1': 'pkg.module:Plugin1'
+                }
+            }
+        """
+        result = {}
+        for section_name in sorted(ep.keys()):
+            result[section_name] = {}
+            for entry_point in sorted(ep[section_name]):
+                name, value = entry_point.split("=")
+                result[section_name][name] = value
+        return result