diff --git a/bundles/org.openhab.automation.pythonscripting/README.md b/bundles/org.openhab.automation.pythonscripting/README.md index b4f92158d8651..ac74309a444bb 100644 --- a/bundles/org.openhab.automation.pythonscripting/README.md +++ b/bundles/org.openhab.automation.pythonscripting/README.md @@ -19,16 +19,16 @@ If you create an empty file called `test.py`, you will see a log line with infor ... [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/openhab/conf/automation/python/test.py' ``` -To enable debug logging, use the [console logging](https://openhab.org/docs/administration/logging.html) commands to +Use the [console logging](https://openhab.org/docs/administration/logging.html) commands to enable debug logging for the automation functionality: ```text log:set DEBUG org.openhab.automation.pythonscripting ``` -## Scripting Basics +### Rules -Lets start with a simple script +Let’s start with a simple script ```python from openhab import rule @@ -40,547 +40,238 @@ class Test: self.logger.info("Rule was triggered") ``` -or another one, using the [scope module](#module-scope) +### `PY` Transformation -```python -from openhab import rule -from openhab.triggers import ItemCommandTrigger +Or as transformation inline script -import scope - -@rule( triggers = [ ItemCommandTrigger("Item1", scope.ON) ] ) -class Test: - def execute(self, module, input): - self.logger.info("Rule was triggered") +```text +String Test "Test [PY(|'String has ' + str(len(input)) + 'characters'):%s]" ``` -::: tip Note -By default, the scope, Registry and logger is automatically imported for UI based rules -::: +### More Scripting -## `PY` Transformation +A complete Documentation about Python Scripting Rules and Transformation Scripts can be found at -openHAB provides several [data transformation services](https://www.openhab.org/addons/#transform) as well as the script transformations, that are available from the framework and need no additional installation. -It allows transforming values using any of the available scripting languages, which means Python Scripting is supported as well. -See the [transformation docs](https://openhab.org/docs/configuration/transformations.html#script-transformation) for more general information on the usage of script transformations. +[>> openHAB Python Scripting <<](https://github.com/openhab/openhab-python/blob/main/README.md) -Use Python Scripting as script transformation by: +including all examples above, much more detailed. -1. Creating a script in the `$OPENHAB_CONF/transform` folder with the `.py` extension. - The script should take one argument `input` and return a value that supports `toString()` or `null`: +## Add-on Administration - ```python - "String has " + str(len(input)) + " characters" - ``` +### Configuration - or +Web based config dialog can be found via Web UI => Settings / Add-on Settings / Python Scripting - ```python - def calc(input): - if input is None: - return 0 +![Pythonscripting configuration](doc/pythonscripting_configuration.png) - return "String has " + str(len(input)) + " characters" - calc(input) - ``` +Additionally, you can configure the Add-on via a config file `/openhab/services/pythonscripting.cfg` like below. -1. Using `PY(.py):%s` as Item state transformation. -1. Passing parameters is also possible by using a URL like syntax: `PY(.py?arg=value)`. - Parameters are injected into the script and can be referenced like variables. - -Simple transformations can also be given as an inline script: `PY(|...)`, e.g. `PY(|"String has " + str(len(input)) + "characters")`. -It should start with the `|` character, quotes within the script may need to be escaped with a backslash `\` when used with another quoted string as in text configurations. - -::: tip Note -By default, the scope, Registry and logger is automatically imported for `PY` Transformation scripts -::: - -## Examples - -### Simple rule - -```python -from openhab import rule, Registry -from openhab.triggers import GenericCronTrigger, ItemStateUpdateTrigger, ItemCommandTrigger, EphemerisCondition, when, onlyif - -import scope - -@rule() -@when("Time cron */5 * * * * ?") -def test1(module, input): - test1.logger.info("Rule 1 was triggered") - -@rule() -@when("Item Item1 received command") -@when("Item Item1 received update") -@onlyif("Today is a holiday") -def test2(module, input): - Registry.getItem("Item2").sendCommand(scope.ON) - -@rule( - triggers = [ GenericCronTrigger("*/5 * * * * ?") ] -) -class Test3: - def execute(self, module, input): - self.logger.info("Rule 3 was triggered") - -@rule( - triggers = [ - ItemStateUpdateTrigger("Item1"), - ItemCommandTrigger("Item1", scope.ON) - ], - conditions = [ - EphemerisCondition("notholiday") - ] -) -class Test4: - def execute(self, module, input): - if Registry.getItem("Item2").postUpdateIfDifferent(scope.OFF): - self.logger.info("Item2 was updated") -``` - -### Query thing status info - -```python -from openhab import logger, Registry - -info = Registry.getThing("zwave:serial_zstick:512").getStatusInfo() -logger.info(info.toString()); -``` - -### Query historic item - -```python -from openhab import logger, Registry -from datetime import datetime - -historicItem = Registry.getItem("Item1").getPersistence().persistedState( datetime.now().astimezone() ) -logger.info( historicItem.getState().toString() ); - -historicItem = Registry.getItem("Item2").getPersistence("jdbc").persistedState( datetime.now().astimezone() ) -logger.info( historicItem.getState().toString() ); -``` - -### Using scope - -Simple usage of jsr223 scope objects - -```python -from openhab import Registry - -from scope import ON - -Registry.getItem("Item1").sendCommand(ON) +```text +# Use scope and import wrapper +# +# This enables a scope module and and import wrapper. +# An scope module is an encapsulated module containing all openHAB jsr223 objects and can be imported with import scope +# Additionally you can run an import like from org.openhab.core import OpenHAB +# +#org.openhab.automation.pythonscripting:scopeEnabled = true + +# Install openHAB Python helper module (requires scope module) +# +# Install openHAB Python helper module to support helper classes like rule, logger, Registry, Timer etc... +# If disabled, the openHAB python helper module can be installed manually by copying it to /conf/automation/python/lib/openhab" +# +#org.openhab.automation.pythonscripting:helperEnabled = true + +# Inject scope and helper objects into rules (requires helper modules) +# +# This injects the scope and helper Registry and logger into rules. +# +# 2 => Auto injection enabled only for UI and Transformation scripts (preferred) +# 1 => Auto injection enabled for all scripts +# 0 => Disable auto injection and use 'import' statements instead +# +#org.openhab.automation.pythonscripting:injectionEnabled = 2 + +# Enables native modules (requires a manually configured venv) +# +# Native modules are sometimes necessary for pip modules which depends on native libraries. +# +#org.openhab.automation.pythonscripting:nativeModules = false + +# Enable dependency tracking +# +# Dependency tracking allows your scripts to automatically reload when one of its dependencies is updated. +# You may want to disable dependency tracking if you plan on editing or updating a shared library, but don't want all +# your scripts to reload until you can test it. +# +#org.openhab.automation.pythonscripting:dependencyTrackingEnabled = true + +# Cache compiled openHAB Python modules (.pyc files) +# +# Cache the openHAB python modules for improved startup performance.
+# Disable this option will result in a slower startup performance, because scripts have to be recompiled on every startup. +# +#org.openhab.automation.pythonscripting:cachingEnabled = true + +# Enable jython emulation +# +# This enables Jython emulation in GraalPy. It is strongly recommended to update code to GraalPy and Python 3 as the emulation can have performance degradation. +# For tips and instructions, please refer to Jython Migration Guide. +# +#org.openhab.automation.pythonscripting:jythonEmulation = false ``` -### Logging +::: tip Configuration note +If you use the marketplace version of this Add-on, it is necessary to use the config file. OpenHAB has a bug which prevents the web based config dialog to work correctly for `kar` file based Add-ons. +::: -There are 3 ways of logging. +### Console -1. using normal print statements. In this case they are redirected to the default openHAB logfile and marked with log level INFO or ERROR +The [openHAB Console](https://www.openhab.org/docs/administration/console.html) provides access to additional features of these Add-on. -```python -import sys +1. `pythonscripting info` is showing you additional data like version numbers, activated features and used path locations

![Pythonscripting info](doc/console_pythonscripting_info.png) -print("log message") +2. `pythonscripting console` provides an interactive python console where you can try live python features

![Pythonscripting console](doc/console_pythonscripting_console.png) -print("error message", file=sys.stderr) +3. `pythonscripting update` allows you to check, list, update or downgrade your helper lib

![Pythonscripting update](doc/console_pythonscripting_update.png) -``` +4. `pythonscripting pip` allows you check, install or remove external python modules.

Check [pip usage](#using-pip-to-install-external-modules) for details -1. using the logging module. Here you get a logging object, already initialized with the prefix "org.openhab.automation.pythonscripting" +5. `pythonscripting typing` generates python type hint stub files.

Check [python autocompletion](#enable-pyhton-autocompletion) for details -```python -from openhab import logging +### Enabling VEnv -logging.info("info message") +VEnv based python runtimes are optional, but needed to provide support for additional modules via 'pip' and for native modules. To activate this feature, simply follow the steps below. -logging.error("error message") -``` +1. Login into [openHAB console](https://www.openhab.org/docs/administration/console.html) and check your current pythonscripting environment by calling `pythonscripting info`

Important values are: -1. using the rule based logging module. Here you get a logging object, already initialized with the prefix "org.openhab.automation.pythonscripting.\" +- `GraalVM version: 24.2.1` +- `VEnv path: /openhab/userdata/cache/org.openhab.automation.pythonscripting/venv`

![Add-on informations](doc/venv_info.png) -```python -from openhab import rule -from openhab.triggers import GenericCronTrigger +These values are needed during the next step -@rule( triggers = [ GenericCronTrigger("*/5 * * * * ?") ] ) -class Test: - def execute(self, module, input): - self.logger.info("Rule was triggered") -``` +2. Download graalpy-community and create venv -## Decorators + ```shell + # The downloaded graalpy-community tar.gz must match your operating system (linux, windows or macos), your architecture (amd64, aarch64) and your "GraalVM version" of openHAB + wget -qO- https://github.com/oracle/graalpython/releases/download/graal-24.2.1/graalpy-community-24.2.1-linux-amd64.tar.gz | gunzip | tar xvf - + cd graalpy-community-24.2.1-linux-amd64/ -### decorator @rule + # The venv target dir must match your "VEnv path" of openHAB + ./bin/graalpy -m venv /openhab/userdata/cache/org.openhab.automation.pythonscripting/venv + ``` -The decorator will register the decorated class as a rule. -It will wrap and extend the class with the following functionalities +3. Install 'patchelf' which is needed for native module support in graalpy (optional). -- Register the class or function as a rule -- If name is not provided, a fallback name in the form "{filename}.{function_or_classname}" is created -- Triggers can be added with argument "triggers", with a function called "buildTriggers" (only classes) or with an [@when decorator](#decorator-when) -- Conditions can be added with argument "conditions", with a function called "buildConditions" (only classes) or with an [@onlyif decorator](#decorator-onlyif) -- The execute function is wrapped within a try / except to provide meaningful error logs -- A logger object (self.logger or {functionname}.logger) with the prefix "org.automation.pythonscripting.{filename}.{function_or_classname}" is available -- You can enable a profiler to analyze runtime with argument "profile=1" -- Every run is logging total runtime and trigger reasons + ``` + apt-get install patchelf + # zypper install patchelf + # yum install patchelf + ``` -```python -from openhab import rule -from openhab.triggers import GenericCronTrigger +After these steps, venv setup is done and will be detected automatically during next openHAB restart. -@rule( triggers = [ GenericCronTrigger("*/5 * * * * ?") ] ) -class Test: - def execute(self, module, input): - self.logger.info("Rule 3 was triggered") -``` +::: tip VEnv note +Theoretically you can create venvs with a native python installation too. But it is strongly recommended to use graalpy for it. It will install a "special" version of pip in this venv, which will install patched python modules if available. This increases the compatibility of python modules with graalpython. -```text -2025-01-09 09:35:11.002 [INFO ] [tomation.pythonscripting.demo1.Test2] - Rule executed in 0.1 ms [Item: Item1] -2025-01-09 09:35:15.472 [INFO ] [tomation.pythonscripting.demo1.Test1] - Rule executed in 0.1 ms [Other: TimerEvent] -``` - -**'execute'** callback **'input'** parameter - -Depending on which trigger type is used, corresponding [event objects](https://www.openhab.org/javadoc/latest/org/openhab/core/items/events/itemevent) are passed via the "input" parameter - -The type of the event can also be queried via [AbstractEvent.getTopic](https://www.openhab.org/javadoc/latest/org/openhab/core/events/abstractevent) - -### decorator @when - -```python -@when("Item Test_String_1 changed from 'old test string' to 'new test string'") -@when("Item gTest_Contact_Sensors changed") -@when("Member of gTest_Contact_Sensors changed from ON to OFF") -@when("Descendent of gTest_Contact_Sensors changed from OPEN to CLOSED") - -@when("Item Test_Switch_2 received update ON") -@when("Member of gTest_Switches received update") - -@when("Item Test_Switch_1 received command") -@when("Item Test_Switch_2 received command OFF") - -@when("Thing hue:device:default:lamp1 received update ONLINE") +In container environments, you should mount the 'graalpy' folder to, because the venv is using symbolik links. +::: -@when("Thing hue:device:default:lamp1 changed from ONLINE to OFFLINE") +### Using pip to install external modules -@when("Channel hue:device:default:lamp1:color triggered START") +As first, you must enable [VEnv](#enabling-venv). After this is enabled, you can use pip in 2 ways. -@when("System started") -@when("System reached start level 50") +1. Using the pythonscripting console

![Pythonscripting pip install](doc/console_pythonscripting_pip_install.png)![Pythonscripting pip install](doc/console_pythonscripting_pip_list.png)![Pythonscripting pip install](doc/console_pythonscripting_pip_show.png) -@when("Time cron 55 55 5 * * ?") -@when("Time is midnight") -@when("Time is noon") +2. Using venv pip on your host system -@when("Time is 10:50") + ``` + /openhab/userdata/cache/org.openhab.automation.pythonscripting/venv/bin/pip install requests + ``` -@when("Datetime is Test_Datetime_1") -@when("Datetime is Test_Datetime_2 time only") +### Enable Python Autocompletion -@when("Item added") -@when("Item removed") -@when("Item updated") +Before you can enable autocompletion, you must generate the required type hint stub files. Login into [openHAB console](https://www.openhab.org/docs/administration/console.html) and run -@when("Thing added") -@when("Thing removed") -@when("Thing updated") ``` - -### decorator @onlyif - -```python -@onlyif("Item Test_Switch_2 equals ON") -@onlyif("Today is a holiday") -@onlyif("It's not a holiday") -@onlyif("Tomorrow is not a holiday") -@onlyif("Today plus 1 is weekend") -@onlyif("Today minus 1 is weekday") -@onlyif("Today plus 3 is a weekend") -@onlyif("Today offset -3 is a weekend") -@onylyf("Today minus 3 is not a holiday") -@onlyif("Yesterday was in dayset") -@onlyif("Time 9:00 to 14:00") +pythonscripting typing ``` -## Modules +This will scan your current openHAB instance, including all installed Add-ons, for public java class methods and create corresponding python type hint stub files. -### module scope +![Pythonscripting typing](doc/console_pythonscripting_typing.png) -The scope module encapsulates all [default jsr223 objects/presents](https://www.openhab.org/docs/configuration/jsr223.html#default-preset-importpreset-not-required) into a new object. -You can use it like below +The files are stored in the folder `/openhab/conf/automation/python/typings/`. -```python -from scope import * # this makes all jsr223 objects available +As a final step, the folders `/openhab/conf/automation/python/libs/` and `/openhab>/conf/automation/python/typings/` must be added as "extraPaths" in your IDE. -print(ON) -``` +![Pythonscripting autocompletion](doc/ide_autocompletion.png) -```python -from scope import ON, OFF # this imports specific jsr223 objects +## Typical log errors -print(ON) -``` +### Graal python language not initialized. ... -```python -import scope # this imports just the module - -print(scope.ON) +```log +2025-07-25 12:10:06.001 [ERROR] [g.internal.PythonScriptEngineFactory] - Graal python language not initialized. Restart openhab to initialize available graal languages properly. ``` -You can also import additional [jsr223 presents](https://www.openhab.org/docs/configuration/jsr223.html#rulesimple-preset) like - -```python -from scope import RuleSimple -from scope import RuleSupport -from scope import RuleFactories -from scope import ScriptAction -from scope import cache -from scope import osgi -``` +This can happen after a new Add-on installation, if JavaScript Scripting is active at the same time. -Additionally you can import all Java classes from 'org.openhab' package like +Just restart openhab to initialize available graal languages properly. -```python -from org.openhab.core import OpenHAB +### User timezone 'XYZ' is different than openhab regional timezone ... -print(str(OpenHAB.getVersion())) +```log +2025-07-22 09:15:53.705 [WARN ] [g.internal.PythonScriptEngineFactory] - User timezone 'Europe/London' is different than openhab regional timezone 'Europe/Berlin'. Python Scripting is running with timezone 'Europe/London'. ``` -### module openhab - -| Class | Usage | Description | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| rule | @rule( name=None, description=None, tags=None, triggers=None, conditions=None, profile=None) | [Rule decorator](#decorator-rule) to wrap a custom class into a rule | -| logger | logger.info, logger.warn ... | Logger object with prefix 'org.automation.pythonscripting.{filename}' | -| Registry | see [Registry class](#class-registry) | Static Registry class used to get items, things or channels | -| Timer | see [Timer class](#class-timer) | Static Timer class to create, start and stop timers | -| Set | see [Set class](#class-set) | Set object | - -### module openhab.actions - -| Class | Usage | Description | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| Audio | see [openHAB Audio API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/audio) | | -| BusEvent | see [openHAB BusEvent API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/busevent) | | -| Ephemeris | see [openHAB Ephemeris API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/ephemeris) | | -| Exec | see [openHAB Exec API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/exec) | e.g. Exec.executeCommandLine(timedelta(seconds=1), "whoami") | -| HTTP | see [openHAB HTTP API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/http) | | -| Log | see [openHAB Log API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/log) | | -| Ping | see [openHAB Ping API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/ping) | | -| ScriptExecution | see [openHAB ScriptExecution API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/scriptexecution) | | -| Semantic | see [openHAB Semantic API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/semantic) | | -| Things | see [openHAB Things API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/things) | | -| Transformation | see [openHAB Transformation API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/transformation) | | -| Voice | see [openHAB Voice API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/voice) | | -| NotificationAction | | e.g. NotificationAction.sendNotification("test\@test.org", "Window is open") | - -### module openhab.triggers - -| Class | Usage | Description | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| when | @when(term_as_string) | [When trigger decorator](#decorator-when) to create a trigger by a term | -| onlyif | @onlyif(term_as_string) | [Onlyif condition decorator](#decorator-onlyif) to create a condition by a term | -| ChannelEventTrigger | ChannelEventTrigger(channel_uid, event=None, trigger_name=None) | | -| ItemStateUpdateTrigger | ItemStateUpdateTrigger(item_name, state=None, trigger_name=None) | | -| ItemStateChangeTrigger | ItemStateChangeTrigger(item_name, state=None, previous_state=None, trigger_name=None) | | -| ItemCommandTrigger | ItemCommandTrigger(item_name, command=None, trigger_name=None) | | -| GroupStateUpdateTrigger | GroupStateUpdateTrigger(group_name, state=None, trigger_name=None) | | -| GroupStateChangeTrigger | GroupStateChangeTrigger(group_name, state=None, previous_state=None, trigger_name=None)| | -| GroupCommandTrigger | GroupCommandTrigger(group_name, command=None, trigger_name=None) | | -| ThingStatusUpdateTrigger | ThingStatusUpdateTrigger(thing_uid, status=None, trigger_name=None) | | -| ThingStatusChangeTrigger | ThingStatusChangeTrigger(thing_uid, status=None, previous_status=None, trigger_name=None)| | -| SystemStartlevelTrigger | SystemStartlevelTrigger(startlevel, trigger_name=None) | for startlevel see [openHAB StartLevelService API](https://www.openhab.org/javadoc/latest/org/openhab/core/service/startlevelservice#) | -| GenericCronTrigger | GenericCronTrigger(cron_expression, trigger_name=None) | | -| TimeOfDayTrigger | TimeOfDayTrigger(time, trigger_name=None) | | -| DateTimeTrigger | DateTimeTrigger(cron_expression, trigger_name=None) | | -| PWMTrigger | PWMTrigger(cron_expression, trigger_name=None) | | -| GenericEventTrigger | GenericEventTrigger(event_source, event_types, event_topic="\*/\*", trigger_name=None) | | -| ItemEventTrigger | ItemEventTrigger(event_types, item_name=None, trigger_name=None) | | -| ThingEventTrigger | ThingEventTrigger(event_types, thing_uid=None, trigger_name=None) | | -| | | | -| ItemStateCondition | ItemStateCondition(item_name, operator, state, condition_name=None) | | -| EphemerisCondition | EphemerisCondition(dayset, offset=0, condition_name=None) | | -| TimeOfDayCondition | TimeOfDayCondition(start_time, end_time, condition_name=None) | | -| IntervalCondition | IntervalCondition(min_interval, condition_name=None) | | - -## Classes - -### class Registry - -| Function | Usage | Return Value | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| getThing | getThing(uid) | [Thing](#class-thing) | -| getChannel | getChannel(uid) | [Channel](#class-channel) | -| getItemMetadata | getItemMetadata(item_or_item_name, namespace) | [openHAB Metadata](https://www.openhab.org/javadoc/latest/org/openhab/core/items/metadata) | -| setItemMetadata | setItemMetadata(item_or_item_name, namespace, value, configuration=None) | [openHAB Metadata](https://www.openhab.org/javadoc/latest/org/openhab/core/items/metadata) | -| removeItemMetadata | removeItemMetadata(item_or_item_name, namespace = None) | [openHAB Metadata](https://www.openhab.org/javadoc/latest/org/openhab/core/items/metadata) | -| getItemState | getItemState(item_name, default = None) | [openHAB State](https://www.openhab.org/javadoc/latest/org/openhab/core/types/state) | -| getItem | getItem(item_name) | [Item](#class-item) or [GroupItem](#class-groupitem) | -| resolveItem | resolveItem(item_or_item_name) | [Item](#class-item) or [GroupItem](#class-groupitem) | -| addItem | addItem(item_config) | [Item](#class-item) or [GroupItem](#class-groupitem) | -| safeItemName | safeItemName(item_name) | | - -### class Item - -Item is a wrapper around [openHAB Item](https://www.openhab.org/javadoc/latest/org/openhab/core/items/item) with additional functionality. - -| Function | Usage | Return Value | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| postUpdate | postUpdate(state) | | -| postUpdateIfDifferent | postUpdateIfDifferent(state) | | -| sendCommand | sendCommand(command) | | -| sendCommandIfDifferent | sendCommandIfDifferent(command) | | -| getPersistence | getPersistence(service_id = None) | [ItemPersistence](#class-itempersistence) | -| getSemantic | getSemantic() | [ItemSemantic](#class-itemsemantic) | -| <...> | see [openHAB Item API](https://www.openhab.org/javadoc/latest/org/openhab/core/items/item) | | - -### class GroupItem - -GroupItem is an extended [Item](#class-item) which wraps results from getAllMembers & getMembers into [Items](#class-item) - -### class ItemPersistence - -ItemPersistence is a wrapper around [openHAB PersistenceExtensions](https://www.openhab.org/javadoc/latest/org/openhab/core/persistence/extensions/persistenceextensions). The parameters 'item' and 'serviceId', as part of the Wrapped Java API, are not needed, because they are inserted automatically. - -| Function | Usage | Description | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| getStableMinMaxState | getStableMinMaxState(time_slot, end_time = None) | Average calculation which takes into account the values depending on their duration | -| getStableState | getStableState(time_slot, end_time = None) | Average calculation which takes into account the values depending on their duration | -| <...> | see [openHAB PersistenceExtensions API](https://www.openhab.org/javadoc/latest/org/openhab/core/persistence/extensions/persistenceextensions) | | - -### class ItemSemantic - -ItemSemantic is a wrapper around [openHAB Semantics](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/semantics). The parameters 'item', as part of the Wrapped Java API, is not needed because it is inserted automatically. - -| Function | Usage | -| ------------------------ | ------------------------------------------------------------------------------------- | -| <...> | see [openHAB Semantics API](https://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/semantics) | - -### class Thing - -Thing is a wrapper around [openHAB Thing](https://www.openhab.org/javadoc/latest/org/openhab/core/thing/thing). - -| Function | Usage | -| ------------------------ | ------------------------------------------------------------------------------------- | -| <...> | see [openHAB Thing API](https://www.openhab.org/javadoc/latest/org/openhab/core/thing/thing) | - -### class Channel - -Channel is a wrapper around [openHAB Channel](https://www.openhab.org/javadoc/latest/org/openhab/core/thing/type/channelgrouptype). - -| Function | Usage | -| ------------------------ | ------------------------------------------------------------------------------------- | -| <...> | see [openHAB Channel API](https://www.openhab.org/javadoc/latest/org/openhab/core/thing/type/channelgrouptype) | - -### class Timer - -| Function | Usage | Description | -| ------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| createTimeout | createTimeout(duration, callback, args=[], kwargs={}, old_timer = None, max_count = 0 ) | Create a timer that will run callback with arguments args and keyword arguments kwargs, after duration seconds have passed. If old_timer from e.g previous call is provided, it will be stopped if not already triggered. If max_count together with old_timer is provided, then 'max_count' times the old timer will be stopped and recreated, before the callback will be triggered immediately | +These error happens if timezone settings are provided in several ways and some of them are different. -### class Set +- Check that your EXTRA_JAVA_OPTS="-Duser.timezone=" setting is matching your openHAB regional setting. +- Additionally the ENVIRONMENT variable 'TZ', if provided, must match your openHAB regional setting. -This is a helper class which makes it possible to use a Python 'set' as an argument for Java class method calls +e.g. in openHABian this can be changed in /etc/default/openhab -## Others +or for containers, this can be provided as a additional environment variable. -### Threading +### Can't install pip modules. VEnv not enabled. -Thread or timer objects which was started by itself should be registered in the lifecycleTracker to be cleaned during script unload. - -```python -import scope -import threading - -class Timer(theading.Timer): - def __init__(self, duration, callback): - super().__init__(duration, callback) - - def shutdown(self): - if not self.is_alive(): - return - self.cancel() - self.join() - -def test(): - print("timer triggered") - -job = Timer(60, test) -job.start() - -scope.lifecycleTracker.addDisposeHook(job.shutdown) +```log +2025-07-22 09:19:05.759 [ERROR] [rnal.PythonScriptEngineConfiguration] - Can't install pip modules. VEnv not enabled. ``` -Timer objects created via `openhab.Timer.createTimeout`, however, automatically register in the disposeHook and are cleaned on script unload. - -```python -from openhab import Timer - -def test(): - print("timer triggered") - -Timer.createTimeout(60, test) -``` - -Below is a complex example of 2 sensor values that are expected to be transmitted in a certain time window (e.g. one after the other). - -After the first state change, the timer wait 5 seconds, before it updates the final target value. -If the second value arrives before this time frame, the final target value is updated immediately. +You configured preinstalled pip modules, but the mandatory VEnv setup is not initialized or detected. Please confirm the correct setup, by following the steps about [Enabling VEnv](#enabling-venv) -```python -from openhab import rule, Registry -from openhab.triggers import ItemStateChangeTrigger - -@rule( - triggers = [ - ItemStateChangeTrigger("Room_Temperature_Value"), - ItemStateChangeTrigger("Room_Humidity_Value") - ] -) -class UpdateInfo: - def __init__(self): - self.update_timer = None - - def updateInfoMessage(self): - msg = "{}{} °C, {} %".format(Registry.getItemState("Room_Temperature_Value").format("%.1f"), Registry.getItemState("Room_Temperature_Value").format("%.0f")) - Registry.getItem("Room_Info").postUpdate(msg) - self.update_timer = None - - def execute(self, module, input): - self.update_timer = Timer.createTimeout(5, self.updateInfoMessage, old_timer = self.update_timer, max_count=2 ) +### Exception during helper lib initialisation +```log +2025-07-20 09:15:05.100 [ERROR] [rnal.PythonScriptEngineConfiguration] - Exception during helper lib initialisation ``` -### Python <=> Java conversion - -In addition to standard [value type mappings](https://www.graalvm.org/python/docs/#mapping-types-between-python-and-other-languages), the following type mappings are available. - -| Python class | Java class | -| ------------------------- | ------------- | -| datetime with timezone | ZonedDateTime | -| datetime without timezone | Instant | -| timedelta | Duration | -| list | Set | -| Item | Item | - -### Typical log errors - -#### Exception during helper lib initialisation - There were problems during the deployment of the helper libs. A typical error is an insufficient permission. The folder "conf/automation/python/" must be writeable by openHAB. -#### Failed to inject import wrapper +### Failed to inject import wrapper for engine ... + +```log +2025-07-20 10:01:17.211 [ERROR] [cripting.internal.PythonScriptEngine] - Failed to inject import wrapper for engine +``` -The reading the Python source file "conf/automation/python/lib/openhab/\_\_wrapper\_\_py" failed. +The reading the Python source file "conf/automation/python/lib/openhab/\_\_wrapper\_\_.py" failed. This could either a permission/owner problem or a problem during deployment of the helper libs. You should check that this file exists and it is readable by openHAB. You should also check your logs for a message related to the helper lib deployment by just grep for "helper lib". -### Limitations +### SystemError, Option python.NativeModules is set to 'true' and a second GraalPy context ... + +```log +SystemError, Option python.NativeModules is set to 'true' and a second GraalPy context attempted to load a native module '' from path '.so'. At least one context in this process runs with 'IsolateNativeModules' set to false. Depending on the order of context creation, this means some contexts in the process cannot use native module, all other contexts must fall back and set python.NativeModules to 'false' to run native extensions in LLVM mode. This is recommended only for extensions included in the Python standard library. Running a 3rd party extension in LLVM mode requires a custom build of the extension and is generally discouraged due to compatibility reasons. +``` + +These errors can occur if you use a native library in your external module but forgot to enable "native modules". Check the [Add-on configuration](#configuration) and enable "native modules". + +## Limitations -- GraalPy can't handle arguments in constructors of Java objects. Means you can't instantiate a Java object in Python with a parameter. +- GraalPy can't handle arguments in constructors of Java objects. Means you can't instantiate a Java object in Python with a parameter. https://github.com/oracle/graalpython/issues/367 diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_console.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_console.png new file mode 100644 index 0000000000000..ce3f2fce52101 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_console.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_info.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_info.png new file mode 100644 index 0000000000000..946701d47cd02 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_info.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_install.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_install.png new file mode 100644 index 0000000000000..d02253ea4e1d5 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_install.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_list.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_list.png new file mode 100644 index 0000000000000..ab34132d9972f Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_list.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_show.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_show.png new file mode 100644 index 0000000000000..3479bb02c22f4 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_pip_show.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_typing.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_typing.png new file mode 100644 index 0000000000000..4551d809fb484 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_typing.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_update.png b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_update.png new file mode 100644 index 0000000000000..eb09fa6a597c9 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/console_pythonscripting_update.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/ide_autocompletion.png b/bundles/org.openhab.automation.pythonscripting/doc/ide_autocompletion.png new file mode 100644 index 0000000000000..f9fcf603cd569 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/ide_autocompletion.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/pythonscripting_configuration.png b/bundles/org.openhab.automation.pythonscripting/doc/pythonscripting_configuration.png new file mode 100644 index 0000000000000..b0ab4d66a98dc Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/pythonscripting_configuration.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/doc/venv_info.png b/bundles/org.openhab.automation.pythonscripting/doc/venv_info.png new file mode 100644 index 0000000000000..b181e7ffbd294 Binary files /dev/null and b/bundles/org.openhab.automation.pythonscripting/doc/venv_info.png differ diff --git a/bundles/org.openhab.automation.pythonscripting/pom.xml b/bundles/org.openhab.automation.pythonscripting/pom.xml index 3d557552e2421..df5126faced5a 100644 --- a/bundles/org.openhab.automation.pythonscripting/pom.xml +++ b/bundles/org.openhab.automation.pythonscripting/pom.xml @@ -15,7 +15,7 @@ openHAB Add-ons :: Bundles :: Automation :: Python Scripting - v1.0.0 + 1.0.8 @@ -31,6 +31,22 @@ + + org.codehaus.mojo + properties-maven-plugin + 1.0.0 + + + + write-project-properties + + generate-resources + + ${project.build.outputDirectory}/build.properties + + + + org.apache.maven.plugins maven-scm-plugin diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/feature/feature.xml b/bundles/org.openhab.automation.pythonscripting/src/main/feature/feature.xml index 74a9fc1d09a36..9840d182a0089 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/feature/feature.xml +++ b/bundles/org.openhab.automation.pythonscripting/src/main/feature/feature.xml @@ -4,24 +4,24 @@ openhab-runtime-base - mvn:org.openhab.osgiify/org.graalvm.llvm.llvm-api/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.polyglot.polyglot/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.python.python-language/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.python.python-resources/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.regex.regex/24.2.1 mvn:org.openhab.osgiify/org.graalvm.sdk.collections/24.2.1 mvn:org.openhab.osgiify/org.graalvm.sdk.jniutils/24.2.1 mvn:org.openhab.osgiify/org.graalvm.sdk.nativeimage/24.2.1 mvn:org.openhab.osgiify/org.graalvm.sdk.word/24.2.1 mvn:org.openhab.osgiify/org.graalvm.shadowed.icu4j/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.llvm.llvm-api/24.2.1 mvn:org.openhab.osgiify/org.graalvm.shadowed.json/24.2.1 mvn:org.openhab.osgiify/org.graalvm.shadowed.xz/24.2.1 mvn:org.openhab.osgiify/org.graalvm.tools.profiler-tool/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-api/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.polyglot.polyglot/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-runtime/24.2.1 mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-compiler/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.regex.regex/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.python.python-resources/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.python.python-language/24.2.1 mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-nfi/24.2.1 mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-nfi-libffi/24.2.1 - mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-runtime/24.2.1 + mvn:org.openhab.osgiify/org.graalvm.truffle.truffle-api/24.2.1 mvn:org.openhab.addons.bundles/org.openhab.automation.pythonscripting/${project.version} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngine.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngine.java index 688c44882acbe..54c486982a133 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngine.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngine.java @@ -17,19 +17,18 @@ import java.io.File; import java.io.IOException; -import java.nio.file.AccessMode; -import java.nio.file.FileSystems; +import java.io.InputStream; +import java.io.OutputStream; import java.nio.file.Files; -import java.nio.file.LinkOption; import java.nio.file.Path; import java.nio.file.Paths; import java.time.Duration; import java.time.Instant; +import java.time.OffsetDateTime; import java.time.ZonedDateTime; import java.util.Arrays; import java.util.HashSet; import java.util.List; -import java.util.Properties; import java.util.Set; import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.Condition; @@ -46,19 +45,26 @@ import org.graalvm.polyglot.Context; import org.graalvm.polyglot.Engine; import org.graalvm.polyglot.HostAccess; +import org.graalvm.polyglot.Language; +import org.graalvm.polyglot.PolyglotException; import org.graalvm.polyglot.Source; import org.graalvm.polyglot.Value; import org.graalvm.polyglot.io.IOAccess; +import org.openhab.automation.pythonscripting.internal.context.ContextInput; +import org.openhab.automation.pythonscripting.internal.context.ContextOutput; +import org.openhab.automation.pythonscripting.internal.context.ContextOutputLogger; import org.openhab.automation.pythonscripting.internal.fs.DelegatingFileSystem; -import org.openhab.automation.pythonscripting.internal.fs.watch.PythonDependencyTracker; -import org.openhab.automation.pythonscripting.internal.graal.GraalPythonScriptEngine; -import org.openhab.automation.pythonscripting.internal.scriptengine.InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable; -import org.openhab.automation.pythonscripting.internal.scriptengine.helper.LifecycleTracker; -import org.openhab.automation.pythonscripting.internal.scriptengine.helper.LogOutputStream; -import org.openhab.automation.pythonscripting.internal.wrapper.ScriptExtensionModuleProvider; -import org.openhab.core.OpenHAB; +import org.openhab.automation.pythonscripting.internal.provider.LifecycleTracker; +import org.openhab.automation.pythonscripting.internal.provider.ScriptExtensionModuleProvider; +import org.openhab.automation.pythonscripting.internal.scriptengine.InvocationInterceptingPythonScriptEngine; +import org.openhab.automation.pythonscripting.internal.scriptengine.graal.GraalPythonScriptEngine; import org.openhab.core.automation.module.script.ScriptExtensionAccessor; -import org.openhab.core.items.Item; +import org.openhab.core.library.types.DateTimeType; +import org.openhab.core.library.types.DecimalType; +import org.openhab.core.library.types.OnOffType; +import org.openhab.core.library.types.StringType; +import org.openhab.core.types.State; +import org.openhab.core.types.UnDefType; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.slf4j.event.Level; @@ -69,11 +75,15 @@ * @author Holger Hees - Initial contribution * @author Jeff James - Initial contribution */ -public class PythonScriptEngine - extends InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable - implements Lock { +public class PythonScriptEngine extends InvocationInterceptingPythonScriptEngine implements Lock { private final Logger logger = LoggerFactory.getLogger(PythonScriptEngine.class); + public static final String CONTEXT_KEY_ENGINE_LOGGER_OUTPUT = "ctx.engine-logger-output"; + public static final String CONTEXT_KEY_ENGINE_LOGGER_INPUT = "ctx.engine-logger-input"; + private static final String CONTEXT_KEY_SCRIPT_FILENAME = "javax.script.filename"; + + private static final String PYTHON_OPTION_ENGINE_WARNINTERPRETERONLY = "engine.WarnInterpreterOnly"; + private static final String SYSTEM_PROPERTY_ATTACH_LIBRARY_FAILURE_ACTION = "polyglotimpl.AttachLibraryFailureAction"; private static final String PYTHON_OPTION_PYTHONPATH = "python.PythonPath"; @@ -84,34 +94,38 @@ public class PythonScriptEngine private static final String PYTHON_OPTION_CHECKHASHPYCSMODE = "python.CheckHashPycsMode"; private static final String PYTHON_OPTION_ALWAYSRUNEXCEPTHOOK = "python.AlwaysRunExcepthook"; + private static final String PYTHON_OPTION_EXECUTABLE = "python.Executable"; + // private static final String PYTHON_OPTION_PYTHONHOME = "python.PythonHome"; + // private static final String PYTHON_OPTION_SYSPREFIX = "python.SysPrefix"; + private static final String PYTHON_OPTION_NATIVEMODULES = "python.NativeModules"; + private static final String PYTHON_OPTION_ISOLATENATIVEMODULES = "python.IsolateNativeModules"; + private static final String PYTHON_OPTION_CACHEDIR = "python.PyCachePrefix"; - private static final String PYTHON_CACHEDIR_PATH = Paths - .get(OpenHAB.getUserDataFolder(), "cache", PythonScriptEngine.class.getPackageName(), "cachedir") - .toString(); private static final int STACK_TRACE_LENGTH = 5; - public static final String LOGGER_INIT_NAME = "__logger_init__"; + private static final String LOGGER_INIT_NAME = "__logger_init__"; /** Shared Polyglot {@link Engine} across all instances of {@link PythonScriptEngine} */ - private static final Engine ENGINE = Engine.newBuilder().allowExperimentalOptions(true) - .option("engine.WarnInterpreterOnly", "false").build(); + private static Engine engine = Engine.newBuilder() + // disable warning about fallback runtime (is only available in graalvm) + .option(PYTHON_OPTION_ENGINE_WARNINTERPRETERONLY, Boolean.toString(false)).build(); + + static { + // disable warning about missing TruffleAttach library (is only available in graalvm) + System.getProperties().setProperty(SYSTEM_PROPERTY_ATTACH_LIBRARY_FAILURE_ACTION, "ignore"); + } + + // private static final boolean isPosix = FileSystems.getDefault().supportedFileAttributeViews().contains("posix"); /** Provides unlimited host access as well as custom translations from Python to Java Objects */ private static final HostAccess HOST_ACCESS = HostAccess.newBuilder(HostAccess.ALL) - // Translate python datetime with timezone to java.time.ZonedDateTime - .targetTypeMapping(Value.class, ZonedDateTime.class, - v -> v.hasMember("ctime") && v.hasMember("isoformat") && v.hasMember("tzinfo") - && !v.getMember("tzinfo").isNull(), - v -> ZonedDateTime.parse(v.invokeMember("isoformat").asString()), - HostAccess.TargetMappingPrecedence.LOW) + .targetTypeMapping(Value.class, ZonedDateTime.class, v -> v.hasMember("ctime") && v.hasMember("isoformat"), + v -> PythonScriptEngine.parseDatetime(v), HostAccess.TargetMappingPrecedence.LOW) - // Translate python datetime without timezone to java.time.Instant - .targetTypeMapping(Value.class, Instant.class, - v -> v.hasMember("ctime") && v.hasMember("isoformat") && v.hasMember("tzinfo") - && v.getMember("tzinfo").isNull(), - v -> Instant.parse(v.invokeMember("isoformat").asString() + "Z"), - HostAccess.TargetMappingPrecedence.LOW) + // Translate python datetime java.time.Instant + .targetTypeMapping(Value.class, Instant.class, v -> v.hasMember("ctime") && v.hasMember("isoformat"), + v -> PythonScriptEngine.parseDatetime(v).toInstant(), HostAccess.TargetMappingPrecedence.LOW) // Translate python timedelta to java.time.Duration .targetTypeMapping(Value.class, Duration.class, @@ -120,70 +134,56 @@ public class PythonScriptEngine v -> Duration.ofNanos(Math.round(v.invokeMember("total_seconds").asDouble() * 1000000000)), HostAccess.TargetMappingPrecedence.LOW) - // Translate python item to org.openhab.core.items.Item - .targetTypeMapping(Value.class, Item.class, v -> v.hasMember("raw_item"), - v -> v.getMember("raw_item").as(Item.class), HostAccess.TargetMappingPrecedence.LOW) - // Translate python array to java.util.Set .targetTypeMapping(Value.class, Set.class, v -> v.hasArrayElements(), PythonScriptEngine::transformArrayToSet, HostAccess.TargetMappingPrecedence.LOW) + // Translate python values to State + .targetTypeMapping(Value.class, State.class, null, PythonScriptEngine::transformValueToState, + HostAccess.TargetMappingPrecedence.LOW) + .build(); /** {@link Lock} synchronization of multi-thread access */ private final Lock lock = new ReentrantLock(); - // these fields start as null because they are populated on first use - private @Nullable Consumer scriptDependencyListener; - private final ScriptExtensionModuleProvider scriptExtensionModuleProvider; - private final LifecycleTracker lifecycleTracker; - private PythonScriptEngineConfiguration pythonScriptEngineConfiguration; private boolean initialized = false; - private boolean closed = false; - private final LogOutputStream scriptOutputStream; - private final LogOutputStream scriptErrorStream; + private String engineIdentifier = ""; + + private final ContextOutput scriptOutputStream; + private final ContextOutput scriptErrorStream; + private final ContextInput scriptInputStream; + + private final DelegatingFileSystem delegatingFileSystem; + + private final ScriptExtensionModuleProvider scriptExtensionModuleProvider; + private final LifecycleTracker lifecycleTracker; /** * Creates an implementation of ScriptEngine {@code (& Invocable)}, wrapping the contained engine, * that tracks the script lifecycle and provides hooks for scripts to do so too. */ - public PythonScriptEngine(PythonDependencyTracker pythonDependencyTracker, - PythonScriptEngineConfiguration pythonScriptEngineConfiguration) { + public PythonScriptEngine(PythonScriptEngineConfiguration pythonScriptEngineConfiguration, + PythonScriptEngineFactory pythonScriptEngineFactory) { this.pythonScriptEngineConfiguration = pythonScriptEngineConfiguration; - scriptOutputStream = new LogOutputStream(logger, Level.INFO); - scriptErrorStream = new LogOutputStream(logger, Level.ERROR); + this.scriptOutputStream = new ContextOutput(new ContextOutputLogger(logger, Level.INFO)); + this.scriptErrorStream = new ContextOutput(new ContextOutputLogger(logger, Level.ERROR)); + this.scriptInputStream = new ContextInput(null); - lifecycleTracker = new LifecycleTracker(); - scriptExtensionModuleProvider = new ScriptExtensionModuleProvider(); + this.lifecycleTracker = new LifecycleTracker(); + this.scriptExtensionModuleProvider = new ScriptExtensionModuleProvider(); - // disable warning about missing TruffleAttach library (is only available in graalvm) - Properties props = System.getProperties(); - props.setProperty(SYSTEM_PROPERTY_ATTACH_LIBRARY_FAILURE_ACTION, "ignore"); + this.delegatingFileSystem = new DelegatingFileSystem(pythonScriptEngineConfiguration.getTempDirectory()); Context.Builder contextConfig = Context.newBuilder(GraalPythonScriptEngine.LANGUAGE_ID) // .out(scriptOutputStream) // .err(scriptErrorStream) // - .allowIO(IOAccess.newBuilder() // - .allowHostSocketAccess(true) // - .fileSystem(new DelegatingFileSystem(FileSystems.getDefault().provider()) { - @Override - public void checkAccess(Path path, Set modes, - LinkOption... linkOptions) throws IOException { - if (pythonScriptEngineConfiguration.isDependencyTrackingEnabled()) { - if (path.startsWith(PythonScriptEngineFactory.PYTHON_LIB_PATH)) { - Consumer localScriptDependencyListener = scriptDependencyListener; - if (localScriptDependencyListener != null) { - localScriptDependencyListener.accept(path.toString()); - } - } - } - super.checkAccess(path, modes, linkOptions); - } - }).build()) // + .in(scriptInputStream) // + .allowIO(IOAccess.newBuilder().allowHostSocketAccess(true).fileSystem(delegatingFileSystem).build()) // .allowHostAccess(HOST_ACCESS) // // usage of .allowAllAccess(true) includes // - allowCreateThread(true) @@ -211,138 +211,199 @@ public void checkAccess(Path path, Set modes, String.valueOf(pythonScriptEngineConfiguration.isJythonEmulation())) // Set python path to point to sources stored in - .option(PYTHON_OPTION_PYTHONPATH, PythonScriptEngineFactory.PYTHON_LIB_PATH.toString() - + File.pathSeparator + PythonScriptEngineFactory.PYTHON_DEFAULT_PATH.toString()); + .option(PYTHON_OPTION_PYTHONPATH, PythonScriptEngineConfiguration.PYTHON_LIB_PATH.toString() + + File.pathSeparator + PythonScriptEngineConfiguration.PYTHON_DEFAULT_PATH.toString()); + + if (pythonScriptEngineConfiguration.isVEnvEnabled()) { + @SuppressWarnings("null") + String venvExecutable = pythonScriptEngineConfiguration.getVEnvExecutable().toString(); + contextConfig = contextConfig.option(PYTHON_OPTION_EXECUTABLE, venvExecutable); + // Path venvPath = this.pythonScriptEngineConfiguration.getVEnvDirectory(); + // .option(PYTHON_OPTION_PYTHONHOME, venvPath.toString()) // + // .option(PYTHON_OPTION_SYSPREFIX, venvPath.toString()) // + + if (pythonScriptEngineConfiguration.isNativeModulesEnabled()) { + contextConfig = contextConfig.option(PYTHON_OPTION_NATIVEMODULES, Boolean.toString(true)) // + .option(PYTHON_OPTION_ISOLATENATIVEMODULES, Boolean.toString(true)); + } + } if (pythonScriptEngineConfiguration.isCachingEnabled()) { contextConfig.option(PYTHON_OPTION_DONTWRITEBYTECODEFLAG, Boolean.toString(false)) // - .option(PYTHON_OPTION_CACHEDIR, PYTHON_CACHEDIR_PATH); + .option(PYTHON_OPTION_CACHEDIR, pythonScriptEngineConfiguration.getBytecodeDirectory().toString()); } else { contextConfig.option(PYTHON_OPTION_DONTWRITEBYTECODEFLAG, Boolean.toString(true)) // // causes the interpreter to always assume hash-based pycs are valid .option(PYTHON_OPTION_CHECKHASHPYCSMODE, "never"); } - delegate = GraalPythonScriptEngine.create(ENGINE, contextConfig); + init(engine, contextConfig, pythonScriptEngineFactory); } @Override - protected void beforeInvocation() { + protected void beforeInvocation() throws PolyglotException { lock.lock(); - logger.debug("Lock acquired before invocation."); + logger.debug("Lock acquired before invocation for engine '{}'", this.engineIdentifier); if (initialized) { return; } - logger.debug("Initializing GraalPython script engine..."); - - ScriptContext ctx = getScriptContext(); + ScriptContext ctx = getContext(); // these are added post-construction, so we need to fetch them late String engineIdentifier = (String) ctx.getAttribute(CONTEXT_KEY_ENGINE_IDENTIFIER); - if (engineIdentifier == null) { - throw new IllegalStateException("Failed to retrieve engine identifier from engine bindings"); - } - - ScriptExtensionAccessor scriptExtensionAccessor = (ScriptExtensionAccessor) ctx - .getAttribute(CONTEXT_KEY_EXTENSION_ACCESSOR); - if (scriptExtensionAccessor == null) { - throw new IllegalStateException("Failed to retrieve script extension accessor from engine bindings"); + if (engineIdentifier != null) { + this.engineIdentifier = engineIdentifier; + } else { + logger.warn("Failed to retrieve script identifier"); } - Consumer scriptDependencyListener = (Consumer) ctx - .getAttribute(CONTEXT_KEY_DEPENDENCY_LISTENER); - if (scriptDependencyListener == null) { - logger.warn( - "Failed to retrieve script script dependency listener from engine bindings. Script dependency tracking will be disabled."); + logger.debug("Initializing GraalPython script engine '{}' ...", this.engineIdentifier); + + if (pythonScriptEngineConfiguration.isDependencyTrackingEnabled()) { + @SuppressWarnings("unchecked") + Consumer scriptDependencyListener = (Consumer) ctx + .getAttribute(CONTEXT_KEY_DEPENDENCY_LISTENER); + if (scriptDependencyListener == null) { + // Can happen for script engines, created directly via PythonScriptEngineFactory and not via + // ScriptEngineManager + logger.debug("No dependency listener found. Script dependency tracking disabled for engine '{}'.", + this.engineIdentifier); + } else { + this.delegatingFileSystem.setAccessConsumer(new Consumer() { + @Override + public void accept(Path path) { + String pathAsString = path.toString(); + // convert cache path to real path + if (pathAsString.endsWith(".pyc")) { + // SOURCE .graalpy-232-311.pyc + // TARGET .py + int pos = pathAsString.indexOf(PythonScriptEngineConfiguration.PYTHON_LIB_PATH.toString()); + if (pos != -1) { + pathAsString = pathAsString.substring(pos, pathAsString.length() - 4); + int indexof = pathAsString.lastIndexOf("."); + pathAsString = pathAsString.substring(0, indexof); + path = Paths.get(pathAsString + ".py"); + } + } + if (path.startsWith(PythonScriptEngineConfiguration.PYTHON_LIB_PATH)) { + // logger.info("REGISTER PATH: {} of engine {}", path, + // PythonScriptEngine.this.engineIdentifier); + scriptDependencyListener.accept(path.toString()); + } + } + }); + } } - this.scriptDependencyListener = scriptDependencyListener; if (pythonScriptEngineConfiguration.isScopeEnabled()) { - // Wrap the "import" function to also allow loading modules from the ScriptExtensionModuleProvider - BiFunction, Object> wrapImportFn = (name, fromlist) -> scriptExtensionModuleProvider - .locatorFor(delegate.getPolyglotContext(), engineIdentifier, scriptExtensionAccessor) - .locateModule(name, fromlist); - delegate.getBindings(ScriptContext.ENGINE_SCOPE).put(ScriptExtensionModuleProvider.IMPORT_PROXY_NAME, - wrapImportFn); - try { - String wrapperContent = new String( - Files.readAllBytes(PythonScriptEngineFactory.PYTHON_WRAPPER_FILE_PATH)); - delegate.getPolyglotContext().eval(Source.newBuilder(GraalPythonScriptEngine.LANGUAGE_ID, - wrapperContent, PythonScriptEngineFactory.PYTHON_WRAPPER_FILE_PATH.toString()).build()); - - // inject scope, Registry and logger - if (!pythonScriptEngineConfiguration.isInjection(PythonScriptEngineConfiguration.INJECTION_DISABLED) - && (ctx.getAttribute("javax.script.filename") == null || pythonScriptEngineConfiguration - .isInjection(PythonScriptEngineConfiguration.INJECTION_ENABLED_FOR_ALL_SCRIPTS))) { - String injectionContent = "import scope\nfrom openhab import Registry, logger"; - delegate.getPolyglotContext().eval(Source - .newBuilder(GraalPythonScriptEngine.LANGUAGE_ID, injectionContent, "").build()); + ScriptExtensionAccessor scriptExtensionAccessor = (ScriptExtensionAccessor) ctx + .getAttribute(CONTEXT_KEY_EXTENSION_ACCESSOR); + if (scriptExtensionAccessor == null) { + // Can happen for script engines, created directly via PythonScriptEngineFactory and not via + // ScriptEngineManager + logger.debug("No Script accessor found. Scope injection disabled for engine {}", this.engineIdentifier); + } else { + // Wrap the "import" function to also allow loading modules from the ScriptExtensionModuleProvider + BiFunction, Object> wrapImportFn = (name, + fromlist) -> scriptExtensionModuleProvider + .locatorFor(getPolyglotContext(), this.engineIdentifier, scriptExtensionAccessor) + .locateModule(name, fromlist); + getBindings(ScriptContext.ENGINE_SCOPE).put(ScriptExtensionModuleProvider.IMPORT_PROXY_NAME, + wrapImportFn); + try { + String wrapperContent = new String( + Files.readAllBytes(PythonScriptEngineConfiguration.PYTHON_WRAPPER_FILE_PATH)); + getPolyglotContext() + .eval(Source + .newBuilder(GraalPythonScriptEngine.LANGUAGE_ID, wrapperContent, + PythonScriptEngineConfiguration.PYTHON_WRAPPER_FILE_PATH.toString()) + .build()); + + // inject scope, Registry and logger + if (!pythonScriptEngineConfiguration.isInjection(PythonScriptEngineConfiguration.INJECTION_DISABLED) + && (ctx.getAttribute(CONTEXT_KEY_SCRIPT_FILENAME) == null || pythonScriptEngineConfiguration + .isInjection(PythonScriptEngineConfiguration.INJECTION_ENABLED_FOR_ALL_SCRIPTS))) { + String injectionContent = "import scope\nfrom openhab import Registry, logger"; + getPolyglotContext().eval( + Source.newBuilder(GraalPythonScriptEngine.LANGUAGE_ID, injectionContent, "") + .build()); + } + } catch (IOException e) { + logger.error("Failed to inject import wrapper for engine '{}'", this.engineIdentifier, e); + throw new IllegalArgumentException("Failed to inject import wrapper", e); } - } catch (IOException e) { - logger.error("Failed to inject import wrapper", e); - throw new IllegalArgumentException("Failed to inject import wrapper", e); } } - // logger initialization, for non file based scripts, has to be delayed, because ruleUID is not available yet - if (ctx.getAttribute("javax.script.filename") == null) { - Runnable wrapperLoggerFn = () -> setScriptLogger(); - delegate.getBindings(ScriptContext.ENGINE_SCOPE).put(LOGGER_INIT_NAME, wrapperLoggerFn); + InputStream input = (InputStream) ctx.getAttribute(CONTEXT_KEY_ENGINE_LOGGER_INPUT); + if (input != null) { + scriptInputStream.setInputStream(input); + } + + OutputStream output = (OutputStream) ctx.getAttribute(CONTEXT_KEY_ENGINE_LOGGER_OUTPUT); + if (output != null) { + scriptOutputStream.setOutputStream(output); + scriptErrorStream.setOutputStream(output); } else { - setScriptLogger(); + // logger initialization, for non file based scripts, has to be delayed, because ruleUID is not + // available yet + if (ctx.getAttribute(CONTEXT_KEY_SCRIPT_FILENAME) == null) { + Runnable wrapperLoggerFn = () -> setScriptLogger(); + getBindings(ScriptContext.ENGINE_SCOPE).put(LOGGER_INIT_NAME, wrapperLoggerFn); + } else { + setScriptLogger(); + } } initialized = true; } @Override - protected String beforeInvocation(String source) { - source = super.beforeInvocation(source); - + protected @Nullable String beforeInvocation(@Nullable String source) { // Happens for Transform and UI based rules (eval and compile) // and has to be evaluate every time, because of changing and late injected ruleUID - if (delegate.getBindings(ScriptContext.ENGINE_SCOPE).get(LOGGER_INIT_NAME) != null) { + if (getBindings(ScriptContext.ENGINE_SCOPE).get(LOGGER_INIT_NAME) != null) { return LOGGER_INIT_NAME + "()\n" + source; } - return source; } @Override - protected Object afterInvocation(Object obj) { + protected @Nullable Object afterInvocation(@Nullable Object obj) { lock.unlock(); - logger.debug("Lock released after invocation."); - return super.afterInvocation(obj); + logger.debug("Lock released after invocation for engine '{}'.", this.engineIdentifier); + return obj; } @Override - protected Exception afterThrowsInvocation(Exception e) { + protected E afterThrowsInvocation(E e) { + Throwable cause = e.getCause(); // OPS4J Pax Logging holds a reference to the exception, which causes the PythonScriptEngine to not be // removed from heap by garbage collection and causing a memory leak. // Therefore, don't pass the exceptions itself to the logger, but only their message! if (e instanceof ScriptException) { // PolyglotException will always be wrapped into ScriptException and they will be visualized in // org.openhab.core.automation.module.script.internal.ScriptEngineManagerImpl - if (scriptErrorStream.getLogger().isDebugEnabled()) { - scriptErrorStream.getLogger().debug("Failed to execute script (PolyglotException): {}", - stringifyThrowable(e.getCause())); + if (logger.isDebugEnabled()) { + logger.debug("Failed to execute script (PolyglotException) for engine '{}': {}", this.engineIdentifier, + stringifyThrowable(cause == null ? e : cause)); } - } else if (e.getCause() instanceof IllegalArgumentException) { - scriptErrorStream.getLogger().error("Failed to execute script (IllegalArgumentException): {}", - stringifyThrowable(e.getCause())); + } else if (cause != null && e.getCause() instanceof IllegalArgumentException) { + logger.error("Failed to execute script (IllegalArgumentException) for engine '{}': {}", + this.engineIdentifier, stringifyThrowable(cause)); } lock.unlock(); - - return super.afterThrowsInvocation(e); + logger.debug("Lock cleaned after an exception for engine '{}'.", this.engineIdentifier); + return e; } @Override // collect JSR223 (scope) variables separately, because they are delivered via 'import scope' - public void put(String key, Object value) { - if ("javax.script.filename".equals(key)) { + public void put(@Nullable String key, @Nullable Object value) { + if (CONTEXT_KEY_SCRIPT_FILENAME.equals(key)) { super.put(key, value); } else { // use a custom lifecycleTracker to handle dispose hook before polyglot context is closed @@ -350,10 +411,15 @@ public void put(String key, Object value) { if ("lifecycleTracker".equals(key)) { value = lifecycleTracker; } - if (pythonScriptEngineConfiguration.isScopeEnabled()) { - scriptExtensionModuleProvider.put(key, value); + if (key != null && value != null) { + if (pythonScriptEngineConfiguration.isScopeEnabled()) { + scriptExtensionModuleProvider.put(key, value); + } else { + super.put(key, value); + } } else { - super.put(key, value); + throw new IllegalArgumentException( + "Null value for key: " + key + ", value: " + value + " not supported"); } } } @@ -361,7 +427,7 @@ public void put(String key, Object value) { @Override public void lock() { lock.lock(); - logger.debug("Lock acquired."); + logger.debug("Lock acquired for engine '{}'.", this.engineIdentifier); } @Override @@ -372,46 +438,67 @@ public void lockInterruptibly() throws InterruptedException { @Override public boolean tryLock() { boolean acquired = lock.tryLock(); - logger.debug("{}", acquired ? "Lock acquired." : "Lock not acquired."); + logger.debug("{} for engine '{}'", acquired ? "Lock acquired." : "Lock not acquired.", this.engineIdentifier); return acquired; } @Override - public boolean tryLock(long l, TimeUnit timeUnit) throws InterruptedException { + public boolean tryLock(long l, @Nullable TimeUnit timeUnit) throws InterruptedException { boolean acquired = lock.tryLock(l, timeUnit); - logger.debug("{}", acquired ? "Lock acquired." : "Lock not acquired."); + logger.debug("{} for engine '{}'", acquired ? "Lock acquired." : "Lock not acquired.", this.engineIdentifier); return acquired; } @Override public void unlock() { lock.unlock(); - logger.debug("Lock released."); + logger.debug("Lock released for engine '{}'.", this.engineIdentifier); + } + + @Override + public Object invokeFunction(String name, Object... objects) throws ScriptException, NoSuchMethodException { + if ("scriptUnloaded".equals(name)) { + /* + * is called from + * => org.openhab.core.automation.module.script.internal.ScriptEngineManagerImpl:removeEngine + * + * must be skipped, because ScriptTransformationService:disposeScriptEngine is calling engine.close several + * times before. Specially if the + * same script is used for more then 1 transformations. If the engine is already closed, the script + * "scriptUnloaded" will fail. + */ + return null; + } else { + return super.invokeFunction(name, objects); + } } @Override - public void close() throws Exception { + public void close() { + /* + * is called from + * => org.openhab.core.automation.module.script.ScriptTransformationService:disposeScriptEngine + * => org.openhab.core.automation.module.script.internal.ScriptEngineManagerImpl:removeEngine + */ + lock.lock(); - if (!closed) { + if (!isClosed()) { try { this.lifecycleTracker.dispose(); + logger.debug("LifecycleTracker for engine '{}' disposed.", this.engineIdentifier); } catch (Exception e) { - logger.warn("Ignoreable exception during dispose: {}", stringifyThrowable(e)); + logger.warn("Ignoreable exception during LifecycleTracker dispose for engine '{}': {}", + this.engineIdentifier, stringifyThrowable(e)); } - logger.debug("Engine disposed."); try { super.close(); + logger.debug("Engine '{}' closed.", this.engineIdentifier); } catch (Exception e) { - logger.warn("Ignoreable exception during close: {}", stringifyThrowable(e)); + logger.warn("Ignoreable exception during close of engine '{}': {}", this.engineIdentifier, + stringifyThrowable(e)); } - - logger.debug("Engine closed."); - - closed = true; - } else { - logger.debug("Engine already disposed and closed."); } lock.unlock(); @@ -428,14 +515,14 @@ public Condition newCondition() { * Therefore, the logger needs to be initialized on the first use after script engine creation. */ private void setScriptLogger() { - ScriptContext ctx = getScriptContext(); - Object fileName = ctx.getAttribute("javax.script.filename"); + ScriptContext ctx = getContext(); + Object fileName = ctx.getAttribute(CONTEXT_KEY_SCRIPT_FILENAME); Object ruleUID = ctx.getAttribute("ruleUID"); - Object ohEngineIdentifier = ctx.getAttribute("oh.engine-identifier"); + Object ohEngineIdentifier = ctx.getAttribute(CONTEXT_KEY_ENGINE_IDENTIFIER); String identifier = "stack"; if (fileName != null) { - identifier = fileName.toString().replaceAll("^.*[/\\\\]", ""); + identifier = fileName.toString().replaceAll("^.*[/\\\\]", "").replaceAll(".py", ""); } else if (ruleUID != null) { identifier = ruleUID.toString(); } else if (ohEngineIdentifier != null) { @@ -446,16 +533,8 @@ private void setScriptLogger() { Logger scriptLogger = LoggerFactory.getLogger("org.openhab.automation.pythonscripting." + identifier); - scriptOutputStream.setLogger(scriptLogger); - scriptErrorStream.setLogger(scriptLogger); - } - - private ScriptContext getScriptContext() { - ScriptContext ctx = delegate.getContext(); - if (ctx == null) { - throw new IllegalStateException("Failed to retrieve script context"); - } - return ctx; + scriptOutputStream.setOutputStream(new ContextOutputLogger(scriptLogger, Level.INFO)); + scriptErrorStream.setOutputStream(new ContextOutputLogger(scriptLogger, Level.ERROR)); } private String stringifyThrowable(Throwable throwable) { @@ -468,11 +547,57 @@ private String stringifyThrowable(Throwable throwable) { } private static Set transformArrayToSet(Value value) { - Set set = new HashSet<>(); - for (int i = 0; i < value.getArraySize(); ++i) { - Value element = value.getArrayElement(i); - set.add(element.asString()); + try { + Set set = new HashSet<>(); + for (int i = 0; i < value.getArraySize(); ++i) { + Value element = value.getArrayElement(i); + set.add(element.isString() ? element.asString() : element.toString()); + } + return set; + } catch (Exception e) { + String msg = "Can't convert python value '" + value.toString() + "' (" + value.getClass() + + ") to a java.util.Set\n" + e.getClass().getSimpleName() + ": " + e.getMessage(); + throw new IllegalArgumentException(msg, e); } - return set; + } + + private static State transformValueToState(Value value) { + try { + if (value.isBoolean()) { + // logger.info("VALUE: OnOffType {}", value.asBoolean()); + return OnOffType.from(value.asBoolean()); + } else if (value.isNumber()) { + // logger.info("VALUE: DecimalType {}", value.toString()); + return DecimalType.valueOf(value.toString()); + } else if (value.hasMember("ctime") && value.hasMember("isoformat")) { + // logger.info("VALUE: DateTimeType"); + return new DateTimeType(PythonScriptEngine.parseDatetime(value)); + } else if (value.isString()) { + // logger.info("VALUE: StringType {}", value.asString()); + return StringType.valueOf(value.asString()); + } else if (value.isNull()) { + // logger.info("VALUE: UnDefType.NULL {}", value.toString()); + return UnDefType.NULL; + } else { + // logger.info("VALUE: FALLBACK {}", value.toString()); + return StringType.valueOf(value.toString()); + } + } catch (Exception e) { + String msg = "Can't convert python value '" + value.toString() + "' (" + value.getClass() + + ") to an org.openhab.core.types.State object\n" + e.getClass().getSimpleName() + ": " + + e.getMessage(); + throw new IllegalArgumentException(msg, e); + } + } + + private static ZonedDateTime parseDatetime(Value value) { + return ZonedDateTime.parse(value.invokeMember("isoformat").asString() + + (!value.hasMember("tzinfo") || value.getMember("tzinfo").isNull() + ? OffsetDateTime.now().getOffset().getId() + : "")); + } + + public static @Nullable Language getLanguage() { + return engine.getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineConfiguration.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineConfiguration.java index 5fd03957a31dc..0cbc2f4eb830d 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineConfiguration.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineConfiguration.java @@ -12,13 +12,32 @@ */ package org.openhab.automation.pythonscripting.internal; +import java.io.IOException; +import java.io.InputStream; +import java.lang.module.ModuleDescriptor.Version; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.List; import java.util.Map; +import java.util.Properties; +import java.util.stream.Collectors; import org.eclipse.jdt.annotation.NonNullByDefault; -import org.openhab.core.config.core.ConfigParser; +import org.eclipse.jdt.annotation.Nullable; +import org.graalvm.polyglot.Context; +import org.openhab.core.OpenHAB; +import org.openhab.core.automation.module.script.ScriptEngineFactory; +import org.openhab.core.config.core.Configuration; +import org.osgi.framework.FrameworkUtil; +import org.osgi.service.component.annotations.Activate; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.ObjectMapper; + /** * Processes Python Configuration Parameters. * @@ -29,62 +48,224 @@ public class PythonScriptEngineConfiguration { private final Logger logger = LoggerFactory.getLogger(PythonScriptEngineConfiguration.class); - private static final String CFG_INJECTION_ENABLED = "injectionEnabled"; - private static final String CFG_HELPER_ENABLED = "helperEnabled"; - private static final String CFG_SCOPE_ENABLED = "scopeEnabled"; - private static final String CFG_DEPENDENCY_TRACKING_ENABLED = "dependencyTrackingEnabled"; - private static final String CFG_CACHING_ENABLED = "cachingEnabled"; - private static final String CFG_JYTHON_EMULATION = "jythonEmulation"; + private static final String SYSTEM_PROPERTY_POLYGLOT_ENGINE_USERRESOURCECACHE = "polyglot.engine.userResourceCache"; + + private static final String SYSTEM_PROPERTY_JAVA_IO_TMPDIR = "java.io.tmpdir"; + + public static final String PATH_SEPARATOR = FileSystems.getDefault().getSeparator(); + public static final String RESOURCE_SEPARATOR = "/"; + + public static final Path PYTHON_DEFAULT_PATH = Paths.get(OpenHAB.getConfigFolder(), "automation", "python"); + public static final Path PYTHON_LIB_PATH = PYTHON_DEFAULT_PATH.resolve("lib"); + public static final Path PYTHON_TYPINGS_PATH = PYTHON_DEFAULT_PATH.resolve("typings"); + public static final Path PYTHON_OPENHAB_LIB_PATH = PYTHON_LIB_PATH.resolve("openhab"); + + public static final Path PYTHON_WRAPPER_FILE_PATH = PYTHON_OPENHAB_LIB_PATH.resolve("__wrapper__.py"); + public static final Path PYTHON_INIT_FILE_PATH = PYTHON_OPENHAB_LIB_PATH.resolve("__init__.py"); public static final int INJECTION_DISABLED = 0; public static final int INJECTION_ENABLED_FOR_ALL_SCRIPTS = 1; public static final int INJECTION_ENABLED_FOR_NON_FILE_BASED_SCRIPTS = 2; - private int injectionEnabled = 0; - private boolean helperEnabled = false; - private boolean scopeEnabled = false; - private boolean dependencyTrackingEnabled = false; - private boolean cachingEnabled = false; - private boolean jythonEmulation = false; + + // The variable names must match the configuration keys in config.xml + public static class PythonScriptingConfiguration { + public boolean scopeEnabled = true; + public boolean helperEnabled = true; + public int injectionEnabled = INJECTION_ENABLED_FOR_NON_FILE_BASED_SCRIPTS; + public boolean dependencyTrackingEnabled = true; + public boolean cachingEnabled = true; + public boolean jythonEmulation = false; + public boolean nativeModules = false; + public String pipModules = ""; + } + + private PythonScriptingConfiguration configuration = new PythonScriptingConfiguration(); + private Path bytecodeDirectory; + private Path tempDirectory; + private Path venvDirectory; + private @Nullable Path venvExecutable = null; + + private Version bundleVersion = Version + .parse(FrameworkUtil.getBundle(PythonScriptEngineConfiguration.class).getVersion().toString()); + + private Version graalVersion = Version.parse("0.0.0"); + private Version providedHelperLibVersion = Version.parse("0.0.0"); + private @Nullable Version installedHelperLibVersion = null; + + public static Version parseHelperLibVersion(@Nullable String version) throws IllegalArgumentException { + // substring(1) => remove leading 'v' + return Version.parse(version != null && version.startsWith("v") ? version.substring(1) : version); + } + + @Activate + public PythonScriptEngineConfiguration(Map config, PythonScriptEngineFactory factory) { + Path userdataDir = Paths.get(OpenHAB.getUserDataFolder()); + + String tmpDir = System.getProperty(SYSTEM_PROPERTY_JAVA_IO_TMPDIR); + if (tmpDir != null) { + tempDirectory = Paths.get(tmpDir); + } else { + tempDirectory = userdataDir.resolve("tmp"); + } + + try { + InputStream is = PythonScriptEngineConfiguration.class + .getResourceAsStream(RESOURCE_SEPARATOR + "build.properties"); + if (is != null) { + Properties p = new Properties(); + p.load(is); + String version = p.getProperty("helperlib.version"); + providedHelperLibVersion = parseHelperLibVersion(version); + version = FrameworkUtil.getBundle(Context.class).getVersion().toString(); + graalVersion = Version.parse(version); + } + } catch (IOException e) { + throw new IllegalArgumentException("Unable to load build.properties"); + } + + String packageName = PythonScriptEngineConfiguration.class.getPackageName(); + packageName = packageName.substring(0, packageName.lastIndexOf(".")); + Path bindingDirectory = userdataDir.resolve("cache").resolve(packageName); + + Properties props = System.getProperties(); + props.setProperty(SYSTEM_PROPERTY_POLYGLOT_ENGINE_USERRESOURCECACHE, bindingDirectory.toString()); + bytecodeDirectory = PythonScriptEngineHelper.initDirectory(bindingDirectory.resolve("resources")); + venvDirectory = PythonScriptEngineHelper.initDirectory(bindingDirectory.resolve("venv")); + + Path venvPythonBin = venvDirectory.resolve("bin").resolve("graalpy"); + if (Files.exists(venvPythonBin)) { + venvExecutable = venvPythonBin; + } + + installedHelperLibVersion = PythonScriptEngineHelper.initHelperLib(this, providedHelperLibVersion); + + this.update(config, factory); + } /** * Update configuration * * @param config Configuration parameters to apply to ScriptEngine + * @param initial */ - void update(Map config) { + public void modified(Map config, ScriptEngineFactory factory) { + boolean oldScopeEnabled = configuration.scopeEnabled; + boolean oldInjectionEnabled = !isInjection(PythonScriptEngineConfiguration.INJECTION_DISABLED); + boolean oldDependencyTrackingEnabled = isDependencyTrackingEnabled(); + + this.update(config, factory); + + if (oldScopeEnabled != isScopeEnabled()) { + logger.info("{} scope for Python Scripting. Please resave your scripts to apply this change.", + isScopeEnabled() ? "Enabled" : "Disabled"); + } + if (oldInjectionEnabled != !isInjection(PythonScriptEngineConfiguration.INJECTION_DISABLED)) { + logger.info("{} injection for Python Scripting. Please resave your UI-based scripts to apply this change.", + !isInjection(PythonScriptEngineConfiguration.INJECTION_DISABLED) ? "Enabled" : "Disabled"); + } + if (oldDependencyTrackingEnabled != isDependencyTrackingEnabled()) { + logger.info("{} dependency tracking for Python Scripting. Please resave your scripts to apply this change.", + isDependencyTrackingEnabled() ? "Enabled" : "Disabled"); + } + } + + private void update(Map config, ScriptEngineFactory factory) { logger.trace("Python Script Engine Configuration: {}", config); - this.scopeEnabled = ConfigParser.valueAsOrElse(config.get(CFG_SCOPE_ENABLED), Boolean.class, true); - this.helperEnabled = ConfigParser.valueAsOrElse(config.get(CFG_HELPER_ENABLED), Boolean.class, true); - this.injectionEnabled = ConfigParser.valueAsOrElse(config.get(CFG_INJECTION_ENABLED), Integer.class, - INJECTION_ENABLED_FOR_NON_FILE_BASED_SCRIPTS); - this.dependencyTrackingEnabled = ConfigParser.valueAsOrElse(config.get(CFG_DEPENDENCY_TRACKING_ENABLED), - Boolean.class, true); - this.cachingEnabled = ConfigParser.valueAsOrElse(config.get(CFG_CACHING_ENABLED), Boolean.class, true); - this.jythonEmulation = ConfigParser.valueAsOrElse(config.get(CFG_JYTHON_EMULATION), Boolean.class, false); + String oldPipModules = configuration.pipModules; + configuration = new Configuration(config).as(PythonScriptingConfiguration.class); + + if (!oldPipModules.equals(configuration.pipModules)) { + PythonScriptEngineHelper.initPipModules(this, factory); + } + } + + public void setHelperLibVersion(Version version) { + installedHelperLibVersion = version; } public boolean isScopeEnabled() { - return scopeEnabled; + return configuration.scopeEnabled; } public boolean isHelperEnabled() { - return helperEnabled; + return configuration.helperEnabled; } public boolean isInjection(int type) { - return injectionEnabled == type; + return configuration.injectionEnabled == type; } public boolean isDependencyTrackingEnabled() { - return dependencyTrackingEnabled; + return configuration.dependencyTrackingEnabled; } public boolean isCachingEnabled() { - return cachingEnabled; + return configuration.cachingEnabled; } public boolean isJythonEmulation() { - return jythonEmulation; + return configuration.jythonEmulation; + } + + public boolean isNativeModulesEnabled() { + return configuration.nativeModules; + } + + public String getPIPModules() { + return configuration.pipModules; + } + + public Path getBytecodeDirectory() { + return bytecodeDirectory; + } + + public Path getTempDirectory() { + return tempDirectory; + } + + public Path getVEnvDirectory() { + return venvDirectory; + } + + public @Nullable Path getVEnvExecutable() { + return venvExecutable; + } + + public boolean isVEnvEnabled() { + return venvExecutable != null; + } + + public Version getBundleVersion() { + return bundleVersion; + } + + public Version getGraalVersion() { + return graalVersion; + } + + public Version getProvidedHelperLibVersion() { + return providedHelperLibVersion; + } + + public @Nullable Version getInstalledHelperLibVersion() { + return installedHelperLibVersion; + } + + /** + * Returns the current configuration as a map. + * This is used to display the configuration in the console. + */ + public Map getConfigurations() { + ObjectMapper objectMapper = new ObjectMapper(); + Map objectMap = objectMapper.convertValue(configuration, + new TypeReference>() { + }); + return objectMap.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey, entry -> { + if (entry.getValue() instanceof List listValue) { + return listValue.stream().map(Object::toString).collect(Collectors.joining("\n")); + } + return entry.getValue().toString(); + })); } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineFactory.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineFactory.java index f5af3dbe09a61..7b4f426f8aef3 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineFactory.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineFactory.java @@ -12,39 +12,24 @@ */ package org.openhab.automation.pythonscripting.internal; -import java.io.BufferedReader; -import java.io.File; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.lang.module.ModuleDescriptor.Version; -import java.net.URL; -import java.nio.charset.StandardCharsets; -import java.nio.file.FileSystems; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.Paths; +import java.time.ZoneId; import java.util.Arrays; -import java.util.Comparator; -import java.util.Enumeration; import java.util.List; import java.util.Map; import java.util.Map.Entry; -import java.util.regex.Matcher; -import java.util.regex.Pattern; -import java.util.stream.Collectors; -import java.util.stream.Stream; import javax.script.ScriptEngine; import org.eclipse.jdt.annotation.NonNullByDefault; import org.eclipse.jdt.annotation.Nullable; -import org.openhab.automation.pythonscripting.internal.fs.watch.PythonDependencyTracker; -import org.openhab.core.OpenHAB; +import org.graalvm.polyglot.Language; +import org.openhab.automation.pythonscripting.internal.fs.PythonDependencyTracker; +import org.openhab.automation.pythonscripting.internal.scriptengine.graal.GraalPythonScriptEngine.ScriptEngineProvider; import org.openhab.core.automation.module.script.ScriptDependencyTracker; import org.openhab.core.automation.module.script.ScriptEngineFactory; import org.openhab.core.config.core.ConfigurableService; +import org.openhab.core.i18n.TimeZoneProvider; import org.osgi.framework.Constants; -import org.osgi.framework.FrameworkUtil; import org.osgi.service.component.annotations.Activate; import org.osgi.service.component.annotations.Component; import org.osgi.service.component.annotations.Deactivate; @@ -59,41 +44,43 @@ * @author Holger Hees - Initial contribution * @author Jeff James - Initial contribution */ -@Component(service = ScriptEngineFactory.class, configurationPid = "org.openhab.automation.pythonscripting", property = Constants.SERVICE_PID - + "=org.openhab.automation.pythonscripting") -@ConfigurableService(category = "automation", label = "Python Scripting", description_uri = "automation:pythonscripting") +@Component(service = { ScriptEngineFactory.class, PythonScriptEngineFactory.class }, // + configurationPid = "org.openhab.automation.pythonscripting", // + property = Constants.SERVICE_PID + "=org.openhab.automation.pythonscripting") +@ConfigurableService(category = "automation", label = "Python Scripting", description_uri = PythonScriptEngineFactory.CONFIG_DESCRIPTION_URI) @NonNullByDefault -public class PythonScriptEngineFactory implements ScriptEngineFactory { +public class PythonScriptEngineFactory implements ScriptEngineFactory, ScriptEngineProvider { private final Logger logger = LoggerFactory.getLogger(PythonScriptEngineFactory.class); - private static final String RESOURCE_SEPARATOR = "/"; - - public static final Path PYTHON_DEFAULT_PATH = Paths.get(OpenHAB.getConfigFolder(), "automation", "python"); - public static final Path PYTHON_LIB_PATH = PYTHON_DEFAULT_PATH.resolve("lib"); - - private static final Path PYTHON_OPENHAB_LIB_PATH = PYTHON_LIB_PATH.resolve("openhab"); - - public static final Path PYTHON_WRAPPER_FILE_PATH = PYTHON_OPENHAB_LIB_PATH.resolve("__wrapper__.py"); - private static final Path PYTHON_INIT_FILE_PATH = PYTHON_OPENHAB_LIB_PATH.resolve("__init__.py"); - + public static final String CONFIG_DESCRIPTION_URI = "automation:pythonscripting"; public static final String SCRIPT_TYPE = "application/x-python3"; - private final List scriptTypes = Arrays.asList("py", SCRIPT_TYPE); + private final List scriptTypes = Arrays.asList("py", SCRIPT_TYPE); private final PythonDependencyTracker pythonDependencyTracker; - private final PythonScriptEngineConfiguration pythonScriptEngineConfiguration; + private final PythonScriptEngineConfiguration configuration; + + private final @Nullable Language language; @Activate public PythonScriptEngineFactory(final @Reference PythonDependencyTracker pythonDependencyTracker, - Map config) { + final @Reference TimeZoneProvider timeZoneProvider, Map config) { logger.debug("Loading PythonScriptEngineFactory"); this.pythonDependencyTracker = pythonDependencyTracker; - this.pythonScriptEngineConfiguration = new PythonScriptEngineConfiguration(); + this.configuration = new PythonScriptEngineConfiguration(config, this); - modified(config); + this.language = PythonScriptEngine.getLanguage(); + if (this.language == null) { + logger.error( + "Graal Python language not initialized. Restart openHAB to initialize available Graal languages properly."); + } - if (this.pythonScriptEngineConfiguration.isHelperEnabled()) { - initHelperLib(); + String defaultTimezone = ZoneId.systemDefault().getId(); + String providerTimezone = timeZoneProvider.getTimeZone().getId(); + if (!defaultTimezone.equals(providerTimezone)) { + logger.warn( + "User timezone '{}' is different than openhab regional timezone '{}'. Python Scripting is running with timezone '{}'.", + defaultTimezone, providerTimezone, defaultTimezone); } } @@ -103,8 +90,8 @@ public void cleanup() { } @Modified - protected void modified(Map config) { - this.pythonScriptEngineConfiguration.update(config); + protected void modified(Map config) { + this.configuration.modified(config, this); } @Override @@ -124,105 +111,23 @@ public void scopeValues(ScriptEngine scriptEngine, Map scopeValu if (!scriptTypes.contains(scriptType)) { return null; } - return new PythonScriptEngine(pythonDependencyTracker, pythonScriptEngineConfiguration); + return createScriptEngine(); } @Override - public @Nullable ScriptDependencyTracker getDependencyTracker() { - return pythonDependencyTracker; + public @Nullable ScriptEngine createScriptEngine() { + if (language == null) { + return null; + } + return new PythonScriptEngine(configuration, this); } - private void initHelperLib() { - try { - String pathSeparator = FileSystems.getDefault().getSeparator(); - String resourceLibPath = PYTHON_OPENHAB_LIB_PATH.toString() - .substring(PYTHON_DEFAULT_PATH.toString().length()) + pathSeparator; - if (!RESOURCE_SEPARATOR.equals(pathSeparator)) { - resourceLibPath = resourceLibPath.replace(pathSeparator, RESOURCE_SEPARATOR); - } - - if (Files.exists(PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH)) { - try (Stream files = Files.list(PYTHON_OPENHAB_LIB_PATH)) { - if (files.count() > 0) { - Pattern pattern = Pattern.compile("__version__\\s*=\\s*\"([0-9]+\\.[0-9]+\\.[0-9]+)\"", - Pattern.CASE_INSENSITIVE); - - Version includedVersion = null; - try (InputStream is = PythonScriptEngineFactory.class.getClassLoader().getResourceAsStream( - resourceLibPath + PYTHON_INIT_FILE_PATH.getFileName().toString())) { - try (InputStreamReader isr = new InputStreamReader(is); - BufferedReader reader = new BufferedReader(isr)) { - String fileContent = reader.lines().collect(Collectors.joining(System.lineSeparator())); - Matcher includedMatcher = pattern.matcher(fileContent); - if (includedMatcher.find()) { - includedVersion = Version.parse(includedMatcher.group(1)); - } - } - } - - Version currentVersion = null; - String fileContent = Files.readString(PYTHON_INIT_FILE_PATH, StandardCharsets.UTF_8); - Matcher currentMatcher = pattern.matcher(fileContent); - if (currentMatcher.find()) { - currentVersion = Version.parse(currentMatcher.group(1)); - } - - if (currentVersion == null) { - logger.warn("Unable to detect installed helper lib version. Skip installing helper libs."); - return; - } else if (includedVersion == null) { - logger.error("Unable to detect provided helper lib version. Skip installing helper libs."); - return; - } else if (currentVersion.compareTo(includedVersion) >= 0) { - logger.info("Newest helper lib version is deployed."); - return; - } - } - } - } - - logger.info("Deploy helper libs into {}.", PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH); - - if (Files.exists(PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH)) { - try (Stream paths = Files.walk(PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH)) { - paths.sorted(Comparator.reverseOrder()).map(Path::toFile).forEach(File::delete); - } - } - - initDirectory(PythonScriptEngineFactory.PYTHON_DEFAULT_PATH); - initDirectory(PythonScriptEngineFactory.PYTHON_LIB_PATH); - initDirectory(PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH); - - Enumeration resourceFiles = FrameworkUtil.getBundle(PythonScriptEngineFactory.class) - .findEntries(resourceLibPath, "*.py", true); - - while (resourceFiles.hasMoreElements()) { - URL resourceFile = resourceFiles.nextElement(); - String resourcePath = resourceFile.getPath(); - - try (InputStream is = PythonScriptEngineFactory.class.getClassLoader() - .getResourceAsStream(resourcePath)) { - Path target = PythonScriptEngineFactory.PYTHON_OPENHAB_LIB_PATH - .resolve(resourcePath.substring(resourcePath.lastIndexOf(RESOURCE_SEPARATOR) + 1)); - - Files.copy(is, target); - File file = target.toFile(); - file.setReadable(true, false); - file.setWritable(true, true); - } - } - } catch (Exception e) { - logger.error("Exception during helper lib initialisation", e); - } + @Override + public @Nullable ScriptDependencyTracker getDependencyTracker() { + return pythonDependencyTracker; } - private void initDirectory(Path path) { - File directory = path.toFile(); - if (!directory.exists()) { - directory.mkdir(); - directory.setExecutable(true, false); - directory.setReadable(true, false); - directory.setWritable(true, true); - } + public PythonScriptEngineConfiguration getConfiguration() { + return this.configuration; } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineHelper.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineHelper.java new file mode 100644 index 0000000000000..afaaf67f1b409 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/PythonScriptEngineHelper.java @@ -0,0 +1,300 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal; + +import java.io.BufferedInputStream; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.lang.module.ModuleDescriptor.Version; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.Comparator; +import java.util.Enumeration; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.eclipse.jdt.annotation.Nullable; +import org.openhab.core.automation.module.script.ScriptEngineFactory; +import org.osgi.framework.FrameworkUtil; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Processes Python Configuration Parameters. + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class PythonScriptEngineHelper { + + private final static Logger logger = LoggerFactory.getLogger(PythonScriptEngineHelper.class); + + private static final Pattern VERSION_PATTERN = Pattern.compile("__version__\\s*=\\s*\"([^\"]*)\"", + Pattern.CASE_INSENSITIVE); + + public static void initPipModules(PythonScriptEngineConfiguration configuration, ScriptEngineFactory factory) { + String pipModulesConfig = configuration.getPIPModules().strip(); + if (pipModulesConfig.isEmpty()) { + return; + } + + if (!configuration.isVEnvEnabled()) { + logger.error("Can't install pip modules. VEnv not enabled."); + return; + } + + List pipModules = Arrays.stream(pipModulesConfig.split(",")).map(String::trim) + .filter(module -> !module.isEmpty()).collect(Collectors.toList()); + + if (pipModules.isEmpty()) { + return; + } + + final String pipCode = """ + import subprocess + import sys + + command_list = [sys.executable, "-m", "pip", "install"] + pipModules + proc = subprocess.run(command_list, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, check=False) + if proc.returncode != 0: + print(proc.stdout) + exit(1) + """; + + ScriptEngine engine = factory.createScriptEngine(PythonScriptEngineFactory.SCRIPT_TYPE); + if (engine != null) { + engine.getContext().setAttribute("pipModules", pipModules, ScriptContext.ENGINE_SCOPE); + try { + logger.info("Checking for pip module{} '{}'", pipModules.size() > 1 ? "s" : "", + configuration.getPIPModules()); + engine.eval(pipCode); + } catch (ScriptException e) { + logger.warn("Error installing pip module{}", pipModules.size() > 1 ? "s" : ""); + logger.trace("TRACE:", unwrap(e)); + } + } else { + logger.warn("Can't install pip modules. No script engine available."); + } + } + + public static @Nullable Version initHelperLib(PythonScriptEngineConfiguration configuration, + Version providedHelperLibVersion) { + Version installedHelperLibVersion = null; + + if (!configuration.isHelperEnabled()) { + return installedHelperLibVersion; + } + + logger.info("Checking for helper libs version '{}'", configuration.getProvidedHelperLibVersion()); + + String resourceLibPath = PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH.toString() + .substring(PythonScriptEngineConfiguration.PYTHON_DEFAULT_PATH.toString().length()) + + PythonScriptEngineConfiguration.PATH_SEPARATOR; + if (!PythonScriptEngineConfiguration.RESOURCE_SEPARATOR + .equals(PythonScriptEngineConfiguration.PATH_SEPARATOR)) { + resourceLibPath = resourceLibPath.replace(PythonScriptEngineConfiguration.PATH_SEPARATOR, + PythonScriptEngineConfiguration.RESOURCE_SEPARATOR); + } + + if (Files.exists(PythonScriptEngineConfiguration.PYTHON_INIT_FILE_PATH)) { + try { + String content = Files.readString(PythonScriptEngineConfiguration.PYTHON_INIT_FILE_PATH, + StandardCharsets.UTF_8); + Matcher currentMatcher = VERSION_PATTERN.matcher(content); + if (currentMatcher.find()) { + installedHelperLibVersion = Version.parse(currentMatcher.group(1)); + @SuppressWarnings("null") + int compareResult = installedHelperLibVersion.compareTo(providedHelperLibVersion); + if (compareResult >= 0) { + if (compareResult > 0) { + logger.info("Newer helper libs version '{}' already installed.", installedHelperLibVersion); + } + return installedHelperLibVersion; + } + } else { + logger.warn("Unable to parse current version. Proceed as if it was not installed."); + } + } catch (IOException | IllegalArgumentException e) { + logger.warn("Unable to detect current version. Proceed as if it was not installed."); + } + } + + if (installedHelperLibVersion != null) { + logger.info("Update helper libs version '{}' to version {}.", installedHelperLibVersion, + providedHelperLibVersion); + } else { + logger.info("Install helper libs version {}.", providedHelperLibVersion); + } + + try { + Path bakLibPath = preProcessHelperLibUpdate(); + + try { + Enumeration resourceFiles = FrameworkUtil.getBundle(PythonScriptEngineHelper.class) + .findEntries(resourceLibPath, "*.py", true); + + while (resourceFiles.hasMoreElements()) { + URL resourceFile = resourceFiles.nextElement(); + String resourcePath = resourceFile.getPath(); + + ClassLoader clsLoader = PythonScriptEngineHelper.class.getClassLoader(); + if (clsLoader != null) { + try (InputStream is = clsLoader.getResourceAsStream(resourcePath)) { + Path target = PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH + .resolve(resourcePath.substring( + resourcePath.lastIndexOf(PythonScriptEngineConfiguration.RESOURCE_SEPARATOR) + + 1)); + + Files.copy(is, target); + initFile(target); + } + } else { + throw new IllegalArgumentException("Class loader is null"); + } + } + + installedHelperLibVersion = postProcessHelperLibUpdateOnSuccess(providedHelperLibVersion, bakLibPath); + } catch (Exception e) { + postProcessHelperLibUpdateOnFailure(bakLibPath); + throw e; + } + } catch (Exception e) { + logger.error("Exception during helper lib initialisation", e); + } + + return installedHelperLibVersion; + } + + public static void installHelperLib(String remoteUrl, Version remoteVersion, + PythonScriptEngineConfiguration configuration) throws URISyntaxException, IOException { + Path bakLibPath = null; + try { + bakLibPath = preProcessHelperLibUpdate(); + URL zipfileUrl = new URI(remoteUrl).toURL(); + InputStream in = new BufferedInputStream(zipfileUrl.openStream(), 1024); + ZipInputStream stream = new ZipInputStream(in); + byte[] buffer = new byte[1024]; + ZipEntry entry; + while ((entry = stream.getNextEntry()) != null) { + if (!entry.getName().contains("/src/") || entry.isDirectory()) { + continue; + } + + int read; + StringBuilder sb = new StringBuilder(); + while ((read = stream.read(buffer, 0, 1024)) >= 0) { + sb.append(new String(buffer, 0, read)); + } + + Path target = PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH + .resolve(new File(entry.getName()).getName()); + Files.write(target, sb.toString().getBytes()); + initFile(target); + } + + Version version = postProcessHelperLibUpdateOnSuccess(remoteVersion, bakLibPath); + configuration.setHelperLibVersion(version); + } catch (IOException | URISyntaxException e) { + postProcessHelperLibUpdateOnFailure(bakLibPath); + throw e; + } + } + + private static @Nullable Path preProcessHelperLibUpdate() throws IOException { + Path bakLibPath = null; + // backup old lib folder before update + if (Files.exists(PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH)) { + bakLibPath = PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH.getParent() + .resolve(PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH.getFileName() + ".bak"); + Files.move(PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH, bakLibPath); + } + initDirectory(PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH); + return bakLibPath; + } + + private static Version postProcessHelperLibUpdateOnSuccess(Version version, @Nullable Path bakLibPath) + throws IOException { + String content = Files.readString(PythonScriptEngineConfiguration.PYTHON_INIT_FILE_PATH, StandardCharsets.UTF_8) + .trim(); + Matcher currentMatcher = VERSION_PATTERN.matcher(content); + content = currentMatcher.replaceAll("__version__ = \"" + version.toString() + "\""); + Files.writeString(PythonScriptEngineConfiguration.PYTHON_INIT_FILE_PATH, content, StandardCharsets.UTF_8); + + if (bakLibPath != null) { + // cleanup old files + try (var dirStream = Files.walk(bakLibPath)) { + dirStream.map(Path::toFile).sorted(Comparator.reverseOrder()).forEach(File::delete); + } + } + + return version; + } + + private static void postProcessHelperLibUpdateOnFailure(@Nullable Path bakLibPath) throws IOException { + // cleanup new files + try (var dirStream = Files.walk(PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH)) { + dirStream.map(Path::toFile).sorted(Comparator.reverseOrder()).forEach(File::delete); + } + + if (bakLibPath != null) { + // restore old files + Files.move(bakLibPath, PythonScriptEngineConfiguration.PYTHON_OPENHAB_LIB_PATH); + } + } + + /** + * Unwraps the cause of an exception, if it has one. + * + * Since a user cares about the _Python_ stack trace of the throwable, not + * the details of where openHAB called it. + */ + private static Throwable unwrap(Throwable e) { + Throwable cause = e.getCause(); + if (cause != null) { + return cause; + } + return e; + } + + private static void initFile(Path path) { + File file = path.toFile(); + file.setReadable(true, false); + file.setWritable(true, true); + } + + public static Path initDirectory(Path path) { + File directory = path.toFile(); + if (!directory.exists()) { + directory.mkdirs(); + directory.setExecutable(true, false); + directory.setReadable(true, false); + directory.setWritable(true, true); + } + return path; + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/PythonConsoleCommandExtension.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/PythonConsoleCommandExtension.java new file mode 100644 index 0000000000000..84c949c1189c3 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/PythonConsoleCommandExtension.java @@ -0,0 +1,389 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.SortedSet; +import java.util.UUID; + +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.eclipse.jdt.annotation.Nullable; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngine; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory; +import org.openhab.automation.pythonscripting.internal.console.handler.InfoCmd; +import org.openhab.automation.pythonscripting.internal.console.handler.TypingCmd; +import org.openhab.automation.pythonscripting.internal.console.handler.UpdateCmd; +import org.openhab.core.automation.module.script.ScriptEngineContainer; +import org.openhab.core.automation.module.script.ScriptEngineFactory; +import org.openhab.core.automation.module.script.ScriptEngineManager; +import org.openhab.core.config.core.ConfigDescriptionRegistry; +import org.openhab.core.io.console.Console; +import org.openhab.core.io.console.ConsoleCommandCompleter; +import org.openhab.core.io.console.StringsCompleter; +import org.openhab.core.io.console.extensions.AbstractConsoleCommandExtension; +import org.openhab.core.io.console.extensions.ConsoleCommandExtension; +import org.osgi.service.component.annotations.Activate; +import org.osgi.service.component.annotations.Component; +import org.osgi.service.component.annotations.Reference; + +/** + * The {@link PythonConsoleCommandExtension} class + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +@Component(service = ConsoleCommandExtension.class) +public class PythonConsoleCommandExtension extends AbstractConsoleCommandExtension implements ConsoleCommandCompleter { + private static final String INFO = "info"; + private static final String CONSOLE = "console"; + private static final String TYPING = "typing"; + private static final String PIP = "pip"; + private static final String PIP_INSTALL = "install"; + private static final String PIP_UNINSTALL = "uninstall"; + private static final String PIP_SHOW = "show"; + private static final String PIP_LIST = "list"; + private static final String UPDATE = "update"; + private static final String UPDATE_LIST = "list"; + private static final String UPDATE_CHECK = "check"; + private static final String UPDATE_INSTALL = "install"; + + private static final List COMMANDS = List.of(INFO, CONSOLE, UPDATE, TYPING); + private static final List UPDATE_COMMANDS = List.of(UPDATE_LIST, UPDATE_CHECK, UPDATE_INSTALL); + private static final List PIP_COMMANDS = List.of(PIP_INSTALL, PIP_UNINSTALL, PIP_SHOW, PIP_LIST); + + private final ScriptEngineManager scriptEngineManager; + private final PythonScriptEngineFactory pythonScriptEngineFactory; + private final ConfigDescriptionRegistry configDescriptionRegistry; + private final PythonScriptEngineConfiguration pythonScriptEngineConfiguration; + + private final String scriptType; + + @Activate + public PythonConsoleCommandExtension( // + @Reference ScriptEngineManager scriptEngineManager, // + @Reference PythonScriptEngineFactory pythonScriptEngineFactory, // + @Reference ConfigDescriptionRegistry configDescriptionRegistry) { + super("pythonscripting", "Python Scripting console utilities."); + this.scriptEngineManager = scriptEngineManager; + this.pythonScriptEngineFactory = pythonScriptEngineFactory; + this.pythonScriptEngineConfiguration = pythonScriptEngineFactory.getConfiguration(); + this.scriptType = PythonScriptEngineFactory.SCRIPT_TYPE; + this.configDescriptionRegistry = configDescriptionRegistry; + } + + @Override + public @Nullable ConsoleCommandCompleter getCompleter() { + return this; + } + + @Override + public List getUsages() { + ArrayList usages = new ArrayList(); + usages.add(buildCommandUsage(INFO, "displays information about Python Scripting add-on")); + usages.add(buildCommandUsage(CONSOLE, "starts an interactive python console")); + usages.add(getUpdateUsage()); + if (pythonScriptEngineConfiguration.isVEnvEnabled()) { + usages.add(getPipUsage()); + } + usages.add(buildCommandUsage(TYPING, "create type hint stub files")); + return usages; + } + + public String getUpdateUsage() { + return buildCommandUsage(UPDATE + " <" + String.join("|", UPDATE_COMMANDS) + ">", "update helper lib module"); + } + + public String getPipUsage() { + return buildCommandUsage(PIP + " <" + String.join("|", PIP_COMMANDS) + "> [optional pip specific arguments]", + "manages python modules"); + } + + @Override + public boolean complete(String[] args, int cursorArgumentIndex, int cursorPosition, List candidates) { + StringsCompleter completer = new StringsCompleter(); + SortedSet strings = completer.getStrings(); + if (cursorArgumentIndex == 0) { + strings.addAll(COMMANDS); + if (pythonScriptEngineConfiguration.isVEnvEnabled()) { + strings.add(PIP); + } + } else if (cursorArgumentIndex == 1) { + if (PIP.equals(args[0])) { + strings.addAll(PIP_COMMANDS); + } else if (UPDATE.equals(args[0])) { + strings.addAll(UPDATE_COMMANDS); + } + } + + return strings.isEmpty() ? false : completer.complete(args, cursorArgumentIndex, cursorPosition, candidates); + } + + @Override + public void execute(String[] args, Console console) { + if (args.length > 0) { + String command = args[0]; + switch (command) { + case "--help": + case "-h": + printUsage(console); + break; + case INFO: + info(console); + break; + case CONSOLE: + startConsole(console, Arrays.copyOfRange(args, 1, args.length)); + break; + case UPDATE: + executeUpdate(console, Arrays.copyOfRange(args, 1, args.length)); + break; + case TYPING: + executeTyping(console); + break; + case PIP: + if (pythonScriptEngineConfiguration.isVEnvEnabled()) { + executePip(console, Arrays.copyOfRange(args, 1, args.length)); + break; + } + default: + console.println("Unknown command '" + command + "'"); + printUsage(console); + break; + } + } else { + printUsage(console); + } + } + + private void info(Console console) { + new InfoCmd(pythonScriptEngineConfiguration, console).show(configDescriptionRegistry); + } + + private void startConsole(Console console, String[] args) { + final String startInteractiveSessionCode = """ + import readline # optional, will allow Up/Down/History in the console + import code + + vars = globals().copy() + vars.update(locals()) + shell = code.InteractiveConsole(vars) + try: + shell.interact() + except SystemExit: + pass + """; + + executePython(console, engine -> engine.eval(startInteractiveSessionCode), true); + } + + private void executeUpdate(Console console, String[] args) { + if (args.length == 0) { + console.println("Missing update action"); + console.printUsage(getUpdateUsage()); + } else if (UPDATE_COMMANDS.indexOf(args[0]) == -1) { + console.println("Unknown update action '" + args[0] + "'"); + console.printUsage(getUpdateUsage()); + } else { + UpdateCmd cmd = new UpdateCmd(pythonScriptEngineConfiguration, console); + switch (args[0]) { + case UPDATE_LIST: + cmd.updateList(); + break; + case UPDATE_CHECK: + cmd.updateCheck(); + break; + case UPDATE_INSTALL: + if (args.length <= 1) { + console.println("Missing release name"); + console.printUsage("pythonscripting update install <\"latest\"|version>"); + } else { + cmd.updateInstall(args[1]); + } + break; + } + } + } + + private void executeTyping(Console console) { + try { + if (!confirmAction(console, "You are about creating python type hint stub files in '" + + PythonScriptEngineConfiguration.PYTHON_TYPINGS_PATH + "'.")) { + return; + } + new TypingCmd(new TypingCmd.Logger(console)).build(); + } catch (Exception e) { + throw new IllegalArgumentException(e); + } + } + + private void executePip(Console console, String[] args) { + if (args.length == 0) { + console.println("Missing pip action"); + console.printUsage(getPipUsage()); + } else if (PIP_COMMANDS.indexOf(args[0]) == -1) { + console.println("Unknown pip action '" + args[1] + "'"); + console.printUsage(getPipUsage()); + } else { + ArrayList params = new ArrayList(Arrays.asList(args)); + + if (PIP_UNINSTALL.equals(args[0]) && args.length >= 2) { + if (!confirmAction(console, "You are uninstalling python modules.")) { + return; + } + params.add(1, "-y"); + } + + final String pipCode = """ + import subprocess + import sys + + command_list = [sys.executable, "-m", "pip"] + PARAMS + with subprocess.Popen(command_list, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) as proc: + for line in proc.stdout: + print(line.rstrip()) + """; + + executePython(console, engine -> { + engine.getContext().setAttribute("PARAMS", params, ScriptContext.ENGINE_SCOPE); + return engine.eval(pipCode); + }, false); + } + } + + private void printLoadingMessage(Console console, boolean show) { + String loadingMessage = "Loading Python script engine..."; + if (show) { + console.print(loadingMessage); + } else { + // Clear the loading message + console.print("\r" + " ".repeat(loadingMessage.length()) + "\r"); + } + } + + /* + * Create Python engine. + * + * withFullContext = true => means a full openHAB-managed script engine with scoped variables + * including any injected required modules. + */ + private @Nullable Object executePython(Console console, EngineEvalFunction process, boolean withFullContext) { + String scriptIdentifier = "python-console-" + UUID.randomUUID().toString(); + ScriptEngine engine = null; + + try { + printLoadingMessage(console, true); + + if (withFullContext) { + ScriptEngineContainer container = scriptEngineManager.createScriptEngine(scriptType, scriptIdentifier); + if (container != null) { + engine = container.getScriptEngine(); + } + } else { + engine = pythonScriptEngineFactory.createScriptEngine(scriptType); + if (engine != null) { + engine.getContext().setAttribute(ScriptEngineFactory.CONTEXT_KEY_ENGINE_IDENTIFIER, + scriptIdentifier, ScriptContext.ENGINE_SCOPE); + } + } + + if (engine == null) { + console.println("Error: Unable to create python script engine."); + return null; + } + + printLoadingMessage(console, false); + + engine.getContext().setAttribute(PythonScriptEngine.CONTEXT_KEY_ENGINE_LOGGER_INPUT, + createInputStream(console), ScriptContext.ENGINE_SCOPE); + engine.getContext().setAttribute(PythonScriptEngine.CONTEXT_KEY_ENGINE_LOGGER_OUTPUT, System.out, + ScriptContext.ENGINE_SCOPE); + + return process.apply(engine); + } catch (ScriptException e) { + console.println("Error: " + e.getMessage()); + return null; + } finally { + if (withFullContext) { + scriptEngineManager.removeEngine(scriptIdentifier); + } else { + if (engine instanceof AutoCloseable closeable) { + try { + closeable.close(); + } catch (Exception e) { + console.println("Error while closing script engine. " + e.getMessage()); + } + } + } + } + } + + private boolean confirmAction(Console console, String msg) { + try { + console.readLine("\n" + msg + "\n\nPress Enter to confirm or Ctrl+C to cancel.", null); + console.println(""); + return true; + } catch (IOException e) { + console.println("Error: " + e.getMessage()); + return false; + } catch (RuntimeException e) { + console.println("Operation cancelled."); + return false; + } + } + + private InputStream createInputStream(Console console) { + return new InputStream() { + byte @Nullable [] buffer = null; + int pos = 0; + + @SuppressWarnings("null") + @Override + public int read() throws IOException { + if (pos < 0) { + pos = 0; + return -1; + } else if (buffer == null) { + assert pos == 0; + try { + String line = console.readLine("", null); + buffer = line.getBytes(StandardCharsets.UTF_8); + } catch (Exception e) { + return -1; + } + } + if (pos == buffer.length) { + buffer = null; + pos = -1; + return '\n'; + } else { + return buffer[pos++]; + } + } + }; + } + + @FunctionalInterface + public interface EngineEvalFunction { + @Nullable + Object apply(ScriptEngine e) throws ScriptException; + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/InfoCmd.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/InfoCmd.java new file mode 100644 index 0000000000000..d9c9f80e227e0 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/InfoCmd.java @@ -0,0 +1,113 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console.handler; + +import java.lang.module.ModuleDescriptor.Version; +import java.net.URI; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.graalvm.polyglot.Language; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngine; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory; +import org.openhab.core.config.core.ConfigDescription; +import org.openhab.core.config.core.ConfigDescriptionParameter; +import org.openhab.core.config.core.ConfigDescriptionRegistry; +import org.openhab.core.io.console.Console; + +/** + * Info command implementation + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class InfoCmd { + private final PythonScriptEngineConfiguration config; + private final Console console; + + public InfoCmd(PythonScriptEngineConfiguration config, Console console) { + this.config = config; + this.console = console; + } + + public void show(ConfigDescriptionRegistry registry) { + console.println("Python Scripting Environment:"); + console.println("======================================"); + console.println(" Runtime:"); + console.println(" Bundle version: " + config.getBundleVersion()); + console.println(" GraalVM version: " + config.getGraalVersion()); + Language language = PythonScriptEngine.getLanguage(); + console.println(" Python version: " + (language != null ? language.getVersion() : "unavailable")); + Version version = config.getInstalledHelperLibVersion(); + console.println(" Helper lib version: " + (version != null ? version.toString() : "disabled")); + console.println(" VEnv state: " + (config.isVEnvEnabled() ? "enabled" : "disabled")); + console.println(" Type hints: " + + (Files.isDirectory(PythonScriptEngineConfiguration.PYTHON_TYPINGS_PATH) ? "available" + : "not yet created")); + console.println(""); + console.println(" Directories:"); + console.println(" Scripts: " + PythonScriptEngineConfiguration.PYTHON_DEFAULT_PATH); + console.println(" Libraries: " + PythonScriptEngineConfiguration.PYTHON_LIB_PATH); + console.println(" Typing: " + PythonScriptEngineConfiguration.PYTHON_TYPINGS_PATH); + Path tempDirectory = config.getTempDirectory(); + console.println(" Temp: " + tempDirectory.toString()); + Path venvDirectory = config.getVEnvDirectory(); + console.println(" VEnv: " + venvDirectory.toString()); + + console.println(""); + console.println("Python Scripting Add-on Configuration:"); + console.println("======================================"); + ConfigDescription configDescription = registry + .getConfigDescription(URI.create(PythonScriptEngineFactory.CONFIG_DESCRIPTION_URI)); + + if (configDescription == null) { + console.println("No configuration found for Python Scripting add-on. This is probably a bug."); + return; + } + + List parameters = configDescription.getParameters(); + Map configMap = config.getConfigurations(); + configDescription.getParameters().forEach(parameter -> { + if (parameter.getGroupName() == null) { + console.println(" " + parameter.getName() + ": " + configMap.get(parameter.getName())); + } + }); + configDescription.getParameterGroups().forEach(group -> { + String groupLabel = group.getLabel(); + if (groupLabel == null) { + groupLabel = group.getName(); + } + console.println(" " + groupLabel); + parameters.forEach(parameter -> { + if (!group.getName().equals(parameter.getGroupName())) { + return; + } + console.print(" " + parameter.getName() + ": "); + String value = configMap.get(parameter.getName()); + if (value == null) { + console.println("not set"); + } else if (value.contains("\n")) { + console.println(" (multiline)"); + console.println(" " + value.replace("\n", "\n ")); + } else { + console.println(value); + } + }); + console.println(""); + }); + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/TypingCmd.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/TypingCmd.java new file mode 100644 index 0000000000000..1b81d023d23c5 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/TypingCmd.java @@ -0,0 +1,173 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console.handler; + +import java.io.File; +import java.io.IOException; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassCollector; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassCollector.ClassContainer; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassConverter; +import org.openhab.core.io.console.Console; + +/** + * Update command implementations + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class TypingCmd { + private final Logger logger; + + private static final String PATH_SEPARATOR = FileSystems.getDefault().getSeparator(); + + public TypingCmd(Logger logger) { + this.logger = logger; + } + + public void build() throws Exception { + Path outputPath = PythonScriptEngineConfiguration.PYTHON_TYPINGS_PATH; + + // Cleanup Directory + if (Files.isDirectory(outputPath)) { + try (Stream paths = Files.walk(outputPath)) { + paths.sorted(Comparator.reverseOrder()).map(Path::toFile).forEach(File::delete); + } + } + ClassCollector collector = new ClassCollector(logger); + + Map fileContainerMap = new HashMap(); + Set imports = new HashSet(); + // Collect Bundle Classes + Map bundleClassMap = collector.collectBundleClasses("org.openhab"); + for (ClassContainer container : bundleClassMap.values()) { + ClassConverter converter = new ClassConverter(container); + String classBody = converter.build(); + imports.addAll(converter.getImports()); + dumpClassContentToFile(classBody, container, outputPath, fileContainerMap); + } + + imports = imports.stream().filter(i -> !i.startsWith("org.openhab")).collect(Collectors.toSet()); + + Map reflectionClassMap = collector.collectReflectionClasses(imports); + for (ClassContainer container : reflectionClassMap.values()) { + ClassConverter converter = new ClassConverter(container); + String classBody = converter.build(); + dumpClassContentToFile(classBody, container, outputPath, fileContainerMap); + } + + // Generate __init__.py Files + dumpInit(outputPath.toString(), fileContainerMap); + + logger.info(bundleClassMap.size() + " bundle and " + reflectionClassMap.size() + " java classes processed"); + logger.info("Total of " + (bundleClassMap.size() + reflectionClassMap.size()) + " type hint files create in '" + + outputPath + "'"); + } + + public void dumpInit(String path, Map fileContainerMap) throws IOException { + File root = new File(path); + File[] list = root.listFiles(); + if (list != null) { + ArrayList files = new ArrayList(); + for (File f : list) { + if (f.isDirectory()) { + dumpInit(f.getAbsolutePath(), fileContainerMap); + } else { + files.add(f); + } + } + + if (!files.isEmpty()) { + StringBuilder initBody = new StringBuilder(); + // List modules = new ArrayList(); + for (File file : files) { + if (file.toString().endsWith("__init__.py")) { + continue; + } + ClassContainer container = fileContainerMap.get(file.toString()); + initBody.append("from .__" + container.getPythonClassName().toLowerCase() + "__ import " + + container.getPythonClassName() + "\n"); + } + + String packageUrl = path.replace(".", PATH_SEPARATOR) + "/__init__.py"; + dumpContentToFile(initBody.toString(), Paths.get(packageUrl)); + } + } + } + + private void dumpClassContentToFile(String classBody, ClassContainer container, Path outputPath, + Map fileContainerMap) throws IOException { + if (classBody.isEmpty()) { + return; + } + + String modulePath = container.getPythonModuleName().replace(".", PATH_SEPARATOR); + Path path = outputPath.resolve(modulePath) + .resolve("__" + container.getPythonClassName().toLowerCase() + "__.py"); + + fileContainerMap.put(path.toString(), container); + + dumpContentToFile(classBody, path); + } + + private void dumpContentToFile(String content, Path path) throws IOException { + Path parent = path.getParent(); + File directory = parent.toFile(); + if (!directory.exists()) { + directory.mkdirs(); + } + Files.write(path, content.getBytes()); + } + + public static class Logger { + private Object logger; + + public Logger(Console console) { + this.logger = console; + } + + public Logger(org.slf4j.Logger logger) { + this.logger = logger; + } + + public void info(String s) { + if (logger instanceof Console console) { + console.println("INFO: " + s); + } else { + ((org.slf4j.Logger) logger).info("{}", s); + } + } + + public void warn(String s) { + if (logger instanceof Console console) { + console.println("WARN: " + s); + } else { + ((org.slf4j.Logger) logger).warn("{}", s); + } + } + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/UpdateCmd.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/UpdateCmd.java new file mode 100644 index 0000000000000..cd917018f298d --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/UpdateCmd.java @@ -0,0 +1,204 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console.handler; + +import java.io.IOException; +import java.lang.module.ModuleDescriptor.Version; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.net.http.HttpResponse.BodyHandlers; +import java.time.OffsetDateTime; +import java.time.format.DateTimeFormatter; +import java.util.Locale; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.eclipse.jdt.annotation.Nullable; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineHelper; +import org.openhab.core.io.console.Console; + +import com.google.gson.JsonArray; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParser; + +/** + * Update command implementations + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class UpdateCmd { + private static final String UPDATE_RELEASES_URL = "https://api.github.com/repos/openhab/openhab-python/releases"; + private static final String UPDATE_LATEST_URL = "https://api.github.com/repos/openhab/openhab-python/releases/latest"; + + private final PythonScriptEngineConfiguration config; + private final Console console; + + public UpdateCmd(PythonScriptEngineConfiguration config, Console console) { + this.config = config; + this.console = console; + } + + public void updateList() { + JsonElement rootElement = getReleaseData(UPDATE_RELEASES_URL); + if (rootElement != null) { + Version installedVersion = config.getInstalledHelperLibVersion(); + Version providedVersion = config.getProvidedHelperLibVersion(); + console.println("Version Released Active"); + console.println("----------------------------------------------"); + if (rootElement.isJsonArray()) { + JsonArray list = rootElement.getAsJsonArray(); + for (JsonElement element : list.asList()) { + String tagName = element.getAsJsonObject().get("tag_name").getAsString(); + String publishString = element.getAsJsonObject().get("published_at").getAsString(); + + boolean isInstalled = false; + try { + Version availableVersion = PythonScriptEngineConfiguration.parseHelperLibVersion(tagName); + if (availableVersion.equals(installedVersion)) { + isInstalled = true; + } else if (availableVersion.compareTo(providedVersion) < 0) { + continue; + } + + DateTimeFormatter df = DateTimeFormatter.ofPattern("dd.MM.yyyy HH:mm:ss", Locale.getDefault()); + OffsetDateTime publishDate = OffsetDateTime.parse(publishString); + console.println( + String.format("%-19s", tagName) + " " + String.format("%-19s", df.format(publishDate)) + + " " + String.format("%-6s", (isInstalled ? "*" : " "))); + } catch (IllegalArgumentException e) { + // ignore not parseable version + } + } + } else { + console.println("Fetching releases failed. Invalid data"); + } + } + } + + public void updateCheck() { + Version installedVersion = config.getInstalledHelperLibVersion(); + if (installedVersion == null) { + console.println("Helper libs disabled. Skipping update."); + } else { + JsonElement rootElement = getReleaseData(UPDATE_LATEST_URL); + if (rootElement != null) { + JsonElement tagName = rootElement.getAsJsonObject().get("tag_name"); + Version latestVersion = PythonScriptEngineConfiguration.parseHelperLibVersion(tagName.getAsString()); + if (latestVersion.compareTo(installedVersion) > 0) { + console.println("Update from version '" + installedVersion + "' to version '" + + latestVersion.toString() + "' available."); + } else { + console.println("Latest version '" + installedVersion + "' already installed."); + } + } + } + } + + public void updateInstall(String requestedVersionString) { + JsonObject releaseObj = null; + Version releaseVersion = null; + if ("latest".equals(requestedVersionString)) { + JsonElement rootElement = getReleaseData(UPDATE_LATEST_URL); + if (rootElement != null) { + releaseObj = rootElement.getAsJsonObject();// + JsonElement tagName = releaseObj.get("tag_name"); + releaseVersion = PythonScriptEngineConfiguration.parseHelperLibVersion(tagName.getAsString()); + } + } else { + try { + Version requestedVersion = PythonScriptEngineConfiguration + .parseHelperLibVersion(requestedVersionString); + JsonElement rootElement = getReleaseData(UPDATE_RELEASES_URL); + if (rootElement != null) { + if (rootElement.isJsonArray()) { + JsonArray list = rootElement.getAsJsonArray(); + for (JsonElement element : list.asList()) { + JsonElement tagName = element.getAsJsonObject().get("tag_name"); + try { + releaseVersion = PythonScriptEngineConfiguration + .parseHelperLibVersion(tagName.getAsString()); + } catch (IllegalArgumentException e) { + continue; + } + if (releaseVersion.compareTo(requestedVersion) == 0) { + releaseObj = element.getAsJsonObject(); + break; + } + } + } + } + } catch (IllegalArgumentException e) { + // continue, if no version was found + } + } + + if (releaseObj != null && releaseVersion != null) { + Version installedVersion = config.getInstalledHelperLibVersion(); + Version providedVersion = config.getProvidedHelperLibVersion(); + if (releaseVersion.compareTo(providedVersion) < 0) { + console.println("Outdated version '" + releaseVersion.toString() + "' not supported"); + releaseVersion = null; + } else if (installedVersion != null) { + if (releaseVersion.compareTo(installedVersion) < 0) { + // Don't install older version, if 'latest' was requested + if ("latest".equals(requestedVersionString)) { + console.println("Newer Version '" + installedVersion.toString() + "' already installed"); + releaseVersion = null; + } + } else if (releaseVersion.equals(installedVersion)) { + console.println("Version '" + releaseVersion.toString() + "' already installed"); + releaseVersion = null; + } + } + + if (releaseVersion != null) { + String zipballUrl = releaseObj.get("zipball_url").getAsString(); + + try { + PythonScriptEngineHelper.installHelperLib(zipballUrl, releaseVersion, config); + console.println("Version '" + releaseVersion.toString() + "' installed successfully"); + } catch (URISyntaxException | IOException e) { + console.println("Fetching release zip '" + zipballUrl + "' file failed. "); + throw new IllegalArgumentException(e); + } + } + } else { + console.println("Version '" + requestedVersionString + "' not found. "); + } + } + + private @Nullable JsonElement getReleaseData(String url) { + try { + HttpClient client = HttpClient.newHttpClient(); + HttpRequest request = HttpRequest.newBuilder().uri(URI.create(url)) + .header("Accept", "application/vnd.github+json").GET().build(); + + HttpResponse response = client.send(request, BodyHandlers.ofString()); + if (response.statusCode() == 200) { + JsonElement obj = JsonParser.parseString(response.body()); + return obj; + } else { + console.println("Fetching releases failed. Status code is " + response.statusCode()); + } + + } catch (IOException | InterruptedException e) { + console.println("Fetching releases failed. Request interrupted " + e.getLocalizedMessage()); + } + return null; + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassCollector.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassCollector.java new file mode 100644 index 0000000000000..271377dab95df --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassCollector.java @@ -0,0 +1,313 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console.handler.typing; + +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.lang.reflect.Parameter; +import java.lang.reflect.Type; +import java.net.URL; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Enumeration; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.stream.Collectors; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.openhab.automation.pythonscripting.internal.console.handler.TypingCmd.Logger; +import org.osgi.framework.Bundle; +import org.osgi.framework.FrameworkUtil; + +/** + * Collects classes + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class ClassCollector { + private final Logger logger; + + public ClassCollector(Logger logger) { + this.logger = logger; + } + + public Map collectBundleClasses(String packageName) throws Exception { + List> clsList = new ArrayList>(); + Bundle bundle = FrameworkUtil.getBundle(ClassCollector.class); + Bundle[] bundles = bundle.getBundleContext().getBundles(); + for (Bundle b : bundles) { + List> bundleClsList = new ArrayList>(); + Enumeration entries = b.findEntries(packageName.replace(".", "/"), "*.class", true); + if (entries != null) { + while (entries.hasMoreElements()) { + String entry = entries.nextElement().toString(); + String clsName = entry.substring(entry.indexOf(packageName.replace(".", "/"))); + if (clsName.indexOf("/internal") != -1) { + continue; + } + clsName = clsName.replace(".class", "").replace("/", "."); + try { + Class cls = Class.forName(clsName); + if (!Modifier.isPublic(cls.getModifiers())) { + continue; + } + bundleClsList.add(cls); + } catch (ClassNotFoundException e) { + logger.warn("BUNDLE: " + b + " class " + clsName + " not found"); + } + } + } + if (!bundleClsList.isEmpty()) { + logger.info("BUNDLE: " + b + " with " + bundleClsList.size() + " classes processed"); + clsList.addAll(bundleClsList); + } + } + return processClasses(clsList); + } + + public Map collectReflectionClasses(Set imports) throws Exception { + List> clsList = new ArrayList>(); + for (String imp : imports) { + if (imp.startsWith("__")) { + continue; + } + + try { + clsList.add(Class.forName(imp)); + } catch (ClassNotFoundException e) { + logger.warn("Class " + imp + " not found"); + } + } + return processClasses(clsList); + } + + private Map processClasses(List> clsList) { + Map result = new HashMap(); + for (Class cls : clsList) { + result.put(cls.getName(), new ClassContainer(cls)); + } + return result; + } + + public static class ClassContainer { + private Class cls; + private List fields; + private Map methods = new HashMap(); + + private String pythonClassName; + private String pythonModuleName; + + public ClassContainer(Class cls) { + this.cls = cls; + + String packageName = cls.getName(); + this.pythonClassName = ClassContainer.parsePythonClassName(packageName); + this.pythonModuleName = ClassContainer.parsePythonModuleName(packageName); + + fields = Arrays.stream(cls.getDeclaredFields()).filter( + method -> Modifier.isPublic(method.getModifiers()) && Modifier.isStatic(method.getModifiers())) + .collect(Collectors.toList()); + + Constructor[] constructors = cls.getConstructors(); + for (Constructor constructor : constructors) { + String uid = constructor.getName(); + MethodContainer methodContainer; + if (!this.methods.containsKey(uid)) { + methodContainer = new MethodContainer(constructor); + this.methods.put(uid, methodContainer); + } + this.methods.get(uid).addParametersFrom(constructor); + } + + List methods = Arrays.stream(cls.getDeclaredMethods()).filter( + method -> Modifier.isPublic(method.getModifiers()) && !Modifier.isVolatile(method.getModifiers())) + .collect(Collectors.toList()); + for (Method method : methods) { + String uid = method.getName(); + MethodContainer methodContainer; + if (!this.methods.containsKey(uid)) { + methodContainer = new MethodContainer(method); + this.methods.put(uid, methodContainer); + } + this.methods.get(uid).addParametersFrom(method); + } + } + + public Class getRelatedClass() { + return cls; + } + + public List getFields() { + return fields; + } + + public List getMethods() { + return new ArrayList(methods.values()); + } + + public String getPythonClassName() { + return pythonClassName; + } + + public String getPythonModuleName() { + return pythonModuleName; + } + + public static String parsePythonClassName(String name) { + String className = name.substring(name.lastIndexOf(".") + 1); + if (className.contains("$")) { + className = className.replace("$", "_"); + } + return className; + } + + public static String parsePythonModuleName(String name) { + return name.substring(0, name.lastIndexOf(".")); + } + } + + public static class MethodContainer { + int modifier; + String methodName; + String rawStringRepresentation; + boolean isConstructor = false; + int mandatoryParameterCount = 999; + + List returnTypes = new ArrayList(); + List> returnClasses = new ArrayList>(); + List args = new ArrayList(); + + public MethodContainer(Constructor constructor) { + this.modifier = constructor.getModifiers(); + this.methodName = ClassContainer.parsePythonClassName(constructor.getName()); + this.rawStringRepresentation = constructor.toString(); + this.isConstructor = true; + } + + public MethodContainer(Method method) { + this.modifier = method.getModifiers(); + this.methodName = method.getName(); + this.rawStringRepresentation = method.toString(); + this.returnTypes.add(method.getGenericReturnType()); + this.returnClasses.add(method.getReturnType()); + } + + public void addParametersFrom(Constructor constructor) { + init(constructor.getParameters(), constructor.getGenericParameterTypes()); + } + + public void addParametersFrom(Method method) { + returnTypes.add(method.getGenericReturnType()); + returnClasses.add(method.getReturnType()); + + init(method.getParameters(), method.getGenericParameterTypes()); + } + + private void init(Parameter[] parameters, Type[] gpType) { + for (int i = 0; i < parameters.length; i++) { + if (args.size() <= i) { + args.add(new ParameterContainer(parameters[i], gpType[i])); + } else { + args.get(i).addParameter(gpType[i]); + } + } + + if (parameters.length < mandatoryParameterCount) { + mandatoryParameterCount = parameters.length; + } + for (int i = mandatoryParameterCount; i < args.size(); i++) { + args.get(i).markAsOptional(); + } + } + + public String getPythonMethodName() { + return methodName; + } + + public boolean isConstructor() { + return this.isConstructor; + } + + public int getModifiers() { + return modifier; + } + + public String getRawStringRepresentation() { + return rawStringRepresentation; + } + + public int getReturnTypeCount() { + return returnTypes.size(); + } + + public Type getGenericReturnType(int index) { + return returnTypes.get(index); + } + + public Class getReturnType(int index) { + return returnClasses.get(index); + } + + public List getParameters() { + return args; + } + + public int getParameterCount() { + return args.size(); + } + + public ParameterContainer getParameter(int index) { + return args.get(index); + } + } + + public static class ParameterContainer { + Parameter parameter; + boolean isOptional = false; + List types = new ArrayList(); + + public ParameterContainer(Parameter arg, Type type) { + this.parameter = arg; + this.types.add(type); + } + + public void addParameter(Type type) { + types.add(type); + } + + public String getName() { + return parameter.getName(); + } + + public int getTypeCount() { + return types.size(); + } + + public Type getGenericType(int index) { + return types.get(index); + } + + public void markAsOptional() { + isOptional = true; + } + + public boolean isOptional() { + return isOptional; + } + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassConverter.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassConverter.java new file mode 100644 index 0000000000000..9a1778f11d744 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/console/handler/typing/ClassConverter.java @@ -0,0 +1,576 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.console.handler.typing; + +import java.io.IOException; +import java.lang.reflect.Field; +import java.lang.reflect.GenericArrayType; +import java.lang.reflect.Modifier; +import java.lang.reflect.ParameterizedType; +import java.lang.reflect.Type; +import java.lang.reflect.TypeVariable; +import java.lang.reflect.WildcardType; +import java.math.BigDecimal; +import java.math.BigInteger; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.Dictionary; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Hashtable; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.eclipse.jdt.annotation.Nullable; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassCollector.ClassContainer; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassCollector.MethodContainer; +import org.openhab.automation.pythonscripting.internal.console.handler.typing.ClassCollector.ParameterContainer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Converts a Java class to a Python class stub + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class ClassConverter { + private final Logger logger = LoggerFactory.getLogger(ClassConverter.class); + + private static Pattern classMatcher = Pattern + .compile("(?:(?:super|extends) )?([a-z0-9\\.\\$_]+|\\?)(?:<.*?>|\\[\\])?$", Pattern.CASE_INSENSITIVE); + + private static Pattern instanceMatcher = Pattern.compile("([a-z0-9\\.\\$_]+)[;]{0,1}@[a-z0-9]+", + Pattern.CASE_INSENSITIVE); + + private static String baseUrl; + static { + // Version version = FrameworkUtil.getBundle(OpenHAB.class).getVersion(); + String v = "latest"; // version.getQualifier() == null || version.getQualifier().isEmpty() ? version.toString() + // : "latest"; + baseUrl = "https://www.openhab.org/javadoc/" + v + "/"; + } + + private final Map classGenerics; + private final Map imports; + private final ClassContainer container; + + public ClassConverter(ClassContainer container) { + this.container = container; + this.classGenerics = new HashMap(); + this.imports = new HashMap(); + } + + public List getImports() { + return new ArrayList(this.imports.keySet()); + } + + public String build() throws IOException, ClassNotFoundException { + // Class head + StringBuilder classBody = new StringBuilder(); + classBody.append(buildClassHead()); + + // Class documentation + String doc = buildClassDocumentationBlock(); + if (doc != null) { + classBody.append(doc); + classBody.append("\n"); + } + + // Class fields + classBody.append(buildClassFields()); + + // Class methods + List methods = container.getMethods(); + Collections.sort(methods, new Comparator() { + @Override + public int compare(MethodContainer o1, MethodContainer o2) { + return o1.getPythonMethodName().compareTo(o2.getPythonMethodName()); + } + }); + for (MethodContainer method : methods) { + + classBody.append(buildClassMethod(method)); + } + + // Class imports + if (!imports.isEmpty()) { + classBody.insert(0, "\n\n"); + classBody.insert(0, buildClassImports()); + } + + return classBody.toString(); + } + + private String buildClassHead() { + Type type = container.getRelatedClass().getGenericSuperclass(); + if (type != null) { + classGenerics.putAll(collectGenerics(type)); + } + + StringBuilder builder = new StringBuilder(); + builder.append("class " + container.getPythonClassName()); + List parentTypes = new ArrayList(); + Class parentClass = container.getRelatedClass().getSuperclass(); + if (parentClass != null) { + String pythonType = convertBaseJavaToPythonType(new JavaType(parentClass.getName()), classGenerics); + if (!"object".equals(pythonType)) { + parentTypes.add(pythonType); + } + } + Class[] parentInterfaces = container.getRelatedClass().getInterfaces(); + for (Class parentInterface : parentInterfaces) { + String pythonType = convertBaseJavaToPythonType(new JavaType(parentInterface.getName()), classGenerics); + parentTypes.add(pythonType); + } + if (parentTypes.isEmpty()) { + if (container.getRelatedClass().isInterface()) { + parentTypes.add("Protocol"); + imports.put("__typing.Protocol", "from typing import Protocol"); + } + } + if (!parentTypes.isEmpty()) { + builder.append("(" + String.join(", ", parentTypes) + ")"); + } + builder.append(":\n"); + return builder.toString(); + } + + private String buildClassFields() { + StringBuilder builder = new StringBuilder(); + for (Field field : container.getFields()) { + try { + JavaType javaType = collectJavaTypes(field.getGenericType(), classGenerics); + String type = convertJavatToPythonType(javaType, classGenerics); + String value = ""; + Class t = field.getType(); + + if (Collection.class.isAssignableFrom(field.getType())) { + List values = new ArrayList(); + if (field.get(null) instanceof Collection col) { + for (Object val : col) { + values.add(convertFieldValue(val)); + } + } + value = "[" + String.join(",", values) + "]"; + } else if (t == short.class) { + value = convertFieldValue(field.getShort(null)); + } else if (t == int.class) { + value = convertFieldValue(field.getInt(null)); + } else if (t == long.class) { + value = convertFieldValue(field.getLong(null)); + } else if (t == float.class) { + value = convertFieldValue(field.getFloat(null)); + } else if (t == double.class) { + value = convertFieldValue(field.getDouble(null)); + } else if (t == boolean.class) { + value = convertFieldValue(field.getBoolean(null)); + } else if (t == char.class) { + value = convertFieldValue(field.getChar(null)); + } else { + value = convertFieldValue(field.get(null)); + } + + builder.append(" " + field.getName() + ": " + type + " = " + value + "\n"); + } catch (IllegalArgumentException | IllegalAccessException e) { + logger.warn("Cant convert static field {} of class {}", container.getRelatedClass().getName(), + field.getName(), e); + } + } + if (!builder.isEmpty()) { + builder.append("\n"); + } + + return builder.toString(); + } + + private String buildClassMethod(MethodContainer method) { + HashMap localGenerics = new HashMap(this.classGenerics); + // Collect generics + for (int i = 0; i < method.getReturnTypeCount(); i++) { + localGenerics.putAll(collectGenerics(method.getGenericReturnType(i))); + } + for (ParameterContainer p : method.getParameters()) { + for (int i = 0; i < p.getTypeCount(); i++) { + localGenerics.putAll(collectGenerics(p.getGenericType(i))); + } + } + + // Collect arguments + List arguments = new ArrayList(); + if (!Modifier.isStatic(method.getModifiers())) { + arguments.add("self"); + } + for (ParameterContainer p : method.getParameters()) { + Set parameterTypes = new HashSet(); + for (int i = 0; i < p.getTypeCount(); i++) { + JavaType javaType = collectJavaTypes(p.getGenericType(i), localGenerics); + String t = convertJavatToPythonType(javaType, localGenerics); + parameterTypes.add(t); + } + List sorted = parameterTypes.stream().sorted().collect(Collectors.toList()); + arguments.add(p.getName() + ": " + String.join(" | ", sorted) + (p.isOptional ? " = None" : "")); + } + + // Build method + StringBuilder builder = new StringBuilder(); + if (Modifier.isStatic(method.getModifiers())) { + builder.append(" @staticmethod\n"); + } + String methodName = method.isConstructor() ? "__init__" : method.getPythonMethodName(); + builder.append(" def " + methodName + "(" + String.join(", ", arguments) + ")"); + + // Build return value + if (method.getReturnTypeCount() > 0) { + // Collect Return types + Set returnTypes = new HashSet(); + for (int i = 0; i < method.getReturnTypeCount(); i++) { + JavaType javaType = collectJavaTypes(method.getGenericReturnType(i), localGenerics); + String t = convertJavatToPythonType(javaType, localGenerics); + returnTypes.add(t); + } + List sortedReturnTypes = returnTypes.stream().sorted().collect(Collectors.toList()); + + builder.append(" -> " + String.join(" | ", sortedReturnTypes)); + } + + // Finalize method + builder.append(":\n"); + String doc = buildMethodDocumentationBlock(method); + if (doc != null) { + builder.append(doc); + } else { + builder.append(" pass\n"); + } + builder.append("\n"); + + return builder.toString(); + } + + private Object buildClassImports() { + StringBuilder builder = new StringBuilder(); + HashSet hashSet = new HashSet<>(imports.values()); + ArrayList sortedImports = new ArrayList<>(hashSet); + Collections.sort(sortedImports, new Comparator() { + @Override + public int compare(String o1, String o2) { + if (o1.length() > o2.length()) { + return 1; + } + if (o1.length() < o2.length()) { + return -1; + } + return o1.compareTo(o2); + } + }); + for (String importLine : sortedImports) { + builder.append(importLine + "\n"); + } + return builder.toString(); + } + + private JavaType collectJavaTypes(Type genericType, Map generics) { + JavaType javaType = null; + if (genericType instanceof TypeVariable) { + TypeVariable typeVar = (TypeVariable) genericType; + String type = generics.get(typeVar.getTypeName()); + if (type != null) { + javaType = new JavaType(type); + } else { + javaType = new JavaType("?"); + } + } else if (genericType instanceof ParameterizedType) { + ParameterizedType paramType = (ParameterizedType) genericType; + // System.out.println("ParameterizedType | " + _type + " | " + _type.getRawType().getTypeName() + " | " + // + _type.getActualTypeArguments()[0]); + javaType = collectJavaTypes(paramType.getRawType(), generics); + for (Type type : paramType.getActualTypeArguments()) { + javaType.addSubType(collectJavaTypes(type, generics)); + } + } else if (genericType instanceof WildcardType) { + WildcardType wildcardType = (WildcardType) genericType; + // System.out.println("WildcardType | " + _type); + if (wildcardType.getUpperBounds().length > 0) { + javaType = collectJavaTypes(wildcardType.getUpperBounds()[0], generics); + } else if (wildcardType.getLowerBounds().length > 0) { + javaType = collectJavaTypes(wildcardType.getLowerBounds()[0], generics); + } else { + javaType = new JavaType("java.lang.Object"); + } + } else if (genericType instanceof GenericArrayType) { + GenericArrayType genericArrayType = (GenericArrayType) genericType; + // System.out.println("GenericArrayType | " + _type); + javaType = new JavaType("java.util.List"); + javaType.addSubType(collectJavaTypes(genericArrayType.getGenericComponentType(), generics)); + } else { + // System.out.println("OtherType | " + genericType.getTypeName()); + String type = genericType.getTypeName(); + JavaType activeJavaType = null; + while (type.endsWith("[]")) { + JavaType currentJavaType = new JavaType("java.util.List"); + if (activeJavaType == null) { + javaType = activeJavaType = currentJavaType; + } else { + activeJavaType.addSubType(currentJavaType); + } + activeJavaType = currentJavaType; + type = type.substring(0, type.length() - 2); + } + JavaType currentJavaType = new JavaType(type); + if (javaType == null) { + javaType = currentJavaType; + } else { + activeJavaType.addSubType(currentJavaType); + } + } + + return javaType; + } + + private String convertJavatToPythonType(JavaType javaType, Map generics) { + switch (javaType.getType()) { + case "char": + return "str"; + case "long": + case "int": + case "short": + return "int"; + case "double": + case "float": + return "float"; + case "byte": + return "bytes"; + case "boolean": + return "bool"; + case "void": + return "None"; + case "?": + if (javaType.hasSubTypes(1)) { + return convertJavatToPythonType(javaType.getSubType(0), generics); + } + return "object"; + } + + try { + Class cls = Class.forName(javaType.getType()); + if (!cls.equals(java.lang.Object.class)) { + if (Byte.class.equals(cls)) { + return "bytes"; + } else if (Double.class.equals(cls) || Float.class.equals(cls)) { + return "float"; + } else if (BigDecimal.class.equals(cls) || BigInteger.class.equals(cls) || Long.class.equals(cls) + || Integer.class.equals(cls) || Short.class.equals(cls)) { + return "int"; + } else if (Number.class.equals(cls)) { + return "float | int"; + } else if (Dictionary.class.equals(cls) || Hashtable.class.equals(cls) || Map.class.equals(cls) + || HashMap.class.equals(cls)) { + if (javaType.hasSubTypes(2)) { + return "dict[" + convertJavatToPythonType(javaType.getSubType(0), generics) + "," + + convertJavatToPythonType(javaType.getSubType(1), generics) + "]"; + } + return "dict"; + } else if (Collection.class.equals(cls) || List.class.equals(cls) || Set.class.equals(cls)) { + if (javaType.hasSubTypes(1)) { + return "list[" + convertJavatToPythonType(javaType.getSubType(0), generics) + "]"; + } + return "list"; + } else if (Iterable.class.equals(cls)) { + if (javaType.hasSubTypes(1)) { + return "iter[" + convertJavatToPythonType(javaType.getSubType(0), generics) + "]"; + } + return "iter"; + } else if (cls.equals(Class.class)) { + if (javaType.hasSubTypes(1)) { + return "type[" + convertJavatToPythonType(javaType.getSubType(0), generics) + "]"; + } + return "type"; + } + } + } catch (ClassNotFoundException e) { + } + + return convertBaseJavaToPythonType(javaType, generics); + } + + private String convertBaseJavaToPythonType(JavaType javaType, Map generics) { + switch (javaType.getType()) { + case "java.lang.String": + return "str"; + case "java.lang.Object": + return "object"; + case "java.time.ZonedDateTime": + imports.put("__java.time.ZonedDateTime", "from datetime import datetime"); + return "datetime"; + case "java.time.Instant": + imports.put("__java.time.Instant", "from datetime import datetime"); + return "datetime"; + default: + String typeName = cleanClassName(javaType.getType()); + if (typeName.contains(".")) { + String className = ClassContainer.parsePythonClassName(typeName); + // Handle import + if (!className.equals(container.getPythonClassName())) { + String moduleName = ClassContainer.parsePythonModuleName(typeName); + imports.put(typeName, "from " + moduleName + " import " + className); + return className; + } + return "\"" + className + "\""; + } else if (typeName.length() == 1) { + String newTypeName = generics.get(typeName); + if (newTypeName != null) { + return newTypeName; + } + } + return typeName; + } + } + + private String convertFieldValue(@Nullable Object value) { + if (value == null) { + return "None"; + } + if (value instanceof Number) { + return value.toString(); + } + if (value instanceof Boolean) { + return ((Boolean) value) ? "True" : "False"; + } + if (value instanceof String) { + return "\"" + value.toString() + "\""; + } + String valueAsString = value.toString(); + Matcher matcher = instanceMatcher.matcher(valueAsString); + if (matcher.find()) { + valueAsString = "<" + matcher.group(1) + ">"; + } + return "\"" + valueAsString + "\""; + } + + private Map collectGenerics(Type genericType) { + return collectGenerics(genericType, new HashMap()); + } + + private Map collectGenerics(Type genericType, Map generics) { + if (genericType instanceof TypeVariable) { + TypeVariable typeVar = (TypeVariable) genericType; + if (!generics.containsKey(typeVar.getTypeName())) { + generics.put(typeVar.getTypeName(), cleanClassName(typeVar.getBounds()[0].getTypeName())); + } + } else if (genericType instanceof ParameterizedType) { + ParameterizedType paramType = (ParameterizedType) genericType; + generics.putAll(collectGenerics(paramType.getRawType(), generics)); + for (Type typeArg : paramType.getActualTypeArguments()) { + generics.putAll(collectGenerics(typeArg, generics)); + } + } else if (genericType instanceof WildcardType) { + WildcardType wildcardType = (WildcardType) genericType; + if (wildcardType.getUpperBounds().length > 0) { + for (Type upperType : wildcardType.getUpperBounds()) { + generics.putAll(collectGenerics(upperType, generics)); + } + } else if (wildcardType.getLowerBounds().length > 0) { + for (Type lowerType : wildcardType.getLowerBounds()) { + generics.putAll(collectGenerics(lowerType, generics)); + } + } + } + return generics; + } + + private String cleanClassName(String className) { + Matcher matcher = classMatcher.matcher(className); + if (matcher.find() && !className.equals(matcher.group(1))) { + // System.out.println("parseSubType: " + container.getRelatedClass().getName() + " | " + // + _type.getTypeName() + " | " + javaSubType + " | " + matcher.group(1)); + return matcher.group(1); + } + return className; + } + + private @Nullable String buildClassDocumentationBlock() { + if (!container.getPythonModuleName().startsWith("org.openhab.core")) { + return null; + } + + String classUrl = baseUrl + + container.getRelatedClass().getName().toLowerCase().replace(".", "/").replace("$", "."); + + StringBuilder builder = new StringBuilder(); + builder.append(" \"\"\"\n"); + builder.append(" Java class: ").append(container.getRelatedClass().getName()).append("\n\n"); + builder.append(" Java doc: ").append(classUrl); + builder.append("\n"); + builder.append(" \"\"\"\n"); + return builder.toString(); + } + + private @Nullable String buildMethodDocumentationBlock(MethodContainer method) { + if (!container.getPythonModuleName().startsWith("org.openhab.core")) { + return null; + } + + String classUrl = baseUrl + + container.getRelatedClass().getName().toLowerCase().replace(".", "/").replace("$", "."); + + StringBuilder builder = new StringBuilder(); + builder.append(" \"\"\"\n"); + builder.append(" Java doc url:\n"); + + String functionRepresentation = method.getRawStringRepresentation(); + Pattern pattern = Pattern.compile("([^\\.]+\\([^\\)]*\\))", Pattern.CASE_INSENSITIVE); + Matcher matcher1 = pattern.matcher(functionRepresentation); + if (matcher1.find()) { + functionRepresentation = matcher1.group(1); + } + functionRepresentation = functionRepresentation.replaceAll("<>\\?", "").replace("$", "."); + // System.out.println(classUrl + "#" + functionRepresentation); + builder.append(" ").append(classUrl).append("#").append(functionRepresentation); + + builder.append("\n"); + builder.append(" \"\"\"\n"); + return builder.toString(); + } + + public static class JavaType { + private final String type; + private final List subTypes = new ArrayList(); + + public JavaType(String type) { + this.type = type; + } + + public void addSubType(JavaType subType) { + this.subTypes.add(subType); + } + + public String getType() { + return this.type; + } + + public boolean hasSubTypes(int neededSize) { + return this.subTypes.size() >= neededSize; + } + + public JavaType getSubType(int index) { + return this.subTypes.get(index); + } + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextInput.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextInput.java new file mode 100644 index 0000000000000..7917ac48860b4 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextInput.java @@ -0,0 +1,52 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.context; + +import java.io.IOException; +import java.io.InputStream; + +import org.eclipse.jdt.annotation.NonNullByDefault; +import org.eclipse.jdt.annotation.Nullable; + +/** + * ContextInput wraps an @nullable InputStream, used as Standard Input for pythonscripting + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class ContextInput extends InputStream { + private @Nullable InputStream stream; + + public ContextInput(@Nullable InputStream stream) { + this.stream = stream; + } + + public void setInputStream(@Nullable InputStream stream) { + this.stream = stream; + } + + @Override + public void close() throws IOException { + if (stream != null) { + stream.close(); + } + } + + @Override + public int read() throws IOException { + if (stream != null) { + return stream.read(); + } + return -1; + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutput.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutput.java new file mode 100644 index 0000000000000..8af33cbdce289 --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutput.java @@ -0,0 +1,51 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.context; + +import java.io.IOException; +import java.io.OutputStream; + +import org.eclipse.jdt.annotation.NonNullByDefault; + +/** + * ContextOutput wraps an @nullable OutputStream, used as Standard Output for pythonscripting + * + * @author Holger Hees - Initial contribution + */ +@NonNullByDefault +public class ContextOutput extends OutputStream { + private OutputStream stream; + + public ContextOutput(OutputStream stream) { + this.stream = stream; + } + + public void setOutputStream(OutputStream stream) { + this.stream = stream; + } + + @Override + public void close() throws IOException { + stream.close(); + } + + @Override + public void flush() throws IOException { + stream.flush(); + } + + @Override + public void write(int b) throws IOException { + stream.write(b); + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LogOutputStream.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutputLogger.java similarity index 88% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LogOutputStream.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutputLogger.java index bff0ab70864a0..a57120fbb123a 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LogOutputStream.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/context/ContextOutputLogger.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.scriptengine.helper; +package org.openhab.automation.pythonscripting.internal.context; import java.io.OutputStream; @@ -19,24 +19,24 @@ import org.slf4j.event.Level; /** - * LogOutputStream implementation + * ContextOutputLogger is the default way of handling the Standard Output for pythonscripting. * * @author Holger Hees - Initial contribution */ @NonNullByDefault -public class LogOutputStream extends OutputStream { +public class ContextOutputLogger extends OutputStream { private static final int DEFAULT_BUFFER_LENGTH = 2048; private static final String LINE_SEPARATOR = System.lineSeparator(); private static final int LINE_SEPARATOR_SIZE = LINE_SEPARATOR.length(); - private Logger logger; - private Level level; - private int bufLength; private byte[] buf; private int count; - public LogOutputStream(Logger logger, Level level) { + private Logger logger; + private Level level; + + public ContextOutputLogger(Logger logger, Level level) { this.logger = logger; this.level = level; @@ -45,14 +45,6 @@ public LogOutputStream(Logger logger, Level level) { count = 0; } - public void setLogger(Logger logger) { - this.logger = logger; - } - - public Logger getLogger() { - return logger; - } - @Override public void write(int b) { // don't log nulls diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/DelegatingFileSystem.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/DelegatingFileSystem.java index 104a02bc40c1d..4bf8a3ddb83e1 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/DelegatingFileSystem.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/DelegatingFileSystem.java @@ -16,7 +16,9 @@ import java.net.URI; import java.nio.channels.SeekableByteChannel; import java.nio.file.AccessMode; +import java.nio.file.CopyOption; import java.nio.file.DirectoryStream; +import java.nio.file.FileSystems; import java.nio.file.LinkOption; import java.nio.file.OpenOption; import java.nio.file.Path; @@ -25,19 +27,31 @@ import java.nio.file.spi.FileSystemProvider; import java.util.Map; import java.util.Set; +import java.util.function.Consumer; +import org.eclipse.jdt.annotation.NonNull; import org.graalvm.polyglot.io.FileSystem; /** * Delegate wrapping a {@link FileSystem} * - * @author Holger Hees - Initial contribution (Reused from jsscripting) + * @author Holger Hees - Initial contribution */ public class DelegatingFileSystem implements FileSystem { - private FileSystemProvider delegate; + // Inspiration from + // https://github.com/oracle/graal/blob/master/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/FileSystems.java + private final FileSystemProvider delegate = FileSystems.getDefault().provider(); + private final Path tmpDir; - public DelegatingFileSystem(FileSystemProvider delegate) { - this.delegate = delegate; + private volatile Path userDir; + private volatile Consumer consumer; + + public DelegatingFileSystem(Path tmpDir) { + this.tmpDir = tmpDir; + } + + public void setAccessConsumer(Consumer<@NonNull Path> consumer) { + this.consumer = consumer; } @Override @@ -66,29 +80,116 @@ public void delete(Path path) throws IOException { } @Override - public SeekableByteChannel newByteChannel(Path path, Set options, FileAttribute... attrs) - throws IOException { - return delegate.newByteChannel(path, options, attrs); + public void copy(Path source, Path target, CopyOption... options) throws IOException { + delegate.copy(resolveRelative(source), resolveRelative(target), options); } @Override - public DirectoryStream newDirectoryStream(Path dir, DirectoryStream.Filter filter) - throws IOException { - return delegate.newDirectoryStream(dir, filter); + public void move(Path source, Path target, CopyOption... options) throws IOException { + delegate.move(resolveRelative(source), resolveRelative(target), options); } @Override - public Path toAbsolutePath(Path path) { - return path.toAbsolutePath(); + public void createLink(Path link, Path existing) throws IOException { + delegate.createLink(resolveRelative(link), resolveRelative(existing)); } @Override - public Path toRealPath(Path path, LinkOption... linkOptions) throws IOException { - return path.toRealPath(linkOptions); + public void createSymbolicLink(Path link, Path target, FileAttribute... attrs) throws IOException { + delegate.createSymbolicLink(resolveRelative(link), target, attrs); + } + + @Override + public Path readSymbolicLink(Path link) throws IOException { + return delegate.readSymbolicLink(resolveRelative(link)); } @Override public Map readAttributes(Path path, String attributes, LinkOption... options) throws IOException { return delegate.readAttributes(path, attributes, options); } + + @Override + public void setAttribute(Path path, String attribute, Object value, LinkOption... options) throws IOException { + delegate.setAttribute(path, attribute, value, options); + } + + @Override + public Path toAbsolutePath(Path path) { + if (path.isAbsolute()) { + return path; + } + Path cwd = userDir; + if (cwd == null) { + return path.toAbsolutePath(); + } else { + return cwd.resolve(path); + } + } + + @Override + public void setCurrentWorkingDirectory(Path currentWorkingDirectory) { + userDir = currentWorkingDirectory; + } + + @Override + public Path toRealPath(Path path, LinkOption... linkOptions) throws IOException { + return resolveRelative(path).toRealPath(linkOptions); + } + + @Override + public Path getTempDirectory() { + return tmpDir; + } + + @Override + public boolean isSameFile(Path path1, Path path2, LinkOption... options) throws IOException { + if (isFollowLinks(options)) { + Path absolutePath1 = resolveRelative(path1); + Path absolutePath2 = resolveRelative(path2); + return delegate.isSameFile(absolutePath1, absolutePath2); + } else { + return delegate.isSameFile(path1, path2); + } + } + + @Override + public SeekableByteChannel newByteChannel(Path path, Set options, FileAttribute... attrs) + throws IOException { + final Path resolved = resolveRelative(path); + if (consumer != null) { + consumer.accept(resolved); + } + try { + return delegate.newFileChannel(resolved, options, attrs); + } catch (UnsupportedOperationException uoe) { + return delegate.newByteChannel(resolved, options, attrs); + } + } + + @Override + public DirectoryStream newDirectoryStream(Path dir, DirectoryStream.Filter filter) + throws IOException { + Path cwd = userDir; + Path resolvedPath; + if (!dir.isAbsolute() && cwd != null) { + resolvedPath = cwd.resolve(dir); + } else { + resolvedPath = dir; + } + return delegate.newDirectoryStream(resolvedPath, filter); + } + + private Path resolveRelative(Path path) { + return !path.isAbsolute() && userDir != null ? toAbsolutePath(path) : path; + } + + private boolean isFollowLinks(final LinkOption... linkOptions) { + for (LinkOption lo : linkOptions) { + if (lo == LinkOption.NOFOLLOW_LINKS) { + return false; + } + } + return true; + } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonDependencyTracker.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonDependencyTracker.java similarity index 92% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonDependencyTracker.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonDependencyTracker.java index 0be4ff9c4e2dc..ceb34b816c27f 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonDependencyTracker.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonDependencyTracker.java @@ -10,10 +10,10 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.fs.watch; +package org.openhab.automation.pythonscripting.internal.fs; import org.eclipse.jdt.annotation.NonNullByDefault; -import org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; import org.openhab.core.automation.module.script.ScriptDependencyTracker; import org.openhab.core.automation.module.script.rulesupport.loader.AbstractScriptDependencyTracker; import org.openhab.core.service.WatchService; @@ -35,7 +35,7 @@ public class PythonDependencyTracker extends AbstractScriptDependencyTracker { @Activate public PythonDependencyTracker(@Reference(target = WatchService.CONFIG_WATCHER_FILTER) WatchService watchService) { - super(watchService, PythonScriptEngineFactory.PYTHON_LIB_PATH.toString()); + super(watchService, PythonScriptEngineConfiguration.PYTHON_LIB_PATH.toString()); } @Deactivate diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonScriptFileWatcher.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonScriptFileWatcher.java similarity index 82% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonScriptFileWatcher.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonScriptFileWatcher.java index 754ae7d9d59a9..8a36c338321c1 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/watch/PythonScriptFileWatcher.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/fs/PythonScriptFileWatcher.java @@ -10,12 +10,13 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.fs.watch; +package org.openhab.automation.pythonscripting.internal.fs; import java.nio.file.Path; import java.util.Optional; import org.eclipse.jdt.annotation.NonNullByDefault; +import org.openhab.automation.pythonscripting.internal.PythonScriptEngineConfiguration; import org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory; import org.openhab.core.automation.module.script.ScriptDependencyTracker; import org.openhab.core.automation.module.script.ScriptEngineManager; @@ -33,7 +34,8 @@ * * @author Holger Hees - Initial contribution (Reused from jsscripting) */ -@Component(immediate = true, service = { ScriptFileWatcher.class, ScriptDependencyTracker.Listener.class }) +@Component(immediate = true, service = { ScriptFileWatcher.class, ScriptDependencyTracker.Listener.class, + PythonScriptFileWatcher.class }) @NonNullByDefault public class PythonScriptFileWatcher extends AbstractScriptFileWatcher { @Activate @@ -42,12 +44,13 @@ public PythonScriptFileWatcher( final @Reference ScriptEngineManager manager, final @Reference ReadyService readyService, final @Reference StartLevelService startLevelService) { super(watchService, manager, readyService, startLevelService, - PythonScriptEngineFactory.PYTHON_DEFAULT_PATH.toString(), true); + PythonScriptEngineConfiguration.PYTHON_DEFAULT_PATH.toString(), true); } @Override protected Optional getScriptType(Path scriptFilePath) { - if (!scriptFilePath.startsWith(PythonScriptEngineFactory.PYTHON_LIB_PATH)) { + if (!scriptFilePath.startsWith(PythonScriptEngineConfiguration.PYTHON_LIB_PATH) + && !scriptFilePath.startsWith(PythonScriptEngineConfiguration.PYTHON_TYPINGS_PATH)) { Optional scriptFileSuffix = super.getScriptType(scriptFilePath); if (scriptFileSuffix.isPresent() && "py".equals(scriptFileSuffix.get())) { return Optional.of(PythonScriptEngineFactory.SCRIPT_TYPE); diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/AbstractScriptExtensionProvider.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/AbstractScriptExtensionProvider.java similarity index 96% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/AbstractScriptExtensionProvider.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/AbstractScriptExtensionProvider.java index b7f13feaa3904..d38c1ceba2387 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/AbstractScriptExtensionProvider.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/AbstractScriptExtensionProvider.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.scope; +package org.openhab.automation.pythonscripting.internal.provider; import java.util.Collection; import java.util.HashMap; @@ -66,6 +66,7 @@ public Collection getTypes() { return types.keySet(); } + @SuppressWarnings("null") @Override public @Nullable Object get(String scriptIdentifier, String type) throws IllegalArgumentException { Map forScript = idToTypes.computeIfAbsent(scriptIdentifier, k -> new HashMap<>()); diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LifecycleTracker.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/LifecycleTracker.java similarity index 92% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LifecycleTracker.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/LifecycleTracker.java index 688d9bc3a5bd5..ce7d11c1d47ce 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/helper/LifecycleTracker.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/LifecycleTracker.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.scriptengine.helper; +package org.openhab.automation.pythonscripting.internal.provider; import java.util.ArrayList; import java.util.List; diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/OSGiScriptExtensionProvider.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/OSGiScriptExtensionProvider.java similarity index 94% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/OSGiScriptExtensionProvider.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/OSGiScriptExtensionProvider.java index 847da61e15549..2085dddf6b1b4 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scope/OSGiScriptExtensionProvider.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/OSGiScriptExtensionProvider.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.scope; +package org.openhab.automation.pythonscripting.internal.provider; import org.eclipse.jdt.annotation.NonNullByDefault; import org.openhab.core.automation.module.script.ScriptExtensionProvider; diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ScriptExtensionModuleProvider.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/ScriptExtensionModuleProvider.java similarity index 95% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ScriptExtensionModuleProvider.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/ScriptExtensionModuleProvider.java index 1a70c5c5ced7a..dd1c7fc4b19d7 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ScriptExtensionModuleProvider.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/provider/ScriptExtensionModuleProvider.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.wrapper; +package org.openhab.automation.pythonscripting.internal.provider; import java.util.ArrayList; import java.util.HashMap; @@ -95,4 +95,8 @@ public ModuleLocator locatorFor(Context ctx, String engineIdentifier, public void put(String key, Object value) { this.globals.put(key, value); } + + public static interface ModuleLocator { + Map locateModule(String name, List fromlist); + } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/DelegatingScriptEngineWithInvocableAndCompilableAndAutocloseable.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/DelegatingScriptEngineWithInvocableAndCompilableAndAutocloseable.java deleted file mode 100644 index ebc8daea361f1..0000000000000 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/DelegatingScriptEngineWithInvocableAndCompilableAndAutocloseable.java +++ /dev/null @@ -1,141 +0,0 @@ -/* - * Copyright (c) 2010-2025 Contributors to the openHAB project - * - * See the NOTICE file(s) distributed with this work for additional - * information. - * - * This program and the accompanying materials are made available under the - * terms of the Eclipse Public License 2.0 which is available at - * http://www.eclipse.org/legal/epl-2.0 - * - * SPDX-License-Identifier: EPL-2.0 - */ -package org.openhab.automation.pythonscripting.internal.scriptengine; - -import java.io.Reader; - -import javax.script.Bindings; -import javax.script.Compilable; -import javax.script.CompiledScript; -import javax.script.Invocable; -import javax.script.ScriptContext; -import javax.script.ScriptEngine; -import javax.script.ScriptEngineFactory; -import javax.script.ScriptException; - -/** - * {@link ScriptEngine} implementation that delegates to a supplied ScriptEngine instance. Allows overriding specific - * methods. - * - * @author Holger Hees - Initial contribution (Reused from jsscripting) - */ -public abstract class DelegatingScriptEngineWithInvocableAndCompilableAndAutocloseable - implements ScriptEngine, Invocable, Compilable, AutoCloseable { - protected T delegate; - - @Override - public Object eval(String s, ScriptContext scriptContext) throws ScriptException { - return delegate.eval(s, scriptContext); - } - - @Override - public Object eval(Reader reader, ScriptContext scriptContext) throws ScriptException { - return delegate.eval(reader, scriptContext); - } - - @Override - public Object eval(String s) throws ScriptException { - return delegate.eval(s); - } - - @Override - public Object eval(Reader reader) throws ScriptException { - return delegate.eval(reader); - } - - @Override - public Object eval(String s, Bindings bindings) throws ScriptException { - return delegate.eval(s, bindings); - } - - @Override - public Object eval(Reader reader, Bindings bindings) throws ScriptException { - return delegate.eval(reader, bindings); - } - - @Override - public void put(String s, Object o) { - delegate.put(s, o); - } - - @Override - public Object get(String s) { - return delegate.get(s); - } - - @Override - public Bindings getBindings(int i) { - return delegate.getBindings(i); - } - - @Override - public void setBindings(Bindings bindings, int i) { - delegate.setBindings(bindings, i); - } - - @Override - public Bindings createBindings() { - return delegate.createBindings(); - } - - @Override - public ScriptContext getContext() { - return delegate.getContext(); - } - - @Override - public void setContext(ScriptContext scriptContext) { - delegate.setContext(scriptContext); - } - - @Override - public ScriptEngineFactory getFactory() { - return delegate.getFactory(); - } - - @Override - public Object invokeMethod(Object o, String s, Object... objects) - throws ScriptException, NoSuchMethodException, IllegalArgumentException { - return delegate.invokeMethod(o, s, objects); - } - - @Override - public Object invokeFunction(String s, Object... objects) throws ScriptException, NoSuchMethodException { - return delegate.invokeFunction(s, objects); - } - - @Override - public T getInterface(Class aClass) { - return delegate.getInterface(aClass); - } - - @Override - public T getInterface(Object o, Class aClass) { - return delegate.getInterface(o, aClass); - } - - @Override - public CompiledScript compile(String s) throws ScriptException { - return delegate.compile(s); - } - - @Override - public CompiledScript compile(Reader reader) throws ScriptException { - return delegate.compile(reader); - } - - @Override - public void close() throws Exception { - delegate.close(); - } -} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingPythonScriptEngine.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingPythonScriptEngine.java new file mode 100644 index 0000000000000..aa37b28dea91d --- /dev/null +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingPythonScriptEngine.java @@ -0,0 +1,99 @@ +/* + * Copyright (c) 2010-2025 Contributors to the openHAB project + * + * See the NOTICE file(s) distributed with this work for additional + * information. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ +package org.openhab.automation.pythonscripting.internal.scriptengine; + +import java.lang.reflect.UndeclaredThrowableException; + +import javax.script.CompiledScript; +import javax.script.ScriptContext; +import javax.script.ScriptException; + +import org.eclipse.jdt.annotation.NonNull; +import org.graalvm.polyglot.PolyglotException; +import org.openhab.automation.pythonscripting.internal.scriptengine.graal.GraalPythonScriptEngine; + +/** + * Interception of calls, either before Invocation, or upon a {@link ScriptException} being + * thrown. + * + * @author Holger Hees - Initial contribution + */ +public abstract class InvocationInterceptingPythonScriptEngine extends GraalPythonScriptEngine { + protected abstract void beforeInvocation() throws PolyglotException; + + protected abstract String beforeInvocation(String source); + + protected abstract Object afterInvocation(Object obj); + + protected abstract E afterThrowsInvocation(@NonNull E e); + + @Override + public Object eval(String s, ScriptContext scriptContext) throws ScriptException { + try { + beforeInvocation(); + return afterInvocation(super.eval(beforeInvocation(s), scriptContext)); + } catch (ScriptException e) { + throw afterThrowsInvocation(e); + } catch (PolyglotException e) { + throw afterThrowsInvocation(toScriptException(e)); + } catch (Exception e) { + throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions + } + } + + @Override + public Object invokeMethod(Object o, String s, Object... objects) throws ScriptException, NoSuchMethodException { + try { + beforeInvocation(); + return afterInvocation(super.invokeMethod(o, s, objects)); + } catch (ScriptException e) { + throw afterThrowsInvocation(e); + } catch (PolyglotException e) { + throw afterThrowsInvocation(toScriptException(e)); + } catch (NoSuchMethodException e) { + throw afterThrowsInvocation(e); + } catch (Exception e) { + throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions + } + } + + @Override + public Object invokeFunction(String s, Object... objects) throws ScriptException, NoSuchMethodException { + try { + beforeInvocation(); + return afterInvocation(super.invokeFunction(s, objects)); + } catch (ScriptException e) { + throw afterThrowsInvocation(e); + } catch (PolyglotException e) { + throw afterThrowsInvocation(toScriptException(e)); + } catch (NoSuchMethodException e) { + throw afterThrowsInvocation(e); + } catch (Exception e) { + throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions + } + } + + @Override + public CompiledScript compile(String s) throws ScriptException { + try { + beforeInvocation(); + return (CompiledScript) afterInvocation(super.compile(beforeInvocation(s))); + } catch (ScriptException e) { + throw afterThrowsInvocation(e); + } catch (PolyglotException e) { + throw afterThrowsInvocation(toScriptException(e)); + } catch (Exception e) { + throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions + } + } +} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable.java deleted file mode 100644 index 23219a458c128..0000000000000 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable.java +++ /dev/null @@ -1,196 +0,0 @@ -/* - * Copyright (c) 2010-2025 Contributors to the openHAB project - * - * See the NOTICE file(s) distributed with this work for additional - * information. - * - * This program and the accompanying materials are made available under the - * terms of the Eclipse Public License 2.0 which is available at - * http://www.eclipse.org/legal/epl-2.0 - * - * SPDX-License-Identifier: EPL-2.0 - */ -package org.openhab.automation.pythonscripting.internal.scriptengine; - -import java.io.Reader; -import java.lang.reflect.UndeclaredThrowableException; - -import javax.script.Bindings; -import javax.script.Compilable; -import javax.script.CompiledScript; -import javax.script.Invocable; -import javax.script.ScriptContext; -import javax.script.ScriptEngine; -import javax.script.ScriptException; - -/** - * Delegate allowing AOP-style interception of calls, either before Invocation, or upon a {@link ScriptException} being - * thrown. - * - * @param The delegate class - * @author Holger Hees - Initial contribution (Reused from jsscripting) - */ -public abstract class InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable - extends DelegatingScriptEngineWithInvocableAndCompilableAndAutocloseable { - - protected String beforeInvocation(String source) { - beforeInvocation(); - return source; - } - - protected Reader beforeInvocation(Reader reader) { - beforeInvocation(); - return reader; - } - - protected void beforeInvocation() { - } - - protected Object afterInvocation(Object obj) { - return obj; - } - - protected Exception afterThrowsInvocation(Exception e) { - return e; - } - - @Override - public Object eval(String s, ScriptContext scriptContext) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(s), scriptContext)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object eval(Reader reader, ScriptContext scriptContext) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(reader), scriptContext)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object eval(String s) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(s))); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object eval(Reader reader) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(reader))); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object eval(String s, Bindings bindings) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(s), bindings)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object eval(Reader reader, Bindings bindings) throws ScriptException { - try { - return afterInvocation(super.eval(beforeInvocation(reader), bindings)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object invokeMethod(Object o, String s, Object... objects) - throws ScriptException, NoSuchMethodException, NullPointerException, IllegalArgumentException { - try { - beforeInvocation(); - return afterInvocation(super.invokeMethod(o, s, objects)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (NoSuchMethodException e) { // Make sure to unlock on exceptions from Invocable.invokeMethod to avoid - // deadlocks - throw (NoSuchMethodException) afterThrowsInvocation(e); - } catch (NullPointerException e) { - throw (NullPointerException) afterThrowsInvocation(e); - } catch (IllegalArgumentException e) { - throw (IllegalArgumentException) afterThrowsInvocation(e); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public Object invokeFunction(String s, Object... objects) - throws ScriptException, NoSuchMethodException, NullPointerException { - try { - beforeInvocation(); - return afterInvocation(super.invokeFunction(s, objects)); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (NoSuchMethodException e) { // Make sure to unlock on exceptions from Invocable.invokeFunction to avoid - // deadlocks - throw (NoSuchMethodException) afterThrowsInvocation(e); - } catch (NullPointerException e) { - throw (NullPointerException) afterThrowsInvocation(e); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public CompiledScript compile(String s) throws ScriptException { - try { - return wrapCompiledScript((CompiledScript) afterInvocation(super.compile(beforeInvocation(s)))); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - @Override - public CompiledScript compile(Reader reader) throws ScriptException { - try { - return wrapCompiledScript((CompiledScript) afterInvocation(super.compile(beforeInvocation(reader)))); - } catch (ScriptException se) { - throw (ScriptException) afterThrowsInvocation(se); - } catch (Exception e) { - throw new UndeclaredThrowableException(afterThrowsInvocation(e)); // Wrap and rethrow other exceptions - } - } - - private CompiledScript wrapCompiledScript(CompiledScript script) { - return new CompiledScript() { - @Override - public ScriptEngine getEngine() { - return InvocationInterceptingScriptEngineWithInvocableAndCompilableAndAutoCloseable.this; - } - - @Override - public Object eval(ScriptContext context) throws ScriptException { - return script.eval(context); - } - }; - } -} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonBindings.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonBindings.java similarity index 72% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonBindings.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonBindings.java index f42554afb5d0f..36ba0e355be16 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonBindings.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonBindings.java @@ -10,7 +10,7 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.graal; +package org.openhab.automation.pythonscripting.internal.scriptengine.graal; import java.util.AbstractMap; import java.util.HashMap; @@ -21,6 +21,7 @@ import javax.script.ScriptEngine; import org.graalvm.polyglot.Context; +import org.graalvm.polyglot.PolyglotException; import org.graalvm.polyglot.Value; /*** @@ -38,57 +39,26 @@ final class GraalPythonBindings extends AbstractMap implements j private ScriptContext scriptContext; private ScriptEngine scriptEngine; + private boolean isClosed = false; + GraalPythonBindings(Context.Builder contextBuilder, ScriptContext scriptContext, ScriptEngine scriptEngine) { this.contextBuilder = contextBuilder; this.scriptContext = scriptContext; this.scriptEngine = scriptEngine; } - GraalPythonBindings(Context context, ScriptContext scriptContext, ScriptEngine scriptEngine) { - this.context = context; - this.scriptContext = scriptContext; - this.scriptEngine = scriptEngine; - - initGlobal(); - } - - private void requireContext() { - if (context == null) { - context = GraalPythonScriptEngine.createDefaultContext(contextBuilder, scriptContext); - - initGlobal(); - } - } - - private void initGlobal() { - this.global = new HashMap<>(); - + public Context getContext() { requireContext(); - - context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID).putMember("__engine__", scriptEngine); - if (scriptContext != null) { - context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID).putMember("__context__", scriptContext); - } + return context; } @Override public Object put(String key, Object v) { requireContext(); - context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID).putMember(key, v); return global.put(key, v); } - @Override - public void clear() { - if (context != null) { - Value binding = context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID); - for (var entry : global.entrySet()) { - binding.removeMember(entry.getKey()); - } - } - } - @Override public Object get(Object key) { requireContext(); @@ -104,9 +74,14 @@ public Object remove(Object key) { return prev; } - public Context getContext() { - requireContext(); - return context; + @Override + public void clear() { + if (context != null) { + Value binding = context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID); + for (var entry : global.entrySet()) { + binding.removeMember(entry.getKey()); + } + } } @Override @@ -115,14 +90,38 @@ public Set> entrySet() { return global.entrySet(); } + /** + * Closes the current context and makes it unusable. + * + * @throws PolyglotException when an error happens in guest language + * @throws IllegalStateException when an operation is performed after closing + */ @Override - public void close() { + public void close() throws PolyglotException, IllegalStateException { if (context != null) { - context.close(); + context.close(true); + // context = null; + // global = null; } + isClosed = true; } - void updateEngineScriptContext(ScriptContext scriptContext) { - this.scriptContext = scriptContext; + public boolean isClosed() { + return isClosed; + } + + private void requireContext() { + if (context == null) { + if (isClosed) { + throw new IllegalStateException("Context already closed"); + } + context = contextBuilder.build(); + global = new HashMap<>(); + + context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID).putMember("__engine__", scriptEngine); + if (scriptContext != null) { + context.getBindings(GraalPythonScriptEngine.LANGUAGE_ID).putMember("__context__", scriptContext); + } + } } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngine.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngine.java similarity index 50% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngine.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngine.java index 21a6da9a72a25..2678675fde32b 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngine.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngine.java @@ -10,12 +10,10 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.graal; +package org.openhab.automation.pythonscripting.internal.scriptengine.graal; import java.io.File; import java.io.IOException; -import java.io.InputStream; -import java.io.OutputStream; import java.io.Reader; import java.lang.reflect.Method; import java.lang.reflect.Modifier; @@ -30,250 +28,115 @@ import javax.script.ScriptException; import org.graalvm.polyglot.Context; -import org.graalvm.polyglot.Context.Builder; import org.graalvm.polyglot.Engine; -import org.graalvm.polyglot.HostAccess; import org.graalvm.polyglot.PolyglotException; import org.graalvm.polyglot.Source; import org.graalvm.polyglot.SourceSection; import org.graalvm.polyglot.Value; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** - * A Graal.Python implementation of the script engine. It provides access to the polyglot context using - * {@link #getPolyglotContext()}. + * A Graal.Python implementation of the script engine. * * @author Holger Hees - Initial contribution * @author Jeff James - Initial contribution */ -public final class GraalPythonScriptEngine extends AbstractScriptEngine +public abstract class GraalPythonScriptEngine extends AbstractScriptEngine implements Compilable, Invocable, AutoCloseable { - public static final String LANGUAGE_ID = "python"; - private static final String POLYGLOT_CONTEXT = "polyglot.context"; - - private static final String PYTHON_OPTION_POSIXMODULEBACKEND = "python.PosixModuleBackend"; - private static final String PYTHON_OPTION_DONTWRITEBYTECODEFLAG = "python.DontWriteBytecodeFlag"; - private static final String PYTHON_OPTION_FORCEIMPORTSITE = "python.ForceImportSite"; - private static final String PYTHON_OPTION_CHECKHASHPYCSMODE = "python.CheckHashPycsMode"; - private final Logger logger = LoggerFactory.getLogger(GraalPythonScriptEngine.class); + private GraalPythonScriptEngineFactory factory; + private GraalPythonBindings bindings; - private final GraalPythonScriptEngineFactory factory; - private final Context.Builder contextConfig; - - GraalPythonScriptEngine(GraalPythonScriptEngineFactory factory) { - this(factory, factory.getPolyglotEngine(), null); + /** + * Creates a new GraalPython script engine from a polyglot Engine instance with a base configuration + * + * @param engine + * + * @param engine the engine to be used for context configurations + * @param contextConfig a base configuration to create new context instances + */ + protected void init(Engine engine, Context.Builder contextConfig, ScriptEngineProvider scriptEngineProvider) { + this.bindings = new GraalPythonBindings(contextConfig.engine(engine), this.context, this); + this.factory = new GraalPythonScriptEngineFactory(engine, scriptEngineProvider); + this.context.setBindings(bindings, ScriptContext.ENGINE_SCOPE); } - GraalPythonScriptEngine(GraalPythonScriptEngineFactory factory, Engine engine, Context.Builder contextConfig) { - Engine engineToUse = (engine != null) ? engine : factory.getPolyglotEngine(); - - Context.Builder contextConfigToUse = contextConfig; - if (contextConfigToUse == null) { - contextConfigToUse = Context.newBuilder(LANGUAGE_ID) // - .allowExperimentalOptions(true) // - .allowAllAccess(true) // - .allowHostAccess(HostAccess.ALL) // - // allow creating python threads - .allowCreateThread(true) // - // allow running Python native extensions - .allowNativeAccess(true) // - // allow exporting Python values to polyglot bindings and accessing Java - // choose the backend for the POSIX module - .option(PYTHON_OPTION_POSIXMODULEBACKEND, "java") // - // equivalent to the Python -B flag - .option(PYTHON_OPTION_DONTWRITEBYTECODEFLAG, "true") // - // Force to automatically import site.py module, to make Python packages available - .option(PYTHON_OPTION_FORCEIMPORTSITE, "true") // - // causes the interpreter to always assume hash-based pycs are valid - .option(PYTHON_OPTION_CHECKHASHPYCSMODE, "never"); - } - this.factory = (factory == null) ? new GraalPythonScriptEngineFactory(engineToUse) : factory; - this.contextConfig = contextConfigToUse.engine(engineToUse); - this.context.setBindings(new GraalPythonBindings(this.contextConfig, this.context, this), - ScriptContext.ENGINE_SCOPE); + /* + * First call will initialize lazy polyglot context + */ + protected Context getPolyglotContext() { + return bindings.getContext(); } - static Context createDefaultContext(Context.Builder builder, ScriptContext ctxt) { - return builder.build(); + protected boolean isClosed() { + return bindings.isClosed(); } /** - * Closes the current context and makes it unusable. Operations performed after closing will - * throw an {@link IllegalStateException}. + * Closes the current context and makes it unusable. + * + * Error happens in guest language will throw an {@link PolyglotException}. + * Operations performed after closing will throw an {@link IllegalStateException}. */ @Override - public void close() { - logger.debug("GraalPythonScriptEngine closed"); - - // "true" to get an exception if something is still running in context - getPolyglotContext().close(true); - } - - /** - * Returns the polyglot engine associated with this script engine. - */ - public Engine getPolyglotEngine() { - return factory.getPolyglotEngine(); - } - - public Context getPolyglotContext() { - return getOrCreateGraalPythonBindings(context).getContext(); + public void close() throws ScriptException, IllegalStateException { + try { + bindings.close(); + } catch (PolyglotException e) { + throw toScriptException(e); + } } - static Value evalInternal(Context context, String script) { - return context.eval(Source.newBuilder(LANGUAGE_ID, script, "internal-script").internal(true).buildLiteral()); + @Override + public GraalPythonScriptEngineFactory getFactory() { + return factory; } @Override public Bindings createBindings() { - return new GraalPythonBindings(contextConfig, null, this); + // Creating a new binding to replace the current one is not needed for scripting addons + throw new IllegalArgumentException("Creating new bindings is not supported"); } @Override public void setBindings(Bindings bindings, int scope) { - if (scope == ScriptContext.ENGINE_SCOPE) { - Bindings oldBindings = getBindings(scope); - if (oldBindings instanceof GraalPythonBindings gpBindings) { - gpBindings.updateEngineScriptContext(null); - } - } - super.setBindings(bindings, scope); - if (scope == ScriptContext.ENGINE_SCOPE && bindings instanceof GraalPythonBindings gpBindings) { - gpBindings.updateEngineScriptContext(getContext()); - } + // Setting a new binding to replace the current one is not needed for scripting addons + throw new IllegalArgumentException("Setting bindings is not supported"); } @Override public Object eval(Reader reader, ScriptContext ctxt) throws ScriptException { - return eval(createSource(read(reader), ctxt), ctxt); - } - - static String read(Reader reader) throws ScriptException { - final StringBuilder builder = new StringBuilder(); - final char[] buffer = new char[1024]; - try { - try (reader) { - while (true) { - final int count = reader.read(buffer); - if (count == -1) { - break; - } - builder.append(buffer, 0, count); - } - } - return builder.toString(); - } catch (IOException ioex) { - throw new ScriptException(ioex); - } + return eval(read(reader), ctxt); } @Override public Object eval(String script, ScriptContext ctxt) throws ScriptException { - return eval(createSource(script, ctxt), ctxt); - } - - private static Source createSource(String script, ScriptContext ctxt) throws ScriptException { - final Object val = ctxt.getAttribute(ScriptEngine.FILENAME); - if (val == null) { - return Source.newBuilder(LANGUAGE_ID, script, "").buildLiteral(); - } else { - try { - return Source.newBuilder(LANGUAGE_ID, new File(val.toString())).content(script).build(); - } catch (IOException ioex) { - throw new ScriptException(ioex); - } - } + return eval(createSource(script, ctxt)); } - private Object eval(Source source, ScriptContext scriptContext) throws ScriptException { - GraalPythonBindings engineBindings = getOrCreateGraalPythonBindings(scriptContext); - Context polyglotContext = engineBindings.getContext(); + private Object eval(Source source) throws ScriptException { try { - return polyglotContext.eval(source).as(Object.class); + return getPolyglotContext().eval(source).as(Object.class); } catch (PolyglotException e) { throw toScriptException(e); } } - private static ScriptException toScriptException(PolyglotException ex) { - ScriptException sex; - if (ex.isHostException()) { - Throwable hostException = ex.asHostException(); - // ScriptException (unlike almost any other exception) does not - // accept Throwable cause (requires the cause to be Exception) - Exception cause; - if (hostException instanceof Exception) { - cause = (Exception) hostException; - } else { - cause = new Exception(hostException); - } - // Make the host exception accessible through the cause chain - sex = new ScriptException(cause); - // Re-use the stack-trace of PolyglotException (with guest-language stack-frames) - sex.setStackTrace(ex.getStackTrace()); - } else { - SourceSection sourceSection = ex.getSourceLocation(); - if (sourceSection != null && sourceSection.isAvailable()) { - Source source = sourceSection.getSource(); - String fileName = source.getPath(); - if (fileName == null) { - fileName = source.getName(); - } - int lineNo = sourceSection.getStartLine(); - int columnNo = sourceSection.getStartColumn(); - sex = new ScriptException(ex.getMessage(), fileName, lineNo, columnNo); - sex.initCause(ex); - } else { - sex = new ScriptException(ex); - } - } - return sex; - } - - private GraalPythonBindings getOrCreateGraalPythonBindings(ScriptContext scriptContext) { - Bindings engineB = scriptContext.getBindings(ScriptContext.ENGINE_SCOPE); - if (engineB instanceof GraalPythonBindings) { - return ((GraalPythonBindings) engineB); - } else { - GraalPythonBindings bindings = new GraalPythonBindings(createContext(engineB), scriptContext, this); - bindings.putAll(engineB); - return bindings; - } - } - - private Context createContext(Bindings engineB) { - Object ctx = engineB.get(POLYGLOT_CONTEXT); - if (!(ctx instanceof Context)) { - ctx = createDefaultContext(contextConfig, context); - engineB.put(POLYGLOT_CONTEXT, ctx); - } - return (Context) ctx; - } - - @Override - public GraalPythonScriptEngineFactory getFactory() { - return factory; - } - @Override public Object invokeMethod(Object thiz, String name, Object... args) throws ScriptException, NoSuchMethodException { if (thiz == null) { throw new IllegalArgumentException("thiz is not a valid object."); } - GraalPythonBindings engineBindings = getOrCreateGraalPythonBindings(context); - Value thisValue = engineBindings.getContext().asValue(thiz); - if (!thisValue.canInvokeMember(name)) { - if (!thisValue.hasMember(name)) { - throw noSuchMethod(name); - } else { - throw notCallable(name); - } - } try { + Value thisValue = getPolyglotContext().asValue(thiz); + if (!thisValue.canInvokeMember(name)) { + if (!thisValue.hasMember(name)) { + throw new NoSuchMethodException(name); + } else { + throw new NoSuchMethodException(name + " is not a function"); + } + } return thisValue.invokeMember(name, args).as(Object.class); } catch (PolyglotException e) { throw toScriptException(e); @@ -282,29 +145,19 @@ public Object invokeMethod(Object thiz, String name, Object... args) throws Scri @Override public Object invokeFunction(String name, Object... args) throws ScriptException, NoSuchMethodException { - GraalPythonBindings engineBindings = getOrCreateGraalPythonBindings(context); - Value function = engineBindings.getContext().getBindings(LANGUAGE_ID).getMember(name); - - if (function == null) { - throw noSuchMethod(name); - } else if (!function.canExecute()) { - throw notCallable(name); - } try { + Value function = getPolyglotContext().getBindings(LANGUAGE_ID).getMember(name); + if (function == null) { + throw new NoSuchMethodException(name); + } else if (!function.canExecute()) { + throw new NoSuchMethodException(name + " is not a function"); + } return function.execute(args).as(Object.class); } catch (PolyglotException e) { throw toScriptException(e); } } - private static NoSuchMethodException noSuchMethod(String name) throws NoSuchMethodException { - throw new NoSuchMethodException(name); - } - - private static NoSuchMethodException notCallable(String name) throws NoSuchMethodException { - throw new NoSuchMethodException(name + " is not a function"); - } - @Override public T getInterface(Class clasz) { checkInterface(clasz); @@ -322,23 +175,9 @@ public T getInterface(Object thiz, Class clasz) { return getInterfaceInner(thisValue, clasz); } - private static void checkInterface(Class clasz) { - if (clasz == null || !clasz.isInterface()) { - throw new IllegalArgumentException("interface Class expected in getInterface"); - } - } - - private static void checkThis(Value thiz) { - if (thiz.isHostObject() || !thiz.hasMembers()) { - throw new IllegalArgumentException("getInterface cannot be called on non-script object"); - } - } - - private static T getInterfaceInner(Value thiz, Class iface) { - if (!isInterfaceImplemented(iface, thiz)) { - return null; - } - return thiz.as(iface); + @Override + public CompiledScript compile(Reader reader) throws ScriptException { + return compile(read(reader)); } @Override @@ -347,14 +186,14 @@ public CompiledScript compile(String script) throws ScriptException { return compile(source); } - @Override - public CompiledScript compile(Reader reader) throws ScriptException { - Source source = createSource(read(reader), getContext()); - return compile(source); - } - private CompiledScript compile(Source source) throws ScriptException { - checkSyntax(source); + try { + // Syntax check + getPolyglotContext().parse(source); + } catch (PolyglotException pex) { + throw toScriptException(pex); + } + return new CompiledScript() { @Override public ScriptEngine getEngine() { @@ -363,35 +202,60 @@ public ScriptEngine getEngine() { @Override public Object eval(ScriptContext ctx) throws ScriptException { - return GraalPythonScriptEngine.this.eval(source, ctx); + return GraalPythonScriptEngine.this.eval(source); } }; } - private void checkSyntax(Source source) throws ScriptException { + private static void checkInterface(Class clasz) { + if (clasz == null || !clasz.isInterface()) { + throw new IllegalArgumentException("interface Class expected in getInterface"); + } + } + + private static void checkThis(Value thiz) { + if (thiz.isHostObject() || !thiz.hasMembers()) { + throw new IllegalArgumentException("getInterface cannot be called on non-script object"); + } + } + + private static T getInterfaceInner(Value thiz, Class iface) { + if (!isInterfaceImplemented(iface, thiz)) { + return null; + } + return thiz.as(iface); + } + + private static String read(Reader reader) throws ScriptException { + final StringBuilder builder = new StringBuilder(); + final char[] buffer = new char[1024]; try { - getPolyglotContext().parse(source); - } catch (PolyglotException pex) { - throw toScriptException(pex); + try (reader) { + while (true) { + final int count = reader.read(buffer); + if (count == -1) { + break; + } + builder.append(buffer, 0, count); + } + } + return builder.toString(); + } catch (IOException ioex) { + throw new ScriptException(ioex); } } - /** - * Creates a new GraalPython script engine from a polyglot Engine instance with a base configuration - * for new polyglot {@link Context} instances. Polyglot context instances can be accessed from - * {@link ScriptContext} instances using {@link #getPolyglotContext()}. The - * {@link Builder#out(OutputStream) out},{@link Builder#err(OutputStream) err} and - * {@link Builder#in(InputStream) in} stream configuration are not inherited from the provided - * polyglot context config. Instead {@link ScriptContext} output and input streams are used. - * - * @param engine the engine to be used for context configurations or null if a - * default engine should be used. - * @param newContextConfig a base configuration to create new context instances or - * null if the default configuration should be used to construct new - * context instances. - */ - public static GraalPythonScriptEngine create(Engine engine, Context.Builder newContextConfig) { - return new GraalPythonScriptEngine(null, engine, newContextConfig); + private static Source createSource(String script, ScriptContext ctxt) throws ScriptException { + final Object val = ctxt.getAttribute(ScriptEngine.FILENAME); + if (val == null) { + return Source.newBuilder(LANGUAGE_ID, script, "").buildLiteral(); + } else { + try { + return Source.newBuilder(LANGUAGE_ID, new File(val.toString())).content(script).build(); + } catch (IOException ioex) { + throw new ScriptException(ioex); + } + } } private static boolean isInterfaceImplemented(final Class iface, final Value obj) { @@ -412,4 +276,47 @@ private static boolean isInterfaceImplemented(final Class iface, final Value } return true; } + + protected static ScriptException toScriptException(PolyglotException ex) { + ScriptException sex; + if (ex.isHostException()) { + Throwable hostException = ex.asHostException(); + // ScriptException (unlike almost any other exception) does not + // accept Throwable cause (requires the cause to be Exception) + Exception cause; + if (hostException instanceof Exception) { + cause = (Exception) hostException; + } else { + cause = new Exception(hostException); + } + // Make the host exception accessible through the cause chain + sex = new ScriptException(cause); + // Re-use the stack-trace of PolyglotException (with guest-language stack-frames) + sex.setStackTrace(ex.getStackTrace()); + } else { + SourceSection sourceSection = ex.getSourceLocation(); + if (sourceSection != null && sourceSection.isAvailable()) { + Source source = sourceSection.getSource(); + String fileName = source.getPath(); + if (fileName == null) { + fileName = source.getName(); + } + int lineNo = sourceSection.getStartLine(); + int columnNo = sourceSection.getStartColumn(); + sex = new ScriptException(ex.getMessage(), fileName, lineNo, columnNo); + sex.initCause(ex); + } else { + sex = new ScriptException(ex); + } + } + return sex; + } + + private static Value evalInternal(Context context, String script) { + return context.eval(Source.newBuilder(LANGUAGE_ID, script, "internal-script").internal(true).buildLiteral()); + } + + public static interface ScriptEngineProvider { + ScriptEngine createScriptEngine(); + } } diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngineFactory.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngineFactory.java similarity index 66% rename from bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngineFactory.java rename to bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngineFactory.java index ff8d5cd8d5c76..58ab83e983780 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/graal/GraalPythonScriptEngineFactory.java +++ b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/scriptengine/graal/GraalPythonScriptEngineFactory.java @@ -10,9 +10,8 @@ * * SPDX-License-Identifier: EPL-2.0 */ -package org.openhab.automation.pythonscripting.internal.graal; +package org.openhab.automation.pythonscripting.internal.scriptengine.graal; -import java.lang.ref.WeakReference; import java.util.List; import java.util.Objects; @@ -21,6 +20,7 @@ import org.graalvm.polyglot.Engine; import org.graalvm.polyglot.Language; +import org.openhab.automation.pythonscripting.internal.scriptengine.graal.GraalPythonScriptEngine.ScriptEngineProvider; /** * A Graal.Python implementation of the script engine factory to exposes metadata describing of the engine class. @@ -29,46 +29,23 @@ * @author Jeff James - Initial contribution */ public final class GraalPythonScriptEngineFactory implements ScriptEngineFactory { - private WeakReference defaultEngine; - private final Engine userDefinedEngine; - private static final String ENGINE_NAME = "Graal.py"; private static final String NAME = "python3"; private static final String[] EXTENSIONS = { "py" }; - public GraalPythonScriptEngineFactory() { - this.userDefinedEngine = null; - - defaultEngine = new WeakReference<>(createDefaultEngine()); - } - - public GraalPythonScriptEngineFactory(Engine engine) { - this.defaultEngine = null; - this.userDefinedEngine = engine; - } + private final Engine engine; + private final Language language; + private ScriptEngineProvider scriptEngineProvider; - private static Engine createDefaultEngine() { - return Engine.newBuilder() // - .allowExperimentalOptions(true) // - .option("engine.WarnInterpreterOnly", "false") // - .build(); + public GraalPythonScriptEngineFactory(Engine engine, ScriptEngineProvider scriptEngineProvider) { + this.engine = engine; + this.scriptEngineProvider = scriptEngineProvider; + this.language = this.engine.getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); } - /** - * Returns the underlying polyglot engine. - */ public Engine getPolyglotEngine() { - if (userDefinedEngine != null) { - return userDefinedEngine; - } else { - Engine engine = defaultEngine == null ? null : defaultEngine.get(); - if (engine == null) { - engine = createDefaultEngine(); - defaultEngine = new WeakReference<>(engine); - } - return engine; - } + return engine; } @Override @@ -78,7 +55,7 @@ public String getEngineName() { @Override public String getEngineVersion() { - return getPolyglotEngine().getVersion(); + return engine.getVersion(); } @Override @@ -88,25 +65,21 @@ public List getExtensions() { @Override public List getMimeTypes() { - Language language = getPolyglotEngine().getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); return List.copyOf(language.getMimeTypes()); } @Override public List getNames() { - Language language = getPolyglotEngine().getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); return List.of(language.getName(), GraalPythonScriptEngine.LANGUAGE_ID, language.getImplementationName()); } @Override public String getLanguageName() { - Language language = getPolyglotEngine().getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); return language.getName(); } @Override public String getLanguageVersion() { - Language language = getPolyglotEngine().getLanguages().get(GraalPythonScriptEngine.LANGUAGE_ID); return language.getVersion(); } @@ -123,8 +96,8 @@ public Object getParameter(String key) { } @Override - public GraalPythonScriptEngine getScriptEngine() { - return new GraalPythonScriptEngine(this); + public ScriptEngine getScriptEngine() { + return scriptEngineProvider.createScriptEngine(); } @Override diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ModuleLocator.java b/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ModuleLocator.java deleted file mode 100644 index ccf80a73fd95e..0000000000000 --- a/bundles/org.openhab.automation.pythonscripting/src/main/java/org/openhab/automation/pythonscripting/internal/wrapper/ModuleLocator.java +++ /dev/null @@ -1,28 +0,0 @@ -/* - * Copyright (c) 2010-2025 Contributors to the openHAB project - * - * See the NOTICE file(s) distributed with this work for additional - * information. - * - * This program and the accompanying materials are made available under the - * terms of the Eclipse Public License 2.0 which is available at - * http://www.eclipse.org/legal/epl-2.0 - * - * SPDX-License-Identifier: EPL-2.0 - */ -package org.openhab.automation.pythonscripting.internal.wrapper; - -import java.util.List; -import java.util.Map; - -import org.eclipse.jdt.annotation.NonNullByDefault; - -/** - * Locates modules from a module name - * - * @author Holger Hees - Initial contribution - */ -@NonNullByDefault -public interface ModuleLocator { - Map locateModule(String name, List fromlist); -} diff --git a/bundles/org.openhab.automation.pythonscripting/src/main/resources/OH-INF/config/config.xml b/bundles/org.openhab.automation.pythonscripting/src/main/resources/OH-INF/config/config.xml index 4f2d41fc45e12..122595b05d921 100644 --- a/bundles/org.openhab.automation.pythonscripting/src/main/resources/OH-INF/config/config.xml +++ b/bundles/org.openhab.automation.pythonscripting/src/main/resources/OH-INF/config/config.xml @@ -27,18 +27,16 @@ true - + - If disabled, the openHAB Python helper module can be installed manually by copying it to /conf/automation/python/lib/openhab" + Install openHAB Python helper module to support helper classes like rule, logger, Registry, Timer etc...
+ If disabled, the openHAB python helper module can be installed manually by copying it to /conf/automation/python/lib/openhab" ]]> true - - + + This injects the scope and helper Registry and logger into rules. @@ -46,8 +44,24 @@ 2 + + + + == followed by standard + python pip version constraint, such as "tzdata==2025.2".]]> + + + true + + + + Native modules are sometimes necessary for pip modules which depends on native libraries. + false + true + - + Dependency tracking allows your scripts to automatically reload when one of its dependencies is updated. You may want to disable dependency tracking if you plan on editing or updating a shared library, but don't want all your scripts to reload until you can test it. @@ -55,7 +69,7 @@ true - + Disable this option will result in a slower startup performance, because scripts have to be recompiled on every startup.