Skip to content

Releases: Chaffelson/nipyapi

Property configuration helpers and multi-asset parameter support

25 Jan 14:09
v1.5.0
7f85f00

Choose a tag to compare

nipyapi v1.5.0 Release Notes

Release Date: January 25, 2026

Property configuration helpers and multi-asset parameter support.

New Features

Canvas Module

  • prepare_processor_config() - New helper to validate property keys against processor descriptors before setting. Prevents accidental creation of dynamic properties by requiring explicit allow_dynamic=True for non-static properties.

  • prepare_controller_config() - Same validation for controller services - validates property keys exist in descriptors before setting.

  • update_controller() error recovery - When auto_disable=True, if the update fails after disabling the controller, the function now attempts to re-enable the controller before raising the exception.

Parameters Module

  • prepare_parameter_with_asset() multi-asset support - Enhanced to support multiple assets via new assets parameter. Pass a list of asset dicts to link multiple files (e.g., JMS client JARs) to a single parameter. Backwards compatible with existing single-asset usage.

CLI Improvements

  • Smarter PAGER handling - CLI now only sets PAGER=cat when help is requested or in non-interactive/CI environments, preserving user's pager preference for normal interactive use.

Improvements

  • Deterministic error messages - Property validation errors now use sorted keys for consistent, diff-friendly output.

v1.4.1

18 Jan 09:02
811aa9c

Choose a tag to compare

What's Changed

  • fix: verify_config now checks processors in child process groups by @Chaffelson in #404

Full Changelog: v1.4.0...v1.4.1

v1.4.0

16 Jan 16:17
a8b7f19

Choose a tag to compare

nipyapi 1.4.0

CLI exit codes, bulletins descendants, batch parameter updates, and throughput stats.

Highlights

  • CLI error exit codes - CLI now exits with code 1 when operations fail, enabling reliable scripting
  • Batched parameter updates - configure_inherited_params() now updates all parameters per context in a single API call
  • Throughput stats - get_status() includes flowfiles_in/out, bytes_in/out, bytes_read/written

What's New

CLI

  • Standardized error exit codes for script reliability
  • Error field convention documented for CI functions

Bulletins

  • get_bulletin_board(): New descendants=True parameter for child process group bulletins

Canvas

  • list_all_controllers(): Added greedy and identifier_type parameters

CI Module

  • get_status(): Added throughput stats, fixed bulletin collection
  • configure_inherited_params(): Batched updates reduce API overhead
  • cleanup(), verify_config(): Added error key for CLI exit code support

Parameters

  • prepare_parameter(): Clarified value=None vs omitted value semantics
  • list_orphaned_contexts(): Find unbound parameter contexts
  • rename_parameter_context(): Rename parameter contexts

Layout

  • PORT_QUEUE_BOX_WIDTH: New constant (240px)

Bug Fixes

  • Fixed parameter value removal handling (value=None vs omitted)
  • Fixed bulletin board to include controller service bulletins

Install

pip install nipyapi==1.4.0
# With CLI
pip install "nipyapi[cli]==1.4.0"

Links

Entity Resolution and Scheduling Improvements

08 Jan 08:47
944d82a

Choose a tag to compare

Consistent API patterns across canvas, versioning, and parameters modules.

Entity Resolution (nipyapi.utils)

  • resolve_entity(): New utility for consistent entity resolution - accepts object, ID, or name across 31 refactored functions
  • resolve_schedule_state(): Standardized bool/string to state normalization for scheduling functions
  • parse_bool(): Extracted utility for CLI boolean parameter handling (fixes --flag=false being truthy)

Canvas Module Enhancements

  • Port scheduling: New get_port() and schedule_port() functions for input/output port management
  • String state scheduling: schedule_controller() and schedule_all_controllers() now accept string states ('ENABLED', 'DISABLED') in addition to bool
  • INVALID controller fix: schedule_all_controllers() now correctly skips INVALID controllers when enabling, avoiding 30-second timeouts
  • create_controller(): Fixed to set component name in initial creation request (was returning stale object)
  • Flexible entity input: 17 canvas functions now accept object, ID, or name via resolve_entity()
  • get_controller(): Added greedy parameter for name matching control
  • Optional location: create_process_group() and create_funnel() no longer require location parameter

Versioning Module Enhancements

  • Flexible entity input: 9 versioning functions now accept object, ID, or name via resolve_entity()

Parameters Module Enhancements

  • Flexible entity input: 5 parameters functions now accept object, ID, or name via resolve_entity()

Deprecations

  • Removed deprecated get_variable_registry() and update_variable_registry() (use parameter contexts)

Breaking Changes

  • Invalid scheduled values now raise ValueError instead of AssertionError (more appropriate exception type)

Infrastructure

  • Consolidated _run_verification_request() helper for controller verification pattern
  • Added PortEntity to config.py registered_filters for filter_obj support
  • Updated contributing guide with CLI boolean parameter handling examples

CLI and CI/CD Automation

31 Dec 18:07
v1.2.0
c4127de

Choose a tag to compare

v1.2.0 - CLI and CI/CD Automation

CLI and CI/CD automation release

Command-line interface for shell scripting and pipeline integration (tested against NiFi 2.7.2)

Command-Line Interface

  • Optional CLI install (pip install nipyapi[cli]): Shell access to all nipyapi modules via Google Fire
  • JSON output by default with automatic CI platform detection (GitHub Actions, GitLab CI)
  • Profile selection via --profile flag or environment variables
  • --version flag and verbosity controls (-v, -vv, -vvv)

CI Operations Module (nipyapi.ci)

26 automation-focused commands for flow deployment pipelines:

  • verify_config, ensure_registry, deploy_flow, start_flow, stop_flow
  • list_flows, list_registry_flows, get_status, get_flow_versions
  • commit_flow, detach_flow, get_flow_diff, revert_flow, change_flow_version
  • configure_params, configure_inherited_params, export_parameters, upload_asset
  • export_flow_definition, import_flow_definition, cleanup, purge_flowfiles
  • list_nars, upload_nar, delete_nar, resolve_git_ref

New Modules

  • nipyapi.bulletins: Bulletin retrieval with filtering; clear_all_bulletins for NiFi 2.7.0+
  • nipyapi.extensions: NAR management, Python processor lifecycle, bundle version switching
  • nipyapi.layout: Canvas positioning with below(), fork(), automatic flow layout, grid alignment
  • nipyapi.nifi_registry: NiFi Registry operations extracted from versioning module

Canvas Module Enhancements

  • FlowFile inspection: list_flowfiles, get_flowfile_details, get_flowfile_content, peek_flowfiles
  • Component state: get_processor_state, clear_processor_state, get_controller_state, clear_controller_state
  • schedule_processor now supports RUN_ONCE and DISABLED states
  • schedule_all_controllers now waits for target state
  • get_processor_docs, get_controller_service_docs for processor discovery
  • get_flow_components returns connected subgraph for flow analysis

Parameters Module Enhancements

  • get_parameter_context_hierarchy with include_bindings and include_parameters options

Utilities Enhancements

  • fs_write supports binary=True for raw byte content
  • is_uuid for UUID format detection
  • format_timestamp for NiFi-compatible timestamps

Documentation

  • CLI reference (docs/cli.rst)
  • CI operations guide with examples (docs/ci.rst)
  • NAR management guide (docs/extensions.rst)

Infrastructure

  • Updated NiFi and Registry API clients to 2.7.2
  • New NiFi 2.7.2 API features: bulletin stack traces, listen ports, clear bulletins
  • Simplified revert_flow_ver() using VCI process_group_revision

Installation

pip install nipyapi==1.2.0
pip install "nipyapi[cli]==1.2.0"  # With CLI

Related

v1.1.0 - GitHub CI/CD Integration & Bearer Token Authentication

08 Dec 23:13
v1.1.0
4507a52

Choose a tag to compare

What's New

GitHub Flow Registry Support

Native support for NiFi's GitHub Flow Registry Client, enabling CI/CD workflows for versioned NiFi flows:

  • Git-specific versioning helpers: list_git_registry_buckets, get_git_registry_bucket, list_git_registry_flows, get_git_registry_flow, list_git_registry_flow_versions, deploy_git_registry_flow, update_git_flow_ver
  • Registry client management: ensure_registry_client and update_registry_client for idempotent configuration
  • Enhanced revert: revert_flow_ver now accepts wait=True for synchronous operation

Profile System Enhancements

  • "env" profile: Configure nipyapi entirely from environment variables - no profiles file required

    nipyapi.profiles.switch('env') # Reads NIFI_API_ENDPOINT, NIFI_BEARER_TOKEN, etc.

    • Bearer token authentication: Simplest auth method - just URL + token
      export NIFI_API_ENDPOINT=https://nifi.example.com/nifi-api
      export NIFI_BEARER_TOKEN=eyJhbGciOiJSUzI1NiIs...

    Canvas Improvements

  • Bulk controller service operations: schedule_all_controllers(pg_id, scheduled) for enabling/disabling all controller services

  • Processor updates: update_processor now accepts name param and auto_stop for handling running processors

Bug Fixes

  • Fixed test_create_controller leaving orphaned controller services
  • revert_flow_ver now refreshes revision internally to prevent stale errors

Installation

pip install --upgrade nipyapi

Related Projects

Full Changelog: v1.0.1...v1.1.0

Infrastructure and documentation fixes for post-1.0 release

10 Nov 17:06
v1.0.1
107e324

Choose a tag to compare

What's Changed

Infrastructure Fixes

  • ReadTheDocs configuration: Fixed .readthedocs.yaml to properly install package with docs extras, ensuring setuptools-scm is available for dynamic version detection. Stable docs now display correct version instead of 0.19.
  • Pylint 3.x compatibility: Updated pylintrc with ignore-paths regex patterns to properly exclude generated code directories (nipyapi/nifi/, nipyapi/registry/, _version.py) in CI environments using pylint 3.x.
  • CI workflow optimization: Reordered GitHub Actions steps to upload coverage reports before make test-dist runs, preventing make clean from deleting coverage.xml before codecov upload.

Documentation Updates

  • Branch references: Updated all documentation links from NiFi2x development branch to main default branch (README badges, contribution guide links, Sphinx source code references).
  • LICENSE: Added full Apache 2.0 license text to repository root.
  • Development workflow: Added .cursor/ to .gitignore for IDE-specific files.

Full Changelog: v1.0.0...v1.0.1

v1.0.0 - Migrate to NiFI 2.x, drops support for NiFi 1.x on main

28 Sep 19:07
v1.0.0
f05687d

Choose a tag to compare

1.0.0 (2025-09-28)


Major migration to Apache NiFi/Registry 2.x (tested against 2.5.0). Drops 1.x support on main
IMPORTANT To continue working with NiFi-1.x you should pin your requirements to nipyapi<1.0.0

Breaking Changes - Action Required

  • Function renaming: Upstream API specification changes result in operation IDs now using suffixed names (e.g., update_run_status1) and some other functions are also renamed
  • Authentication and configuration overhaul: Significant changes to align with modern API standards and upstream API changes
  • Users must review and update authentication patterns - legacy configuration methods will be different

Profile Management System

  • Extensible file format (YAML/JSON) with environment variable overrides and sane defaults - familiar workflow like AWS CLI
  • Intelligent authentication method detection: OIDC, mTLS, and Basic authentication based on available configuration parameters
  • Built-in profiles for common deployment patterns: single-user, secure-ldap, secure-mtls, secure-oidc
  • 15+ configurable parameters (URLs, credentials, certificates, SSL settings) with NIPYAPI_PROFILE environment variable
  • Profile switching with nipyapi.profiles.switch() configures endpoints, authentication, and SSL settings in single function call

Automated Development Workflow

  • Comprehensive Makefile targets for all key development and release processes
  • End-to-end automation: entire client generation and testing sequence from test certificates to final integration tests
  • GitHub Actions CI with full Docker NiFi integration tests and coverage reporting
  • Smart certificate regeneration and optimized rebuild flows to avoid unnecessary infrastructure cycling

Quick Start and Migration Tools

  • Sandbox Docker environment for testing different authentication mechanisms with make sandbox target
  • FDLC example retained and modernized to demonstrate proper multi-environment workflows (single-user dev, secure-ldap prod)
  • Comprehensive migration guide (docs/migration.rst) for upgrading from NiPyAPI 0.x/NiFi 1.x to 1.x/2.x

Core Technical Improvements

  • Documentation system rebuild: Complete Sphinx overhaul with individual pages for all Core client, NiFi APIs and Registry APIs - with a flat API structure with optimal navigation
  • Test coverage expansion: Comprehensive test suite with profile-driven automation rather than manual reconfiguration
  • Pre-commit checks: Automated code quality with trailing whitespace, debug statements, flake8, and pylint hooks
  • Modern dependency management: Migrated to python -m build, replaced ruamel.yaml with PyYAML, explicit urllib3/certifi/requests inclusion
  • Enhanced documentation: Direct GitHub source code links with line-level precision, standardized docstrings throughout generated clients
  • Profile-driven testing: Deprecated complex tox regression suite in favor of spec-driven single version testing
  • Legacy pruning: Removed Python 2.7 and NiFi 1.x references, deprecated template-era dependencies (lxml, xmltodict)
  • Codecov migration: Switched from Coveralls with pytest-cov integration and build process automation
  • Enhanced convenience functions: Improved set_endpoint and various ensure_* object functions
  • Certificate handling improvements: Resolved user complexity with automatic CA certificate setup and validation
  • Extensive authentication documentation: OIDC setup instructions, Safari keychain guidance, development vs production practices
  • API augmentation system: Client build-time patching for upstream API issues (currently: enum handling and missing security schemes)

Additional Details

New Contributors

Full Changelog: v0.22.0...v1.0.0

v0.22.0

25 Mar 10:53
e14d2f5

Choose a tag to compare

What's Changed

  • Updates to supported Python versions, handing Template deprecation in NiFi-2.x, and Windows development support by @Chaffelson in #381
  • Add support for mTLS by @Chaffelson in #382
  • Updated nifi and registry low level clients to v1.28.1 by @Chaffelson in #383

Full Changelog: v0.21.0...v0.22.0

v0.21.0

08 Oct 13:43
3010452

Choose a tag to compare

What's Changed

  • Fix API model generator by @michaelarnauts in #356
  • issue-360: handle -9 error messages better by @ottobackwards in #361
  • Handle plain text response types so json values are correctly returned by @michaelarnauts in #358
  • update clients to 1.27.0 by @Chaffelson in #365
  • Simplified the use of setattr in recurse_flow, flatten, and list_all_by_kind methods in nipyapi/canvas.py.
  • Added support for key_password in the Configuration class and its usage in nipyapi/nifi/rest.py and nipyapi/registry/rest.py.
  • Fixed the method to retrieve HTTP headers in nipyapi/nifi/rest.py and nipyapi/registry/rest.py.
  • Updated pylintrc to match more modern python standards
  • Fixed issue #326 where the latest flow version was not being deployed by default
  • Fixes nipyapi.nifi.ProcessGroupsApi.upload_process_group_with_http_info() incomplete #310
  • VersionedReportingTask added with appropriate functions
  • Move docker requirement to extras to avoid dependency install during standard usage
  • Set latest python3 version to 3.12
  • Added NiFi-2.0.0-M4 to testing suite, appears to work but compatibility not guaranteed using currently 1.x clients
  • Deprecate usage of py.test in favour of newer pytest.
  • Update readme

New Contributors

Full Changelog: v0.20.0...v0.21.0