Problem Statement

Currently, Higress is increasingly using Golang filters deployed through Istio EnvoyFilter CRD. However, there's a critical limitation: EnvoyFilter sorting logic and WasmPlugin sorting logic are completely isolated, making it impossible to achieve mixed ordering between them. For example, placing a rag-mcp Golang filter after jwt-auth plugin cannot be precisely controlled.

Proposed Solution

Extend the EnvoyFilter CRD with wasmPhase and wasmPriority fields to enable mixed sorting with WasmPlugins while maintaining full backward compatibility.

Implementation Details

1. CRD Extension

Add new fields to EnvoyFilter spec in manifests/charts/base/crds/crd-all.gen.yaml 1 :

wasmPhase:
  description: Determines where in the filter chain this EnvoyFilter should be injected for mixed sorting with WasmPlugins
  enum: [UNSPECIFIED_PHASE, AUTHN, AUTHZ, STATS]
  type: string
wasmPriority:
  description: Determines ordering of EnvoyFilter in the same wasmPhase for mixed sorting with WasmPlugins
  type: integer

2. Core Sorting Logic Enhancement

Modify the initEnvoyFilters function in pilot/pkg/model/push_context.go 2 to implement mixed sorting:

sort.Slice(envoyFilterConfigs, func(i, j int) bool {
    ifilter := envoyFilterConfigs[i].Spec.(*networking.EnvoyFilter)
    jfilter := envoyFilterConfigs[j].Spec.(*networking.EnvoyFilter)
    
    // Check if wasmPhase is set
    iHasWasmPhase := ifilter.WasmPhase != extensions.PluginPhase_UNSPECIFIED_PHASE && ifilter.WasmPhase != 0
    jHasWasmPhase := jfilter.WasmPhase != extensions.PluginPhase_UNSPECIFIED_PHASE && jfilter.WasmPhase != 0
    
    // Both have wasmPhase: use mixed sorting
    if iHasWasmPhase && jHasWasmPhase {
        if ifilter.WasmPhase != jfilter.WasmPhase {
            return phaseOrder(ifilter.WasmPhase) < phaseOrder(jfilter.WasmPhase)
        }
        if ifilter.WasmPriority != jfilter.WasmPriority {
            return ifilter.WasmPriority > jfilter.WasmPriority
        }
    }
    
    // Only one has wasmPhase: prioritize the one with wasmPhase
    if iHasWasmPhase && !jHasWasmPhase {
        return true
    }
    if !iHasWasmPhase && jHasWasmPhase {
        return false
    }
    
    // Neither has wasmPhase: use original sorting logic
    if ifilter.Priority != jfilter.Priority {
        return ifilter.Priority < jfilter.Priority
    }
    // ... rest of original logic
})

3. LDS Generation Integration

Update the LDS generation flow in pilot/pkg/xds/lds.go 3 to apply mixed sorting before generating listeners.

4. Filter Chain Mixed Sorting

Implement mixed sorting in the HTTP filter chain, integrating with existing WasmPlugin handling from pilot/pkg/networking/core/v1alpha3/extension/wasmplugin.go 4 .

Usage Example

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: custom-authn-filter
spec:
  wasmPhase: AUTHN          # Enable mixed sorting
  wasmPriority: 100         # Mixed sorting priority
  priority: 10              # Traditional priority (backward compatible)
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.custom_authn
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.custom_authn.v3.CustomAuthn

Benefits

1. Full Backward Compatibility: EnvoyFilters without new fields maintain original behavior
2. Flexible Control: Selective enablement of mixed sorting via new fields
3. Unified Ordering: True mixed sorting between EnvoyFilter and WasmPlugin
4. Performance Optimized: Unified processing before LDS generation

Implementation Roadmap

1. Phase 1: Add new fields to CRD and data structures
2. Phase 2: Implement mixed sorting logic with feature flag
3. Phase 3: Comprehensive testing and documentation
4. Phase 4: Enable by default

Call for Contributors

We're seeking community contributions to implement this feature. Key areas needing work:

• CRD schema updates and validation
• PushContext sorting algorithm implementation
• LDS generation integration
• Comprehensive test coverage
• Documentation and examples

Labels

enhancement envoy-filter wasm-plugin sorting mixed-ordering

Note: This implementation maintains complete backward compatibility while providing the flexibility needed for mixed filter ordering scenarios in Higress deployments.

Notes

The proposed solution leverages existing infrastructure in the Istio control plane, specifically the PushContext's configuration processing capabilities and the LDS generation pipeline. The mixed sorting logic integrates seamlessly with current WasmPlugin handling through the PopAppend mechanism already in place.

Wiki pages you might want to explore:

• Control Plane (istiod) (higress-group/istio)
• Configuration Processing (higress-group/istio)

Citations

File: manifests/charts/base/crds/crd-all.gen.yaml (L3021-3030)

          spec:
            description: 'Customizing Envoy configuration generated by Istio. See
              more details at: https://istio.io/docs/reference/config/networking/envoy-filter.html'
            properties:
              configPatches:
                description: One or more patches with match conditions.
                items:
                  properties:
                    applyTo:
                      enum:

File: pilot/pkg/model/push_context.go (L2017-2033)

	sort.Slice(envoyFilterConfigs, func(i, j int) bool {
		ifilter := envoyFilterConfigs[i].Spec.(*networking.EnvoyFilter)
		jfilter := envoyFilterConfigs[j].Spec.(*networking.EnvoyFilter)
		if ifilter.Priority != jfilter.Priority {
			return ifilter.Priority < jfilter.Priority
		}
		// If priority is same fallback to name and creation timestamp, else use priority.
		// If creation time is the same, then behavior is nondeterministic. In this case, we can
		// pick an arbitrary but consistent ordering based on name and namespace, which is unique.
		// CreationTimestamp is stored in seconds, so this is not uncommon.
		if envoyFilterConfigs[i].CreationTimestamp != envoyFilterConfigs[j].CreationTimestamp {
			return envoyFilterConfigs[i].CreationTimestamp.Before(envoyFilterConfigs[j].CreationTimestamp)
		}
		in := envoyFilterConfigs[i].Name + "." + envoyFilterConfigs[i].Namespace
		jn := envoyFilterConfigs[j].Name + "." + envoyFilterConfigs[j].Namespace
		return in < jn
	})

File: pilot/pkg/networking/core/v1alpha3/extension/wasmplugin.go (L49-60)

// PopAppend takes a list of filters and a set of WASM plugins, keyed by phase. It will remove all
// plugins of a provided phase from the WASM plugin set and append them to the list of filters
func PopAppend(list []*hcm.HttpFilter,
	filterMap map[extensions.PluginPhase][]*model.WasmPluginWrapper,
	phase extensions.PluginPhase,
) []*hcm.HttpFilter {
	for _, ext := range filterMap[phase] {
		list = append(list, toEnvoyHTTPFilter(ext))
	}
	delete(filterMap, phase)
	return list
}

Problem Statement

Currently, Higress is increasingly using Golang filters deployed through Istio EnvoyFilter CRD. However, there's a critical limitation: EnvoyFilter sorting logic and WasmPlugin sorting logic are completely isolated, making it impossible to achieve mixed ordering between them. For example, placing a rag-mcp Golang filter after jwt-auth plugin cannot be precisely controlled.

Proposed Solution

Extend the EnvoyFilter CRD with wasmPhase and wasmPriority fields to enable mixed sorting with WasmPlugins while maintaining full backward compatibility.

Implementation Details

1. CRD Extension

Add new fields to EnvoyFilter spec in manifests/charts/base/crds/crd-all.gen.yaml 1 :

wasmPhase:
  description: Determines where in the filter chain this EnvoyFilter should be injected for mixed sorting with WasmPlugins
  enum: [UNSPECIFIED_PHASE, AUTHN, AUTHZ, STATS]
  type: string
wasmPriority:
  description: Determines ordering of EnvoyFilter in the same wasmPhase for mixed sorting with WasmPlugins
  type: integer

2. Core Sorting Logic Enhancement

Modify the initEnvoyFilters function in pilot/pkg/model/push_context.go 2 to implement mixed sorting:

sort.Slice(envoyFilterConfigs, func(i, j int) bool {
    ifilter := envoyFilterConfigs[i].Spec.(*networking.EnvoyFilter)
    jfilter := envoyFilterConfigs[j].Spec.(*networking.EnvoyFilter)
    
    // Check if wasmPhase is set
    iHasWasmPhase := ifilter.WasmPhase != extensions.PluginPhase_UNSPECIFIED_PHASE && ifilter.WasmPhase != 0
    jHasWasmPhase := jfilter.WasmPhase != extensions.PluginPhase_UNSPECIFIED_PHASE && jfilter.WasmPhase != 0
    
    // Both have wasmPhase: use mixed sorting
    if iHasWasmPhase && jHasWasmPhase {
        if ifilter.WasmPhase != jfilter.WasmPhase {
            return phaseOrder(ifilter.WasmPhase) < phaseOrder(jfilter.WasmPhase)
        }
        if ifilter.WasmPriority != jfilter.WasmPriority {
            return ifilter.WasmPriority > jfilter.WasmPriority
        }
    }
    
    // Only one has wasmPhase: prioritize the one with wasmPhase
    if iHasWasmPhase && !jHasWasmPhase {
        return true
    }
    if !iHasWasmPhase && jHasWasmPhase {
        return false
    }
    
    // Neither has wasmPhase: use original sorting logic
    if ifilter.Priority != jfilter.Priority {
        return ifilter.Priority < jfilter.Priority
    }
    // ... rest of original logic
})

3. LDS Generation Integration

Update the LDS generation flow in pilot/pkg/xds/lds.go 3 to apply mixed sorting before generating listeners.

4. Filter Chain Mixed Sorting

Implement mixed sorting in the HTTP filter chain, integrating with existing WasmPlugin handling from pilot/pkg/networking/core/v1alpha3/extension/wasmplugin.go 4 .

Usage Example

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: custom-authn-filter
spec:
  wasmPhase: AUTHN          # Enable mixed sorting
  wasmPriority: 100         # Mixed sorting priority
  priority: 10              # Traditional priority (backward compatible)
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.custom_authn
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.custom_authn.v3.CustomAuthn

Benefits

1. Full Backward Compatibility: EnvoyFilters without new fields maintain original behavior
2. Flexible Control: Selective enablement of mixed sorting via new fields
3. Unified Ordering: True mixed sorting between EnvoyFilter and WasmPlugin
4. Performance Optimized: Unified processing before LDS generation

Implementation Roadmap

1. Phase 1: Add new fields to CRD and data structures
2. Phase 2: Implement mixed sorting logic with feature flag
3. Phase 3: Comprehensive testing and documentation
4. Phase 4: Enable by default

Call for Contributors

We're seeking community contributions to implement this feature. Key areas needing work:

• CRD schema updates and validation
• PushContext sorting algorithm implementation
• LDS generation integration
• Comprehensive test coverage
• Documentation and examples

Labels

enhancement envoy-filter wasm-plugin sorting mixed-ordering

Note: This implementation maintains complete backward compatibility while providing the flexibility needed for mixed filter ordering scenarios in Higress deployments.

Notes

The proposed solution leverages existing infrastructure in the Istio control plane, specifically the PushContext's configuration processing capabilities and the LDS generation pipeline. The mixed sorting logic integrates seamlessly with current WasmPlugin handling through the PopAppend mechanism already in place.

Wiki pages you might want to explore:

• Control Plane (istiod) (higress-group/istio)
• Configuration Processing (higress-group/istio)

Citations

File: manifests/charts/base/crds/crd-all.gen.yaml (L3021-3030)

          spec:
            description: 'Customizing Envoy configuration generated by Istio. See
              more details at: https://istio.io/docs/reference/config/networking/envoy-filter.html'
            properties:
              configPatches:
                description: One or more patches with match conditions.
                items:
                  properties:
                    applyTo:
                      enum:

File: pilot/pkg/model/push_context.go (L2017-2033)

	sort.Slice(envoyFilterConfigs, func(i, j int) bool {
		ifilter := envoyFilterConfigs[i].Spec.(*networking.EnvoyFilter)
		jfilter := envoyFilterConfigs[j].Spec.(*networking.EnvoyFilter)
		if ifilter.Priority != jfilter.Priority {
			return ifilter.Priority < jfilter.Priority
		}
		// If priority is same fallback to name and creation timestamp, else use priority.
		// If creation time is the same, then behavior is nondeterministic. In this case, we can
		// pick an arbitrary but consistent ordering based on name and namespace, which is unique.
		// CreationTimestamp is stored in seconds, so this is not uncommon.
		if envoyFilterConfigs[i].CreationTimestamp != envoyFilterConfigs[j].CreationTimestamp {
			return envoyFilterConfigs[i].CreationTimestamp.Before(envoyFilterConfigs[j].CreationTimestamp)
		}
		in := envoyFilterConfigs[i].Name + "." + envoyFilterConfigs[i].Namespace
		jn := envoyFilterConfigs[j].Name + "." + envoyFilterConfigs[j].Namespace
		return in < jn
	})

File: pilot/pkg/networking/core/v1alpha3/extension/wasmplugin.go (L49-60)

// PopAppend takes a list of filters and a set of WASM plugins, keyed by phase. It will remove all
// plugins of a provided phase from the WASM plugin set and append them to the list of filters
func PopAppend(list []*hcm.HttpFilter,
	filterMap map[extensions.PluginPhase][]*model.WasmPluginWrapper,
	phase extensions.PluginPhase,
) []*hcm.HttpFilter {
	for _, ext := range filterMap[phase] {
		list = append(list, toEnvoyHTTPFilter(ext))
	}
	delete(filterMap, phase)
	return list
}