Help: Issues integrating Code Node with Knowledge Base Node and Output Length Limit in Knowledge Pipelines

[finrod86921812]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I am currently using the Knowledge Pipeline feature in the Enterprise version of Dify and have encountered two issues. I would appreciate any assistance:

Issue 1: Code Node output not recognized by Knowledge Base Node

Scenario Description:
In a Knowledge Pipeline, I use the General Chunker to segment a document. I then intend to use a Code Node to perform a custom merge on the segmented chunks. Finally, I want to pass this processed merged chunk list to the Knowledge Base Node for indexing.

Problem:
After processing in the Code Node, I cannot see or connect to the Code Node’s output in the Knowledge Base Node.

My Code:

def main(chunk_list: list[str]) :
    max_chunk_size = 500
    overlap_size = 100
    
    merged_results = []
    current_chunk = ""
    
    for chunk in chunk_list:
        if not current_chunk:
            # 如果当前是空的，直接加入
            current_chunk = chunk
        else:
            # 尝试合并，中间用 \n\n 连接
            potential_chunk = current_chunk + "\n\n" + chunk
            
            if len(potential_chunk) <= max_chunk_size:
                # 如果合并后未超过限制，更新当前chunk
                current_chunk = potential_chunk
            else:
                # 如果超过了，保存当前结果
                merged_results.append(current_chunk)
                
                # 处理重叠：从上一个chunk的末尾截取 overlap_size 长度的内容
                # 注意：这里假设 chunk 之间是切分好的，overlap 主要来源于上一个合并块的尾部
                overlap_content = current_chunk[-overlap_size:] if len(current_chunk) > overlap_size else current_chunk
                
                # 新的 chunk 以重叠内容开头，拼接新内容
                current_chunk = overlap_content + "\n\n" + chunk

    # 别忘了处理最后一个累积的 chunk
    if current_chunk:
        merged_results.append(current_chunk)
        
    return {
        "result": merged_results,
    }

Questions:

• How should the Code Node output correctly to be recognized and connected by the Knowledge Base Node?
• Are there specific Schema or output format requirements I am missing?

Issue 2: Code Node Output List Length Limit (Enterprise)

Error Message:
The length of output variable result must be less than 30 elements.

Attempted Solution:
Referencing the community issue #5423, I added environment variables to the sandbox configuration in my values.yaml:

sandbox:
  extraEnv:
    - name: CODE_MAX_STRING_ARRAY_LENGTH
       value: "10000"
    - name: CODE_MAX_OBJECT_ARRAY_LENGTH
       value: "10000"

After running helm upgrade, the configuration does not seem to have taken effect, and the error persists.

Questions:
Are the environment variable names or configuration location correct?

2. Additional context or comments

No response

[dosubot[bot]]

Hi @finrod86921812! I'm Dosu and I’m helping the dify team.

For the Code Node output to be recognized by the Knowledge Base Node, make sure your function returns a dictionary with the expected key (like "result") and that the output matches the chunk structure required by your dataset type. For plain text datasets, a list of strings is fine. For hierarchical or QA datasets, you need to return a list of objects matching the expected schema (e.g., each item should have fields like parent_content and child_contents for hierarchical, or question and answer for QA). Also, ensure the variable name you output matches what the Knowledge Base Node is configured to receive (usually via index_chunk_variable_selector) source.

About the output length limit: the default is 30 elements, controlled by the environment variables CODE_MAX_STRING_ARRAY_LENGTH and CODE_MAX_OBJECT_ARRAY_LENGTH. These should be set in your Helm values.yaml under the sandbox.extraEnv section. After updating, you must fully restart the sandbox pod for changes to take effect. To verify the variables are set, use kubectl exec <sandbox-pod> -- env and check the values inside the running container source. If the limit still doesn't change, double-check the variable names, placement, and that the pod was restarted—not just reloaded. There is no code-level logging or endpoint that exposes these values; they must be injected into the container environment source.

If you still hit the limit, check if your deployment uses a different environment variable (like WORKFLOW_VARIABLE_TRUNCATION_ARRAY_LENGTH), as some recent versions enforce the limit via this variable instead source.

Let me know if you need example code for a specific chunk structure or more details on Helm configuration!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[finrod86921812]

I've dug into the Dify source code to investigate this issue, and I found some limitations.

My Original Question
I was trying to use a Code Node to output data that can be connected to a Knowledge Base Node's chunksInput field. My code looked like:

def main(result: list[str]) -> dict:
    return {
        "chunks": ['11111','2222','3333']
    }

But the output variable couldn't be selected by the Knowledge Base Node, even though it was a list[str] which should match the general structure requirement.

My Follow-up Test
Then I tried creating a custom Tool (workflow as tool) containing just one Code Node:

def main():
    return {
        "result": ['111','222','333','444'],
    }

The tool's output variables shows as:

**result**   array[string]

But when I compare it with the official General Chunker, its output variables shows as:

**result**   array[string]
Schema for general structure (v1) - array of strings

The difference is that my outputs don't have the schema type label, so they cannot be selected by the Knowledge Base Node.

What I Found in the Source Code
1. Knowledge Base Node's Variable Filter
In web/app/components/workflow/nodes/knowledge-base/panel.tsx:58-71, the Knowledge Base Node filters variables by schemaType:

const filterVar = useCallback((variable: Var) => {
  if (!data.chunk_structure)
    return false
  switch (data.chunk_structure) {
    case ChunkStructureEnum.general:
      return variable.schemaType === 'general_structure' ||
             variable.schemaType === 'multimodal_general_structure'
    // ...
  }
}, [data.chunk_structure])

Only variables with schemaType === 'general_structure' or schemaType === 'multimodal_general_structure' can be selected.

2. Code Node Outputs Don't Have schemaType
In web/app/components/workflow/nodes/_base/components/variable/utils.ts:420-431, Code Node outputs are defined as:

case BlockEnum.Code: {
  const { outputs } = data as CodeNodeType
  res.vars = outputs
    ? Object.keys(outputs).map((key) => {
        return {
          variable: key,
          type: outputs[key].type,
          // NOTE: No schemaType property!
        }
      })
    : []
  break
}

Code Node outputs don't have a schemaType property at all, so they will always be filtered out.

3. Workflow-as-Tool Also Lacks $ref Support
When I checked how custom tools (workflow as tool) generate their output_schema in api/core/tools/workflow_as_tool/provider.py:169-176:

properties = {}
for output in outputs:
    if output.variable not in reserved_keys:
        properties[output.variable] = {
            "type": output.value_type,
            "description": "",
        }
output_schema = {"type": "object", "properties": properties}

This only sets the type field, but doesn't add a $ref to reference the schema definition.

4. Official Plugin Does It Correctly
The official General Chunker plugin (dify-official-plugins/tools/general_chunk/tools/general.yaml) uses:

output_schema:
  type: object
  properties:
    result:
      $ref: "https://dify.ai/schemas/v1/general_structure.json"

This $ref is what makes the schema type label appear in the UI.

So my question is: are there any solutions or workarounds to make a custom node or tool output that can be connected to a Knowledge Base Node?

Regarding the output length limit issue, I have confirmed that I restarted the pod, and I entered the container and used echo $CODE_MAX_OBJECT_ARRAY_LENGTH and echo $CODE_MAX_STRING_ARRAY_LENGTH to print the values. I confirmed that the environment variables have taken effect, but the problem still persists.

[dosubot[bot]]

You're correct—custom Code Node and workflow-as-tool outputs can't be selected by the Knowledge Base Node because they lack the required schemaType (like general_structure) or $ref in their output schema. The Knowledge Base Node UI filters variables strictly by schemaType, and only official plugins like General Chunker include the $ref that marks their outputs as compatible. There’s currently no built-in way to add schemaType or $ref to custom outputs without modifying the Dify codebase itself. If you want to patch this, you’d need to extend both the backend (to allow specifying $ref/schemaType in output schemas) and the frontend (to let users pick or assign a schemaType for outputs) source.

Partial workarounds like using the reserved json output or restructuring with Template/Parameter Extractor nodes won’t make the variable selectable as a Knowledge Base input—they just let you pass data between nodes, not satisfy the schemaType filter.

For the output length limit: the variable you need to set is WORKFLOW_VARIABLE_TRUNCATION_ARRAY_LENGTH, not CODE_MAX_STRING_ARRAY_LENGTH or CODE_MAX_OBJECT_ARRAY_LENGTH. Set WORKFLOW_VARIABLE_TRUNCATION_ARRAY_LENGTH in your environment (e.g., values.yaml under sandbox.extraEnv for Helm), then fully restart the sandbox pod. You can confirm it’s set by running kubectl exec -- env and checking the value source.

If you want to try a code patch or need more details on the schemaType extension, let me know!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[finrod86921812]

Your method didn’t work.
I eventually found the solution. It turns out the configuration differs between Dify Enterprise and the Community version.
I added the following environment variables under the api section in values.yaml:

api:  
    extraEnv:
      - name: CODE_MAX_STRING_ARRAY_LENGTH
        value: "10000"
      - name: CODE_MAX_OBJECT_ARRAY_LENGTH
        value: "10000"
      - name: CODE_MAX_NUMBER_ARRAY_LENGTH
        value: "10000"

After applying this configuration, the error no longer occurs.