update

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

.github/pull_request_template.md

@@ -1,34 +1,32 @@
-# Checklist:
+# Summary
+
+Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
+
+> [!Tip]
+> Close issue syntax: `Fixes #<issue number>` or `Resolves #<issue number>`, see [documentation](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more details.
+
+
+# Screenshots
+
+<table>
+  <tr>
+  <td>Before: </td>
+  <td>After: </td>
+  </tr>
+  <tr>
+  <td>...</td>
+  <td>...</td>
+  </tr>
+</table>
+
+# Checklist
 
 > [!IMPORTANT]  
 > Please review the checklist below before submitting your pull request.
 
-- [ ] Please open an issue before creating a PR or link to an existing issue
-- [ ] I have performed a self-review of my own code
-- [ ] I have commented my code, particularly in hard-to-understand areas
-- [ ] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods
-
-# Description
-
-Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. Close issue syntax: `Fixes #<issue number>`, see [documentation](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more details.
-
-Fixes 
-
-## Type of Change
-
-- [ ] Bug fix (non-breaking change which fixes an issue)
-- [ ] New feature (non-breaking change which adds functionality)
-- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
 - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs)
-- [ ] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
-- [ ] Dependency upgrade
-
-# Testing Instructions
-
-Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
-
-- [ ] Test A
-- [ ] Test B
-
-
+- [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
+- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
+- [x] I've updated the documentation accordingly.
+- [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods
 

api/Dockerfile

@@ -55,7 +55,7 @@ RUN apt-get update \
     && echo "deb http://deb.debian.org/debian testing main" > /etc/apt/sources.list \
     && apt-get update \
     # For Security
-    && apt-get install -y --no-install-recommends expat=2.6.3-2 libldap-2.5-0=2.5.18+dfsg-3+b1 perl=5.40.0-7 libsqlite3-0=3.46.1-1 zlib1g=1:1.3.dfsg+really1.3.1-1+b1 \
+    && apt-get install -y --no-install-recommends expat=2.6.4-1 libldap-2.5-0=2.5.18+dfsg-3+b1 perl=5.40.0-7 libsqlite3-0=3.46.1-1 zlib1g=1:1.3.dfsg+really1.3.1-1+b1 \
     # install a chinese font to support the use of tools like matplotlib
     && apt-get install -y fonts-noto-cjk \
     && apt-get autoremove -y \

api/core/model_runtime/model_providers/azure_openai/llm/llm.py

@@ -113,7 +113,7 @@ class AzureOpenAILargeLanguageModel(_CommonAzureOpenAI, LargeLanguageModel):
         try:
             client = AzureOpenAI(**self._to_credential_kwargs(credentials))
 
-            if model.startswith("o1"):
+            if "o1" in model:
                 client.chat.completions.create(
                     messages=[{"role": "user", "content": "ping"}],
                     model=model,
@@ -311,7 +311,7 @@ class AzureOpenAILargeLanguageModel(_CommonAzureOpenAI, LargeLanguageModel):
         prompt_messages = self._clear_illegal_prompt_messages(model, prompt_messages)
 
         block_as_stream = False
-        if model.startswith("o1"):
+        if "o1" in model:
             if stream:
                 block_as_stream = True
                 stream = False
@@ -404,7 +404,7 @@ class AzureOpenAILargeLanguageModel(_CommonAzureOpenAI, LargeLanguageModel):
                                 ]
                             )
 
-        if model.startswith("o1"):
+        if "o1" in model:
             system_message_count = len([m for m in prompt_messages if isinstance(m, SystemPromptMessage)])
             if system_message_count > 0:
                 new_prompt_messages = []
@@ -653,7 +653,7 @@ class AzureOpenAILargeLanguageModel(_CommonAzureOpenAI, LargeLanguageModel):
             tokens_per_message = 4
             # if there's a name, the role is omitted
             tokens_per_name = -1
-        elif model.startswith("gpt-35-turbo") or model.startswith("gpt-4") or model.startswith("o1"):
+        elif model.startswith("gpt-35-turbo") or model.startswith("gpt-4") or "o1" in model:
             tokens_per_message = 3
             tokens_per_name = 1
         else:

api/core/model_runtime/model_providers/gitee_ai/llm/Qwen2.5-72B-Instruct.yaml

@@ -0,0 +1,95 @@
+model: Qwen2.5-72B-Instruct
+label:
+  zh_Hans: Qwen2.5-72B-Instruct
+  en_US: Qwen2.5-72B-Instruct
+model_type: llm
+features:
+  - agent-thought
+  - tool-call
+  - stream-tool-call
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: max_tokens
+    use_template: max_tokens
+    label:
+      en_US: "Max Tokens"
+      zh_Hans: "最大Token数"
+    type: int
+    default: 512
+    min: 1
+    required: true
+    help:
+      en_US: "The maximum number of tokens that can be generated by the model varies depending on the model."
+      zh_Hans: "模型可生成的最大 token 个数，不同模型上限不同。"
+
+  - name: temperature
+    use_template: temperature
+    label:
+      en_US: "Temperature"
+      zh_Hans: "采样温度"
+    type: float
+    default: 0.7
+    min: 0.0
+    max: 1.0
+    precision: 1
+    required: true
+    help:
+      en_US: "The randomness of the sampling temperature control output. The temperature value is within the range of [0.0, 1.0]. The higher the value, the more random and creative the output; the lower the value, the more stable it is. It is recommended to adjust either top_p or temperature parameters according to your needs to avoid adjusting both at the same time."
+      zh_Hans: "采样温度控制输出的随机性。温度值在 [0.0, 1.0] 范围内，值越高，输出越随机和创造性；值越低，输出越稳定。建议根据需求调整 top_p 或 temperature 参数，避免同时调整两者。"
+
+  - name: top_p
+    use_template: top_p
+    label:
+      en_US: "Top P"
+      zh_Hans: "Top P"
+    type: float
+    default: 0.7
+    min: 0.0
+    max: 1.0
+    precision: 1
+    required: true
+    help:
+      en_US: "The value range of the sampling method is [0.0, 1.0]. The top_p value determines that the model selects tokens from the top p% of candidate words with the highest probability; when top_p is 0, this parameter is invalid. It is recommended to adjust either top_p or temperature parameters according to your needs to avoid adjusting both at the same time."
+      zh_Hans: "采样方法的取值范围为 [0.0,1.0]。top_p 值确定模型从概率最高的前p%的候选词中选取 tokens；当 top_p 为 0 时，此参数无效。建议根据需求调整 top_p 或 temperature 参数，避免同时调整两者。"
+
+  - name: top_k
+    use_template: top_k
+    label:
+      en_US: "Top K"
+      zh_Hans: "Top K"
+    type: int
+    default: 50
+    min: 0
+    max: 100
+    required: true
+    help:
+      en_US: "The value range is [0,100], which limits the model to only select from the top k words with the highest probability when choosing the next word at each step. The larger the value, the more diverse text generation will be."
+      zh_Hans: "取值范围为 [0,100]，限制模型在每一步选择下一个词时，只从概率最高的前 k 个词中选取。数值越大，文本生成越多样。"
+
+  - name: frequency_penalty
+    use_template: frequency_penalty
+    label:
+      en_US: "Frequency Penalty"
+      zh_Hans: "频率惩罚"
+    type: float
+    default: 0
+    min: -1.0
+    max: 1.0
+    precision: 1
+    required: false
+    help:
+      en_US: "Used to adjust the frequency of repeated content in automatically generated text. Positive numbers reduce repetition, while negative numbers increase repetition. After setting this parameter, if a word has already appeared in the text, the model will decrease the probability of choosing that word for subsequent generation."
+      zh_Hans: "用于调整自动生成文本中重复内容的频率。正数减少重复，负数增加重复。设置此参数后，如果一个词在文本中已经出现过，模型在后续生成中选择该词的概率会降低。"
+
+  - name: user
+    use_template: text
+    label:
+      en_US: "User"
+      zh_Hans: "用户"
+    type: string
+    required: false
+    help:
+      en_US: "Used to track and differentiate conversation requests from different users."
+      zh_Hans: "用于追踪和区分不同用户的对话请求。"

api/core/model_runtime/model_providers/gitee_ai/llm/_position.yaml

@@ -1,3 +1,4 @@
+- Qwen2.5-72B-Instruct
 - Qwen2-7B-Instruct
 - Qwen2-72B-Instruct
 - Yi-1.5-34B-Chat

api/core/model_runtime/model_providers/gitee_ai/llm/llm.py

@@ -6,6 +6,7 @@ from core.model_runtime.entities.message_entities import (
     PromptMessage,
     PromptMessageTool,
 )
+from core.model_runtime.entities.model_entities import ModelFeature
 from core.model_runtime.model_providers.openai_api_compatible.llm.llm import OAIAPICompatLargeLanguageModel
 
 
@@ -28,14 +29,13 @@ class GiteeAILargeLanguageModel(OAIAPICompatLargeLanguageModel):
         user: Optional[str] = None,
     ) -> Union[LLMResult, Generator]:
         self._add_custom_parameters(credentials, model, model_parameters)
-        return super()._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream)
+        return super()._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream, user)
 
     def validate_credentials(self, model: str, credentials: dict) -> None:
         self._add_custom_parameters(credentials, model, None)
         super().validate_credentials(model, credentials)
 
-    @staticmethod
-    def _add_custom_parameters(credentials: dict, model: str, model_parameters: dict) -> None:
+    def _add_custom_parameters(self, credentials: dict, model: str, model_parameters: dict) -> None:
         if model is None:
             model = "bge-large-zh-v1.5"
 
@@ -45,3 +45,7 @@ class GiteeAILargeLanguageModel(OAIAPICompatLargeLanguageModel):
             credentials["mode"] = LLMMode.COMPLETION.value
         else:
             credentials["mode"] = LLMMode.CHAT.value
+
+        schema = self.get_model_schema(model, credentials)
+        if ModelFeature.TOOL_CALL in schema.features or ModelFeature.MULTI_TOOL_CALL in schema.features:
+            credentials["function_calling_type"] = "tool_call"

api/core/model_runtime/model_providers/minimax/llm/abab7-chat-preview.yaml

@@ -0,0 +1,46 @@
+model: abab7-chat-preview
+label:
+  en_US: Abab7-chat-preview
+model_type: llm
+features:
+  - agent-thought
+  - tool-call
+  - stream-tool-call
+model_properties:
+  mode: chat
+  context_size: 245760
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    min: 0.01
+    max: 1
+    default: 0.1
+  - name: top_p
+    use_template: top_p
+    min: 0.01
+    max: 1
+    default: 0.95
+  - name: max_tokens
+    use_template: max_tokens
+    required: true
+    default: 2048
+    min: 1
+    max: 245760
+  - name: mask_sensitive_info
+    type: boolean
+    default: true
+    label:
+      zh_Hans: 隐私保护
+      en_US: Moderate
+    help:
+      zh_Hans: 对输出中易涉及隐私问题的文本信息进行打码，目前包括但不限于邮箱、域名、链接、证件号、家庭住址等，默认true，即开启打码
+      en_US: Mask the sensitive info of the generated content, such as email/domain/link/address/phone/id..
+  - name: presence_penalty
+    use_template: presence_penalty
+  - name: frequency_penalty
+    use_template: frequency_penalty
+pricing:
+  input: '0.1'
+  output: '0.1'
+  unit: '0.001'
+  currency: RMB

api/core/model_runtime/model_providers/minimax/llm/llm.py

@@ -34,6 +34,7 @@ from core.model_runtime.model_providers.minimax.llm.types import MinimaxMessage
 
 class MinimaxLargeLanguageModel(LargeLanguageModel):
     model_apis = {
+        "abab7-chat-preview": MinimaxChatCompletionPro,
         "abab6.5s-chat": MinimaxChatCompletionPro,
         "abab6.5-chat": MinimaxChatCompletionPro,
         "abab6-chat": MinimaxChatCompletionPro,

api/core/model_runtime/model_providers/siliconflow/llm/Internvl2-26b.yaml

@@ -0,0 +1,84 @@
+model: OpenGVLab/InternVL2-26B
+label:
+  en_US: OpenGVLab/InternVL2-26B
+model_type: llm
+features:
+  - vision
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/Internvl2-8b.yaml

@@ -0,0 +1,84 @@
+model: Pro/OpenGVLab/InternVL2-8B
+label:
+  en_US: Pro/OpenGVLab/InternVL2-8B
+model_type: llm
+features:
+  - vision
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/_position.yaml

@@ -1,16 +1,18 @@
+- Tencent/Hunyuan-A52B-Instruct
 - Qwen/Qwen2.5-72B-Instruct
 - Qwen/Qwen2.5-32B-Instruct
 - Qwen/Qwen2.5-14B-Instruct
 - Qwen/Qwen2.5-7B-Instruct
+- Qwen/Qwen2.5-Coder-32B-Instruct
 - Qwen/Qwen2.5-Coder-7B-Instruct
 - Qwen/Qwen2.5-Math-72B-Instruct
-- Qwen/Qwen2-72B-Instruct
-- Qwen/Qwen2-57B-A14B-Instruct
-- Qwen/Qwen2-7B-Instruct
+- Qwen/Qwen2-VL-72B-Instruct
 - Qwen/Qwen2-1.5B-Instruct
+- Pro/Qwen/Qwen2-VL-7B-Instruct
+- OpenGVLab/InternVL2-Llama3-76B
+- OpenGVLab/InternVL2-26B
+- Pro/OpenGVLab/InternVL2-8B
 - deepseek-ai/DeepSeek-V2.5
-- deepseek-ai/DeepSeek-V2-Chat
-- deepseek-ai/DeepSeek-Coder-V2-Instruct
 - THUDM/glm-4-9b-chat
 - 01-ai/Yi-1.5-34B-Chat-16K
 - 01-ai/Yi-1.5-9B-Chat-16K
@@ -20,9 +22,6 @@
 - meta-llama/Meta-Llama-3.1-405B-Instruct
 - meta-llama/Meta-Llama-3.1-70B-Instruct
 - meta-llama/Meta-Llama-3.1-8B-Instruct
-- meta-llama/Meta-Llama-3-70B-Instruct
-- meta-llama/Meta-Llama-3-8B-Instruct
 - google/gemma-2-27b-it
 - google/gemma-2-9b-it
-- mistralai/Mistral-7B-Instruct-v0.2
-- mistralai/Mixtral-8x7B-Instruct-v0.1
+- deepseek-ai/DeepSeek-V2-Chat

api/core/model_runtime/model_providers/siliconflow/llm/deepdeek-coder-v2-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '1.33'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/deepseek-v2-chat.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '1.33'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/deepseek-v2.5.yaml

@@ -4,6 +4,8 @@ label:
 model_type: llm
 features:
   - agent-thought
+  - tool-call
+  - stream-tool-call
 model_properties:
   mode: chat
   context_size: 32768
@@ -32,6 +34,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '1.33'
   output: '1.33'

api/core/model_runtime/model_providers/siliconflow/llm/gemma-2-27b-it.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '1.26'
   output: '1.26'

api/core/model_runtime/model_providers/siliconflow/llm/gemma-2-9b-it.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/glm4-9b-chat.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/hunyuan-a52b-instruct.yaml

@@ -0,0 +1,84 @@
+model: Tencent/Hunyuan-A52B-Instruct
+label:
+  en_US: Tencent/Hunyuan-A52B-Instruct
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/internlm2_5-20b-chat.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '1'
   output: '1'

api/core/model_runtime/model_providers/siliconflow/llm/internlm2_5-7b-chat.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/internvl2-llama3-76b.yaml

@@ -0,0 +1,84 @@
+model: OpenGVLab/InternVL2-Llama3-76B
+label:
+  en_US: OpenGVLab/InternVL2-Llama3-76B
+model_type: llm
+features:
+  - vision
+model_properties:
+  mode: chat
+  context_size: 8192
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/llm.py

@@ -29,6 +29,9 @@ class SiliconflowLargeLanguageModel(OAIAPICompatLargeLanguageModel):
         user: Optional[str] = None,
     ) -> Union[LLMResult, Generator]:
         self._add_custom_parameters(credentials)
+        # {"response_format": "json_object"} need convert to {"response_format": {"type": "json_object"}}
+        if "response_format" in model_parameters:
+            model_parameters["response_format"] = {"type": model_parameters.get("response_format")}
         return super()._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream)
 
     def validate_credentials(self, model: str, credentials: dict) -> None:

api/core/model_runtime/model_providers/siliconflow/llm/meta-mlama-3-70b-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '4.13'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/meta-mlama-3-8b-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '0'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/meta-mlama-3.1-405b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '21'
   output: '21'

api/core/model_runtime/model_providers/siliconflow/llm/meta-mlama-3.1-70b-instruct.yaml

@@ -6,7 +6,7 @@ features:
   - agent-thought
 model_properties:
   mode: chat
-  context_size: 32768
+  context_size: 8192
 parameter_rules:
   - name: temperature
     use_template: temperature
@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '4.13'
   output: '4.13'

api/core/model_runtime/model_providers/siliconflow/llm/meta-mlama-3.1-8b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2-57b-a14b-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '1.26'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/qwen2-72b-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '4.13'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/qwen2-7b-instruct.yaml

@@ -37,3 +37,4 @@ pricing:
   output: '0'
   unit: '0.000001'
   currency: RMB
+deprecated: true

api/core/model_runtime/model_providers/siliconflow/llm/qwen2-vl-72b-instruct.yaml

@@ -0,0 +1,84 @@
+model: Qwen/Qwen2-VL-72B-Instruct
+label:
+  en_US: Qwen/Qwen2-VL-72B-Instruct
+model_type: llm
+features:
+  - vision
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/qwen2-vl-7b-Instruct.yaml

@@ -0,0 +1,84 @@
+model: Pro/Qwen/Qwen2-VL-7B-Instruct
+label:
+  en_US: Pro/Qwen/Qwen2-VL-7B-Instruct
+model_type: llm
+features:
+  - vision
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 2000
+    min: 1
+    max: 2000
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '21'
+  output: '21'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-14b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0.7'
   output: '0.7'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-32b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '1.26'
   output: '1.26'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-72b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '4.13'
   output: '4.13'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-7b-instruct.yaml

@@ -32,6 +32,18 @@ parameter_rules:
     required: false
   - name: frequency_penalty
     use_template: frequency_penalty
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-coder-32b-instruct.yaml

@@ -0,0 +1,84 @@
+model: Qwen/Qwen2.5-Coder-32B-Instruct
+label:
+  en_US: Qwen/Qwen2.5-Coder-32B-Instruct
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 32768
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 0.3
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 用于控制随机性和多样性的程度。具体来说，temperature值控制了生成文本时对每个候选词的概率分布进行平滑的程度。较高的temperature值会降低概率分布的峰值，使得更多的低概率词被选择，生成结果更加多样化；而较低的temperature值则会增强概率分布的峰值，使得高概率词更容易被选择，生成结果更加确定。
+      en_US: Used to control the degree of randomness and diversity. Specifically, the temperature value controls the degree to which the probability distribution of each candidate word is smoothed when generating text. A higher temperature value will reduce the peak value of the probability distribution, allowing more low-probability words to be selected, and the generated results will be more diverse; while a lower temperature value will enhance the peak value of the probability distribution, making it easier for high-probability words to be selected. , the generated results are more certain.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 8192
+    min: 1
+    max: 8192
+    help:
+      zh_Hans: 用于指定模型在生成内容时token的最大数量，它定义了生成的上限，但不保证每次都会生成到这个数量。
+      en_US: It is used to specify the maximum number of tokens when the model generates content. It defines the upper limit of generation, but does not guarantee that this number will be generated every time.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 0.8
+    min: 0.1
+    max: 0.9
+    help:
+      zh_Hans: 生成过程中核采样方法概率阈值，例如，取值为0.8时，仅保留概率加起来大于等于0.8的最可能token的最小集合作为候选集。取值范围为（0,1.0)，取值越大，生成的随机性越高；取值越低，生成的确定性越高。
+      en_US: The probability threshold of the kernel sampling method during the generation process. For example, when the value is 0.8, only the smallest set of the most likely tokens with a sum of probabilities greater than or equal to 0.8 is retained as the candidate set. The value range is (0,1.0). The larger the value, the higher the randomness generated; the lower the value, the higher the certainty generated.
+  - name: top_k
+    type: int
+    min: 0
+    max: 99
+    label:
+      zh_Hans: 取样数量
+      en_US: Top k
+    help:
+      zh_Hans: 生成时，采样候选集的大小。例如，取值为50时，仅将单次生成中得分最高的50个token组成随机采样的候选集。取值越大，生成的随机性越高；取值越小，生成的确定性越高。
+      en_US: The size of the sample candidate set when generated. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form a randomly sampled candidate set. The larger the value, the higher the randomness generated; the smaller the value, the higher the certainty generated.
+  - name: seed
+    required: false
+    type: int
+    default: 1234
+    label:
+      zh_Hans: 随机种子
+      en_US: Random seed
+    help:
+      zh_Hans: 生成时使用的随机数种子，用户控制模型生成内容的随机性。支持无符号64位整数，默认值为 1234。在使用seed时，模型将尽可能生成相同或相似的结果，但目前不保证每次生成的结果完全相同。
+      en_US: The random number seed used when generating, the user controls the randomness of the content generated by the model. Supports unsigned 64-bit integers, default value is 1234. When using seed, the model will try its best to generate the same or similar results, but there is currently no guarantee that the results will be exactly the same every time.
+  - name: repetition_penalty
+    required: false
+    type: float
+    default: 1.1
+    label:
+      zh_Hans: 重复惩罚
+      en_US: Repetition penalty
+    help:
+      zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
+      en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
+  - name: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
+pricing:
+  input: '1.26'
+  output: '1.26'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-coder-7b-instruct.yaml

@@ -66,7 +66,17 @@ parameter_rules:
       zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
       en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
   - name: response_format
-    use_template: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '0'
   output: '0'

api/core/model_runtime/model_providers/siliconflow/llm/qwen2.5-math-72b-instruct.yaml

@@ -66,7 +66,17 @@ parameter_rules:
       zh_Hans: 用于控制模型生成时的重复度。提高repetition_penalty时可以降低模型生成的重复度。1.0表示不做惩罚。
       en_US: Used to control the repeatability when generating models. Increasing repetition_penalty can reduce the duplication of model generation. 1.0 means no punishment.
   - name: response_format
-    use_template: response_format
+    label:
+      zh_Hans: 回复格式
+      en_US: Response Format
+    type: string
+    help:
+      zh_Hans: 指定模型必须输出的格式
+      en_US: specifying the format that the model must output
+    required: false
+    options:
+      - text
+      - json_object
 pricing:
   input: '4.13'
   output: '4.13'

api/core/model_runtime/model_providers/siliconflow/speech2text/funaudio-sense-voice-small.yaml

@@ -0,0 +1,5 @@
+model: FunAudioLLM/SenseVoiceSmall
+model_type: speech2text
+model_properties:
+  file_upload_limit: 1
+  supported_file_extensions: mp3,wav

api/core/model_runtime/model_providers/siliconflow/speech2text/sense-voice-small.yaml

@@ -3,3 +3,4 @@ model_type: speech2text
 model_properties:
   file_upload_limit: 1
   supported_file_extensions: mp3,wav
+deprecated: true

api/core/model_runtime/model_providers/volcengine_maas/legacy/volc_sdk/common.py

@@ -1,5 +1,6 @@
 import json
 import random
+from collections import UserDict
 from datetime import datetime
 
 
@@ -10,9 +11,9 @@ class ChatRole:
     FUNCTION = "function"
 
 
-class _Dict(dict):
-    __setattr__ = dict.__setitem__
-    __getattr__ = dict.__getitem__
+class _Dict(UserDict):
+    __setattr__ = UserDict.__setitem__
+    __getattr__ = UserDict.__getitem__
 
     def __missing__(self, key):
         return None

api/core/rag/datasource/vdb/elasticsearch/elasticsearch_vector.py

@@ -178,6 +178,7 @@ class ElasticSearchVector(BaseVector):
                         Field.VECTOR.value: {  # Make sure the dimension is correct here
                             "type": "dense_vector",
                             "dims": dim,
+                            "index": True,
                             "similarity": "cosine",
                         },
                         Field.METADATA_KEY.value: {

api/core/tools/provider/_position.yaml

@@ -78,3 +78,4 @@
 - regex
 - trello
 - vanna
+- fal

api/core/tools/provider/builtin/audio/_assets/icon.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="200" height="200" viewBox="0 0 200 200" fill="none">
+<path d="M167.358 102.395C167.358 117.174 157.246 129.18 144.61 131.027H137.861C125.225 129.18 115.113 117.174 115.113 102.395H100.792C100.792 123.637 115.118 142.106 133.653 145.801V164.276H147.139V145.801C165.674 142.106 180 124.558 180 102.4H167.358V102.395ZM154.717 62.677C154.717 53.4397 147.979 46.9765 140.396 46.9765C138.523 46.9446 136.663 47.3273 134.924 48.1024C133.185 48.8775 131.603 50.0294 130.27 51.4909C128.936 52.9524 127.878 54.6943 127.157 56.6148C126.436 58.5354 126.066 60.5962 126.07 62.677V78.3775H154.717V70.4478V62.677ZM126.07 102.395C126.07 111.632 132.813 118.095 140.396 118.095C142.269 118.127 144.13 117.744 145.868 116.969C147.607 116.194 149.189 115.042 150.523 113.581C151.856 112.119 152.914 110.377 153.635 108.457C154.356 106.536 154.726 104.475 154.722 102.395V86.694H126.07V102.395ZM92.1297 45.8938L70.4796 21.7595L69.4235 20.5865L59.604 20L68.3674 20.5865L67.3113 21.7654L64.1429 25.2961L63.6149 25.8826L64.1429 27.0614L66.2552 29.4133L77.8723 42.3631H54.1099C35.1 43.5361 20.3146 61.1896 20.3146 81.7874V83.5527H28.2354V81.7932C28.2354 65.8992 39.8525 52.3628 54.1099 51.1899H77.8723L66.2552 64.1338L64.671 65.8992L64.1429 67.0722L63.6149 67.6645L64.1429 68.251L68.3674 72.9606L68.8954 73.5471L69.4235 72.9606L74.1759 67.6645L92.1297 47.6591L92.6578 47.0727L92.1297 45.8938ZM20 95.8496V118.213H30.033V107.034H50.099V168.821H40.066V180H70.165V168.821H60.132V107.034H80.198V118.213H90.231V95.8496H20Z" fill="#FF0099"/>
+</svg>

api/core/tools/provider/builtin/audio/audio.py

@@ -0,0 +1,6 @@
+from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
+
+
+class AudioToolProvider(BuiltinToolProviderController):
+    def _validate_credentials(self, credentials: dict) -> None:
+        pass

api/core/tools/provider/builtin/audio/audio.yaml

@@ -0,0 +1,11 @@
+identity:
+  author: hjlarry
+  name: audio
+  label:
+    en_US: Audio
+  description:
+    en_US: A tool for tts and asr.
+    zh_Hans: 一个用于文本转语音和语音转文本的工具。
+  icon: icon.svg
+  tags:
+    - utilities

api/core/tools/provider/builtin/audio/tools/asr.py

@@ -0,0 +1,69 @@
+import io
+from typing import Any
+
+from core.file.enums import FileType
+from core.file.file_manager import download
+from core.model_manager import ModelManager
+from core.model_runtime.entities.model_entities import ModelType
+from core.tools.entities.common_entities import I18nObject
+from core.tools.entities.tool_entities import ToolInvokeMessage, ToolParameter, ToolParameterOption
+from core.tools.tool.builtin_tool import BuiltinTool
+from services.model_provider_service import ModelProviderService
+
+
+class ASRTool(BuiltinTool):
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]) -> list[ToolInvokeMessage]:
+        file = tool_parameters.get("audio_file")
+        if file.type != FileType.AUDIO:
+            return [self.create_text_message("not a valid audio file")]
+        audio_binary = io.BytesIO(download(file))
+        audio_binary.name = "temp.mp3"
+        provider, model = tool_parameters.get("model").split("#")
+        model_manager = ModelManager()
+        model_instance = model_manager.get_model_instance(
+            tenant_id=self.runtime.tenant_id,
+            provider=provider,
+            model_type=ModelType.SPEECH2TEXT,
+            model=model,
+        )
+        text = model_instance.invoke_speech2text(
+            file=audio_binary,
+            user=user_id,
+        )
+        return [self.create_text_message(text)]
+
+    def get_available_models(self) -> list[tuple[str, str]]:
+        model_provider_service = ModelProviderService()
+        models = model_provider_service.get_models_by_model_type(
+            tenant_id=self.runtime.tenant_id, model_type="speech2text"
+        )
+        items = []
+        for provider_model in models:
+            provider = provider_model.provider
+            for model in provider_model.models:
+                items.append((provider, model.model))
+        return items
+
+    def get_runtime_parameters(self) -> list[ToolParameter]:
+        parameters = []
+
+        options = []
+        for provider, model in self.get_available_models():
+            option = ToolParameterOption(value=f"{provider}#{model}", label=I18nObject(en_US=f"{model}({provider})"))
+            options.append(option)
+
+        parameters.append(
+            ToolParameter(
+                name="model",
+                label=I18nObject(en_US="Model", zh_Hans="Model"),
+                human_description=I18nObject(
+                    en_US="All available ASR models. You can config model in the Model Provider of Settings.",
+                    zh_Hans="所有可用的 ASR 模型。你可以在设置中的模型供应商里配置。",
+                ),
+                type=ToolParameter.ToolParameterType.SELECT,
+                form=ToolParameter.ToolParameterForm.FORM,
+                required=True,
+                options=options,
+            )
+        )
+        return parameters

api/core/tools/provider/builtin/audio/tools/asr.yaml

@@ -0,0 +1,22 @@
+identity:
+  name: asr
+  author: hjlarry
+  label:
+    en_US: Speech To Text
+description:
+  human:
+    en_US: Convert audio file to text.
+    zh_Hans: 将音频文件转换为文本。
+  llm: Convert audio file to text.
+parameters:
+  - name: audio_file
+    type: file
+    required: true
+    label:
+      en_US: Audio File
+      zh_Hans: 音频文件
+    human_description:
+      en_US: The audio file to be converted.
+      zh_Hans: 要转换的音频文件。
+    llm_description: The audio file to be converted.
+    form: llm

api/core/tools/provider/builtin/audio/tools/tts.py

@@ -0,0 +1,89 @@
+import io
+from typing import Any
+
+from core.model_manager import ModelManager
+from core.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
+from core.tools.entities.common_entities import I18nObject
+from core.tools.entities.tool_entities import ToolInvokeMessage, ToolParameter, ToolParameterOption
+from core.tools.tool.builtin_tool import BuiltinTool
+from services.model_provider_service import ModelProviderService
+
+
+class TTSTool(BuiltinTool):
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]) -> list[ToolInvokeMessage]:
+        provider, model = tool_parameters.get("model").split("#")
+        voice = tool_parameters.get(f"voice#{provider}#{model}")
+        model_manager = ModelManager()
+        model_instance = model_manager.get_model_instance(
+            tenant_id=self.runtime.tenant_id,
+            provider=provider,
+            model_type=ModelType.TTS,
+            model=model,
+        )
+        tts = model_instance.invoke_tts(
+            content_text=tool_parameters.get("text"),
+            user=user_id,
+            tenant_id=self.runtime.tenant_id,
+            voice=voice,
+        )
+        buffer = io.BytesIO()
+        for chunk in tts:
+            buffer.write(chunk)
+
+        wav_bytes = buffer.getvalue()
+        return [
+            self.create_text_message("Audio generated successfully"),
+            self.create_blob_message(
+                blob=wav_bytes,
+                meta={"mime_type": "audio/x-wav"},
+                save_as=self.VariableKey.AUDIO,
+            ),
+        ]
+
+    def get_available_models(self) -> list[tuple[str, str, list[Any]]]:
+        model_provider_service = ModelProviderService()
+        models = model_provider_service.get_models_by_model_type(tenant_id=self.runtime.tenant_id, model_type="tts")
+        items = []
+        for provider_model in models:
+            provider = provider_model.provider
+            for model in provider_model.models:
+                voices = model.model_properties.get(ModelPropertyKey.VOICES, [])
+                items.append((provider, model.model, voices))
+        return items
+
+    def get_runtime_parameters(self) -> list[ToolParameter]:
+        parameters = []
+
+        options = []
+        for provider, model, voices in self.get_available_models():
+            option = ToolParameterOption(value=f"{provider}#{model}", label=I18nObject(en_US=f"{model}({provider})"))
+            options.append(option)
+            parameters.append(
+                ToolParameter(
+                    name=f"voice#{provider}#{model}",
+                    label=I18nObject(en_US=f"Voice of {model}({provider})"),
+                    type=ToolParameter.ToolParameterType.SELECT,
+                    form=ToolParameter.ToolParameterForm.FORM,
+                    options=[
+                        ToolParameterOption(value=voice.get("mode"), label=I18nObject(en_US=voice.get("name")))
+                        for voice in voices
+                    ],
+                )
+            )
+
+        parameters.insert(
+            0,
+            ToolParameter(
+                name="model",
+                label=I18nObject(en_US="Model", zh_Hans="Model"),
+                human_description=I18nObject(
+                    en_US="All available TTS models. You can config model in the Model Provider of Settings.",
+                    zh_Hans="所有可用的 TTS 模型。你可以在设置中的模型供应商里配置。",
+                ),
+                type=ToolParameter.ToolParameterType.SELECT,
+                form=ToolParameter.ToolParameterForm.FORM,
+                required=True,
+                options=options,
+            ),
+        )
+        return parameters

api/core/tools/provider/builtin/audio/tools/tts.yaml

@@ -0,0 +1,22 @@
+identity:
+  name: tts
+  author: hjlarry
+  label:
+    en_US: Text To Speech
+description:
+  human:
+    en_US: Convert text to audio file.
+    zh_Hans: 将文本转换为音频文件。
+  llm: Convert text to audio file.
+parameters:
+  - name: text
+    type: string
+    required: true
+    label:
+      en_US: Text
+      zh_Hans: 文本
+    human_description:
+      en_US: The text to be converted.
+      zh_Hans: 要转换的文本。
+    llm_description: The text to be converted.
+    form: llm

api/core/tools/provider/builtin/email/email.py

@@ -0,0 +1,7 @@
+from core.tools.provider.builtin.email.tools.send_mail import SendMailTool
+from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
+
+
+class SmtpProvider(BuiltinToolProviderController):
+    def _validate_credentials(self, credentials: dict) -> None:
+        SendMailTool()

api/core/tools/provider/builtin/email/email.yaml

@@ -0,0 +1,83 @@
+identity:
+  author: wakaka6
+  name: email
+  label:
+    en_US: email
+    zh_Hans: 电子邮件
+  description:
+    en_US: send email through smtp protocol
+    zh_Hans: 通过smtp协议发送电子邮件
+  icon: icon.svg
+  tags:
+    - utilities
+credentials_for_provider:
+  email_account:
+    type: text-input
+    required: true
+    label:
+      en_US: email account
+      zh_Hans: 邮件账号
+    placeholder:
+      en_US: input you email account
+      zh_Hans: 输入你的邮箱账号
+    help:
+      en_US: email account
+      zh_Hans: 邮件账号
+  email_password:
+    type: secret-input
+    required: true
+    label:
+      en_US: email password
+      zh_Hans: 邮件密码
+    placeholder:
+      en_US: email password
+      zh_Hans: 邮件密码
+    help:
+      en_US: email password
+      zh_Hans: 邮件密码
+  smtp_server:
+    type: text-input
+    required: true
+    label:
+      en_US: smtp server
+      zh_Hans: 发信smtp服务器地址
+    placeholder:
+      en_US: smtp server
+      zh_Hans: 发信smtp服务器地址
+    help:
+      en_US: smtp server
+      zh_Hans: 发信smtp服务器地址
+  smtp_port:
+    type: text-input
+    required: true
+    label:
+      en_US: smtp server port
+      zh_Hans: 发信smtp服务器端口
+    placeholder:
+      en_US: smtp server port
+      zh_Hans: 发信smtp服务器端口
+    help:
+      en_US: smtp server port
+      zh_Hans: 发信smtp服务器端口
+  encrypt_method:
+    type: select
+    required: true
+    options:
+      - value: NONE
+        label:
+          en_US: NONE
+          zh_Hans: 无加密
+      - value: SSL
+        label:
+          en_US: SSL
+          zh_Hans: SSL加密
+      - value: TLS
+        label:
+          en_US: START TLS
+          zh_Hans: START TLS加密
+    label:
+      en_US: encrypt method
+      zh_Hans: 加密方式
+    help:
+      en_US: smtp server encrypt method
+      zh_Hans: 发信smtp服务器加密方式

api/core/tools/provider/builtin/email/tools/send.py

@@ -0,0 +1,53 @@
+import logging
+import smtplib
+import ssl
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+
+from pydantic import BaseModel
+
+
+class SendEmailToolParameters(BaseModel):
+    smtp_server: str
+    smtp_port: int
+
+    email_account: str
+    email_password: str
+
+    sender_to: str
+    subject: str
+    email_content: str
+    encrypt_method: str
+
+
+def send_mail(parmas: SendEmailToolParameters):
+    timeout = 60
+    msg = MIMEMultipart("alternative")
+    msg["From"] = parmas.email_account
+    msg["To"] = parmas.sender_to
+    msg["Subject"] = parmas.subject
+    msg.attach(MIMEText(parmas.email_content, "plain"))
+    msg.attach(MIMEText(parmas.email_content, "html"))
+
+    ctx = ssl.create_default_context()
+
+    if parmas.encrypt_method.upper() == "SSL":
+        try:
+            with smtplib.SMTP_SSL(parmas.smtp_server, parmas.smtp_port, context=ctx, timeout=timeout) as server:
+                server.login(parmas.email_account, parmas.email_password)
+                server.sendmail(parmas.email_account, parmas.sender_to, msg.as_string())
+                return True
+        except Exception as e:
+            logging.exception("send email failed: %s", e)
+            return False
+    else:  # NONE or TLS
+        try:
+            with smtplib.SMTP(parmas.smtp_server, parmas.smtp_port, timeout=timeout) as server:
+                if parmas.encrypt_method.upper() == "TLS":
+                    server.starttls(context=ctx)
+                server.login(parmas.email_account, parmas.email_password)
+                server.sendmail(parmas.email_account, parmas.sender_to, msg.as_string())
+                return True
+        except Exception as e:
+            logging.exception("send email failed: %s", e)
+            return False

api/core/tools/provider/builtin/email/tools/send_mail.py

@@ -0,0 +1,66 @@
+import re
+from typing import Any, Union
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.provider.builtin.email.tools.send import (
+    SendEmailToolParameters,
+    send_mail,
+)
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class SendMailTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        """
+        invoke tools
+        """
+        sender = self.runtime.credentials.get("email_account", "")
+        email_rgx = re.compile(r"^[a-zA-Z0-9_-]+@[a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)+$")
+        password = self.runtime.credentials.get("email_password", "")
+        smtp_server = self.runtime.credentials.get("smtp_server", "")
+        if not smtp_server:
+            return self.create_text_message("please input smtp server")
+        smtp_port = self.runtime.credentials.get("smtp_port", "")
+        try:
+            smtp_port = int(smtp_port)
+        except ValueError:
+            return self.create_text_message("Invalid parameter smtp_port(should be int)")
+
+        if not sender:
+            return self.create_text_message("please input sender")
+        if not email_rgx.match(sender):
+            return self.create_text_message("Invalid parameter userid, the sender is not a mailbox")
+
+        receiver_email = tool_parameters["send_to"]
+        if not receiver_email:
+            return self.create_text_message("please input receiver email")
+        if not email_rgx.match(receiver_email):
+            return self.create_text_message("Invalid parameter receiver email, the receiver email is not a mailbox")
+        email_content = tool_parameters.get("email_content", "")
+
+        if not email_content:
+            return self.create_text_message("please input email content")
+
+        subject = tool_parameters.get("subject", "")
+        if not subject:
+            return self.create_text_message("please input email subject")
+
+        encrypt_method = self.runtime.credentials.get("encrypt_method", "")
+        if not encrypt_method:
+            return self.create_text_message("please input encrypt method")
+
+        send_email_params = SendEmailToolParameters(
+            smtp_server=smtp_server,
+            smtp_port=smtp_port,
+            email_account=sender,
+            email_password=password,
+            sender_to=receiver_email,
+            subject=subject,
+            email_content=email_content,
+            encrypt_method=encrypt_method,
+        )
+        if send_mail(send_email_params):
+            return self.create_text_message("send email success")
+        return self.create_text_message("send email failed")

api/core/tools/provider/builtin/email/tools/send_mail.yaml

@@ -0,0 +1,46 @@
+identity:
+  name: send_mail
+  author: wakaka6
+  label:
+    en_US: send email
+    zh_Hans: 发送邮件
+  icon: icon.svg
+description:
+  human:
+    en_US: A tool for sending email
+    zh_Hans: 用于发送邮件
+  llm: A tool for sending email
+parameters:
+  - name: send_to
+    type: string
+    required: true
+    label:
+      en_US: Recipient email account
+      zh_Hans: 收件人邮箱账号
+    human_description:
+      en_US: Recipient email account
+      zh_Hans: 收件人邮箱账号
+    llm_description: Recipient email account
+    form: llm
+  - name: subject
+    type: string
+    required: true
+    label:
+      en_US: email subject
+      zh_Hans: 邮件主题
+    human_description:
+      en_US: email subject
+      zh_Hans: 邮件主题
+    llm_description: email subject
+    form: llm
+  - name: email_content
+    type: string
+    required: true
+    label:
+      en_US: email content
+      zh_Hans: 邮件内容
+    human_description:
+      en_US: email content
+      zh_Hans: 邮件内容
+    llm_description: email content
+    form: llm

api/core/tools/provider/builtin/email/tools/send_mail_batch.py

@@ -0,0 +1,75 @@
+import json
+import re
+from typing import Any, Union
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.provider.builtin.email.tools.send import (
+    SendEmailToolParameters,
+    send_mail,
+)
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class SendMailTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        """
+        invoke tools
+        """
+        sender = self.runtime.credentials.get("email_account", "")
+        email_rgx = re.compile(r"^[a-zA-Z0-9_-]+@[a-zA-Z0-9_-]+(\.[a-zA-Z0-9_-]+)+$")
+        password = self.runtime.credentials.get("email_password", "")
+        smtp_server = self.runtime.credentials.get("smtp_server", "")
+        if not smtp_server:
+            return self.create_text_message("please input smtp server")
+        smtp_port = self.runtime.credentials.get("smtp_port", "")
+        try:
+            smtp_port = int(smtp_port)
+        except ValueError:
+            return self.create_text_message("Invalid parameter smtp_port(should be int)")
+
+        if not sender:
+            return self.create_text_message("please input sender")
+        if not email_rgx.match(sender):
+            return self.create_text_message("Invalid parameter userid, the sender is not a mailbox")
+
+        receivers_email = tool_parameters["send_to"]
+        if not receivers_email:
+            return self.create_text_message("please input receiver email")
+        receivers_email = json.loads(receivers_email)
+        for receiver in receivers_email:
+            if not email_rgx.match(receiver):
+                return self.create_text_message(
+                    f"Invalid parameter receiver email, the receiver email({receiver}) is not a mailbox"
+                )
+        email_content = tool_parameters.get("email_content", "")
+
+        if not email_content:
+            return self.create_text_message("please input email content")
+
+        subject = tool_parameters.get("subject", "")
+        if not subject:
+            return self.create_text_message("please input email subject")
+
+        encrypt_method = self.runtime.credentials.get("encrypt_method", "")
+        if not encrypt_method:
+            return self.create_text_message("please input encrypt method")
+
+        msg = {}
+        for receiver in receivers_email:
+            send_email_params = SendEmailToolParameters(
+                smtp_server=smtp_server,
+                smtp_port=smtp_port,
+                email_account=sender,
+                email_password=password,
+                sender_to=receiver,
+                subject=subject,
+                email_content=email_content,
+                encrypt_method=encrypt_method,
+            )
+            if send_mail(send_email_params):
+                msg[receiver] = "send email success"
+            else:
+                msg[receiver] = "send email failed"
+        return self.create_text_message(json.dumps(msg))

api/core/tools/provider/builtin/email/tools/send_mail_batch.yaml

@@ -0,0 +1,46 @@
+identity:
+  name: send_mail_batch
+  author: wakaka6
+  label:
+    en_US: send email to multiple recipients
+    zh_Hans: 发送邮件给多个收件人
+  icon: icon.svg
+description:
+  human:
+    en_US: A tool for sending email to multiple recipients
+    zh_Hans: 用于发送邮件给多个收件人的工具
+  llm: A tool for sending email to multiple recipients
+parameters:
+  - name: send_to
+    type: string
+    required: true
+    label:
+      en_US: Recipient email account(json list)
+      zh_Hans: 收件人邮箱账号(json list)
+    human_description:
+      en_US: Recipient email account
+      zh_Hans: 收件人邮箱账号
+    llm_description: A list of recipient email account(json format)
+    form: llm
+  - name: subject
+    type: string
+    required: true
+    label:
+      en_US: email subject
+      zh_Hans: 邮件主题
+    human_description:
+      en_US: email subject
+      zh_Hans: 邮件主题
+    llm_description: email subject
+    form: llm
+  - name: email_content
+    type: string
+    required: true
+    label:
+      en_US: email content
+      zh_Hans: 邮件内容
+    human_description:
+      en_US: email content
+      zh_Hans: 邮件内容
+    llm_description: email content
+    form: llm

api/core/tools/provider/builtin/fal/_assets/icon.svg

@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg version="1.1" xmlns="http://www.w3.org/2000/svg" width="32" height="32">
+<path d="M0 0 C3.96 0 7.92 0 12 0 C12.4125 0.928125 12.825 1.85625 13.25 2.8125 C15.56104487 7.02190315 17.49701732 8.49900577 22 10 C22 13.96 22 17.92 22 22 C21.071875 22.4125 20.14375 22.825 19.1875 23.25 C14.97809685 25.56104487 13.50099423 27.49701732 12 32 C8.04 32 4.08 32 0 32 C-0.4125 31.071875 -0.825 30.14375 -1.25 29.1875 C-3.56104487 24.97809685 -5.49701732 23.50099423 -10 22 C-10 18.04 -10 14.08 -10 10 C-9.071875 9.5875 -8.14375 9.175 -7.1875 8.75 C-2.97809685 6.43895513 -1.50099423 4.50298268 0 0 Z M-2 11 C-3.42662219 13.85324437 -3.31033868 15.83454549 -3 19 C-1.20006226 21.69990662 0.083773 23.5418865 3 25 C7.1364408 25.56406011 8.76045933 25.14638597 12.375 22.9375 C15.26054626 20.20817124 15.26054626 20.20817124 15.6875 16.5625 C14.76325283 11.77321919 13.68514918 10.2147046 10 7 C4.54838272 6.02649691 1.87056683 7.12943317 -2 11 Z " fill="#EC0648" transform="translate(10,0)"/>
+</svg>

api/core/tools/provider/builtin/fal/fal.py

@@ -0,0 +1,20 @@
+import requests
+
+from core.tools.errors import ToolProviderCredentialValidationError
+from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
+
+
+class FalProvider(BuiltinToolProviderController):
+    def _validate_credentials(self, credentials: dict) -> None:
+        url = "https://fal.run/fal-ai/flux/dev"
+        headers = {
+            "Authorization": f"Key {credentials.get('fal_api_key')}",
+            "Content-Type": "application/json",
+        }
+        data = {"prompt": "Cat"}
+
+        response = requests.post(url, json=data, headers=headers)
+        if response.status_code == 401:
+            raise ToolProviderCredentialValidationError("FAL API key is invalid")
+        elif response.status_code != 200:
+            raise ToolProviderCredentialValidationError(f"FAL API key validation failed: {response.text}")

api/core/tools/provider/builtin/fal/fal.yaml

@@ -0,0 +1,21 @@
+identity:
+  author: Kalo Chin
+  name: fal
+  label:
+    en_US: FAL
+    zh_CN: FAL
+  description:
+    en_US: The image generation API provided by FAL.
+    zh_CN: FAL 提供的图像生成 API。
+  icon: icon.svg
+  tags:
+    - image
+credentials_for_provider:
+  fal_api_key:
+    type: secret-input
+    required: true
+    label:
+      en_US: FAL API Key
+    placeholder:
+      en_US: Please input your FAL API key
+    url: https://fal.ai/dashboard/keys

api/core/tools/provider/builtin/fal/tools/flux_1_1_pro.py

@@ -0,0 +1,46 @@
+from typing import Any, Union
+
+import requests
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class Flux11ProTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        headers = {
+            "Authorization": f"Key {self.runtime.credentials['fal_api_key']}",
+            "Content-Type": "application/json",
+        }
+
+        prompt = tool_parameters.get("prompt", "")
+        sanitized_prompt = prompt.replace("\\", "")  # Remove backslashes from the prompt which may cause errors
+
+        payload = {
+            "prompt": sanitized_prompt,
+            "image_size": tool_parameters.get("image_size", "landscape_4_3"),
+            "seed": tool_parameters.get("seed"),
+            "sync_mode": tool_parameters.get("sync_mode", False),
+            "num_images": tool_parameters.get("num_images", 1),
+            "enable_safety_checker": tool_parameters.get("enable_safety_checker", True),
+            "safety_tolerance": tool_parameters.get("safety_tolerance", "2"),
+        }
+
+        url = "https://fal.run/fal-ai/flux-pro/v1.1"
+
+        response = requests.post(url, json=payload, headers=headers)
+
+        if response.status_code != 200:
+            return self.create_text_message(f"Got Error Response: {response.text}")
+
+        res = response.json()
+        result = [self.create_json_message(res)]
+
+        for image_info in res.get("images", []):
+            image_url = image_info.get("url")
+            if image_url:
+                result.append(self.create_image_message(image=image_url, save_as=self.VariableKey.IMAGE.value))
+
+        return result

api/core/tools/provider/builtin/fal/tools/flux_1_1_pro.yaml

@@ -0,0 +1,147 @@
+identity:
+  name: flux_1_1_pro
+  author: Kalo Chin
+  label:
+    en_US: FLUX 1.1 [pro]
+    zh_Hans: FLUX 1.1 [pro]
+  icon: icon.svg
+description:
+  human:
+    en_US: FLUX 1.1 [pro] is an enhanced version of FLUX.1 [pro], improved image generation capabilities, delivering superior composition, detail, and artistic fidelity compared to its predecessor.
+    zh_Hans: FLUX 1.1 [pro] 是 FLUX.1 [pro] 的增强版，改进了图像生成能力，与其前身相比，提供了更出色的构图、细节和艺术保真度。
+  llm: This tool generates images from prompts using FAL's FLUX 1.1 [pro] model.
+parameters:
+  - name: prompt
+    type: string
+    required: true
+    label:
+      en_US: Prompt
+      zh_Hans: 提示词
+    human_description:
+      en_US: The text prompt used to generate the image.
+      zh_Hans: 用于生成图片的文字提示词。
+    llm_description: This prompt text will be used to generate the image.
+    form: llm
+  - name: image_size
+    type: select
+    required: false
+    options:
+      - value: square_hd
+        label:
+          en_US: Square HD
+          zh_Hans: 方形高清
+      - value: square
+        label:
+          en_US: Square
+          zh_Hans: 方形
+      - value: portrait_4_3
+        label:
+          en_US: Portrait 4:3
+          zh_Hans: 竖屏 4:3
+      - value: portrait_16_9
+        label:
+          en_US: Portrait 16:9
+          zh_Hans: 竖屏 16:9
+      - value: landscape_4_3
+        label:
+          en_US: Landscape 4:3
+          zh_Hans: 横屏 4:3
+      - value: landscape_16_9
+        label:
+          en_US: Landscape 16:9
+          zh_Hans: 横屏 16:9
+    default: landscape_4_3
+    label:
+      en_US: Image Size
+      zh_Hans: 图片大小
+    human_description:
+      en_US: The size of the generated image.
+      zh_Hans: 生成图像的尺寸。
+    form: form
+  - name: num_images
+    type: number
+    required: false
+    default: 1
+    min: 1
+    max: 1
+    label:
+      en_US: Number of Images
+      zh_Hans: 图片数量
+    human_description:
+      en_US: The number of images to generate.
+      zh_Hans: 要生成的图片数量。
+    form: form
+  - name: safety_tolerance
+    type: select
+    required: false
+    options:
+      - value: "1"
+        label:
+          en_US: "1 (Most strict)"
+          zh_Hans: "1（最严格）"
+      - value: "2"
+        label:
+          en_US: "2"
+          zh_Hans: "2"
+      - value: "3"
+        label:
+          en_US: "3"
+          zh_Hans: "3"
+      - value: "4"
+        label:
+          en_US: "4"
+          zh_Hans: "4"
+      - value: "5"
+        label:
+          en_US: "5"
+          zh_Hans: "5"
+      - value: "6"
+        label:
+          en_US: "6 (Most permissive)"
+          zh_Hans: "6（最宽松）"
+    default: "2"
+    label:
+      en_US: Safety Tolerance
+      zh_Hans: 安全容忍度
+    human_description:
+      en_US: The safety tolerance level for the generated image. 1 being the most strict and 6 being the most permissive.
+      zh_Hans: 生成图像的安全容忍级别，1 为最严格，6 为最宽松。
+    form: form
+  - name: seed
+    type: number
+    required: false
+    min: 0
+    max: 9999999999
+    label:
+      en_US: Seed
+      zh_Hans: 种子
+    human_description:
+      en_US: The same seed and prompt can produce similar images.
+      zh_Hans: 相同的种子和提示词可以产生相似的图像。
+    form: form
+  - name: enable_safety_checker
+    type: boolean
+    required: false
+    default: true
+    label:
+      en_US: Enable Safety Checker
+      zh_Hans: 启用安全检查器
+    human_description:
+      en_US: Enable or disable the safety checker.
+      zh_Hans: 启用或禁用安全检查器。
+    form: form
+  - name: sync_mode
+    type: boolean
+    required: false
+    default: false
+    label:
+      en_US: Sync Mode
+      zh_Hans: 同步模式
+    human_description:
+      en_US: >
+        If set to true, the function will wait for the image to be generated and uploaded before returning the response.
+        This will increase the latency but allows you to get the image directly in the response without going through the CDN.
+      zh_Hans: >
+        如果设置为 true，函数将在生成并上传图像后再返回响应。
+        这将增加函数的延迟，但可以让您直接在响应中获取图像，而无需通过 CDN。
+    form: form

api/core/tools/provider/builtin/fal/tools/flux_1_1_pro_ultra.py

@@ -0,0 +1,47 @@
+from typing import Any, Union
+
+import requests
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class Flux11ProUltraTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        headers = {
+            "Authorization": f"Key {self.runtime.credentials['fal_api_key']}",
+            "Content-Type": "application/json",
+        }
+
+        prompt = tool_parameters.get("prompt", "")
+        sanitized_prompt = prompt.replace("\\", "")  # Remove backslashes from the prompt which may cause errors
+
+        payload = {
+            "prompt": sanitized_prompt,
+            "seed": tool_parameters.get("seed"),
+            "sync_mode": tool_parameters.get("sync_mode", False),
+            "num_images": tool_parameters.get("num_images", 1),
+            "enable_safety_checker": tool_parameters.get("enable_safety_checker", True),
+            "safety_tolerance": str(tool_parameters.get("safety_tolerance", "2")),
+            "aspect_ratio": tool_parameters.get("aspect_ratio", "16:9"),
+            "raw": tool_parameters.get("raw", False),
+        }
+
+        url = "https://fal.run/fal-ai/flux-pro/v1.1-ultra"
+
+        response = requests.post(url, json=payload, headers=headers)
+
+        if response.status_code != 200:
+            return self.create_text_message(f"Got Error Response: {response.text}")
+
+        res = response.json()
+        result = [self.create_json_message(res)]
+
+        for image_info in res.get("images", []):
+            image_url = image_info.get("url")
+            if image_url:
+                result.append(self.create_image_message(image=image_url, save_as=self.VariableKey.IMAGE.value))
+
+        return result

api/core/tools/provider/builtin/fal/tools/flux_1_1_pro_ultra.yaml

@@ -0,0 +1,162 @@
+identity:
+  name: flux_1_1_pro_ultra
+  author: Kalo Chin
+  label:
+    en_US: FLUX 1.1 [pro] ultra
+    zh_Hans: FLUX 1.1 [pro] ultra
+  icon: icon.svg
+description:
+  human:
+    en_US: FLUX 1.1 [pro] ultra is the newest version of FLUX 1.1 [pro], maintaining professional-grade image quality while delivering up to 2K resolution with improved photo realism.
+    zh_Hans: FLUX 1.1 [pro] ultra 是 FLUX 1.1 [pro] 的最新版本，保持了专业级的图像质量，同时以改进的照片真实感提供高达 2K 的分辨率。
+  llm: This tool generates images from prompts using FAL's FLUX 1.1 [pro] ultra model.
+parameters:
+  - name: prompt
+    type: string
+    required: true
+    label:
+      en_US: Prompt
+      zh_Hans: 提示词
+    human_description:
+      en_US: The text prompt used to generate the image.
+      zh_Hans: 用于生成图像的文本提示。
+    llm_description: This prompt text will be used to generate the image.
+    form: llm
+  - name: aspect_ratio
+    type: select
+    required: false
+    options:
+      - value: '21:9'
+        label:
+          en_US: '21:9'
+          zh_Hans: '21:9'
+      - value: '16:9'
+        label:
+          en_US: '16:9'
+          zh_Hans: '16:9'
+      - value: '4:3'
+        label:
+          en_US: '4:3'
+          zh_Hans: '4:3'
+      - value: '1:1'
+        label:
+          en_US: '1:1'
+          zh_Hans: '1:1'
+      - value: '3:4'
+        label:
+          en_US: '3:4'
+          zh_Hans: '3:4'
+      - value: '9:16'
+        label:
+          en_US: '9:16'
+          zh_Hans: '9:16'
+      - value: '9:21'
+        label:
+          en_US: '9:21'
+          zh_Hans: '9:21'
+    default: '16:9'
+    label:
+      en_US: Aspect Ratio
+      zh_Hans: 纵横比
+    human_description:
+      en_US: The aspect ratio of the generated image.
+      zh_Hans: 生成图像的宽高比。
+    form: form
+  - name: num_images
+    type: number
+    required: false
+    default: 1
+    min: 1
+    max: 1
+    label:
+      en_US: Number of Images
+      zh_Hans: 图片数量
+    human_description:
+      en_US: The number of images to generate.
+      zh_Hans: 要生成的图像数量。
+    form: form
+  - name: safety_tolerance
+    type: select
+    required: false
+    options:
+      - value: "1"
+        label:
+          en_US: "1 (Most strict)"
+          zh_Hans: "1（最严格）"
+      - value: "2"
+        label:
+          en_US: "2"
+          zh_Hans: "2"
+      - value: "3"
+        label:
+          en_US: "3"
+          zh_Hans: "3"
+      - value: "4"
+        label:
+          en_US: "4"
+          zh_Hans: "4"
+      - value: "5"
+        label:
+          en_US: "5"
+          zh_Hans: "5"
+      - value: "6"
+        label:
+          en_US: "6 (Most permissive)"
+          zh_Hans: "6（最宽松）"
+    default: '2'
+    label:
+      en_US: Safety Tolerance
+      zh_Hans: 安全容忍度
+    human_description:
+      en_US: The safety tolerance level for the generated image. 1 being the most strict and 6 being the most permissive.
+      zh_Hans: 生成图像的安全容忍级别，1 为最严格，6 为最宽松。
+    form: form
+  - name: seed
+    type: number
+    required: false
+    min: 0
+    max: 9999999999
+    label:
+      en_US: Seed
+      zh_Hans: 种子
+    human_description:
+      en_US: The same seed and prompt can produce similar images.
+      zh_Hans: 相同的种子和提示词可以生成相似的图像。
+    form: form
+  - name: raw
+    type: boolean
+    required: false
+    default: false
+    label:
+      en_US: Raw Mode
+      zh_Hans: 原始模式
+    human_description:
+      en_US: Generate less processed, more natural-looking images.
+      zh_Hans: 生成较少处理、更自然的图像。
+    form: form
+  - name: enable_safety_checker
+    type: boolean
+    required: false
+    default: true
+    label:
+      en_US: Enable Safety Checker
+      zh_Hans: 启用安全检查器
+    human_description:
+      en_US: Enable or disable the safety checker.
+      zh_Hans: 启用或禁用安全检查器。
+    form: form
+  - name: sync_mode
+    type: boolean
+    required: false
+    default: false
+    label:
+      en_US: Sync Mode
+      zh_Hans: 同步模式
+    human_description:
+      en_US: >
+        If set to true, the function will wait for the image to be generated and uploaded before returning the response.
+        This will increase the latency but allows you to get the image directly in the response without going through the CDN.
+      zh_Hans: >
+        如果设置为 true，函数将在生成并上传图像后才返回响应。
+        这将增加延迟，但允许您直接在响应中获取图像，而无需通过 CDN。
+    form: form

api/core/tools/provider/builtin/fal/tools/flux_1_dev.py

@@ -0,0 +1,47 @@
+from typing import Any, Union
+
+import requests
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class Flux1DevTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        headers = {
+            "Authorization": f"Key {self.runtime.credentials['fal_api_key']}",
+            "Content-Type": "application/json",
+        }
+
+        prompt = tool_parameters.get("prompt", "")
+        sanitized_prompt = prompt.replace("\\", "")  # Remove backslashes from the prompt which may cause errors
+
+        payload = {
+            "prompt": sanitized_prompt,
+            "image_size": tool_parameters.get("image_size", "landscape_4_3"),
+            "num_inference_steps": tool_parameters.get("num_inference_steps", 28),
+            "guidance_scale": tool_parameters.get("guidance_scale", 3.5),
+            "seed": tool_parameters.get("seed"),
+            "num_images": tool_parameters.get("num_images", 1),
+            "enable_safety_checker": tool_parameters.get("enable_safety_checker", True),
+            "sync_mode": tool_parameters.get("sync_mode", False),
+        }
+
+        url = "https://fal.run/fal-ai/flux/dev"
+
+        response = requests.post(url, json=payload, headers=headers)
+
+        if response.status_code != 200:
+            return self.create_text_message(f"Got Error Response: {response.text}")
+
+        res = response.json()
+        result = [self.create_json_message(res)]
+
+        for image_info in res.get("images", []):
+            image_url = image_info.get("url")
+            if image_url:
+                result.append(self.create_image_message(image=image_url, save_as=self.VariableKey.IMAGE.value))
+
+        return result

api/core/tools/provider/builtin/fal/tools/flux_1_dev.yaml

@@ -0,0 +1,137 @@
+identity:
+  name: flux_1_dev
+  author: Kalo Chin
+  label:
+    en_US: FLUX.1 [dev]
+    zh_Hans: FLUX.1 [dev]
+  icon: icon.svg
+description:
+  human:
+    en_US: FLUX.1 [dev] is a 12 billion parameter flow transformer that generates high-quality images from text. It is suitable for personal and commercial use.
+    zh_Hans: FLUX.1 [dev] 是一个拥有120亿参数的流动变换模型，可以从文本生成高质量的图像。适用于个人和商业用途。
+  llm: This tool generates images from prompts using FAL's FLUX.1 [dev] model.
+parameters:
+  - name: prompt
+    type: string
+    required: true
+    label:
+      en_US: Prompt
+      zh_Hans: 提示词
+    human_description:
+      en_US: The text prompt used to generate the image.
+      zh_Hans: 用于生成图片的文字提示词。
+    llm_description: This prompt text will be used to generate the image.
+    form: llm
+  - name: image_size
+    type: select
+    required: false
+    options:
+      - value: square_hd
+        label:
+          en_US: Square HD
+          zh_Hans: 方形高清
+      - value: square
+        label:
+          en_US: Square
+          zh_Hans: 方形
+      - value: portrait_4_3
+        label:
+          en_US: Portrait 4:3
+          zh_Hans: 竖屏 4:3
+      - value: portrait_16_9
+        label:
+          en_US: Portrait 16:9
+          zh_Hans: 竖屏 16:9
+      - value: landscape_4_3
+        label:
+          en_US: Landscape 4:3
+          zh_Hans: 横屏 4:3
+      - value: landscape_16_9
+        label:
+          en_US: Landscape 16:9
+          zh_Hans: 横屏 16:9
+    default: landscape_4_3
+    label:
+      en_US: Image Size
+      zh_Hans: 图片大小
+    human_description:
+      en_US: The size of the generated image.
+      zh_Hans: 生成图像的尺寸。
+    form: form
+  - name: num_images
+    type: number
+    required: false
+    default: 1
+    min: 1
+    max: 4
+    label:
+      en_US: Number of Images
+      zh_Hans: 图片数量
+    human_description:
+      en_US: The number of images to generate.
+      zh_Hans: 要生成的图片数量。
+    form: form
+  - name: num_inference_steps
+    type: number
+    required: false
+    default: 28
+    min: 1
+    max: 50
+    label:
+      en_US: Num Inference Steps
+      zh_Hans: 推理步数
+    human_description:
+      en_US: The number of inference steps to perform. More steps produce higher quality but take longer.
+      zh_Hans: 执行的推理步骤数量。更多的步骤可以产生更高质量的结果，但需要更长的时间。
+    form: form
+  - name: guidance_scale
+    type: number
+    required: false
+    default: 3.5
+    min: 0
+    max: 20
+    label:
+      en_US: Guidance Scale
+      zh_Hans: 指导强度
+    human_description:
+      en_US: How closely the model should follow the prompt.
+      zh_Hans: 模型对提示词的遵循程度。
+    form: form
+  - name: seed
+    type: number
+    required: false
+    min: 0
+    max: 9999999999
+    label:
+      en_US: Seed
+      zh_Hans: 种子
+    human_description:
+      en_US: The same seed and prompt can produce similar images.
+      zh_Hans: 相同的种子和提示可以产生相似的图像。
+    form: form
+  - name: enable_safety_checker
+    type: boolean
+    required: false
+    default: true
+    label:
+      en_US: Enable Safety Checker
+      zh_Hans: 启用安全检查器
+    human_description:
+      en_US: Enable or disable the safety checker.
+      zh_Hans: 启用或禁用安全检查器。
+    form: form
+  - name: sync_mode
+    type: boolean
+    required: false
+    default: false
+    label:
+      en_US: Sync Mode
+      zh_Hans: 同步模式
+    human_description:
+      en_US: >
+        If set to true, the function will wait for the image to be generated and uploaded before returning the response.
+        This will increase the latency but allows you to get the image directly in the response without going through the CDN.
+      zh_Hans: >
+        如果设置为 true，函数将在生成并上传图像后再返回响应。
+        这将增加函数的延迟，但可以让您直接在响应中获取图像，而无需通过 CDN。
+    form: form

api/core/tools/provider/builtin/fal/tools/flux_1_pro_new.py

@@ -0,0 +1,47 @@
+from typing import Any, Union
+
+import requests
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class Flux1ProNewTool(BuiltinTool):
+    def _invoke(
+        self, user_id: str, tool_parameters: dict[str, Any]
+    ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        headers = {
+            "Authorization": f"Key {self.runtime.credentials['fal_api_key']}",
+            "Content-Type": "application/json",
+        }
+
+        prompt = tool_parameters.get("prompt", "")
+        sanitized_prompt = prompt.replace("\\", "")  # Remove backslashes that may cause errors
+
+        payload = {
+            "prompt": sanitized_prompt,
+            "image_size": tool_parameters.get("image_size", "landscape_4_3"),
+            "num_inference_steps": tool_parameters.get("num_inference_steps", 28),
+            "guidance_scale": tool_parameters.get("guidance_scale", 3.5),
+            "seed": tool_parameters.get("seed"),
+            "num_images": tool_parameters.get("num_images", 1),
+            "safety_tolerance": tool_parameters.get("safety_tolerance", "2"),
+            "sync_mode": tool_parameters.get("sync_mode", False),
+        }
+
+        url = "https://fal.run/fal-ai/flux-pro/new"
+
+        response = requests.post(url, json=payload, headers=headers)
+
+        if response.status_code != 200:
+            return self.create_text_message(f"Got Error Response: {response.text}")
+
+        res = response.json()
+        result = [self.create_json_message(res)]
+
+        for image_info in res.get("images", []):
+            image_url = image_info.get("url")
+            if image_url:
+                result.append(self.create_image_message(image=image_url, save_as=self.VariableKey.IMAGE.value))
+
+        return result

api/core/tools/provider/builtin/fal/tools/flux_1_pro_new.yaml

@@ -0,0 +1,164 @@
+identity:
+  name: flux_1_pro_new
+  author: Kalo Chin
+  label:
+    en_US: FLUX.1 [pro] new
+    zh_Hans: FLUX.1 [pro] new
+  icon: icon.svg
+description:
+  human:
+    en_US: FLUX.1 [pro] new is an accelerated version of FLUX.1 [pro], maintaining professional-grade image quality while delivering significantly faster generation speeds.
+    zh_Hans: FLUX.1 [pro] new 是 FLUX.1 [pro] 的加速版本，在保持专业级图像质量的同时，大大提高了生成速度。
+  llm: This tool generates images from prompts using FAL's FLUX.1 [pro] new model.
+parameters:
+  - name: prompt
+    type: string
+    required: true
+    label:
+      en_US: Prompt
+      zh_Hans: 提示词
+    human_description:
+      en_US: The text prompt used to generate the image.
+      zh_Hans: 用于生成图像的文本提示。
+    llm_description: This prompt text will be used to generate the image.
+    form: llm
+  - name: image_size
+    type: select
+    required: false
+    options:
+      - value: square_hd
+        label:
+          en_US: Square HD
+          zh_Hans: 正方形高清
+      - value: square
+        label:
+          en_US: Square
+          zh_Hans: 正方形
+      - value: portrait_4_3
+        label:
+          en_US: Portrait 4:3
+          zh_Hans: 竖屏 4:3
+      - value: portrait_16_9
+        label:
+          en_US: Portrait 16:9
+          zh_Hans: 竖屏 16:9
+      - value: landscape_4_3
+        label:
+          en_US: Landscape 4:3
+          zh_Hans: 横屏 4:3
+      - value: landscape_16_9
+        label:
+          en_US: Landscape 16:9
+          zh_Hans: 横屏 16:9
+    default: landscape_4_3
+    label:
+      en_US: Image Size
+      zh_Hans: 图像尺寸
+    human_description:
+      en_US: The size of the generated image.
+      zh_Hans: 生成图像的尺寸。
+    form: form
+  - name: num_images
+    type: number
+    required: false
+    default: 1
+    min: 1
+    max: 1
+    label:
+      en_US: Number of Images
+      zh_Hans: 图像数量
+    human_description:
+      en_US: The number of images to generate.
+      zh_Hans: 要生成的图像数量。
+    form: form
+  - name: num_inference_steps
+    type: number
+    required: false
+    default: 28
+    min: 1
+    max: 50
+    label:
+      en_US: Num Inference Steps
+      zh_Hans: 推理步数
+    human_description:
+      en_US: The number of inference steps to perform. More steps produce higher quality but take longer.
+      zh_Hans: 执行的推理步数。步数越多，质量越高，但所需时间也更长。
+    form: form
+  - name: guidance_scale
+    type: number
+    required: false
+    default: 3.5
+    min: 0
+    max: 20
+    label:
+      en_US: Guidance Scale
+      zh_Hans: 指导强度
+    human_description:
+      en_US: How closely the model should follow the prompt.
+      zh_Hans: 模型对提示词的遵循程度。
+    form: form
+  - name: safety_tolerance
+    type: select
+    required: false
+    options:
+      - value: "1"
+        label:
+          en_US: "1 (Most strict)"
+          zh_Hans: "1（最严格）"
+      - value: "2"
+        label:
+          en_US: "2"
+          zh_Hans: "2"
+      - value: "3"
+        label:
+          en_US: "3"
+          zh_Hans: "3"
+      - value: "4"
+        label:
+          en_US: "4"
+          zh_Hans: "4"
+      - value: "5"
+        label:
+          en_US: "5"
+          zh_Hans: "5"
+      - value: "6"
+        label:
+          en_US: "6 (Most permissive)"
+          zh_Hans: "6（最宽松）"
+    default: "2"
+    label:
+      en_US: Safety Tolerance
+      zh_Hans: 安全容忍度
+    human_description:
+      en_US: >
+        The safety tolerance level for the generated image. 1 being the most strict and 5 being the most permissive.
+      zh_Hans: >
+        生成图像的安全容忍级别。1 是最严格，6 是最宽松。
+    form: form
+  - name: seed
+    type: number
+    required: false
+    min: 0
+    max: 9999999999
+    label:
+      en_US: Seed
+      zh_Hans: 种子
+    human_description:
+      en_US: The same seed and prompt can produce similar images.
+      zh_Hans: 相同的种子和提示词可以生成相似的图像。
+    form: form
+  - name: sync_mode
+    type: boolean
+    required: false
+    default: false
+    label:
+      en_US: Sync Mode
+      zh_Hans: 同步模式
+    human_description:
+      en_US: >
+        If set to true, the function will wait for the image to be generated and uploaded before returning the response.
+        This will increase the latency but allows you to get the image directly in the response without going through the CDN.
+      zh_Hans: >
+        如果设置为 true，函数将在生成并上传图像后才返回响应。
+        这将增加延迟，但允许您直接在响应中获取图像，而无需通过 CDN。
+    form: form

api/core/tools/provider/builtin/fal/tools/wizper.py

@@ -0,0 +1,52 @@
+import io
+import os
+from typing import Any
+
+import fal_client
+
+from core.file.enums import FileAttribute, FileType
+from core.file.file_manager import download, get_attr
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class WizperTool(BuiltinTool):
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]) -> ToolInvokeMessage:
+        audio_file = tool_parameters.get("audio_file")
+        task = tool_parameters.get("task", "transcribe")
+        language = tool_parameters.get("language", "en")
+        chunk_level = tool_parameters.get("chunk_level", "segment")
+        version = tool_parameters.get("version", "3")
+
+        if audio_file.type != FileType.AUDIO:
+            return [self.create_text_message("Not a valid audio file.")]
+
+        api_key = self.runtime.credentials["fal_api_key"]
+
+        os.environ["FAL_KEY"] = api_key
+
+        audio_binary = io.BytesIO(download(audio_file))
+        mime_type = get_attr(file=audio_file, attr=FileAttribute.MIME_TYPE)
+        file_data = audio_binary.getvalue()
+
+        try:
+            audio_url = fal_client.upload(file_data, mime_type)
+
+        except Exception as e:
+            return [self.create_text_message(f"Error uploading audio file: {str(e)}")]
+
+        arguments = {
+            "audio_url": audio_url,
+            "task": task,
+            "language": language,
+            "chunk_level": chunk_level,
+            "version": version,
+        }
+
+        result = fal_client.subscribe(
+            "fal-ai/wizper",
+            arguments=arguments,
+            with_logs=False,
+        )
+
+        return self.create_json_message(result)

api/core/tools/provider/builtin/fal/tools/wizper.yaml

@@ -0,0 +1,489 @@
+identity:
+  name: wizper
+  author: Kalo Chin
+  label:
+    en_US: Wizper
+    zh_Hans: Wizper
+description:
+  human:
+    en_US: Transcribe an audio file using the Whisper model.
+    zh_Hans: 使用 Whisper 模型转录音频文件。
+  llm: Transcribe an audio file using the Whisper model.
+parameters:
+  - name: audio_file
+    type: file
+    required: true
+    label:
+      en_US: Audio File
+      zh_Hans: 音频文件
+    human_description:
+      en_US: "Upload an audio file to transcribe. Supports mp3, mp4, mpeg, mpga, m4a, wav, or webm formats."
+      zh_Hans: "上传要转录的音频文件。支持 mp3、mp4、mpeg、mpga、m4a、wav 或 webm 格式。"
+    llm_description: "Audio file to transcribe. Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, or webm."
+    form: llm
+  - name: task
+    type: select
+    required: true
+    label:
+      en_US: Task
+      zh_Hans: 任务
+    human_description:
+      en_US: "Choose whether to transcribe the audio in its original language or translate it to English"
+      zh_Hans: "选择是以原始语言转录音频还是将其翻译成英语"
+    llm_description: "Task to perform on the audio file. Either transcribe or translate. Default value: 'transcribe'. If 'translate' is selected as the task, the audio will be translated to English, regardless of the language selected."
+    form: form
+    default: transcribe
+    options:
+      - value: transcribe
+        label:
+          en_US: Transcribe
+          zh_Hans: 转录
+      - value: translate
+        label:
+          en_US: Translate
+          zh_Hans: 翻译
+  - name: language
+    type: select
+    required: true
+    label:
+      en_US: Language
+      zh_Hans: 语言
+    human_description:
+      en_US: "Select the primary language spoken in the audio file"
+      zh_Hans: "选择音频文件中使用的主要语言"
+    llm_description: "Language of the audio file."
+    form: form
+    default: en
+    options:
+      - value: af
+        label:
+          en_US: Afrikaans
+          zh_Hans: 南非语
+      - value: am
+        label:
+          en_US: Amharic
+          zh_Hans: 阿姆哈拉语
+      - value: ar
+        label:
+          en_US: Arabic
+          zh_Hans: 阿拉伯语
+      - value: as
+        label:
+          en_US: Assamese
+          zh_Hans: 阿萨姆语
+      - value: az
+        label:
+          en_US: Azerbaijani
+          zh_Hans: 阿塞拜疆语
+      - value: ba
+        label:
+          en_US: Bashkir
+          zh_Hans: 巴什基尔语
+      - value: be
+        label:
+          en_US: Belarusian
+          zh_Hans: 白俄罗斯语
+      - value: bg
+        label:
+          en_US: Bulgarian
+          zh_Hans: 保加利亚语
+      - value: bn
+        label:
+          en_US: Bengali
+          zh_Hans: 孟加拉语
+      - value: bo
+        label:
+          en_US: Tibetan
+          zh_Hans: 藏语
+      - value: br
+        label:
+          en_US: Breton
+          zh_Hans: 布列塔尼语
+      - value: bs
+        label:
+          en_US: Bosnian
+          zh_Hans: 波斯尼亚语
+      - value: ca
+        label:
+          en_US: Catalan
+          zh_Hans: 加泰罗尼亚语
+      - value: cs
+        label:
+          en_US: Czech
+          zh_Hans: 捷克语
+      - value: cy
+        label:
+          en_US: Welsh
+          zh_Hans: 威尔士语
+      - value: da
+        label:
+          en_US: Danish
+          zh_Hans: 丹麦语
+      - value: de
+        label:
+          en_US: German
+          zh_Hans: 德语
+      - value: el
+        label:
+          en_US: Greek
+          zh_Hans: 希腊语
+      - value: en
+        label:
+          en_US: English
+          zh_Hans: 英语
+      - value: es
+        label:
+          en_US: Spanish
+          zh_Hans: 西班牙语
+      - value: et
+        label:
+          en_US: Estonian
+          zh_Hans: 爱沙尼亚语
+      - value: eu
+        label:
+          en_US: Basque
+          zh_Hans: 巴斯克语
+      - value: fa
+        label:
+          en_US: Persian
+          zh_Hans: 波斯语
+      - value: fi
+        label:
+          en_US: Finnish
+          zh_Hans: 芬兰语
+      - value: fo
+        label:
+          en_US: Faroese
+          zh_Hans: 法罗语
+      - value: fr
+        label:
+          en_US: French
+          zh_Hans: 法语
+      - value: gl
+        label:
+          en_US: Galician
+          zh_Hans: 加利西亚语
+      - value: gu
+        label:
+          en_US: Gujarati
+          zh_Hans: 古吉拉特语
+      - value: ha
+        label:
+          en_US: Hausa
+          zh_Hans: 毫萨语
+      - value: haw
+        label:
+          en_US: Hawaiian
+          zh_Hans: 夏威夷语
+      - value: he
+        label:
+          en_US: Hebrew
+          zh_Hans: 希伯来语
+      - value: hi
+        label:
+          en_US: Hindi
+          zh_Hans: 印地语
+      - value: hr
+        label:
+          en_US: Croatian
+          zh_Hans: 克罗地亚语
+      - value: ht
+        label:
+          en_US: Haitian Creole
+          zh_Hans: 海地克里奥尔语
+      - value: hu
+        label:
+          en_US: Hungarian
+          zh_Hans: 匈牙利语
+      - value: hy
+        label:
+          en_US: Armenian
+          zh_Hans: 亚美尼亚语
+      - value: id
+        label:
+          en_US: Indonesian
+          zh_Hans: 印度尼西亚语
+      - value: is
+        label:
+          en_US: Icelandic
+          zh_Hans: 冰岛语
+      - value: it
+        label:
+          en_US: Italian
+          zh_Hans: 意大利语
+      - value: ja
+        label:
+          en_US: Japanese
+          zh_Hans: 日语
+      - value: jw
+        label:
+          en_US: Javanese
+          zh_Hans: 爪哇语
+      - value: ka
+        label:
+          en_US: Georgian
+          zh_Hans: 格鲁吉亚语
+      - value: kk
+        label:
+          en_US: Kazakh
+          zh_Hans: 哈萨克语
+      - value: km
+        label:
+          en_US: Khmer
+          zh_Hans: 高棉语
+      - value: kn
+        label:
+          en_US: Kannada
+          zh_Hans: 卡纳达语
+      - value: ko
+        label:
+          en_US: Korean
+          zh_Hans: 韩语
+      - value: la
+        label:
+          en_US: Latin
+          zh_Hans: 拉丁语
+      - value: lb
+        label:
+          en_US: Luxembourgish
+          zh_Hans: 卢森堡语
+      - value: ln
+        label:
+          en_US: Lingala
+          zh_Hans: 林加拉语
+      - value: lo
+        label:
+          en_US: Lao
+          zh_Hans: 老挝语
+      - value: lt
+        label:
+          en_US: Lithuanian
+          zh_Hans: 立陶宛语
+      - value: lv
+        label:
+          en_US: Latvian
+          zh_Hans: 拉脱维亚语
+      - value: mg
+        label:
+          en_US: Malagasy
+          zh_Hans: 马尔加什语
+      - value: mi
+        label:
+          en_US: Maori
+          zh_Hans: 毛利语
+      - value: mk
+        label:
+          en_US: Macedonian
+          zh_Hans: 马其顿语
+      - value: ml
+        label:
+          en_US: Malayalam
+          zh_Hans: 马拉雅拉姆语
+      - value: mn
+        label:
+          en_US: Mongolian
+          zh_Hans: 蒙古语
+      - value: mr
+        label:
+          en_US: Marathi
+          zh_Hans: 马拉地语
+      - value: ms
+        label:
+          en_US: Malay
+          zh_Hans: 马来语
+      - value: mt
+        label:
+          en_US: Maltese
+          zh_Hans: 马耳他语
+      - value: my
+        label:
+          en_US: Burmese
+          zh_Hans: 缅甸语
+      - value: ne
+        label:
+          en_US: Nepali
+          zh_Hans: 尼泊尔语
+      - value: nl
+        label:
+          en_US: Dutch
+          zh_Hans: 荷兰语
+      - value: nn
+        label:
+          en_US: Norwegian Nynorsk
+          zh_Hans: 新挪威语
+      - value: no
+        label:
+          en_US: Norwegian
+          zh_Hans: 挪威语
+      - value: oc
+        label:
+          en_US: Occitan
+          zh_Hans: 奥克语
+      - value: pa
+        label:
+          en_US: Punjabi
+          zh_Hans: 旁遮普语
+      - value: pl
+        label:
+          en_US: Polish
+          zh_Hans: 波兰语
+      - value: ps
+        label:
+          en_US: Pashto
+          zh_Hans: 普什图语
+      - value: pt
+        label:
+          en_US: Portuguese
+          zh_Hans: 葡萄牙语
+      - value: ro
+        label:
+          en_US: Romanian
+          zh_Hans: 罗马尼亚语
+      - value: ru
+        label:
+          en_US: Russian
+          zh_Hans: 俄语
+      - value: sa
+        label:
+          en_US: Sanskrit
+          zh_Hans: 梵语
+      - value: sd
+        label:
+          en_US: Sindhi
+          zh_Hans: 信德语
+      - value: si
+        label:
+          en_US: Sinhala
+          zh_Hans: 僧伽罗语
+      - value: sk
+        label:
+          en_US: Slovak
+          zh_Hans: 斯洛伐克语
+      - value: sl
+        label:
+          en_US: Slovenian
+          zh_Hans: 斯洛文尼亚语
+      - value: sn
+        label:
+          en_US: Shona
+          zh_Hans: 修纳语
+      - value: so
+        label:
+          en_US: Somali
+          zh_Hans: 索马里语
+      - value: sq
+        label:
+          en_US: Albanian
+          zh_Hans: 阿尔巴尼亚语
+      - value: sr
+        label:
+          en_US: Serbian
+          zh_Hans: 塞尔维亚语
+      - value: su
+        label:
+          en_US: Sundanese
+          zh_Hans: 巽他语
+      - value: sv
+        label:
+          en_US: Swedish
+          zh_Hans: 瑞典语
+      - value: sw
+        label:
+          en_US: Swahili
+          zh_Hans: 斯瓦希里语
+      - value: ta
+        label:
+          en_US: Tamil
+          zh_Hans: 泰米尔语
+      - value: te
+        label:
+          en_US: Telugu
+          zh_Hans: 泰卢固语
+      - value: tg
+        label:
+          en_US: Tajik
+          zh_Hans: 塔吉克语
+      - value: th
+        label:
+          en_US: Thai
+          zh_Hans: 泰语
+      - value: tk
+        label:
+          en_US: Turkmen
+          zh_Hans: 土库曼语
+      - value: tl
+        label:
+          en_US: Tagalog
+          zh_Hans: 他加禄语
+      - value: tr
+        label:
+          en_US: Turkish
+          zh_Hans: 土耳其语
+      - value: tt
+        label:
+          en_US: Tatar
+          zh_Hans: 鞑靼语
+      - value: uk
+        label:
+          en_US: Ukrainian
+          zh_Hans: 乌克兰语
+      - value: ur
+        label:
+          en_US: Urdu
+          zh_Hans: 乌尔都语
+      - value: uz
+        label:
+          en_US: Uzbek
+          zh_Hans: 乌兹别克语
+      - value: vi
+        label:
+          en_US: Vietnamese
+          zh_Hans: 越南语
+      - value: yi
+        label:
+          en_US: Yiddish
+          zh_Hans: 意第绪语
+      - value: yo
+        label:
+          en_US: Yoruba
+          zh_Hans: 约鲁巴语
+      - value: yue
+        label:
+          en_US: Cantonese
+          zh_Hans: 粤语
+      - value: zh
+        label:
+          en_US: Chinese
+          zh_Hans: 中文
+  - name: chunk_level
+    type: select
+    label:
+      en_US: Chunk Level
+      zh_Hans: 分块级别
+    human_description:
+      en_US: "Choose how the transcription should be divided into chunks"
+      zh_Hans: "选择如何将转录内容分成块"
+    llm_description: "Level of the chunks to return."
+    form: form
+    default: segment
+    options:
+      - value: segment
+        label:
+          en_US: Segment
+          zh_Hans: 段
+  - name: version
+    type: select
+    label:
+      en_US: Version
+      zh_Hans: 版本
+    human_description:
+      en_US: "Select which version of the Whisper large model to use"
+      zh_Hans: "选择要使用的 Whisper large 模型版本"
+    llm_description: "Version of the model to use. All of the models are the Whisper large variant."
+    form: form
+    default: "3"
+    options:
+      - value: "3"
+        label:
+          en_US: Version 3
+          zh_Hans: 版本 3

api/core/tools/provider/builtin/podcast_generator/podcast_generator.py

@@ -1,6 +1,7 @@
 from typing import Any
 
 import openai
+from yarl import URL
 
 from core.tools.errors import ToolProviderCredentialValidationError
 from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
@@ -10,6 +11,7 @@ class PodcastGeneratorProvider(BuiltinToolProviderController):
     def _validate_credentials(self, credentials: dict[str, Any]) -> None:
         tts_service = credentials.get("tts_service")
         api_key = credentials.get("api_key")
+        base_url = credentials.get("openai_base_url")
 
         if not tts_service:
             raise ToolProviderCredentialValidationError("TTS service is not specified")
@@ -17,13 +19,16 @@ class PodcastGeneratorProvider(BuiltinToolProviderController):
         if not api_key:
             raise ToolProviderCredentialValidationError("API key is missing")
 
+        if base_url:
+            base_url = str(URL(base_url) / "v1")
+
         if tts_service == "openai":
-            self._validate_openai_credentials(api_key)
+            self._validate_openai_credentials(api_key, base_url)
         else:
             raise ToolProviderCredentialValidationError(f"Unsupported TTS service: {tts_service}")
 
-    def _validate_openai_credentials(self, api_key: str) -> None:
-        client = openai.OpenAI(api_key=api_key)
+    def _validate_openai_credentials(self, api_key: str, base_url: str | None) -> None:
+        client = openai.OpenAI(api_key=api_key, base_url=base_url)
         try:
             # We're using a simple API call to validate the credentials
             client.models.list()

api/core/variables/__init__.py

@@ -17,6 +17,7 @@ from .segments import (
 from .types import SegmentType
 from .variables import (
     ArrayAnyVariable,
+    ArrayFileVariable,
     ArrayNumberVariable,
     ArrayObjectVariable,
     ArrayStringVariable,
@@ -58,4 +59,5 @@ __all__ = [
     "ArrayStringSegment",
     "FileSegment",
     "FileVariable",
+    "ArrayFileVariable",
 ]

api/core/variables/variables.py

@@ -1,9 +1,13 @@
+from collections.abc import Sequence
+from uuid import uuid4
+
 from pydantic import Field
 
 from core.helper import encrypter
 
 from .segments import (
     ArrayAnySegment,
+    ArrayFileSegment,
     ArrayNumberSegment,
     ArrayObjectSegment,
     ArrayStringSegment,
@@ -24,11 +28,12 @@ class Variable(Segment):
     """
 
     id: str = Field(
-        default="",
-        description="Unique identity for variable. It's only used by environment variables now.",
+        default=lambda _: str(uuid4()),
+        description="Unique identity for variable.",
     )
     name: str
     description: str = Field(default="", description="Description of the variable.")
+    selector: Sequence[str] = Field(default_factory=list)
 
 
 class StringVariable(StringSegment, Variable):
@@ -78,3 +83,7 @@ class NoneVariable(NoneSegment, Variable):
 
 class FileVariable(FileSegment, Variable):
     pass
+
+
+class ArrayFileVariable(ArrayFileSegment, Variable):
+    pass

api/core/workflow/entities/variable_pool.py

@@ -95,13 +95,16 @@ class VariablePool(BaseModel):
         if len(selector) < 2:
             raise ValueError("Invalid selector")
 
+        if isinstance(value, Variable):
+            variable = value
         if isinstance(value, Segment):
-            v = value
+            variable = variable_factory.segment_to_variable(segment=value, selector=selector)
         else:
-            v = variable_factory.build_segment(value)
+            segment = variable_factory.build_segment(value)
+            variable = variable_factory.segment_to_variable(segment=segment, selector=selector)
 
         hash_key = hash(tuple(selector[1:]))
-        self.variable_dictionary[selector[0]][hash_key] = v
+        self.variable_dictionary[selector[0]][hash_key] = variable
 
     def get(self, selector: Sequence[str], /) -> Segment | None:
         """

api/core/workflow/nodes/document_extractor/node.py

@@ -143,14 +143,14 @@ def _extract_text_by_file_extension(*, file_content: bytes, file_extension: str)
 
 def _extract_text_from_plain_text(file_content: bytes) -> str:
     try:
-        return file_content.decode("utf-8")
+        return file_content.decode("utf-8", "ignore")
     except UnicodeDecodeError as e:
         raise TextExtractionError("Failed to decode plain text file") from e
 
 
 def _extract_text_from_json(file_content: bytes) -> str:
     try:
-        json_data = json.loads(file_content.decode("utf-8"))
+        json_data = json.loads(file_content.decode("utf-8", "ignore"))
         return json.dumps(json_data, indent=2, ensure_ascii=False)
     except (UnicodeDecodeError, json.JSONDecodeError) as e:
         raise TextExtractionError(f"Failed to decode or parse JSON file: {e}") from e
@@ -159,7 +159,7 @@ def _extract_text_from_json(file_content: bytes) -> str:
 def _extract_text_from_yaml(file_content: bytes) -> str:
     """Extract the content from yaml file"""
     try:
-        yaml_data = yaml.safe_load_all(file_content.decode("utf-8"))
+        yaml_data = yaml.safe_load_all(file_content.decode("utf-8", "ignore"))
         return yaml.dump_all(yaml_data, allow_unicode=True, sort_keys=False)
     except (UnicodeDecodeError, yaml.YAMLError) as e:
         raise TextExtractionError(f"Failed to decode or parse YAML file: {e}") from e
@@ -217,7 +217,7 @@ def _extract_text_from_file(file: File):
 
 def _extract_text_from_csv(file_content: bytes) -> str:
     try:
-        csv_file = io.StringIO(file_content.decode("utf-8"))
+        csv_file = io.StringIO(file_content.decode("utf-8", "ignore"))
         csv_reader = csv.reader(csv_file)
         rows = list(csv_reader)
 

api/core/workflow/nodes/tool/tool_node.py

@@ -1,5 +1,4 @@
 from collections.abc import Mapping, Sequence
-from os import path
 from typing import Any
 
 from sqlalchemy import select
@@ -180,7 +179,6 @@ class ToolNode(BaseNode[ToolNodeData]):
         for response in tool_response:
             if response.type in {ToolInvokeMessage.MessageType.IMAGE_LINK, ToolInvokeMessage.MessageType.IMAGE}:
                 url = str(response.message) if response.message else None
-                ext = path.splitext(url)[1] if url else ".bin"
                 tool_file_id = str(url).split("/")[-1].split(".")[0]
                 transfer_method = response.meta.get("transfer_method", FileTransferMethod.TOOL_FILE)
 
@@ -202,7 +200,6 @@ class ToolNode(BaseNode[ToolNodeData]):
                 )
                 result.append(file)
             elif response.type == ToolInvokeMessage.MessageType.BLOB:
-                # get tool file id
                 tool_file_id = str(response.message).split("/")[-1].split(".")[0]
                 with Session(db.engine) as session:
                     stmt = select(ToolFile).where(ToolFile.id == tool_file_id)
@@ -211,7 +208,6 @@ class ToolNode(BaseNode[ToolNodeData]):
                         raise ValueError(f"tool file {tool_file_id} not exists")
                 mapping = {
                     "tool_file_id": tool_file_id,
-                    "type": FileType.IMAGE,
                     "transfer_method": FileTransferMethod.TOOL_FILE,
                 }
                 file = file_factory.build_from_mapping(
@@ -228,13 +224,8 @@ class ToolNode(BaseNode[ToolNodeData]):
                     tool_file = session.scalar(stmt)
                     if tool_file is None:
                         raise ToolFileError(f"Tool file {tool_file_id} does not exist")
-                if "." in url:
-                    extension = "." + url.split("/")[-1].split(".")[1]
-                else:
-                    extension = ".bin"
                 mapping = {
                     "tool_file_id": tool_file_id,
-                    "type": FileType.IMAGE,
                     "transfer_method": transfer_method,
                     "url": url,
                 }

api/docker/entrypoint.sh

@@ -21,7 +21,7 @@ if [[ "${MODE}" == "worker" ]]; then
   fi
 
   exec celery -A app.celery worker -P ${CELERY_WORKER_CLASS:-gevent} $CONCURRENCY_OPTION --loglevel ${LOG_LEVEL} \
-    -Q ${CELERY_QUEUES:-dataset,generation,mail,ops_trace,app_deletion}
+    -Q ${CELERY_QUEUES:-dataset,mail,ops_trace,app_deletion}
 
 elif [[ "${MODE}" == "beat" ]]; then
   exec celery -A app.celery beat --loglevel ${LOG_LEVEL}

api/factories/file_factory.py

@@ -180,6 +180,20 @@ def _get_remote_file_info(url: str):
     return mime_type, filename, file_size
 
 
+def _get_file_type_by_mimetype(mime_type: str) -> FileType:
+    if "image" in mime_type:
+        file_type = FileType.IMAGE
+    elif "video" in mime_type:
+        file_type = FileType.VIDEO
+    elif "audio" in mime_type:
+        file_type = FileType.AUDIO
+    elif "text" in mime_type or "pdf" in mime_type:
+        file_type = FileType.DOCUMENT
+    else:
+        file_type = FileType.CUSTOM
+    return file_type
+
+
 def _build_from_tool_file(
     *,
     mapping: Mapping[str, Any],
@@ -199,12 +213,13 @@ def _build_from_tool_file(
         raise ValueError(f"ToolFile {mapping.get('tool_file_id')} not found")
 
     extension = "." + tool_file.file_key.split(".")[-1] if "." in tool_file.file_key else ".bin"
+    file_type = mapping.get("type", _get_file_type_by_mimetype(tool_file.mimetype))
 
     return File(
         id=mapping.get("id"),
         tenant_id=tenant_id,
         filename=tool_file.name,
-        type=FileType.value_of(mapping.get("type")),
+        type=file_type,
         transfer_method=transfer_method,
         remote_url=tool_file.original_url,
         related_id=tool_file.id,

api/factories/variable_factory.py

@@ -1,34 +1,65 @@
-from collections.abc import Mapping
+from collections.abc import Mapping, Sequence
 from typing import Any
+from uuid import uuid4
 
 from configs import dify_config
 from core.file import File
-from core.variables import (
+from core.variables.exc import VariableError
+from core.variables.segments import (
     ArrayAnySegment,
     ArrayFileSegment,
     ArrayNumberSegment,
-    ArrayNumberVariable,
     ArrayObjectSegment,
-    ArrayObjectVariable,
     ArraySegment,
     ArrayStringSegment,
-    ArrayStringVariable,
     FileSegment,
     FloatSegment,
-    FloatVariable,
     IntegerSegment,
-    IntegerVariable,
     NoneSegment,
     ObjectSegment,
+    Segment,
+    StringSegment,
+)
+from core.variables.types import SegmentType
+from core.variables.variables import (
+    ArrayAnyVariable,
+    ArrayFileVariable,
+    ArrayNumberVariable,
+    ArrayObjectVariable,
+    ArrayStringVariable,
+    FileVariable,
+    FloatVariable,
+    IntegerVariable,
+    NoneVariable,
     ObjectVariable,
     SecretVariable,
-    Segment,
-    SegmentType,
-    StringSegment,
     StringVariable,
     Variable,
 )
-from core.variables.exc import VariableError
+
+
+class InvalidSelectorError(ValueError):
+    pass
+
+
+class UnsupportedSegmentTypeError(Exception):
+    pass
+
+
+# Define the constant
+SEGMENT_TO_VARIABLE_MAP = {
+    StringSegment: StringVariable,
+    IntegerSegment: IntegerVariable,
+    FloatSegment: FloatVariable,
+    ObjectSegment: ObjectVariable,
+    FileSegment: FileVariable,
+    ArrayStringSegment: ArrayStringVariable,
+    ArrayNumberSegment: ArrayNumberVariable,
+    ArrayObjectSegment: ArrayObjectVariable,
+    ArrayFileSegment: ArrayFileVariable,
+    ArrayAnySegment: ArrayAnyVariable,
+    NoneSegment: NoneVariable,
+}
 
 
 def build_variable_from_mapping(mapping: Mapping[str, Any], /) -> Variable:
@@ -96,3 +127,30 @@ def build_segment(value: Any, /) -> Segment:
             case _:
                 raise ValueError(f"not supported value {value}")
     raise ValueError(f"not supported value {value}")
+
+
+def segment_to_variable(
+    *,
+    segment: Segment,
+    selector: Sequence[str],
+    id: str | None = None,
+    name: str | None = None,
+    description: str = "",
+) -> Variable:
+    if isinstance(segment, Variable):
+        return segment
+    name = name or selector[-1]
+    id = id or str(uuid4())
+
+    segment_type = type(segment)
+    if segment_type not in SEGMENT_TO_VARIABLE_MAP:
+        raise UnsupportedSegmentTypeError(f"not supported segment type {segment_type}")
+
+    variable_class = SEGMENT_TO_VARIABLE_MAP[segment_type]
+    return variable_class(
+        id=id,
+        name=name,
+        description=description,
+        value=segment.value,
+        selector=selector,
+    )

api/poetry.lock

@@ -1,4 +1,4 @@
-# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand.
+# This file is automatically @generated by Poetry 1.8.4 and should not be changed by hand.
 
 [[package]]
 name = "aiohappyeyeballs"
@@ -2389,6 +2389,26 @@ files = [
 [package.extras]
 tests = ["pytest"]
 
+[[package]]
+name = "fal-client"
+version = "0.5.6"
+description = "Python client for fal.ai"
+optional = false
+python-versions = ">=3.8"
+files = [
+    {file = "fal_client-0.5.6-py3-none-any.whl", hash = "sha256:631fd857a3c44753ee46a2eea1e7276471453aca58faac9c3702f744c7c84050"},
+    {file = "fal_client-0.5.6.tar.gz", hash = "sha256:d3afc4b6250023d0ee8437ec504558231d3b106d7aabc12cda8c39883faddecb"},
+]
+
+[package.dependencies]
+httpx = ">=0.21.0,<1"
+httpx-sse = ">=0.4.0,<0.5"
+
+[package.extras]
+dev = ["fal-client[docs,test]"]
+docs = ["sphinx", "sphinx-autodoc-typehints", "sphinx-rtd-theme"]
+test = ["pillow", "pytest", "pytest-asyncio"]
+
 [[package]]
 name = "fastapi"
 version = "0.115.4"
@@ -4018,6 +4038,17 @@ http2 = ["h2 (>=3,<5)"]
 socks = ["socksio (==1.*)"]
 zstd = ["zstandard (>=0.18.0)"]
 
+[[package]]
+name = "httpx-sse"
+version = "0.4.0"
+description = "Consume Server-Sent Event (SSE) messages with HTTPX."
+optional = false
+python-versions = ">=3.8"
+files = [
+    {file = "httpx-sse-0.4.0.tar.gz", hash = "sha256:1e81a3a3070ce322add1d3529ed42eb5f70817f45ed6ec915ab753f961139721"},
+    {file = "httpx_sse-0.4.0-py3-none-any.whl", hash = "sha256:f329af6eae57eaa2bdfd962b42524764af68075ea87370a2de920af5341e318f"},
+]
+
 [[package]]
 name = "huggingface-hub"
 version = "0.16.4"
@@ -8423,29 +8454,29 @@ pyasn1 = ">=0.1.3"
 
 [[package]]
 name = "ruff"
-version = "0.6.9"
+version = "0.7.3"
 description = "An extremely fast Python linter and code formatter, written in Rust."
 optional = false
 python-versions = ">=3.7"
 files = [
-    {file = "ruff-0.6.9-py3-none-linux_armv6l.whl", hash = "sha256:064df58d84ccc0ac0fcd63bc3090b251d90e2a372558c0f057c3f75ed73e1ccd"},
-    {file = "ruff-0.6.9-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:140d4b5c9f5fc7a7b074908a78ab8d384dd7f6510402267bc76c37195c02a7ec"},
-    {file = "ruff-0.6.9-py3-none-macosx_11_0_arm64.whl", hash = "sha256:53fd8ca5e82bdee8da7f506d7b03a261f24cd43d090ea9db9a1dc59d9313914c"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:645d7d8761f915e48a00d4ecc3686969761df69fb561dd914a773c1a8266e14e"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:eae02b700763e3847595b9d2891488989cac00214da7f845f4bcf2989007d577"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7d5ccc9e58112441de8ad4b29dcb7a86dc25c5f770e3c06a9d57e0e5eba48829"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:417b81aa1c9b60b2f8edc463c58363075412866ae4e2b9ab0f690dc1e87ac1b5"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3c866b631f5fbce896a74a6e4383407ba7507b815ccc52bcedabb6810fdb3ef7"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7b118afbb3202f5911486ad52da86d1d52305b59e7ef2031cea3425142b97d6f"},
-    {file = "ruff-0.6.9-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a67267654edc23c97335586774790cde402fb6bbdb3c2314f1fc087dee320bfa"},
-    {file = "ruff-0.6.9-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:3ef0cc774b00fec123f635ce5c547dac263f6ee9fb9cc83437c5904183b55ceb"},
-    {file = "ruff-0.6.9-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:12edd2af0c60fa61ff31cefb90aef4288ac4d372b4962c2864aeea3a1a2460c0"},
-    {file = "ruff-0.6.9-py3-none-musllinux_1_2_i686.whl", hash = "sha256:55bb01caeaf3a60b2b2bba07308a02fca6ab56233302406ed5245180a05c5625"},
-    {file = "ruff-0.6.9-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:925d26471fa24b0ce5a6cdfab1bb526fb4159952385f386bdcc643813d472039"},
-    {file = "ruff-0.6.9-py3-none-win32.whl", hash = "sha256:eb61ec9bdb2506cffd492e05ac40e5bc6284873aceb605503d8494180d6fc84d"},
-    {file = "ruff-0.6.9-py3-none-win_amd64.whl", hash = "sha256:785d31851c1ae91f45b3d8fe23b8ae4b5170089021fbb42402d811135f0b7117"},
-    {file = "ruff-0.6.9-py3-none-win_arm64.whl", hash = "sha256:a9641e31476d601f83cd602608739a0840e348bda93fec9f1ee816f8b6798b93"},
-    {file = "ruff-0.6.9.tar.gz", hash = "sha256:b076ef717a8e5bc819514ee1d602bbdca5b4420ae13a9cf61a0c0a4f53a2baa2"},
+    {file = "ruff-0.7.3-py3-none-linux_armv6l.whl", hash = "sha256:34f2339dc22687ec7e7002792d1f50712bf84a13d5152e75712ac08be565d344"},
+    {file = "ruff-0.7.3-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:fb397332a1879b9764a3455a0bb1087bda876c2db8aca3a3cbb67b3dbce8cda0"},
+    {file = "ruff-0.7.3-py3-none-macosx_11_0_arm64.whl", hash = "sha256:37d0b619546103274e7f62643d14e1adcbccb242efda4e4bdb9544d7764782e9"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5d59f0c3ee4d1a6787614e7135b72e21024875266101142a09a61439cb6e38a5"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:44eb93c2499a169d49fafd07bc62ac89b1bc800b197e50ff4633aed212569299"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6d0242ce53f3a576c35ee32d907475a8d569944c0407f91d207c8af5be5dae4e"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:6b6224af8b5e09772c2ecb8dc9f3f344c1aa48201c7f07e7315367f6dd90ac29"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c50f95a82b94421c964fae4c27c0242890a20fe67d203d127e84fbb8013855f5"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7f3eff9961b5d2644bcf1616c606e93baa2d6b349e8aa8b035f654df252c8c67"},
+    {file = "ruff-0.7.3-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b8963cab06d130c4df2fd52c84e9f10d297826d2e8169ae0c798b6221be1d1d2"},
+    {file = "ruff-0.7.3-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:61b46049d6edc0e4317fb14b33bd693245281a3007288b68a3f5b74a22a0746d"},
+    {file = "ruff-0.7.3-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:10ebce7696afe4644e8c1a23b3cf8c0f2193a310c18387c06e583ae9ef284de2"},
+    {file = "ruff-0.7.3-py3-none-musllinux_1_2_i686.whl", hash = "sha256:3f36d56326b3aef8eeee150b700e519880d1aab92f471eefdef656fd57492aa2"},
+    {file = "ruff-0.7.3-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:5d024301109a0007b78d57ab0ba190087b43dce852e552734ebf0b0b85e4fb16"},
+    {file = "ruff-0.7.3-py3-none-win32.whl", hash = "sha256:4ba81a5f0c5478aa61674c5a2194de8b02652f17addf8dfc40c8937e6e7d79fc"},
+    {file = "ruff-0.7.3-py3-none-win_amd64.whl", hash = "sha256:588a9ff2fecf01025ed065fe28809cd5a53b43505f48b69a1ac7707b1b7e4088"},
+    {file = "ruff-0.7.3-py3-none-win_arm64.whl", hash = "sha256:1713e2c5545863cdbfe2cbce21f69ffaf37b813bfd1fb3b90dc9a6f1963f5a8c"},
+    {file = "ruff-0.7.3.tar.gz", hash = "sha256:e1d1ba2e40b6e71a61b063354d04be669ab0d39c352461f3d789cac68b54a313"},
 ]
 
 [[package]]
@@ -10929,4 +10960,4 @@ cffi = ["cffi (>=1.11)"]
 [metadata]
 lock-version = "2.0"
 python-versions = ">=3.11,<3.13"
-content-hash = "96e2529374144624230bceff43f037d6988d6b0de99bbbcc7ee4b2085eb493fa"
+content-hash = "6b2d4d4e81f7061dced8d585eb6f08422452644368afca0addc31ac99bfd349a"

api/pyproject.toml

@@ -122,6 +122,7 @@ celery = "~5.4.0"
 chardet = "~5.1.0"
 cohere = "~5.2.4"
 dashscope = { version = "~1.17.0", extras = ["tokenizer"] }
+fal-client = "0.5.6"
 flask = "~3.0.1"
 flask-compress = "~1.14"
 flask-cors = "~4.0.0"
@@ -278,4 +279,4 @@ pytest-mock = "~3.14.0"
 optional = true
 [tool.poetry.group.lint.dependencies]
 dotenv-linter = "~0.5.0"
-ruff = "~0.6.9"
+ruff = "~0.7.3"

api/services/dataset_service.py

@@ -1458,6 +1458,7 @@ class SegmentService:
             pre_segment_data_list = []
             segment_data_list = []
             keywords_list = []
+            position = max_position + 1 if max_position else 1
             for segment_item in segments:
                 content = segment_item["content"]
                 doc_id = str(uuid.uuid4())
@@ -1475,7 +1476,7 @@ class SegmentService:
                     document_id=document.id,
                     index_node_id=doc_id,
                     index_node_hash=segment_hash,
-                    position=max_position + 1 if max_position else 1,
+                    position=position,
                     content=content,
                     word_count=len(content),
                     tokens=tokens,
@@ -1490,6 +1491,7 @@ class SegmentService:
                 increment_word_count += segment_document.word_count
                 db.session.add(segment_document)
                 segment_data_list.append(segment_document)
+                position += 1
 
                 pre_segment_data_list.append(segment_document)
                 if "keywords" in segment_item:

api/tasks/document_indexing_task.py

@@ -25,7 +25,9 @@ def document_indexing_task(dataset_id: str, document_ids: list):
     start_at = time.perf_counter()
 
     dataset = db.session.query(Dataset).filter(Dataset.id == dataset_id).first()
-
+    if not dataset:
+        logging.info(click.style("Dataset is not found: {}".format(dataset_id), fg="yellow"))
+        return
     # check document limit
     features = FeatureService.get_features(dataset.tenant_id)
     try:

api/tests/integration_tests/vdb/__mock/baiduvectordb.py

@@ -1,4 +1,5 @@
 import os
+from collections import UserDict
 from unittest.mock import MagicMock
 
 import pytest
@@ -11,7 +12,7 @@ from pymochow.model.table import Table
 from requests.adapters import HTTPAdapter
 
 
-class AttrDict(dict):
+class AttrDict(UserDict):
     def __getattr__(self, item):
         return self.get(item)
 

api/tests/integration_tests/vdb/__mock/upstashvectordb.py

@@ -1,4 +1,5 @@
 import os
+from collections import UserDict
 from typing import Optional
 
 import pytest
@@ -50,7 +51,7 @@ class MockIndex:
         return AttrDict({"dimension": 1024})
 
 
-class AttrDict(dict):
+class AttrDict(UserDict):
     def __getattr__(self, item):
         return self.get(item)
 

api/tests/unit_tests/core/app/segments/test_segment.py

@@ -1,5 +1,5 @@
 from core.helper import encrypter
-from core.variables import SecretVariable, StringSegment
+from core.variables import SecretVariable, StringVariable
 from core.workflow.entities.variable_pool import VariablePool
 from core.workflow.enums import SystemVariableKey
 
@@ -54,4 +54,5 @@ def test_convert_variable_to_segment_group():
     segments_group = variable_pool.convert_template(template)
     assert segments_group.text == "fake-user-id"
     assert segments_group.log == "fake-user-id"
-    assert segments_group.value == [StringSegment(value="fake-user-id")]
+    assert isinstance(segments_group.value[0], StringVariable)
+    assert segments_group.value[0].value == "fake-user-id"

api/tests/unit_tests/core/workflow/nodes/test_document_extractor_node.py

@@ -140,6 +140,17 @@ def test_extract_text_from_plain_text():
     assert text == "Hello, world!"
 
 
+def test_extract_text_from_plain_text_non_utf8():
+    import tempfile
+
+    non_utf8_content = b"Hello, world\xa9."  # \xA9 represents © in Latin-1
+    with tempfile.NamedTemporaryFile(delete=True) as temp_file:
+        temp_file.write(non_utf8_content)
+        temp_file.seek(0)
+        text = _extract_text_from_plain_text(temp_file.read())
+    assert text == "Hello, world."
+
+
 @patch("pypdfium2.PdfDocument")
 def test_extract_text_from_pdf(mock_pdf_document):
     mock_page = Mock()

api/tests/unit_tests/oss/__mock/volcengine_tos.py

@@ -1,4 +1,5 @@
 import os
+from collections import UserDict
 from unittest.mock import MagicMock
 
 import pytest
@@ -14,7 +15,7 @@ from tests.unit_tests.oss.__mock.base import (
 )
 
 
-class AttrDict(dict):
+class AttrDict(UserDict):
     def __getattr__(self, item):
         return self.get(item)
 

docker/.env.example

@@ -689,6 +689,9 @@ TEMPLATE_TRANSFORM_MAX_LENGTH=80000
 CODE_MAX_STRING_ARRAY_LENGTH=30
 CODE_MAX_OBJECT_ARRAY_LENGTH=30
 CODE_MAX_NUMBER_ARRAY_LENGTH=1000
+CODE_EXECUTION_CONNECT_TIMEOUT=10
+CODE_EXECUTION_READ_TIMEOUT=60
+CODE_EXECUTION_WRITE_TIMEOUT=10
 
 # Workflow runtime configuration
 WORKFLOW_MAX_EXECUTION_STEPS=500

docker/docker-compose.yaml

@@ -244,6 +244,9 @@ x-shared-env: &shared-api-worker-env
   RESET_PASSWORD_TOKEN_EXPIRY_MINUTES: ${RESET_PASSWORD_TOKEN_EXPIRY_MINUTES:-5}
   CODE_EXECUTION_ENDPOINT: ${CODE_EXECUTION_ENDPOINT:-http://sandbox:8194}
   CODE_EXECUTION_API_KEY: ${SANDBOX_API_KEY:-dify-sandbox}
+  CODE_EXECUTION_CONNECT_TIMEOUT: ${CODE_EXECUTION_CONNECT_TIMEOUT:-10}
+  CODE_EXECUTION_READ_TIMEOUT: ${CODE_EXECUTION_READ_TIMEOUT:-60}
+  CODE_EXECUTION_WRITE_TIMEOUT: ${CODE_EXECUTION_WRITE_TIMEOUT:-10}
   CODE_MAX_NUMBER: ${CODE_MAX_NUMBER:-9223372036854775807}
   CODE_MIN_NUMBER: ${CODE_MIN_NUMBER:--9223372036854775808}
   CODE_MAX_DEPTH: ${CODE_MAX_DEPTH:-5}

web/Dockerfile

@@ -1,5 +1,5 @@
 # base image
-FROM node:20.11-alpine3.19 AS base
+FROM node:20-alpine3.20 AS base
 LABEL maintainer="takatost@gmail.com"
 
 # if you located in China, you can use aliyun mirror to speed up

web/app/components/base/prompt-editor/constants.tsx

@@ -53,6 +53,6 @@ export const getInputVars = (text: string): ValueSelector[] => {
 export const FILE_EXTS: Record<string, string[]> = {
   [SupportUploadFileTypes.image]: ['JPG', 'JPEG', 'PNG', 'GIF', 'WEBP', 'SVG'],
   [SupportUploadFileTypes.document]: ['TXT', 'MD', 'MARKDOWN', 'PDF', 'HTML', 'XLSX', 'XLS', 'DOCX', 'CSV', 'EML', 'MSG', 'PPTX', 'PPT', 'XML', 'EPUB'],
-  [SupportUploadFileTypes.audio]: ['MP3', 'M4A', 'WAV', 'WEBM', 'AMR'],
+  [SupportUploadFileTypes.audio]: ['MP3', 'M4A', 'WAV', 'WEBM', 'AMR', 'MPGA'],
   [SupportUploadFileTypes.video]: ['MP4', 'MOV', 'MPEG', 'MPGA'],
 }

web/app/components/develop/template/template_advanced_chat.en.mdx

@@ -774,6 +774,10 @@ Chat applications support session persistence, allowing previous chat history to
 <Row>
   <Col>
     ### Request Body
+    Rename the session, the session name is used for display on clients that support multiple sessions.
+
+    ### Path
+    - `conversation_id` (string) Conversation ID
 
     <Properties>
       <Property name='name' type='string' key='name'>
@@ -796,10 +800,10 @@ Chat applications support session persistence, allowing previous chat history to
   </Col>
   <Col sticky>
 
-    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
+    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
 
     ```bash {{ title: 'cURL' }}
-    curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \
+    curl -X POST '${props.appDetail.api_base_url}/conversations/{conversation_id}/name' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer {api_key}' \
     --data-raw '{

web/app/components/develop/template/template_advanced_chat.zh.mdx

@@ -810,6 +810,9 @@ import { Row, Col, Properties, Property, Heading, SubProperty } from '../md.tsx'
   <Col>
     对会话进行重命名，会话名称用于显示在支持多会话的客户端上。
 
+    ### Path
+    - `conversation_id` (string) 会话 ID
+
     ### Request Body
 
     <Properties>
@@ -833,10 +836,10 @@ import { Row, Col, Properties, Property, Heading, SubProperty } from '../md.tsx'
   </Col>
   <Col sticky>
 
-    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
+    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
 
     ```bash {{ title: 'cURL' }}
-    curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \
+    curl -X POST '${props.appDetail.api_base_url}/conversations/{conversation_id}/name' \
     --header 'Authorization: Bearer {api_key}' \
     --header 'Content-Type: application/json' \
     --data-raw '{

web/app/components/develop/template/template_chat.en.mdx

@@ -808,6 +808,10 @@ Chat applications support session persistence, allowing previous chat history to
 <Row>
   <Col>
     ### Request Body
+    Rename the session, the session name is used for display on clients that support multiple sessions.
+
+    ### Path
+    - `conversation_id` (string) Conversation ID
 
     <Properties>
       <Property name='name' type='string' key='name'>
@@ -830,10 +834,10 @@ Chat applications support session persistence, allowing previous chat history to
   </Col>
   <Col sticky>
 
-    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
+    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
 
     ```bash {{ title: 'cURL' }}
-    curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \
+    curl -X POST '${props.appDetail.api_base_url}/conversations/{conversation_id}/name' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer {api_key}' \
     --data-raw '{

web/app/components/develop/template/template_chat.zh.mdx

@@ -824,6 +824,9 @@ import { Row, Col, Properties, Property, Heading, SubProperty } from '../md.tsx'
   <Col>
     对会话进行重命名，会话名称用于显示在支持多会话的客户端上。
 
+    ### Path
+    - `conversation_id` (string) 会话 ID
+
     ### Request Body
 
     <Properties>
@@ -847,10 +850,10 @@ import { Row, Col, Properties, Property, Heading, SubProperty } from '../md.tsx'
   </Col>
   <Col sticky>
 
-    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
+    <CodeGroup title="Request" tag="POST" label="/conversations/:conversation_id/name" targetCode={`curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{ \n "name": "", \n "user": "abc-123"\n}'`}>
 
     ```bash {{ title: 'cURL' }}
-    curl -X POST '${props.appDetail.api_base_url}/conversations/:conversation_id/name' \
+    curl -X POST '${props.appDetail.api_base_url}/conversations/{conversation_id}/name' \
     --header 'Authorization: Bearer {api_key}' \
     --header 'Content-Type: application/json' \
     --data-raw '{

web/i18n/pt-BR/app-log.ts

@@ -86,8 +86,8 @@ const translation = {
   agenteLogDetail: {
     agentMode: 'Modo Agente',
     toolUsed: 'Ferramenta usada',
-    iterações: 'Iterações',
-    iteração: 'Iteração',
+    iterations: 'Iterações',
+    iteration: 'Iteração',
     finalProcessing: 'Processamento Final',
   },
   agentLogDetail: {