Skip to content

Modify prompts of entry and validator#152

Merged
lfnothias merged 1 commit intodevfrom
dev_madina
Apr 2, 2025
Merged

Modify prompts of entry and validator#152
lfnothias merged 1 commit intodevfrom
dev_madina

Conversation

@madina1203
Copy link
Collaborator

@madina1203 madina1203 commented Apr 2, 2025

PR Type

enhancement


Description

  • Enhanced entry agent prompt to clarify role and task.

  • Improved validator prompt for better question validation.

  • Added guidelines for handling unclear or complex questions.

  • Corrected minor typographical errors in prompts.


Changes walkthrough 📝

Relevant files
Enhancement
prompt.py
Refine entry agent prompt for task clarity                             

app/core/agents/entry/prompt.py

  • Clarified entry agent's role in handling queries.
  • Added instructions for classifying and forwarding queries.
  • Corrected typographical error in the prompt.
  • Emphasized non-answering role of the entry agent.
  • +3/-2     
    prompt.py
    Improve validator prompt for robust question validation   

    app/core/agents/validator/prompt.py

  • Enhanced criteria for question validity.
  • Added flexibility for phrasing in validation.
  • Provided guidelines for handling unclear questions.
  • Updated feedback mechanism for invalid questions.
  • +5/-3     

    Need help?
  • Type /help how to ... in the comments thread for any questions about PR-Agent usage.
  • Check out the documentation for more information.
  • @qodo-code-review
    Copy link

    qodo-code-review bot commented Apr 2, 2025

    PR Reviewer Guide 🔍

    (Review updated until commit 9e493a0)

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Instruction Clarity

    Confirm that the newly added instruction about not answering questions directly integrates seamlessly with the existing responsibilities and does not lead to any ambiguity in agent behavior.

    Do not answer the question yourself. Your role is to classify and pass along the request, not to generate final answers. Other team members will handle other queries.
    Validation Consistency

    Review the added validation criteria and flexible phrasing to ensure they are consistent with current schema checks and do not conflict with existing logic.

    -The question is valid if it references any nodes, classes, or properties in the ENPKGeven if indirectly or across multiple relationships.
    -Allow flexibility for phrasing. As long as the topic aligns with schema elements, the question should be considered valid.
    Guidelines for Validity:
    Plant-Specific and Feature-Related Queries: Recognize questions as valid if they involve plant names, metabolites, or features, as long as properties (e.g., has_LCMS, submitted_taxon, has_lab_process, or has_bioassay_results, etc) are mentioned or implied. For instance, questions referencing specific plants, compounds, or taxonomic details should be valid if these attributes can be queried in ENPKG.
    Grouping, Counting, and Comparing Annotations: Allow questions that require counting, grouping, or comparing annotations  if the question mentions properties available in the ENPKG schema (e.g., has_wd_id, has_InChIkey2D, or has_canopus_annotation). These types of questions are valid if they involve any schema properties or identifiers.
    Valid Question Criteria:
    -The question is related to one or more of the ENPKG nodes/entities listed below. As long as the entities in the question exist in the schema, even if the question is complex or spans multiple relationships, it is still valid.
    
    Not Valid Question Criteria:
    -It is completely unrelated to any nodes/entities in ENPKG or the topic cannot be mapped to any known classes or properties in the schema.
    
    If a question is unclear or possibly valid but lacks detail, do not mark it invalidask the user for clarification instead.

    Google DocString Format
    While no new functions or classes were added, ensure that any key functions or modules in future updates include Google-style docstrings as per project guidelines.

    @github-actions
    Copy link
    Contributor

    github-actions bot commented Apr 2, 2025

    Persistent review updated to latest commit 9e493a0

    @qodo-code-review
    Copy link

    qodo-code-review bot commented Apr 2, 2025

    PR Code Suggestions ✨

    Latest suggestions up to 9e493a0
    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    General
    Clarify deferral instructions

    Clarify the agent’s role by specifying conditions for deferring response generation,
    ensuring the intended flow is unambiguous.

    app/core/agents/entry/prompt.py [5]

    -You do not directly answer knowledge-based user queries. Your task is to classify them, and forward it with proper context. Other agents will handle the actual response.
    +You do not directly answer knowledge-based user queries; instead, classify them and forward with necessary context to the appropriate agent for response generation.
    Suggestion importance[1-10]: 5

    __

    Why: The suggestion slightly rewords the deferral instruction for clarity without altering functionality, offering only a minor improvement.

    Low
    Refine validation criteria

    Refine the validation criteria to emphasize clear association with known schema
    elements while still allowing flexible phrasing.

    app/core/agents/validator/prompt.py [10-11]

    -The question is valid if it references any nodes, classes, or properties in the ENPKG — even if indirectly or across multiple relationships.
    -Allow flexibility for phrasing. As long as the topic aligns with schema elements, the question should be considered valid.
    +The question is valid if it clearly references nodes, classes, or properties in the ENPKG—even indirectly—provided the reference unambiguously aligns with known schema elements. Allow flexible phrasing as long as the core topic relates to these elements.
    Suggestion importance[1-10]: 5

    __

    Why: The change refines the wording for validation criteria to enhance clarity; however, it remains a stylistic update with limited impact.

    Low
    Standardize unclear query handling

    Consolidate the instructions for unclear queries to ensure consistency in prompting
    for clarification where needed.

    app/core/agents/validator/prompt.py [19-21]

    -It is completely unrelated to any nodes/entities in ENPKG or the topic cannot be mapped to any known classes or properties in the schema.
    -If a question is unclear or possibly valid but lacks detail, do not mark it invalid — ask the user for clarification instead.
    +It is completely unrelated to any nodes/entities in ENPKG or cannot be mapped to any known classes or properties in the schema. For queries that are unclear or borderline in detail, always ask the user for clarification rather than marking them as invalid.
    Suggestion importance[1-10]: 5

    __

    Why: This suggestion consolidates instructions for handling unclear queries, improving consistency. The improvement is useful but only a minor refinement.

    Low

    Previous suggestions

    Suggestions up to commit 9e493a0
    CategorySuggestion                                                                                                                                    Impact
    General
    Refine unclear query handling

    Refine the instruction for borderline cases to ensure that ambiguous queries
    trigger a request for more detail without being incorrectly invalidated.

    app/core/agents/validator/prompt.py [21]

    -If a question is unclear or possibly valid but lacks detail, do not mark it invalid — ask the user for clarification instead.
    +If a question is unclear or insufficiently detailed, do not mark it as invalid; instead, request further clarification.
    • Apply this suggestion
    Suggestion importance[1-10]: 5

    __

    Why: The suggestion rephrases the instruction for borderline cases to improve clarity without changing the intended behavior, representing a moderate improvement.

    Low
    • More

    @lfnothias lfnothias merged commit 442f8a9 into dev Apr 2, 2025
    1 check passed
    Mattheex added a commit that referenced this pull request Apr 4, 2025
    commit 2d711c5
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Fri Apr 4 10:41:35 2025 +0200
    
        Removed links from sidebar text
    
    commit fef7062
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 16:33:23 2025 +0200
    
        Modified tool spectra
    
    commit eda1816
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 14:25:10 2025 +0200
    
        updated requirements
    
    commit 3e81c44
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 14:18:12 2025 +0200
    
        Updated requirements and postgres
    
    commit 1ef0604
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 14:08:46 2025 +0200
    
        updated requirements
    
    commit 9acb8a6
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 14:04:16 2025 +0200
    
        Modified streamlit
    
    commit fdfb947
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 13:50:00 2025 +0200
    
        Updated tool
    
    commit 6f1f547
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 13:35:07 2025 +0200
    
        updated requirements.txt
    
    commit 0bb80e4
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 13:33:11 2025 +0200
    
        Changed requirements.txt
    
    commit ae39550
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 13:31:23 2025 +0200
    
        Updated requirements
    
    commit ca34859
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 13:28:37 2025 +0200
    
        New versions of packages
    
    commit 7db3374
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 12:28:55 2025 +0200
    
        corrected typo
    
    commit 38c4dfc
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 12:27:32 2025 +0200
    
        corrected typo
    
    commit 0c486d6
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 12:26:04 2025 +0200
    
        Updated imports
    
    commit 9000e1c
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 08:32:03 2025 +0200
    
        Updated streamlit webapp
    
    commit 3ba129a
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 08:18:12 2025 +0200
    
        Updated wrong import
    
    commit ba31848
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Thu Apr 3 08:14:52 2025 +0200
    
        Added new requirements
    
    commit feb45a5
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 22:18:49 2025 +0200
    
        Updated streamlit
    
    commit 0d39e81
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 22:12:15 2025 +0200
    
        Updated requirements
    
    commit 2038148
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:59:53 2025 +0200
    
        Updated requirements for heroku
    
    commit f479659
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:46:05 2025 +0200
    
        Corrected typo in requirements
    
    commit df9572d
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:44:28 2025 +0200
    
        Updated new file format for heroku
    
    commit b880614
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:41:50 2025 +0200
    
        Added runtime file
    
    commit 7617303
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:38:57 2025 +0200
    
        Updated requirements.txt
    
    commit da73d58
    Author: Lucas Pradi <lucaspradi1@gmail.com>
    Date:   Wed Apr 2 21:15:37 2025 +0200
    
        Added heroku deployment files
    
    commit 442f8a9
    Merge: 2140f6c 9e493a0
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Wed Apr 2 21:05:57 2025 +0200
    
        Merge pull request #152 from HolobiomicsLab/dev_madina
    
        Modify prompts of entry and validator
    
    commit 9e493a0
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Wed Apr 2 17:25:07 2025 +0200
    
        Modify prompts of entry and validator
    
    commit 2140f6c
    Merge: b7fc7ed 4867d85
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Wed Apr 2 16:01:25 2025 +0200
    
        Merge pull request #151 from HolobiomicsLab/dev_madina
    
        Dev madina
    
    commit 4867d85
    Merge: 9834acd b836325
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Wed Apr 2 14:49:08 2025 +0200
    
        Merge branch 'main' into dev_madina
    
    commit 9834acd
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Tue Mar 25 13:45:29 2025 +0100
    
        Update docs
    
    commit b836325
    Author: Madina Bekbergenova <140626371+madina1203@users.noreply.github.com>
    Date:   Tue Mar 25 13:04:26 2025 +0100
    
        Update README.md
    
    commit a4cbc0c
    Author: Madina Bekbergenova <140626371+madina1203@users.noreply.github.com>
    Date:   Tue Mar 25 12:38:33 2025 +0100
    
        Update README.md
    
    commit f975806
    Author: Madina Bekbergenova <140626371+madina1203@users.noreply.github.com>
    Date:   Tue Mar 25 10:40:06 2025 +0100
    
        Update README.md
    
    commit 1d78829
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Fri Mar 21 17:01:44 2025 +0100
    
        Minor changes
    
    commit f2693a3
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Fri Mar 21 14:32:12 2025 +0100
    
        Modify tool_spectrum.py to provide url
    
    commit 0dcc8ef
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Fri Mar 21 09:26:05 2025 +0100
    
        New tool updates
    
    commit 5a0ab2b
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Tue Mar 18 15:51:14 2025 +0100
    
        Modify new tool and Interpreter prompt
    
    commit 08890ff
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Sat Mar 15 20:08:46 2025 +0100
    
        Add new tool for Interpreter, evaluation dataset, change prompt of Interpreter agent
    
    commit 9ed8a97
    Author: Madina Bekbergenova <140626371+madina1203@users.noreply.github.com>
    Date:   Fri Mar 14 12:28:36 2025 +0100
    
        Update README.md
    
    commit 4e1ff0b
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Fri Mar 14 12:24:55 2025 +0100
    
        Add information about streamlit to README
    
    commit 49ba3b0
    Author: madina1203 <madina.bekbergenova@etu.univ-cotedazur.fr>
    Date:   Fri Mar 14 12:13:38 2025 +0100
    
        Modify entry agent  prompt to call supervisor in case if file is submitted, add streamlit-modal to requirements and modify streamli_app.py
    
    commit 8b0c01a
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Fri Feb 21 12:43:52 2025 +0100
    
        Update .pr_agent.toml
    
    commit 4f5b889
    Merge: f864751 23dd970
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Mon Feb 17 15:23:39 2025 +0100
    
        Merge pull request #141 from HolobiomicsLab/dev
    
        Fix error sparql_query and add extra LLMs
    
    commit f864751
    Merge: 686370b 6d4898e
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Mon Feb 17 10:29:30 2025 +0100
    
        Merge pull request #139 from HolobiomicsLab/dev
    
        Update pip packages mkdocs
    
    commit 686370b
    Merge: 41b13ce 6fbaac1
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Mon Feb 17 09:42:25 2025 +0100
    
        Merge pull request #138 from HolobiomicsLab/dev
    
        Dev merge with Docs and more LLM support
    
    commit 41b13ce
    Merge: f30df6c 59add66
    Author: Louis-Félix Nothias <louisfelix.nothias@gmail.com>
    Date:   Thu Feb 13 22:41:16 2025 +0100
    
        Merge pull request #131 from holobiomics-lab/dev
    
        Tidying up and Improved README.md
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    2 participants