Issue with Custom Skills in PandasAI when using OpenAI API

[juancolonna]

Hi PandasAI community,

First off, congratulations on maintaining this project—it's amazing!

I was experimenting with the example on using "skills" as described in the official documentation. Instead of using BambooLLM, I’m working with OpenAI. I’ve added the @Skill decorator to create a function that should help me generate a bar plot with names on the vertical axis and rotated 45 degrees. This was intended as a simple test to see if it works.

However, to my surprise, the resulting plot displayed the names on the horizontal axis, with no rotation. It appears that the custom skill I added was ignored. To investigate further, I used print(agent.last_code_executed) to check the actual executed code, and it didn’t match the code I provided in the @Skill function.

Could this be a bug, or is there something I may have configured incorrectly? Any insights or guidance would be greatly appreciated. Thanks

import os
from dotenv import load_dotenv
import pandas as pd
from pandasai import Agent
from pandasai.skills import skill
from pandasai.llm import OpenAI

# Load environment variables from .env file
load_dotenv()

# Get the OpenAI API key
api_key = os.getenv("OPENAI_API_KEY")

# Initialize the OpenAI LLM with the API key
llm = OpenAI(api_token=api_key, model="gpt-4o-mini", temperature=0.0)

employees_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Name": ["John", "Emma", "Liam", "Olivia", "William"],
    "Department": ["HR", "Sales", "IT", "Marketing", "Finance"],
}

salaries_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Salary": [5000, 6000, 4500, 7000, 5500],
}

employees_df = pd.DataFrame(employees_data)
salaries_df = pd.DataFrame(salaries_data)

# Function doc string to give more context to the model for use this skill
@skill
def plot_salaries(names: list[str], salaries: list[int]):
    """
    Displays the bar chart  having name on x-axis and salaries on y-axis
    Args:
        names (list[str]): Employees' names
        salaries (list[int]): Salaries
    """
    # plot bars
    import matplotlib.pyplot as plt

    plt.bar(names, salaries)
    plt.xlabel("Salary")
    plt.ylabel("Employee Name")
    plt.title("Employee Salaries")
    plt.yticks(rotation=45)

agent = Agent([employees_df, salaries_df], memory_size=10, config={"llm": llm})

# Add the skill to the agent
agent.add_skills(plot_salaries)

# Chat with the agent
response = agent.chat("Plot the employee salaries against names")

print(agent.last_code_executed)

Output:

df1 = dfs[0]
df2 = dfs[1]
merged_df = pd.merge(df1, df2, on='EmployeeID')
plt.figure(figsize=(10, 5))
plt.bar(merged_df['Name'], merged_df['Salary'])
plt.xlabel('Name')
plt.ylabel('Salary')
plt.title('Employee Salaries')
plt.savefig('/content/exports/charts/temp_chart.png')
result = {'type': 'plot', 'value': '/content/exports/charts/temp_chart.png'}

[dosubot[bot]]

Hey there, @juancolonna! I'm here to help you with any bugs, questions, or contributions. Let's tackle this together!

The issue you're experiencing with the custom skill being ignored might be due to how the PandasAI agent determines which code to execute. The agent uses the function's docstring to provide context to the model, which influences the decision on when to use the custom skill. If the model's interpretation of the query doesn't align with the skill's docstring, it might not trigger the custom skill as expected ^[1][2].

Here are a few steps you can take to troubleshoot and potentially resolve the issue:

1. Check the Docstring: Ensure that the docstring of your plot_salaries function clearly describes the intended functionality. The model relies on this description to decide when to use the skill.

2. Skill Registration: Verify that the skill is correctly registered with the agent using agent.add_skills(plot_salaries). Ensure there are no naming conflicts or errors during this process ^[1].

3. Logging and Debugging: Utilize the logging functionality in PandasAI to trace the execution path. This can help identify why the custom skill might be ignored. Check the logs for any errors or warnings that might indicate issues with skill execution ^[3].

4. Configuration: Review the configuration settings of the agent, especially those related to the language model (LLM) and verbosity. Ensure that the LLM is correctly configured to interpret and execute custom skills ^[4].

5. Skill Context: Consider modifying the query to more closely match the context provided in the skill's docstring. This might help the model better align the query with the custom skill.

By following these steps, you should be able to better understand and address the issue with your custom skill in PandasAI.

To continue talking to Dosu, mention @dosu.

[juancolonna]

@dosu Thanks, it was a problem with the docstring. Thanks for your help.

[dosubot[bot]]

You're welcome! I'm glad to hear that the issue was resolved. If you have any more questions or need further assistance, feel free to reach out. Happy coding!

To continue talking to Dosu, mention @dosu.