Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse/Replicate SemanticAnalyzer and Enhancer to Agent workflow #811

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

DonggeLiu
Copy link
Collaborator

@DonggeLiu DonggeLiu commented Feb 21, 2025

  1. Reuse SemanticAnalyzer from the one-prompt workflow
  2. Replicate Enhancer from OnePromptEnhancer
  3. Last result can still be BuildResult if build failure.
  4. When Execution Fails, still record BuildResult without RunResult.
  5. Download xs benchmarks to understand why its successful BuildResult was not recorded on final experiment report after its run failure.
  6. Clean up commented code.
  7. Do not print chat_histroy when logging Results, which makes it unreadable.
  8. Stringify WorDir and Benchmark in Results.
  9. Maybe write_result() at the end of every stage, so that the report is always up-to-date.

Next:

  1. Fix log type for easier debugging.
  2. Understand and fix prompt truncation logic error.
  3. Re-run cloud build if a flaky step failed (e.g., apt install).

@DonggeLiu DonggeLiu marked this pull request as draft February 21, 2025 00:30
@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg1 -ag

@DonggeLiu DonggeLiu marked this pull request as ready for review February 23, 2025 08:17
@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg2 -ag

@DonggeLiu DonggeLiu mentioned this pull request Feb 24, 2025
12 tasks
@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg1 -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg2 -ag



class Enhancer(Prototyper):
"""The Agent to generate a simple but valid fuzz target from scratch."""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: fix docstring.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!

# from results import AnalysisResult, BuildResult, Result
# from tool.container_tool import ProjectContainerTool

MAX_ROUND = 10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this used?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove.
It was used by its parent class to control the #rounds for LLM interaction.

pipeline.py Outdated
result_history)
return False

self.logger.warning('[Cycle %d] Last result is unexpeceted: %s',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: fix spelling of unexpected.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg -ag

@DonggeLiu
Copy link
Collaborator Author

/gcbrun exp -n dg1 -ag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants