Navie commands are logged to a jsonl file

[kgilpin]

Update the Client to log Navie commands to a jsonl file.

The log file should include distinct fields for:

• "@" command name, e.g. @plan, @generate
• command options, e.g. /noformat, as an array (options can be split on whitespace)
• the rest of the primary command input (user question) (the actual question, not a file path)
• custom prompt, if any (the actual prompt, not a file path)
• timestamp
• elapsed time

Focus on the plan function initially.

Try not to reparse command strings that have already been parsed.

---

Output format

• Do not reproduce large sections of the existing code.
• Do not emit a "Title" section
• Do not emit an "Implementation details" section
• Do emit an "Example" section, showing how the log facility is used

---

v4

[github-actions]

Problem

Navie commands are recorded, but the current logging mechanism isn't detailed enough to capture specific command attributes and their context in a structured format.

Analysis

To provide better traceability and debugging, the logging process should be enhanced to log commands to a JSONL file with distinct fields for the command name, options, user question, custom prompt, timestamp, and elapsed time. Specifically, this should be implemented in the plan function first, with the idea that similar changes will follow for other functions.

The log entries should be JSON objects with clear keys for each piece of information:

• The command name
• Command options
• The main query (user question)
• Any custom prompt
• A timestamp for when the command was initiated
• The elapsed time for the command execution

The command options and the main query should be captured without reparsing the command strings if they have already been parsed. This requires minor modifications to the data flow in the plan function and possibly some helper functions to centralize logging.

Proposed Changes

1. navie/client.py:

   • Modify the plan function to record the start time before executing the command.
   • Calculate the elapsed time after execution.
   • Capture necessary command details and structure them into a JSON object.
   • Write the JSON object to a log file in JSONL format.
   • Create helper functions if needed to minimize code repetition for logging.

2. navie/editor.py:

   • Adjust the call to the plan function of the Client class to ensure it passes along any custom prompts.
   • Ensure appropriate catching of the command details, splitting options correctly, and passing these details to the Client.

Example

Updating navie/client.py

def plan(self, issue_file, output_file, context_file=None, prompt_file=None, options=None):
    import json
    import time

    log_file = os.path.join(self.work_dir, "plan.log")
    input_file = os.path.join(self.work_dir, "plan.txt")
    jsonl_log_file = os.path.join(self.work_dir, "commands.log.jsonl")

    start_time = time.time()

    with open(issue_file, "r") as issue_f:
        issue_content = issue_f.read()

    with open(input_file, "w") as plan_f:
        input_tokens = ["@plan"]
        if context_file:
            input_tokens.append("/nocontext")
        input_tokens.append(issue_content)
        full_command = " ".join(input_tokens)
        plan_f.write(full_command)

    command = self._build_command(
        input_path=input_file,
        output_path=output_file,
        context_path=context_file,
        prompt_path=prompt_file,
        log_file=log_file,
    )
    
    # Execute the command
    self._execute(command, log_file)

    # Log the command details to JSONL file
    end_time = time.time()
    elapsed_time = end_time - start_time
    log_entry = {
        "command_name": "@plan",
        "options": ["/nocontext"] if context_file else [],
        "user_question": issue_content,
        "custom_prompt": None,  # Add actual prompt content if available
        "timestamp": time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(start_time)),
        "elapsed_time": elapsed_time
    }
    
    with open(jsonl_log_file, "a") as jsonl_f:
        jsonl_f.write(json.dumps(log_entry) + "\n")

This demonstrates how to log specific details of the plan command to a JSONL file. Other functions in the Client class would need similar updates for consistent logging.

Updating navie/editor.py

def plan(self, issue, context=None, options=None, prompt=None, cache=True, auto_context=True):
    work_dir = self._work_dir("plan")
    issue_file = os.path.join(work_dir, "plan.input.txt")
    output_file = os.path.join(work_dir, "plan.md")

    self._log_action("@plan", options, issue)

    def read_output(save_cache):
        with open(output_file, "r") as f:
            self._plan = f.read()

        if save_cache:
            self._save_cache(work_dir, issue, "issue", options, "options", context, "context", prompt, "prompt")

        print(f"  Output is available at {output_file}")

        self._log_response(self._plan, output_file=output_file)

        return self._plan

    if cache and self._all_cache_valid(work_dir, issue, "issue", options, "options", context, "context", prompt, "prompt"):
        print("  Using cached plan")
        return read_output(False)

    with open(issue_file, "w") as f:
        content = []
        if options:
            content.append(options)
        content.append(issue)
        f.write(" ".join(content))

    context_file = self._save_context(work_dir, "plan", context, auto_context)
    prompt_file = self._save_prompt(work_dir, "plan", prompt)

    Client(work_dir, self.temperature, self.token_limit).plan(
        issue_file, output_file, context_file, prompt_file=prompt_file
    )

This is an updated plan function in the Editor class that ensures the necessary data is passed to the Client class for logging.