Title Optimization: improve error handling

[lhkowalski]

Proposed changes:

• Handle suggestion errors and show them using an error notice component (moderation, unclear prompt, service unavailable, etc)
• Handle quota exceeded errors and show the QuotaExceededMessage component
  • when the plan is free, the QuotaExceededMessage will display the upgrade nudge
  • when the plan is paid, the QuotaExceededMessage will display the fair usage notice
• Handle JSON parse errors and show a generic error message using an error notice component
• Keep the "Try again" button only for transient errors

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

• Make sure you are sandboxing the public-api domain
• Make sure you are running with the beta blocks variation
• To test the error handling, we are going to simulate errors on the backend using code snippets on the jetpack-ai-query endpoint handler; add the snippet as the first line of the run_query method

---

Testing the moderation error

• Use the following snippet on the backend:

return new \WP_Error( 'jetpack_ai_moderation', 'This request has been flagged by our moderation system. Please try to rephrase it and try again.', [ 'status' => 422 ] );

• On the block editor, add some content, open the AI Assistant sidebar section and click the Improve title for SEO button
• Confirm you see the proper error message on the UI:

---

Testing the JSON parsing error

• This error can happen from time to time when the backend model fails to produce a valid, complete JSON
• Use the following snippet on the backend (to simulate a suggestion streaming returning an incomplete JSON object; line breaks are relevant, so don't remove it):

		header( 'HTTP/1.1 200 OK' );
		header( 'X-Accel-Buffering: no' );
		header( 'Date: ' . gmdate( 'D, d M Y H:i:s', time() ) . ' GMT' );
		header( 'Content-Type: text/event-stream' );
		header( 'Connection: close' );
		header( 'access-control-allow-origin: *' );
		header( "Cache-Control: no-cache, must-revalidate\n" );
		echo <<<COMPLETION
data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"role":"assistant","content":"","refusal":null},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"content":"[\\n"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"content":" {\\n"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"content":"   "},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-A6LVBpL5q401MjFrd5mY8Siiei3H0","object":"chat.completion.chunk","created":1726075985,"model":"gpt-3.5-turbo-1106","system_fingerprint":"fp_54b3a4486f","choices":[{"index":0,"delta":{"content":" \\""},"logprobs":null,"finish_reason":null}]}

data: [DONE]


COMPLETION;

		return ['body' => '{"usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30}}' ];

• Confirm you see the proper error message on the UI:

---

Testing the quota exceeded error on a free site

• Use the following snippet on the 0-sandbox.php file:

add_filter( 'jetpack_ai_tier_licensed_quantity', function() { return 0; } );
add_filter( 'jetpack_ai_current_period_requests_count', function() { return 19; } );
add_filter( 'jetpack_ai_all_time_requests_count', function() { return 19; } ); // almost on the limit

• Go to the block editor, refresh it but do not click on the Improve title for SEO button yet
• Return to the 0-sandbox.php file and change the snippet to:

add_filter( 'jetpack_ai_tier_licensed_quantity', function() { return 0; } );
add_filter( 'jetpack_ai_current_period_requests_count', function() { return 20; } );
add_filter( 'jetpack_ai_all_time_requests_count', function() { return 20; } );

• Now, on the block editor, without refreshing, click Improve title for SEO
• Confirm you see the proper error message on the UI:

• Confirm that clicking the upgrade link will land you on the proper place (My Jetpack interstitial or checkout page, for Jetpack and Simple sites, in order)

---

Testing the quota exceeded error on a paid site

• Use the following snippet on the 0-sandbox.php file:

add_filter( 'jetpack_ai_tier_licensed_quantity', function() { return 1; } );
add_filter( 'jetpack_ai_current_period_requests_count', function() { return 2999; } ); // almost on the limit
add_filter( 'jetpack_ai_all_time_requests_count', function() { return 45000; } );

• Go to the block editor, refresh it but do not click on the Improve title for SEO button yet
• Return to the 0-sandbox.php file and change the snippet to:

add_filter( 'jetpack_ai_tier_licensed_quantity', function() { return 1; } );
add_filter( 'jetpack_ai_current_period_requests_count', function() { return 3000; } );
add_filter( 'jetpack_ai_all_time_requests_count', function() { return 45000; } );

• Now, on the block editor, without refreshing, click Improve title for SEO
• Confirm you see the proper error message on the UI:

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the update/jetpack-ai-handle-errors-on-title-improvement-tool branch.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack update/jetpack-ai-handle-errors-on-title-improvement-tool
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Follow this PR Review Process:

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Choose a review path based on your changes:
   • A. Team Review: add the "[Status] Needs Team Review" label
     • For most changes, including minor cross-team impacts.
     • Example: Updating a team-specific component or a small change to a shared library.
   • B. Crew Review: add the "[Status] Needs Review" label
     • For significant changes to core functionality.
     • Example: Major updates to a shared library or complex features.
   • C. Both: Start with Team, then request Crew
     • For complex changes or when you need extra confidence.
     • Example: Refactor affecting multiple systems.
3. Get at least one approval before merging.

Still unsure? Reach out in #jetpack-developers for guidance!

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen semi-continuously (PCYsg-Jjm-p2).
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly. The next release is scheduled for October 1, 2024 (scheduled code freeze on September 30, 2024).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[dhasilva]

Suggested change

	const [ error, setError ] = useState< TitleOptimizationError >( null );
	const [ error, setError ] = useState< TitleOptimizationError | null >( null );

[dhasilva]

Suggested change

	const errorMessage = error?.message ? error.message : genericErrorMessage;
	const errorMessage = error.message ? error.message : genericErrorMessage;

[dhasilva]

LGTM, just left a couple type comments