[FEATURE REQUEST] Add Meta Robots Tag Support in Content Editor (Default: noindex, nofollow)

[vishalchopra666]

Description:
It would be highly beneficial to have the ability to add a meta robots tag in the content editor for blog posts, static pages, and other content types. This feature would allow users to control search engine indexing on a per-page or per-post basis, with the default setting being noindex, nofollow. This is particularly important for avoiding thin content issues, such as tags, categories, and pagination pages, which could negatively impact SEO if indexed.

Why is this important?

1. SEO Control: Users can define the indexing behavior for each post or page individually, with the default setting being noindex, nofollow to prevent unwanted content from being indexed by search engines.
2. Avoid Indexing Thin Content: Pages like tags, categories, and pagination often result in thin content that adds little value to search engine results. Preventing these pages from being indexed by default protects the site's SEO health.
3. Better Compliance with Search Engines: Unlike robots.txt, which can be ignored by search engines, the meta robots tag embedded in HTML has a stronger influence on crawling behavior, making it more effective for controlling indexing.

Suggested Implementation:

1. Add a field in the content editor to insert or modify the meta robots tag, with the default setting being noindex, nofollow.
2. Provide global meta robots tag settings that can be applied to all pages, posts, tags, categories, and pagination by default, with the ability for users to override these settings on a per-page or per-post basis.
3. Ensure that the noindex, nofollow tag is automatically applied to tags, categories, and pagination pages unless manually overridden.

Examples:

• Blog post 1: index, follow
• Blog post 2: noindex, nofollow (default)
• Tags page: noindex, nofollow (default)
• Pagination pages: noindex, nofollow (default)

Expected Benefits:

• Improved SEO performance by avoiding indexing of thin content pages, such as pagination, tags, and categories, which can dilute search engine rankings.
• Ensures privacy and prevents accidental indexing of pages by default, giving users full control over content visibility.
• Granular control over which content search engines can index and follow, with an emphasis on user privacy and SEO best practices.

Potential Challenges:

• Ensuring compatibility with existing themes and plugins.
• Providing clear documentation for users on how to utilize this feature effectively and modify the default settings as needed.

Conclusion:
Adding meta robots tag support with a default setting of noindex, nofollow would greatly enhance the flexibility and SEO control that users have over their content, especially in avoiding thin content issues like pagination and tag/category pages. This feature will make the platform even more powerful for bloggers and website owners, ensuring better search engine optimization and content privacy.

Thank you for considering this feature request!

[kuslahne]

It will be great also if we can modify meta tag for canonical URL, as this will decrease chance of duplicate content found by search engine. Maybe change to url friendly something like blabla?pages=5 to blabla/pages/5 a bit easier than add this metatag for editing.

[Joduai]

Since there's no support for custom dashboard and page options I went with adding noindex,follow rules to archive and tag pages leaving default on categories, within my template.
I thought that adding such option as a temporary solution to htmly config could be useful.
Although I'm not sure how to make simple choices without adding tons of new options to config, since there's already plethora of them

edit: I propose moving this thread to discussions since it's not an issue. The more, I'd even rename it to a topic about development changes toward SEO friendliness. HTMLy might be a "simple" blog platform, but it's already powerful, and pushing some changes towards seo might be very beneficial. I personally don't give much damns about it, but such things might have influence on raising a bit user base.