Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't move messages into Finance, Purchases, Travel categories #667

Closed
n1ckd0r opened this issue Feb 28, 2022 · 15 comments
Closed

Can't move messages into Finance, Purchases, Travel categories #667

n1ckd0r opened this issue Feb 28, 2022 · 15 comments

Comments

@n1ckd0r
Copy link

n1ckd0r commented Feb 28, 2022

I have enabled the hidden Finance, Purchases and Travel categories through Simplify, but when messages don't go there automatically there seems to be no way to move messages to those categories, manually or through a rule.

I'd like a way to either move messages to those system categories for messages that gmail misses, or some way to create a separate label that gets bundled with those categories as a workaround.

@leggett
Copy link
Owner

leggett commented Feb 28, 2022

It is possible but you have to manually set it up. I hope to automate it some day but I haven't gotten to it yet.

First, some background:

  • Simplify makes 3 additional categories (Finance, Purchases, and Travel) visible in Gmail's left navigation as links to a search for each category.
  • These categories are from Google Inbox and are still applied to incoming email in the background but Gmail decided to hide them entirely in Gmail's interface.
  • Gmail decided all of the messages in these categories would show up in the "Updates" category (mail in these categories is actually in two categories).
  • It is not easy to modify the email in these categories. (It is actually impossible but there is a work around I'll get to shortly)
  • It is not possible to show them as inbox tabs
  • The limited functionality is why Simplify doesn't show them in the nav by default and labels them as "read-only" in Simplify settings.

A glimmer of hope:

  • A couple months ago, I discovered a way to make these categories more full featured
  • You can create your own label (e.g., Receipts for Purchases) and then filter mail by category (e.g., has the words: "category:purchases") into that label where you can add and remove email freely.
  • If you don't want these emails to also show up in category:updates, the filter should also set the category to "Primary" (the Purchases and Travel category will remain on the email, but for some reason, Finance is removed when you do this).
  • Since this is a label, you can also show the label and then bundle email in the label.
  • The reason that works is that categories are always applied in the backend before any of the user filters are run. So while you can't filter based on labels, you can filter based on categories.
  • I've been testing this for a few months now and it is working flawlessly for me.
  • But I will have to do some work to figure out how to automate setting all this up (as it involves creating labels and filters for each category).

@leggett leggett closed this as completed Feb 28, 2022
@leggett leggett pinned this issue Feb 28, 2022
@n1ckd0r
Copy link
Author

n1ckd0r commented Feb 28, 2022 via email

@MaxLinWorm
Copy link

MaxLinWorm commented Jun 27, 2023

Hey,

Tried the workaround and it worked fine apparently for travel and purchases. However, when I applied it to the finance category (by creating a filter and applying it to existing conversations), categorizing the emails as primary to remove them from the updates category, all emails had the finance category removed (ie I can't find them searching for category:finance). I can only get to them using my own label (ie searching for label:myFinanceLabel).

However, there's no way to put them back to the finance category, as it is read-only (apparently not though =D)

I don't know if the category will still be applied to new incoming email. Will let you know.

@leggett
Copy link
Owner

leggett commented Jun 28, 2023

I'm seeing the same thing on my end... not sure there is a way to avoid this. Either you leave it in category:updates which leaves it also in category:finance OR you move it to primary which appears to remove it from both.

VERY strange that it only does that for finance but not for travel and purchases... not sure why that is.

Was there a reason you wanted to search for category:finance?

@MaxLinWorm
Copy link

Nope, just reporting the bug

@davelavoie
Copy link

davelavoie commented Nov 13, 2023

@leggett I know I'm late here, but I just wanted to share with you that it's possible to assign emails to the hidden categories such as Finance, Purchases and Travel. In Gmail, these categories are actually categorized as labels with names starting with the "^" character, along with others categories such as Low Priority, Important, etc.

For example, it's possible to create a filter so specific emails are assigned to the "Finance" category by setting the "^smartlabel_finance" label. However, as far as I know, it's not possible (or easy at least) to do it in Gmail Standard, but it's possible to do it in Gmail HTML classic (while it still exists). I've done this for years. Here's how I did it:

  1. Go to https://mail.google.com/mail/u/0/h/ so you can browse your inbox in Gmail basic HTML mode.
  2. Click on "Settings", then "Filters" and the "Create a new Filter" button at the bottom of the page.
  3. Create your filter with the criteria you want, as you would have done in Gmail Standard. Then click on "Next Step".
  4. Now here's the tricky step. You'll be redirected to the last step of creating a new filter. You'll need to inspect the page with developer tools, and select the "Choose label" select field. Then, you can manually edit the values of the filters in the HTML code directly. Simply change one of the option, and replace it's value with ^smartlabel_finance , and change it's name with Finance (or anything you want, it's just for the sake of making sure you'll be selecting the right option).
image
  1. If you want, you can even categorize existing emails by checking the "Also apply filter to conversations below".
  2. Click on Create Filter, and voilà! You're now able to categorize emails to the Finance category! You can do the same with all hidden categories such as Purchases and the others. I had a list of all these hidden categories, but I can't seem to find it right now. I'll try to find it if someone is interested.

Please note that in the list of filters, in Gmail HTML classic, the filter will look like if it does nothing:
image
... while in Gmail standard, it says it'll be categorized as Updates:
image
But I assure you that if you search for "category:finance", your emails will be showing there. If you decide to edit the filter (for example by changing the criteria), as long as you don't change the value of the "Categorize as Updates" field, all new emails that fit with the criteria will still be categorized as Finance.

I know Gmail HTML classic is going to be disabled by January 2024, so it's still the time to create a bunch of placeholder filters that can be edited in the future... Maybe it's even possible to make this work with Gmail Standard, but since it's not possible to edit the Select field option values, I don't know how to do it.

I just wanted to share this trick with you, since maybe it can give you some ideas :-)

@leggett
Copy link
Owner

leggett commented Nov 13, 2023

Thanks Dave! Definitely gives me some things to try out. The remaining downside of this is the emails will still show up in the Updates category as well as the correct hidden category. Or do you move them to the desired smart label (category) and categorize them as Primary?

@davelavoie
Copy link

davelavoie commented Nov 16, 2023

Ok, so my reply is a little long, and it might not be the appropriate channel to share all of this, but since I've already spent time researching this in the past, I suppose it could be of use to you, or in fact anyone who really liked Inbox. ;-)


Thanks Dave! Definitely gives me some things to try out. The remaining downside of this is the emails will still show up in the Updates category as well as the correct hidden category. Or do you move them to the desired smart label (category) and categorize them as Primary?

Yeah, unfortunately, the "^smartlabel_finance" label is like a child of the "^smartlabel_notification" label, along with a few other ones:

  • ^smartlabel_notification
    • ^smartlabel_finance
    • ^smartlabel_receipt
    • ^assistive_purchase
    • ^smartlabel_travel
    • ^smartlabel_pure_notif - this is the real "updates" category, but unfortunately, the "category:updates" query is instead an alias for "label:^smartlabel_notification", which includes all the categories above. 😒 Searching for label:^smartlabel_pure_notif indeed returns all updates, minus the finance, receipt and travel ones.

So there's no way to make the emails categorized under the "Finance" category to stand out without some tweaking, since they'll always show as "Updates" (but Simplify allows me to kind of bypass this limitation, as you'll see below)...

Because of this, I ended up, a few years ago, doing the same thing you suggested, I've created four labels named "Finance", "Purchases", "Travel" and "Low Priority", and by using filters, I apply the labels based on their corresponding category. I know it's technically not possible to create a new label with these names since they are system-reserved, but it's actually possible to create labels with these names by using the Gmail HTML classic URL I've shared earlier.

image

I really liked how Inbox was bundling my emails efficiently. When they pulled the plug on Inbox, after investigating how things were working under the hood, I've figured that I was also able to recreate the following categories by using the filters with these rules:

  • Purchases: category:^assistive_purchase OR category:^smartlabel_receipt - while ^assistive_purchase is an alias of category:purchases, I've figured that without the category:^smartlabel_receipt, most of the purchases were missing, to a point that the "category:purchases" is almost useless. Hence I'm adding my custom Purchases label to all emails that fit with one of these two categories
  • Finance: category:finance
  • Travel: category:travel
  • Low Priority: is:lowpriority -category:^smartlabel_receipt -category:^assistive_purchase -category:finance - Here I wanted to recreate the "Low Priority" category from Inbox. The choice of excluding the purchase, receipt and finance categories is a personal one though.
  • Notifications: category:^smartlabel_pure_notif - Since I wanted to separate Finance and Purchases from the "real" updates, later I've discovered the ^smartlabel_pure_notif category, and found out it was really good for listing updates without mixing it up with the other two categories. Logically, when searching for category:updates -category:^smartlabel_pure_notif rule, you'll end up with all other type of updates (which are category:finance and category:purchases)

And finally, the distinction between Personal (category:personal) and Primary (category:primary) categories seems to be that category:primary is almost the same as category:personal in:inbox. I've figured that all emails that weren't part of any other categories were falling in the Personal category, and that all emails under the Primary category are basically personal emails that are still in the inbox. So I decided not to bother with Primary and Personal categories at all, except when I actually want to exclude an email from any category.


The way I filter the Finance, Notifications, Purchases and Low Priority categories, along with Simplify, makes my life a lot easier. Actually, it doesn't bother me if all of these bundles end up under the "Updates" tab, since in my workflow it makes a lot of sense:

image

As you can see, it works very well. The only thing that is missing from Inbox now are the respective icons for each category instead of the mail envelop icon 😅 . That said, I'm definitely going to purchase the annual subscription of Simplify before my trial ends!


Here a link I've used that initially helped me, which could be of help to you. It's a list with a lot of Gmail system labels. Some are missing, such as ^assistive_purchase, but its still the most complete list I've stumbled upon up to now.


And finally, a small trick I've discovered this week while trying to find another alternative to Gmail HTML classic, which is going to be disabled in January. I've found a mobile version of Gmail at https://mail.google.com/mail/mu/mp/ which actually ALLOWS us to assign custom system categories easily to any existing emails. I don't have an actual use for this, but with some reverse engineering, it could be possible to import this feature in Gmail in order to properly assign emails to hidden categories.

In order for it to work, you must have enabled the Default inbox set in Gmail with at least one Category enabled. It'll then allow you to move an email to any hidden category, simply by editing the "data-onclick-arg" parameter:

image image

It looks like Google forgot to put in place any kind of validation before applying the category.... 😁

That's all for now! I hope it gives you food for thought!

@leggett
Copy link
Owner

leggett commented Nov 16, 2023

Love it!! VERY thorough and a couple details I didn't know.

I definitely will bring the icons back to the bundles... I already do for the nav:
image

@leggett
Copy link
Owner

leggett commented Nov 16, 2023

I played some with category:^smartlabel_receipt on a few accounts and have mostly found the opposite of you: there are definitely orders in there that are otherwise missed but it comes with a LOT of false positives. More to consider if/when I automate setting up filters so you can move email in/out of these categories in Gmail.

@davelavoie
Copy link

Yeah, I've compared the results with 3 different accounts, and I can see at least a few differences. The main difference is that category:^assistive_purchase (which is the same as category:purchases) returns emails where Gmail is able to display a summary of the purchase or the shipment at the top of the page, such as this:

image

This is the case for almost all of the emails categorized as ^assistive_purchase and this behavior is the same for all of the accounts I've tested. So in this case, it's almost certain that emails that are categorized under category:purchases are indeed related to purchases.

Regarding the results of category:^smartlabel_receipt, on my business account, almost all my purchase-related emails fall under that category. And none of them is displaying the summary like on the image above. But almost all of them are purchase-related: they are mostly Amazon purchases, shipment notices, and a few ones are newsletter related emails and promotional emails that should have been categorized under promo, but again, I think those emails might have been categorized as purchases because of my own filters.

However, what's interesting is that the emails that end up under the ^smartlabel_receipt category are more often than not emails that weren't sent to me directly. In this specific case, we use Google groups with generic emails such as "[email protected]", so all purchases end up in a shared inbox. They are then redirected to my Gmail inbox via Google Group, but most of the time, the name of the sender is rewritten to something such as 'Amazon.ca' via Purchases, and the sender email is the one of the Google group such as [email protected]. Those emails all display the famous message "You received this message because you are subscribed to the Google Groups 'Purchases' group" in the footer of the email. When it happens, it seems to kill the "assistive purchase" feature that display the order or shipment summary generated by Google at the top of the email.

That might explain why almost all my purchases are only available under the ^smartlabel_receipt category and not the ^assistive_purchase. I couldn't say that I'm an edge case, but I suppose that's not something that will happen with most people. But based on this, I think we can assume the following: both ^smartlabel_receipt and ^assistive_purchase categories are related more or less the same way finance and updates categories are related.

Searching for the following queries returns the following results with various accounts :

Case Category Search Query Account A Account B Account C
# 1 ^smartlabel_receipt category:^smartlabel_receipt 748 emails 280 emails 6443 emails
# 2 ^smartlabel_receipt only category:^smartlabel_receipt AND NOT category:^assistive_purchase 567 emails 233 emails 6281 emails
# 3 ^assistive_purchase category:^assistive_purchase 182 emails 47 emails 239 emails
# 4 ^assistive_purchase only category:^assistive_purchase AND NOT category:^smartlabel_receipt 1 email 0 email 76 emails
# 5 Both categories at once category:^assistive_purchase AND category:^smartlabel_receipt 181 emails 47 emails 163 emails

While it seems possible that an email can be categorized under the ^assistive_purchase category without being categorized under the ^smartlabel_receipt category as well, it seems to be unlikely at best.

Account A and B are accounts where I've never set up any filters with regards to categories. The only email that can be found for the case # 4 of Account A is the oldest email of the ^assistive_purchase category and is dated of 2009, maybe at a time where Google was starting to experiment categories. My first email categorized under the ^smartlabel_receipt is dated of 2010. On the other side, Account C has a lot of filters that could interfere with the native logic of Google regarding the purchases categorization, and some of these filters were added using Google HTML classic after Inbox has been shutdown and after the Purchases and Finance categories disappeared from Gmail, so it might explain why I've got 76 emails that are categorized under ^assistive_purchase only... in this case I really think it's the results of me messing up too much with the filters in "unauthorized" or unexpected ways.

Logically, I think that when an email is categorized under the 'Purchases' category (either manually or through a filter), Google will first apply to it the ^smartlabel_receipt label. Then, if Google is able to display a summary at the top of the email, it'll also add the assistive_purchase label.

So to me, it looks like ^assistive_purchase is somewhat a child of ^smartlabel_receipt, even though the case # 4 does return results where it's possible that "orphaned" emails are categorized under ^assistive_purchase without being also categorized under ^smartlabel_receipt. So while they might not have a parent->child relation directly in the code, there's a process somewhere that binds them together 99.5% of the time.

Now, I'm not sure if it really helps, since category:purchases is still an alias of category:^assistive_purchase, which kind of makes the category:^smartlabel_receipt an illegitimate parent... 😅

@leggett
Copy link
Owner

leggett commented Nov 17, 2023

Love the detail... will read over this carefully this weekend and explore some myself.

Two quick thoughts:

  • ^assistive_purchase is not just emails Gmail can show the purchase summary at the top on. I definitely have emails in that category that don't have a summary card. But you're right that most of the emails in that category have that summary.
  • I have over 4,000 emails in category:purchases -category:^smartlabel_receipt and about 2,000 in category:^smartlabel_receipt -- no idea why it is so different. It might be I trained it or set up some rules that are invisible to me now when Google Inbox was still around.

@leggett
Copy link
Owner

leggett commented Nov 17, 2023

Oh, also...

  • I definitely have order/receipt emails that neither of these categories are catching.
  • A search that has good signal-to-noise in picking up missed emails for my main personal account: subject:(receipt OR order) -category:travel -category:purchases

@davelavoie
Copy link

I have over 4,000 emails in category:purchases -category:^smartlabel_receipt and about 2,000 in category:^smartlabel_receipt -- no idea why it is so different. It might be I trained it or set up some rules that are invisible to me now when Google Inbox was still around.

The differences we're having are mesmerizing. It looks like there's still something we didn't figure out yet.

Oh, also...

  • A search that has good signal-to-noise in picking up missed emails for my main personal account: subject:(receipt OR order) -category:travel -category:purchases

I did the same, and applied the ^smartlabel_receipt category using the trick shared earlier, and it works well!

However I tried to create a filter to target all emails categorized under ^smartlabel_receipt, and apply the ^assistive_purchase category to them, since I really wanted to make it simple and make sure all purchase-related emails are displayed when searching for category:purchases, but unfortunately, that trick didn't work. There's simply no way to assign the ^assistive_purchase category, even when trying to set it manually by using the Gmail mobile link I've shared earlier. I guess we were never allowed to play with this category in the first place....

But the more I'm playing with this, the more I think it's probably too confusing for normal users... A normal user won't open the developer tools and try to create unauthorized labels and filters like I did. The only things for which we still have control easily are the filters... So I'm beginning to think that the best way is probably to create a custom "Purchases" label that is assigned when it matches with a specific filter... And that specific filter will probably change based on user needs.

For example, the filter I use to make sure all purchase-related emails are correctly assigned to my custom label is the following. I'm using French words as well since it's my native language, and in my case I also want to assign all shipment-related emails to the Purchases label, hence I added them. But that could be totally different for someone else...

category:^smartlabel_receipt OR category:^assistive_purchase OR subject:(invoice OR receipt OR payment OR shipped OR order OR bill OR delivered OR purchase OR facture OR reçu OR commande OR livraison OR paiement OR colis OR livré OR achat)

Based on this, I think maybe there should have the option in Simplify so the Purchases link displayed in the nav is somewhat customizable, allowing the user to choose between one of these options available through a select box, so when clicking on the Purchases category, it runs one of the following query:

  • option name: Default Purchases Category
    value: category:purchases
  • option name: Purchases & Receipts Categories
    value: category:^assistive_purchase AND category:^smartlabel_receipt
  • option names: all available labels
    value: let the user decide which label should be queried when clicking on Purchases in the nav
  • option name: Custom Query
    value: an input field where the user can paste his own custom query, or select one of the filters available

Obviously, you're not obligated to do this, it's just a suggestion. I'm able to change the way the Purchases button in the nav works in any case :-)

@leggett
Copy link
Owner

leggett commented Nov 22, 2023

So sorry I wasn't clearer... using a filter and assigning a user label is EXACTLY what I do on my own account and advocate for. Categories are applied before filters run and so you can filter on them (whereas you can not filter on labels).

I hope to add something that either walks a user through creating a filter or just does it for them. One of the fun challenges is internationalizing on any queries if I want to go beyond the category.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants