list index out of range after calling .related_queries() method

[ivanprjcts]

Hi :)

It looks like .related_queries() method is broken.
Sample code:

from pytrends.request import TrendReq


keywords = ['real madrid']
pytrends = TrendReq(hl='en-US', geo='US')
pytrends.build_payload(keywords)
related_queries_df = pytrends.related_queries()

Traceback:

Traceback (most recent call last):
  ...
  File ".../pytrends/request.py", line 450, in related_queries
    req_json['default']['rankedList'][0]['rankedKeyword'])
IndexError: list index out of range

[sparklette]

I have the same problem. My guess is that the API has changed. Is there a workaround?

[tanapa2018]

Received data: {'default': {'rankedList': []}}
pytrends.request: No rankedList data found
pytrends.request: No second rankedList data found for rising queries

On the evening of the 19th, my request was successful, but from the 20th onwards, I got an error: IndexError: list index out of range

[wxqqh]

I have the same problem.

[hsuan619]

same problem here. It has been 2 days.
Maybe both of related_queries() related_topics() are broken.

[SpearsonWa83643]

indeed both related_queries() and related_topics() are not working, Google returns 200 but data returned is {'default': {'rankedList': []}} which causes the index out of range issue.
if api changed why would Google send back 200?
has anyone figured out any patches yet?

[FelipeAlvarezRodrigues]

I have the same problem.
Only by related queries.

[Toffee1878]

I have the very same issue with Related topics & Related queries...

Are there any updates? Same issues when testing locally or via hosted web page.

[Yuhsuant1994]

still facing the same issue... anyone have any workaround?

[linda-tong-dentsu]

same issue here... any updates?

[wakeisle9933]

Not only related_queries(), but also related_topics() are the same

[shawsa-artc]

same here, it's broken

[dbitton]

Same problem here, related_queries() endpoint is broken, does anyone support this ?

[hcdc-ap]

Same here, it's not working

[st1vms]

By inspecting the code I found the reason why it is failing.

This error comes from when retrieving tokens, apparently Google inserted a challenge that gets you flagged as USER_TYPE_SCRAPER whenever you fail it.
Once a token/request is retrieved with this flag, calling the data endpoints will return empty json.

In order to work the server must return tokens with a request having USER_TYPE_LEGIT_USER in it.
So in order to make this library work again, the bot-challenge needs to be solved.

[NickWang322]

same here, any updates?

[douglastehling]

Has anyone managed to solve this problem?

[philippe-heitzmann]

Hi, has anyone managed to solve this for related_queries?

[CHHLeo]

After several tests, I found that google only allowed logged in account to access related queries and related topics, so there is no way to use any scrapper to get those infos now.

[sdil87]

After several tests, I found that google only allowed logged in account to access related queries and related topics, so there is no way to use any scrapper to get those infos now.

This is not entirely accurate. In trendspy, I think I’ve managed to work around this limitation. You can access related queries and topics without needing a logged-in account. I’ve successfully tested it with at least 600 sequential requests (with 1-second pauses), and it worked without issues. That said, if you send too many requests too quickly, you might still run into a 429 error, so pauses are recommended.

Here’s a quick example of how it works:

!pip install trendspy
from trendspy import Trends
tr = Trends()
related = tr.related_queries('python')
print(related)

[CHHLeo]

After several tests, I found that google only allowed logged in account to access related queries and related topics, so there is no way to use any scrapper to get those infos now.

This is not entirely accurate. In trendspy, I think I’ve managed to work around this limitation. You can access related queries and topics without needing a logged-in account. I’ve successfully tested it with at least 600 sequential requests (with 1-second pauses), and it worked without issues. That said, if you send too many requests too quickly, you might still run into a 429 error, so pauses are recommended.

Here’s a quick example of how it works:

!pip install trendspy
from trendspy import Trends
tr = Trends()
related = tr.related_queries('python')
print(related)

It's work! Thanks for your help, you did a great job.

[douglastehling]

pip install trendspy

@sdil87 I'm receiving this error. Do You know what is the problem?

Traceback (most recent call last):
  File "E:\Learning\trendspy\main.py", line 3, in <module>
    related = tr.related_queries('python')
  File "E:\Learning\trendspy\venv\lib\site-packages\trendspy\client.py", line 412, in related_queries
    token, data = self._get_token_data(EMBED_QUERIES_URL, locals(), headers=headers, raise_quota_error=True)
  File "E:\Learning\trendspy\venv\lib\site-packages\trendspy\client.py", line 308, in _get_token_data
    raise TrendsQuotaExceededError()
trendspy.client.TrendsQuotaExceededError: API quota exceeded for related queries/topics. To resolve this, you can try:
1. Use a different referer in request headers:
   tr.related_queries(keyword, headers={'referer': 'https://www.google.com/'})
2. Use a different IP address by configuring a proxy:
   tr.set_proxy('http://proxy:port')
   # or
   tr = Trends(proxy={'http': 'http://proxy:port', 'https': 'https://proxy:port'})
3. Wait before making additional requests

[sdil87]

pip install trendspy

@sdil87 I'm receiving this error. Do You know what is the problem?

Traceback (most recent call last):
  File "E:\Learning\trendspy\main.py", line 3, in <module>
    related = tr.related_queries('python')
  File "E:\Learning\trendspy\venv\lib\site-packages\trendspy\client.py", line 412, in related_queries
    token, data = self._get_token_data(EMBED_QUERIES_URL, locals(), headers=headers, raise_quota_error=True)
  File "E:\Learning\trendspy\venv\lib\site-packages\trendspy\client.py", line 308, in _get_token_data
    raise TrendsQuotaExceededError()
trendspy.client.TrendsQuotaExceededError: API quota exceeded for related queries/topics. To resolve this, you can try:
1. Use a different referer in request headers:
   tr.related_queries(keyword, headers={'referer': 'https://www.google.com/'})
2. Use a different IP address by configuring a proxy:
   tr.set_proxy('http://proxy:port')
   # or
   tr = Trends(proxy={'http': 'http://proxy:port', 'https': 'https://proxy:port'})
3. Wait before making additional requests

This is a HTTP 429 "Too Many Requests" error returned by Google Trends API, indicating you've exceeded their rate limiting thresholds. The error message above provides several approaches you can try to resolve this issue.