[Bug]: Error spam due to prometheus metrics trying to be updated for non-premium user

[mikstew]

What happened?

We are using a non-premium installation of LiteLLM proxy, which means the prometheus metrics are not created based on this code.
It looks like the issue is that this function call is trying to update metrics that don't exist, resulting in a bunch of error spam.

Relevant log output

{"message": "LiteLLM.LoggingError: [Non-Blocking] Exception occurred while success logging Traceback (most recent call last):\n  File \"/usr/local/lib/python3.13/site-packages/litellm/litellm_core_utils/litellm_logging.py\", line 1618, in async_success_handler\n    await callback.async_log_success_event(\n    ...<4 lines>...\n    )\n  File \"/usr/local/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 396, in async_log_success_event\n    self._increment_top_level_request_and_spend_metrics(\n    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n        end_user_id=end_user_id,\n        ^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<7 lines>...\n        enum_values=enum_values,\n        ^^^^^^^^^^^^^^^^^^^^^^^^\n    )\n    ^\n  File \"/usr/local/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 575, in _increment_top_level_request_and_spend_metrics\n    self.litellm_requests_metric.labels(**_labels).inc()\n    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: 'PrometheusLogger' object has no attribute 'litellm_requests_metric'\n", "level": "ERROR", "timestamp": "2025-01-13T17:41:55.490887"}
{"message": "prometheus Layer Error(): Exception occured - 'PrometheusLogger' object has no attribute 'litellm_proxy_total_requests_metric'", "level": "ERROR", "timestamp": "2025-01-13T17:41:55.479797", "stacktrace": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 812, in async_post_call_success_hook\n    self.litellm_proxy_total_requests_metric.labels(**_labels).inc()\n    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: 'PrometheusLogger' object has no attribute 'litellm_proxy_total_requests_metric'"}

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.58.0

Twitter / LinkedIn details

No response

[krrishdholakia]

@mikstew curious why prometheus is being added as a callback.

What does your config look like?

[mikstew]

Yeah, I can't figure out why it's doing that. Here's our config:

litellm_settings: {
    cache: false,
    json_logs: true,
    success_callback: [
        'langfuse'
     ]
}

[krrishdholakia]

able to repro

[mikstew]

We'll revert to an older version until this can get fixed.

[krrishdholakia]

Hey @mikstew do you still see this on latest? i don't see this anymore

[mikstew]

Ahh, I had been watching this issue thread to see if a fix was merged. Let me update and check.

[mikstew]

We updated to helm chart version 0.1.588, which deployed app v1.59.3, and are still seeing the same/similar error.

litellm {"message": "prometheus Layer Error(): Exception occured - 'PrometheusLogger' object has no attribute 'litellm_proxy_total_requests_metric'", "level":
 "ERROR", "timestamp": "2025-01-23T02:01:01.822247", "stacktrace": "Traceback (most recent call last):\n  File \"/usr/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 853, in async_post_call_success_hook\n    self.litellm_proxy_total_requests_metric.labels(**_labels).inc()\n    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: 'PrometheusLogger' object has no attribute 'litellm_proxy_total_requests_metric'"}
litellm {"message": "LiteLLM.LoggingError: [Non-Blocking] Exception occurred while success logging Traceback (most recent call last):\n  File \"/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/litellm_logging.py\", line 1679, in async_success_handler\n    await callback.async_log_success_event(\n    ...
<4 lines>...\n    )\n  File \"/usr/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 431, in async_log_success_event\n    self._increment_top_level_request_and_spend_metrics(\n    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n        end_user_id=end_user_id,\n        ^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<7 lines>...\n        enum_values=enum_values,\n        ^^^^^^^^^^^^^^^^^^^^^^^^\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/integrations/prometheus.py\", line 621, in _increment_top_level_request_and_spend_metrics\n    self.litellm_requests_metric.labels(**_labels).inc()\n    ^
^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: 'PrometheusLogger' object has no attribute 'litellm_requests_metric'\n", "level": "ERROR", "timestamp": "2025-01-23T02:01:01.837357"}

[mikstew]

Hi @krrishdholakia. Were you able to still repro after your change? We're still seeing the error.

[krrishdholakia]

not seeing it - can i see your config? maybe there's a specific flow i need to run

[mikstew]

Here's our config:
(Trimmed down models list to make it easier to read and redacted some random items)

                values: {
                    proxy_config: {
                        model_list: [
                            {
                                model_name: 'gpt-3.5-turbo',
                                litellm_params: {
                                    model: 'gpt-3.5-turbo',
                                    api_key: 'os.environ/OPENAI_API_KEY',
                                }
                            },
                            {
                                model_name: 'gpt-4-turbo',
                                litellm_params: {
                                    model: 'gpt-4-turbo-2024-04-09',
                                    api_key: 'os.environ/OPENAI_API_KEY',
                                }
                            },
                            ... // trimmed down the models list
                        ],
                        litellm_settings: {
                            cache: false,
                            json_logs: true,
                            success_callback: [
                                'langfuse'
                            ]
                        },
                        router_settings: {
                            redis_host: '*****',
                            redis_port: 6379,
                            redis_password: '*****',
                        },
                        environment_variables: {
                            LANGFUSE_HOST: 'https://us.cloud.langfuse.com'
                        },
                        general_settings: {
                            master_key: 'os.environ/LITELLM_MASTER_KEY',
                            alerting: ["slack"],
                            store_model_in_db: true
                        }
                    },
                    masterkey: '*****',
                    environmentSecrets: [ `litellm-${env}` ],
                    db: {
                        useExisting: true,
                        endpoint: '*****',
                        database: 'litellm',
                        secret: {
                            name: `llm-ops-${env}`,
                            usernameKey: 'LITELLM_DB_USER',
                            passwordKey: 'LITELLM_DB_PASSWORD'
                        },
                        deployStandalone: false
                    },
                    resources: {
                        limits: {
                            cpu: '1',
                            memory: '2Gi'
                        },
                        requests: {
                            cpu: '500m',
                            memory: '2Gi'
                        }
                    },
                    autoscaling: {
                        enabled: true,
                        minReplicas: env.includes('prod') ? 3 : 2,
                        maxReplicas: 5,
                        targetCPUUtilizationPercentage: 80
                    },
                    migrationJob: {
                        annotations: {
                            'sidecar.istio.io/inject': 'false'
                        }
                    }
                }