[Feature Request]: Is it possible to add audio only review/event to trigger notification? Frigate 0.14 blueprint

[longhtran91]

Describe what you are trying to accomplish and why in non technical terms
I have a camera that's used for watching the bassinet and I only configured it for crying detection. The crying detection is labeled as alert in Frigate 0.14. I want to be notified when there are alerts of this camera (crying detection only) . Here is the camera

  baby_bassinet:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/baby_bassinet
          input_args: preset-rtsp-restream
          roles:
            - record
            - audio
    motion:
      mask:
        - 1374,55,1868,55,1868,22,1374,22
    review:
      alerts:
        labels:
          - crying
    audio:
      min_volume: 600
      listen:
        - crying
      filters:
        crying:
          threshold: 0.8 #no min_score
    objects:
      track: [] 

Here is an example of the event in mqtt

{
    "type": "end",
    "before": {
        "id": "1737757739.861891-zmzexc",
        "camera": "baby_bassinet",
        "start_time": 1737757739.861891,
        "end_time": null,
        "severity": "alert",
        "thumb_path": "/media/frigate/clips/review/thumb-baby_bassinet-1737757739.861891-zmzexc.webp",
        "data": {
            "detections": [],
            "objects": [],
            "sub_labels": [],
            "zones": [],
            "audio": [
                "crying"
            ]
        }
    },
    "after": {
        "id": "1737757739.861891-zmzexc",
        "camera": "baby_bassinet",
        "start_time": 1737757739.861891,
        "end_time": 1737757742.569357,
        "severity": "alert",
        "thumb_path": "/media/frigate/clips/review/thumb-baby_bassinet-1737757739.861891-zmzexc.webp",
        "data": {
            "detections": [],
            "objects": [],
            "sub_labels": [],
            "zones": [],
            "audio": [
                "crying"
            ]
        }
    }
}

Here is my current configuration in the blueprint

alias: Baby Bassinet Crying 0.14
description: ""
use_blueprint:
  path: SgtBatten/Beta.yaml
  input:
    presence_filter:
      - ""
    camera:
      - camera.baby_bassinet
    notify_device: d50ae4e1f1351cd252adbc2379819954
    message: >-
      {{ label }} detected - {{ camera_name }} at
      {{event['after']['start_time']|timestamp_custom('%H:%M')}}
    attachment: >-
      {{base_url}}/api/frigate{{client_id}}/notifications/{{id}}/event_preview.gif
    review_severity:
      - alert
    attachment_2: ""
    tap_action: >-
      {{base_url}}/api/frigate{{client_id}}/notifications/{{id}}/{{camera}}/master.m3u8
    url_1: >-
      {{base_url}}/api/frigate{{client_id}}/notifications/{{id}}/{{camera}}/master.m3u8
    url_3: /ui-lovelace-minimalist/cctv
    button_3: Open Home Assistant
    base_url: https://ha.domain.com

[SgtBatten]

Does frigate post it in the reviews topic?

[longhtran91]

It does. The event is what I get from reviews topic

[SgtBatten]

what kind of snapshots can you get from these? It doesn't have a detection (object) so the typical id used isn't provided.

I see the thumb path there and may be able to use that.

Also, when you use the current version, what happens? any errors or a picture of the trace.

[longhtran91]

I don't think it has regular snapshot, but here is the url that I see in alert
https://frigate.domain.com/clips/review/thumb-baby_bassinet-1737847133.749676-hfmxgm.webp

I'm using the latest beta in the main branch. The automation is never triggered. Looking at all traces, but I don't see any trace having the payload from this camera, but I'm pretty sure it's in reviews topic

{
  "trace": {
    "last_step": "condition/0/conditions/2/conditions/1",
    "run_id": "2571396d992d33ae1f51fcdd4c176f97",
    "state": "stopped",
    "script_execution": "failed_conditions",
    "timestamp": {
      "start": "2025-01-26T02:21:10.108508+00:00",
      "finish": "2025-01-26T02:21:10.109700+00:00"
    },
    "domain": "automation",
    "item_id": "1737606076055",
    "trigger": "mqtt topic frigate/reviews",
    "trace": {
      "trigger/0": [
        {
          "path": "trigger/0",
          "timestamp": "2025-01-26T02:21:10.109498+00:00",
          "changed_variables": {
            "this": {
              "entity_id": "automation.baby_bassinet_crying_0_14",
              "state": "on",
              "attributes": {
                "id": "1737606076055",
                "last_triggered": null,
                "mode": "parallel",
                "current": 0,
                "max": 10,
                "friendly_name": "Baby Bassinet Crying 0.14"
              },
              "last_changed": "2025-01-23T13:47:14.984388+00:00",
              "last_reported": "2025-01-23T13:47:14.984388+00:00",
              "last_updated": "2025-01-23T13:47:14.984388+00:00",
              "context": {
                "id": "01JJ9PNGN83QH9MQDGK2GSQ2BS",
                "parent_id": null,
                "user_id": null
              }
            },
            "trigger": {
              "id": "frigate-event",
              "idx": "0",
              "alias": null,
              "platform": "mqtt",
              "topic": "frigate/reviews",
              "payload": "{\"type\": \"new\", \"before\": {\"id\": \"1737858069.852171-k1ynyt\", \"camera\": \"front_driveway\", \"start_time\": 1737858069.852171, \"end_time\": null, \"severity\": \"detection\", \"thumb_path\": \"/media/frigate/clips/review/thumb-front_driveway-1737858069.852171-k1ynyt.webp\", \"data\": {\"detections\": [\"1737858067.685219-l690f6\"], \"objects\": [\"car\"], \"sub_labels\": [], \"zones\": [\"street\"], \"audio\": []}}, \"after\": {\"id\": \"1737858069.852171-k1ynyt\", \"camera\": \"front_driveway\", \"start_time\": 1737858069.852171, \"end_time\": null, \"severity\": \"detection\", \"thumb_path\": \"/media/frigate/clips/review/thumb-front_driveway-1737858069.852171-k1ynyt.webp\", \"data\": {\"detections\": [\"1737858067.685219-l690f6\"], \"objects\": [\"car\"], \"sub_labels\": [], \"zones\": [\"street\"], \"audio\": []}}}",
              "qos": 0,
              "description": "mqtt topic frigate/reviews",
              "payload_json": {
                "type": "new",
                "before": {
                  "id": "1737858069.852171-k1ynyt",
                  "camera": "front_driveway",
                  "start_time": 1737858069.852171,
                  "end_time": null,
                  "severity": "detection",
                  "thumb_path": "/media/frigate/clips/review/thumb-front_driveway-1737858069.852171-k1ynyt.webp",
                  "data": {
                    "detections": [
                      "1737858067.685219-l690f6"
                    ],
                    "objects": [
                      "car"
                    ],
                    "sub_labels": [],
                    "zones": [
                      "street"
                    ],
                    "audio": []
                  }
                },
                "after": {
                  "id": "1737858069.852171-k1ynyt",
                  "camera": "front_driveway",
                  "start_time": 1737858069.852171,
                  "end_time": null,
                  "severity": "detection",
                  "thumb_path": "/media/frigate/clips/review/thumb-front_driveway-1737858069.852171-k1ynyt.webp",
                  "data": {
                    "detections": [
                      "1737858067.685219-l690f6"
                    ],
                    "objects": [
                      "car"
                    ],
                    "sub_labels": [],
                    "zones": [
                      "street"
                    ],
                    "audio": []
                  }
                }
              }
            },
            "mqtt_topic": "frigate/reviews",
            "input_camera": [
              "camera.baby_bassinet"
            ],
            "input_camera_name": [
              "baby_bassinet"
            ],
            "camera": "front_driveway",
            "camera_name": "Front Driveway",
            "input_severity": [
              "alert"
            ],
            "severity": "detection",
            "type": "new",
            "input_base_url": "https://ha.domain.com",
            "base_url": "https://ha.domain.com",
            "input_telegram_base_url": "",
            "telegram_base_url": "https://ha.domain.com",
            "update_sub_label": true,
            "input_client_id": "",
            "client_id": "",
            "alert_once": false,
            "final_update": false,
            "ios_live_view": "",
            "android_auto": false,
            "notify_group": "",
            "notify_group_target": "",
            "telegram_chat_id": "",
            "telegram": false,
            "zone_only": false,
            "input_zones": [],
            "zones": [],
            "zone_multi": false,
            "zone_order_enforced": false,
            "input_labels": "",
            "labels": [],
            "presence_entity": [
              ""
            ],
            "disable_times": [],
            "cooldown": 0,
            "timeout": 2,
            "loitering": false,
            "loiter_timer": 0,
            "initial_delay": 0,
            "final_delay": 5,
            "fps": 5,
            "state_only": false,
            "input_entity": "",
            "input_states": [],
            "states_filter": [],
            "custom_filter": true,
            "color": "

[SgtBatten]

Yeah that particular trace says the camera name didn't match but all that's needed to at least start a trace should be an mqtt message in the reviews topic that has a type of "new"

[longhtran91]

I see. It always starts with type "update" instead of "new"

[SgtBatten]

I see. It always starts with type "update" instead of "new"

That seems impractical. Are you certain?

[Skuair]

About this, I started a topic in the frigate discussion repos here: blakeblackshear/frigate#16366

My goal was to create manually an notification automation to include the snapshot of the associated audio event but currently the event ID is not in the mqtt message (only the review event is), so impossible to get the snapshot directly.

You will see in the answers that the dev said it could be done in a next version.
So if it is added in the mqtt response, I think the blueprint could work because the audio is in the review topic, the only change is to take the "audio" array of the response and list | join it for example.

[alexhk90]

Thanks @Skuair for posting this here and on the Frigate repo as well.
I'm seeing the same behaviour with my custom/manual events (i.e. blakeblackshear/frigate#11139) - on the "frigate/reviews" MQTT topic these events generate messages of "type": "update" and then "type": "end", but no messages of "type": "new". As a result, they don't trigger the Frigate Notifications (0.14.0.2w) automation.