fixes for CR

membraneframework · Sep 21, 2023 · 908fb05 · 908fb05
1 parent edcc97f
commit 908fb05
Show file tree

Hide file tree

Showing 3 changed files with 54 additions and 46 deletions.
diff --git a/.formatter.exs b/.formatter.exs
@@ -11,7 +11,7 @@ locals_without_parens =
 [
   inputs: [
     "{lib,test,config,benchmark}/**/*.{ex,exs}",
-    "mix.exs"
+    "*.exs"
   ],
   locals_without_parens: locals_without_parens,
   export: [

diff --git a/README.md b/README.md
@@ -23,6 +23,8 @@ Want a generic media server, instead of building a custom one? Try [Jellyfish](h
 
 If you have questions or need consulting, we're for you at our [Discord](https://discord.gg/nwnfVSY), [forum](https://elixirforum.com/c/elixir-framework-forums/membrane-forum/), [GitHub discussions](https://github.com/orgs/membraneframework/discussions), [X (Twitter)](https://twitter.com/ElixirMembrane) and via [e-mail](mailto:[email protected]).
 
+You can also [follow Membrane on X (Twitter)](https://twitter.com/ElixirMembrane) or [join our Discord](https://discord.gg/nwnfVSY) to be up to date and get involved in the community.
+
 Membrane is maintained by [Software Mansion](swmansion.com).
 
 ## Quick start
@@ -84,7 +86,7 @@ Elements can be organized into a pipeline - a sequence of linked elements that p
 
 ### Membrane packages
 
-To embrace modularity, Membrane is delivered to you in multiple packages, including plugins, formats, core and standalone libraries. The complete list of all the Membrane packages maintained by the Membrane team is available [here](https://github.com/membraneframework/membrane_core/Membrane-packages).
+To embrace modularity, Membrane is delivered to you in multiple packages, including plugins, formats, core and standalone libraries. The complete list of all the Membrane packages maintained by the Membrane team is available [here](#Membrane-packages).
 
 **Plugins**
 
@@ -131,11 +133,11 @@ We welcome everyone to contribute to Membrane. Here are some ways to contribute:
 - Spread the word about Membrane! Even though multimedia are present everywhere today, media dev is still quite niche. Let it be no longer!
 - Create learning materials. We try our best but can cover only a limited number of Membrane use cases.
 - Improve docs. We know it's not the most exciting part, but if you had a hard time understanding the docs, you're the best person to fix them ;)
-- Contribute code - plugins, features and bug fixes. It's best to contact us before, so we can provide our help & assistance, and agree on important matters. For details see the [contribution guide](CONTRIBUTING.md)
+- Contribute code - plugins, features and bug fixes. It's best to contact us before, so we can provide our help & assistance, and agree on important matters. For details see the [contribution guide](CONTRIBUTING.md).
 
 ## Support and questions
 
-If you have any problems with Membrane Framework feel free to contact us via [Discord](https://discord.gg/nwnfVSY), [forum](https://elixirforum.com/c/elixir-framework-forums/membrane-forum/), [GitHub discussions](https://github.com/orgs/membraneframework/discussions), [X (Twitter)](https://twitter.com/ElixirMembrane) or [e-mail](mailto:[email protected]).
+If you have any questions regarding Membrane Framework or need consulting, feel free to contact us via [Discord](https://discord.gg/nwnfVSY), [forum](https://elixirforum.com/c/elixir-framework-forums/membrane-forum/), [GitHub discussions](https://github.com/orgs/membraneframework/discussions), [X (Twitter)](https://twitter.com/ElixirMembrane) or [e-mail](mailto:[email protected]).
 
 ## All packages
 
@@ -147,6 +149,7 @@ If you have any problems with Membrane Framework feel free to contact us via [Di
 | Package | Description | Links |
 | --- | --- | --- |
 | [membrane_core](https://github.com/membraneframework/membrane_core) | The core of the Membrane Framework, advanced multimedia processing framework | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_core.svg)](https://hex.pm/api/packages/membrane_core) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_core/) |
+| [membrane_rtc_engine](https://github.com/jellyfish-dev/membrane_rtc_engine) | [Maintainer: [jellyfish-dev](https://github.com/jellyfish-dev)] Customizable Real-time Communication Engine/SFU library focused on WebRTC. | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtc_engine.svg)](https://hex.pm/api/packages/membrane_rtc_engine) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtc_engine/) |
 | [kino_membrane](https://github.com/membraneframework/kino_membrane) | Utilities for introspecting Membrane pipelines in Livebook | [![Hex.pm](https://img.shields.io/hexpm/v/kino_membrane.svg)](https://hex.pm/api/packages/kino_membrane) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/kino_membrane/) |
 | [docker_membrane](https://github.com/membraneframework-labs/docker_membrane) | [Labs] A docker image based on Ubuntu, with Erlang, Elixir and libraries necessary to test and run the Membrane Framework. |   |
 | [membrane_demo](https://github.com/membraneframework/membrane_demo) | Examples of using the Membrane Framework |   |
@@ -158,7 +161,6 @@ If you have any problems with Membrane Framework feel free to contact us via [Di
 | Package | Description | Links |
 | --- | --- | --- |
 | [membrane_file_plugin](https://github.com/membraneframework/membrane_file_plugin) | Membrane plugin for reading and writing to files | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_file_plugin.svg)](https://hex.pm/api/packages/membrane_file_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_file_plugin/) |
-| [membrane_udp_plugin](https://github.com/membraneframework/membrane_udp_plugin) | Membrane plugin for sending and receiving UDP streams | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_udp_plugin.svg)](https://hex.pm/api/packages/membrane_udp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_udp_plugin/) |
 | [membrane_hackney_plugin](https://github.com/membraneframework/membrane_hackney_plugin) | HTTP sink and source based on Hackney | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_hackney_plugin.svg)](https://hex.pm/api/packages/membrane_hackney_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_hackney_plugin/) |
 | [membrane_scissors_plugin](https://github.com/membraneframework/membrane_scissors_plugin) | Element for cutting off parts of the stream | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_scissors_plugin.svg)](https://hex.pm/api/packages/membrane_scissors_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_scissors_plugin/) |
 | [membrane_tee_plugin](https://github.com/membraneframework/membrane_tee_plugin) | Membrane plugin for splitting data from a single input to multiple outputs | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_tee_plugin.svg)](https://hex.pm/api/packages/membrane_tee_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_tee_plugin/) |
@@ -176,6 +178,8 @@ If you have any problems with Membrane Framework feel free to contact us via [Di
 | [membrane_webrtc_plugin](https://github.com/jellyfish-dev/membrane_webrtc_plugin) | [Maintainer: [jellyfish-dev](https://github.com/jellyfish-dev)] Membrane plugin for sending and receiving media with WebRTC | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_webrtc_plugin.svg)](https://hex.pm/api/packages/membrane_webrtc_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_webrtc_plugin/) |
 | [membrane_rtmp_plugin](https://github.com/membraneframework/membrane_rtmp_plugin) | RTMP server & client | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtmp_plugin.svg)](https://hex.pm/api/packages/membrane_rtmp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtmp_plugin/) |
 | [membrane_http_adaptive_stream_plugin](https://github.com/membraneframework/membrane_http_adaptive_stream_plugin) | Plugin generating manifests for HLS (DASH support planned) | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_http_adaptive_stream_plugin.svg)](https://hex.pm/api/packages/membrane_http_adaptive_stream_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_http_adaptive_stream_plugin/) |
+| [membrane_ice_plugin](https://github.com/jellyfish-dev/membrane_ice_plugin) | [Maintainer: [jellyfish-dev](https://github.com/jellyfish-dev)] Membrane plugin for ICE protocol | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_ice_plugin.svg)](https://hex.pm/api/packages/membrane_ice_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_ice_plugin/) |
+| [membrane_udp_plugin](https://github.com/membraneframework/membrane_udp_plugin) | Membrane plugin for sending and receiving UDP streams | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_udp_plugin.svg)](https://hex.pm/api/packages/membrane_udp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_udp_plugin/) |
 | [membrane_rtp_plugin](https://github.com/membraneframework/membrane_rtp_plugin) | Membrane bins and elements for sending and receiving RTP/SRTP and RTCP/SRTCP streams | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_plugin/) |
 | [membrane_rtp_h264_plugin](https://github.com/membraneframework/membrane_rtp_h264_plugin) | Membrane RTP payloader and depayloader for H264 | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_h264_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_h264_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_h264_plugin/) |
 | [membrane_rtp_vp8_plugin](https://github.com/membraneframework/membrane_rtp_vp8_plugin) | Membrane elements for payloading and depayloading VP8 into RTP | [![Hex.pm](https://img.shields.io/hexpm/v/membrane_rtp_vp8_plugin.svg)](https://hex.pm/api/packages/membrane_rtp_vp8_plugin) [![Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_rtp_vp8_plugin/) |

diff --git a/update_packages_list.exs b/update_packages_list.exs
@@ -7,14 +7,14 @@ packages =
   [
     {:md, "### General"},
     "membrane_core",
+    "membrane_rtc_engine",
     "kino_membrane",
     "docker_membrane",
     "membrane_demo",
     "membrane_tutorials",
     {:md, "### Plugins"},
     {:md, "#### General purpose"},
     "membrane_file_plugin",
-    "membrane_udp_plugin",
     "membrane_hackney_plugin",
     "membrane_scissors_plugin",
     "membrane_tee_plugin",
@@ -29,6 +29,8 @@ packages =
     "membrane_webrtc_plugin",
     "membrane_rtmp_plugin",
     "membrane_http_adaptive_stream_plugin",
+    "membrane_ice_plugin",
+    "membrane_udp_plugin",
     "membrane_rtp_plugin",
     "membrane_rtp_h264_plugin",
     "membrane_rtp_vp8_plugin",
@@ -137,49 +139,51 @@ gh_req_mock = false
 repos =
   ["membraneframework", "membraneframework-labs", "jellyfish-dev"]
   |> Enum.flat_map(fn org ->
-    Process.sleep(gh_req_timeout)
-
-    Req.get!(
-      "https://api.github.com/orgs/#{org}/repos?per_page=100",
-      decode_json: [keys: :atoms]
-    ).body
+    Stream.iterate(1, &(&1 + 1))
+    |> Stream.map(fn page ->
+      Process.sleep(gh_req_timeout)
+      url = "https://api.github.com/orgs/#{org}/repos?per_page=100&page=#{page}"
+      IO.puts("Fetching #{url}")
+      Req.get!(url, decode_json: [keys: :atoms]).body
+    end)
+    |> Enum.take_while(&(&1 != []))
+    |> Enum.flat_map(& &1)
   end)
   |> Map.new(&{&1.name, &1})
 
-
 # find repos from the membraneframework organization that aren't in the list
 
 package_names =
-  packages |> Enum.filter(&(&1.type == :package)) |> Enum.map(& &1.name) |> MapSet.new()
+  packages |> Enum.filter(&(&1.type == :package)) |> MapSet.new(& &1.name)
+
+packages_blacklist = [
+  "circleci-orb",
+  "guide",
+  "design-system",
+  ~r/.*_tutorial/,
+  "membrane_resources",
+  "membrane_gigachad",
+  "static",
+  "membrane_videoroom",
+  ".github",
+  "membraneframework.github.io",
+  "membrane_rtc_engine_timescaledb"
+]
 
 lacking_repos =
   repos
   |> Map.values()
-  |> Enum.filter(&(&1.owner.login == "membraneframework"))
-  |> Enum.map(& &1.name)
-  |> Enum.reject(&(&1 in package_names))
-  |> Enum.reject(
-    &Enum.any?(
-      [
-        "circleci-orb",
-        "guide",
-        "design-system",
-        ~r/.*_tutorial/,
-        "membrane_resources",
-        "membrane_gigachad",
-        "static",
-        "membrane_videoroom",
-        ".github",
-        "membraneframework.github.io"
-      ],
-      fn repo -> &1 =~ repo end
-    )
-  )
+  |> Enum.filter(fn repo ->
+    repo.name not in package_names and
+      repo.owner.login in ["membraneframework", "jellyfish-dev"] and
+      (repo.owner.login == "membraneframework" or repo.name =~ ~r/^membrane_.*/) and
+      not Enum.any?(packages_blacklist, fn name -> repo.name =~ name end)
+  end)
 
 unless Enum.empty?(lacking_repos) do
   Logger.warning("""
-  The following repositories from the membraneframework organization aren't mentioned in the package list:
-  #{Enum.join(lacking_repos, ",\n")}
+  The following repositories aren't mentioned in the package list:
+  #{Enum.map_join(lacking_repos, ",\n", & &1.name)}
   """)
 end
 
@@ -197,23 +201,23 @@ packages =
 
           :error when owner != nil ->
             Process.sleep(gh_req_timeout)
-            IO.puts("Fetching https://api.github.com/repos/#{owner}/#{name}")
-
-            Req.get!("https://api.github.com/repos/#{owner}/#{name}", decode_json: [keys: :atoms]).body
+            url = "https://api.github.com/repos/#{owner}/#{name}"
+            IO.puts("Fetching #{url}")
+            Req.get!(url, decode_json: [keys: :atoms]).body
 
           :error ->
             raise "Package #{inspect(name)} repo not found, please specify owner."
         end
 
       hex = Req.get!("https://hex.pm/api/packages/#{name}", decode_json: [keys: :atoms])
-      hex_present = hex.status == 200
+      is_hex_present = hex.status == 200
 
       Map.merge(package, %{
         owner: repo.owner.login,
         url: repo.html_url,
         description: repo.description,
-        hex_url: if(hex_present, do: hex.body.url),
-        hexdocs_url: if(hex_present, do: hex.body.docs_html_url)
+        hex_url: if(is_hex_present, do: hex.body.url),
+        hexdocs_url: if(is_hex_present, do: hex.body.docs_html_url)
       })
 
     other ->
@@ -229,9 +233,9 @@ header = """
 
 packages_md =
   packages
-  |> Enum.map_reduce(%{header_present: false}, fn
+  |> Enum.map_reduce(%{is_header_present: false}, fn
     %{type: :markdown, content: content}, acc ->
-      {"\n#{content}", %{acc | header_present: false}}
+      {"\n#{content}", %{acc | is_header_present: false}}
 
     %{type: :package} = package, acc ->
       prefix =
@@ -254,11 +258,11 @@ packages_md =
       url = "[#{package.name}](#{package.url})"
 
       result = """
-      #{if acc.header_present, do: "", else: header}\
+      #{if acc.is_header_present, do: "", else: header}\
       | #{url} | #{prefix}#{package.description} | #{hex_badge} #{hexdocs_badge} |\
       """
 
-      {result, %{acc | header_present: true}}
+      {result, %{acc | is_header_present: true}}
   end)
   |> elem(0)
   |> Enum.join("\n")