EOF while parsing a value

[createthis]

This tool works for a while, then I eventually get this error:

thread 'main' panicked at /Users/jesse/.cargo/registry/src/index.crates.io-6f17d22bba15001f/shreddit-0.9.3/src/things/post.rs:168:10:
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: <async_stream::async_stream::AsyncStream<T,U> as futures_core::stream::Stream>::poll_next
   4: shreddit::main::{{closure}}
   5: tokio::runtime::scheduler::current_thread::Context::enter
   6: tokio::runtime::context::scoped::Scoped<T>::set
   7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
   8: tokio::runtime::context::runtime::enter_runtime
   9: shreddit::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Let me know if there is anything I can do to help you debug this. I'm dabbling with rust as part of my job lately.

My version was installed from cargo, I believe:

shreddit 0.9.3

[andrewbanchich]

Hi! That implies that the response from Reddit's API is failing to deserialize into JSON here.

If you'd like to try it out yourself you could debug this by cloning this repo and updating that line to use text() instead of json(), then just dbg!() the response. It would probably be hard for me to do that myself since I've never run into that myself.

It might be a good improvement to shreddit to just use text() instead of json() in every case, then try to parse that as JSON separately so that we can error! log the value if it fails to deserialize to JSON at all. If you're interested in making a PR for this it would be appreciated, but up to you!

[createthis]

@andrewbanchich sorry for the long time replying. I did this:

diff --git a/src/things/post.rs b/src/things/post.rs
index b404e4e..25ed2d9 100644
--- a/src/things/post.rs
+++ b/src/things/post.rs
@@ -167,6 +167,17 @@ pub async fn list(client: &Client, config: &Config) -> impl Stream<Item = Post>
 
     let uri = format!("https://reddit.com/user/{username}/submitted.json{query_params}");
 
+            let res1 = client
+        .get(&uri)
+        .header("User-Agent", user_agent.clone())
+        .send()
+        .await
+        .unwrap()
+        .text()
+        .await
+        .unwrap();
+    dbg!(res1);
+
             let res: PostRes = client
         .get(&uri)
         .header("User-Agent", user_agent.clone())
@@ -177,6 +188,7 @@ pub async fn list(client: &Client, config: &Config) -> impl Stream<Item = Post>
         .await
         .unwrap();
 
+
     match res {
         PostRes::Success { data } => {
 

And I got this result:

[src/things/post.rs:179:5] res1 = ""
thread 'main' panicked at src/things/post.rs:189:10:
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  2024-07-17T16:27:41.906875Z DEBUG  No shreddit.env config file found.
    at src/main.rs:37

  2024-07-17T16:27:42.744979Z  INFO  Shredding Posts...
    at src/main.rs:103

  2024-07-17T16:27:42.745171Z  INFO  Fetching posts...
    at src/things/post.rs:152
    in list

[andrewbanchich]

[src/things/post.rs:179:5] res1 = ""

That's... interesting. Reddit's API is responding with an empty body. Can you try to dbg! send().await.unwrap().headers()?

[createthis]

@andrewbanchich sure:

diff --git a/src/things/post.rs b/src/things/post.rs
index b404e4e..7904876 100644
--- a/src/things/post.rs
+++ b/src/things/post.rs
@@ -167,6 +167,16 @@ pub async fn list(client: &Client, config: &Config) -> impl Stream<Item = Post>
 
     let uri = format!("https://reddit.com/user/{username}/submitted.json{query_params}");
 
+            let res1 = client
+        .get(&uri)
+        .header("User-Agent", user_agent.clone())
+        .send()
+        .await
+        .unwrap();
+
+    let headers = res1.headers();
+    dbg!(headers);
+
             let res: PostRes = client
         .get(&uri)
         .header("User-Agent", user_agent.clone())
@@ -177,6 +187,7 @@ pub async fn list(client: &Client, config: &Config) -> impl Stream<Item = Post>
         .await
         .unwrap();
 
+
     match res {
         PostRes::Success { data } => {
 

output:

  2024-07-18T23:56:54.929813Z  INFO  Fetching posts...
    at src/things/post.rs:152
    in list

[src/things/post.rs:178:5] headers = {
    "x-ratelimit-used": "100",
    "x-ratelimit-remaining": "0.0",
    "x-ratelimit-reset": "185",
    "accept-ranges": "bytes",
    "date": "Thu, 18 Jul 2024 23:56:54 GMT",
    "via": "1.1 varnish",
    "vary": "Accept-Encoding",
    "strict-transport-security": "max-age=31536000; includeSubdomains",
    "x-content-type-options": "nosniff",
    "x-frame-options": "SAMEORIGIN",
    "x-xss-protection": "1; mode=block",
    "set-cookie": "csv=2; Max-Age=63072000; Domain=.reddit.com; Path=/; Secure; SameSite=None",
    "set-cookie": "edgebucket=CwfR1D5v6eLYf2STio; Domain=reddit.com; Max-Age=63071999; Path=/;  secure",
    "cache-control": "private, max-age=3600",
    "server": "snooserv",
    "report-to": "{\"group\": \"w3-reporting-nel\", \"max_age\": 14400, \"include_subdomains\": true,  \"endpoints\": [{ \"url\": \"https://w3-reporting-nel.reddit.com/reports\" }]}, {\"group\": \"w3-reporting\", \"max_age\": 14400, \"include_subdomains\": true, \"endpoints\": [{ \"url\": \"https://w3-reporting.reddit.com/reports\" }]}, {\"group\": \"w3-reporting-csp\", \"max_age\": 14400, \"include_subdomains\": true, \"endpoints\": [{ \"url\": \"https://w3-reporting-csp.reddit.com/reports\" }]}",
    "nel": "{\"report_to\": \"w3-reporting-nel\", \"max_age\": 14400, \"include_subdomains\": false, \"success_fraction\": 1.0, \"failure_fraction\": 1.0}",
    "content-length": "0",
}
thread 'main' panicked at src/things/post.rs:188:10:
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  2024-07-18T23:56:55.260795Z DEBUG  No shreddit.env config file found.
    at src/main.rs:37

[Avamander]

I've stumbled upon the same issue I think.

  2024-10-01T21:30:14.513765Z DEBUG  No shreddit.env config file found.
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/shreddit-0.9.3/src/main.rs:37

  2024-10-01T21:30:15.855340Z  INFO  Shredding Posts...
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/shreddit-0.9.3/src/main.rs:103

  2024-10-01T21:30:15.855402Z  INFO  Fetching posts...
    at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/shreddit-0.9.3/src/things/post.rs:142
    in list

thread 'main' panicked at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/shreddit-0.9.3/src/things/post.rs:168:10:
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace