Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream config toml file uses u64 #14

Open
tasleson opened this issue Nov 13, 2023 · 3 comments · May be fixed by #15
Open

Stream config toml file uses u64 #14

tasleson opened this issue Nov 13, 2023 · 3 comments · May be fixed by #15

Comments

@tasleson
Copy link
Contributor

Each stream in the archive has an associated <stream id>.toml that has the following fields:

pub struct StreamConfig {
    pub name: Option<String>,
    pub source_path: String,
    pub pack_time: toml::value::Datetime,
    pub size: u64,
    pub mapped_size: u64,
    pub packed_size: u64,
    pub thin_id: Option<u32>,
}

A simple example:

use serde_derive::{Deserialize, Serialize};

#[derive(Deserialize, Serialize, PartialEq)]
struct Config {
    big_num: u64,
}


fn main() {
    let cfg_start =  Config {big_num: u64::MAX};
    let string_rep = toml::to_string(&cfg_start).unwrap();
    let cfg_end = toml::from_str(&string_rep).unwrap();
    assert!(cfg_start == cfg_end);
}

The problem is the toml code expresses numeric values as i64. If you try to read back a toml file that has values which exceed 2**63-1 you'll get something like the following:

Error { inner: ErrorInner { kind: NumberInvalid, line: Some(0), col: 10, at: Some(10), message: "", key: [] } }
This is when you're using toml version "0.5.8".

If you use the latest version of toml, you get the following error when you try to serialize:
Error { inner: OutOfRange(Some("u64")) }

@tasleson
Copy link
Contributor Author

From looking at the implementation, it seems like this should be supported...

@tasleson
Copy link
Contributor Author

OK, this is expected, ref. toml-rs/toml#512

We need a different on disk format which supports all data types.

@jthornber
Copy link
Owner

jthornber commented Nov 14, 2023 via email

@tasleson tasleson linked a pull request Nov 14, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants