Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why the shapefile read .shp Slower than shapefile(pyshp) of python? It doesn't make #35

Open
kl402401 opened this issue Apr 28, 2024 · 12 comments

Comments

@kl402401
Copy link

it don't make any sense

@kl402401
Copy link
Author

@tmontaigu

@tmontaigu
Copy link
Owner

tmontaigu commented Apr 28, 2024

What about sharing the code you use ? and how you compile, and the file you try to read

Your issue doesn't make sense

@tmontaigu
Copy link
Owner

@kl402401

@tmontaigu
Copy link
Owner

no info = no bug = no fix

@tmontaigu tmontaigu closed this as not planned Won't fix, can't repro, duplicate, stale May 4, 2024
@kl402401
Copy link
Author

kl402401 commented May 6, 2024

no info = no bug = no fix

sorry ,it maybe not a bug, but i use the pyshp of python to read and iterate through the .shp that it has 96w shp, it just cost 30secs,but the same .shp, use shapefile of rust ,it cost 180secs

@tmontaigu
Copy link
Owner

tmontaigu commented May 6, 2024

I can't do anything about that unless you give some actual info

  • What is the rust code you wrote ?
  • How do you run/compile it (don't forget the --release flag)
  • Are you able to share the file itself ?

@kl402401
Copy link
Author

kl402401 commented May 6, 2024

sorry ,I can't share the file ,it's a secret file.

.it's a road.shp of the beijing that one of the city in china. it may vary big, it has 9.6 million shape in one .shp。

.the rust code just like that, i build it with --release flag:
use std::time::{Instant};
fn main() {
let start = Instant::now();
let mut reader = shapefile::Reader::from_path("D:\road0714out\result1\beijing\road.shp").unwrap();

for result in reader.iter_shapes_and_records() {
    let (shape, record) = result.unwrap();
    println!("Shape: {}, records: ", shape);
    for (name, value) in record {
        println!("\t{}: {:?}, ", name, value);
    }
    println!();
}
let duration = start.elapsed();
let seconds = duration.as_secs();

println!("Execution time in seconds: {}", seconds);

}

@kl402401
Copy link
Author

kl402401 commented May 6, 2024

Execution time in seconds: 1114 this time it cost 1114secs, is that the code problem? that maybe a little slow

@tmontaigu
Copy link
Owner

Does the file have an index file road.shx that goes along with it ?
If so I think I have an idea for the where slowness comes from and know how to fix

@tmontaigu
Copy link
Owner

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

@tmontaigu tmontaigu reopened this May 6, 2024
@kl402401
Copy link
Author

kl402401 commented May 7, 2024

Does the file have an index file road.shx that goes along with it ? If so I think I have an idea for the where slowness comes from and know how to fix

it has a 'road.shx'.

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

That's really great,can you update the project?

@tmontaigu
Copy link
Owner

Fix merged in #36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants