implement area size heuristic for area:id queries #364

tyrasd · 2017-03-02T15:34:03Z

fixes #359. The implementation feels a bit hacky / inefficient (since we need to iterate twice over all areas now), but I didn't see another way to access the area's used_indices property. Any pointers for possible improvements are welcome.

fixes drolbr#359

mmd-osm · 2017-03-02T15:48:57Z

src/overpass_api/statements/area_query.cc

@@ -287,8 +287,20 @@ void Area_Query_Statement::get_ranges

 bool Area_Constraint::delivers_data(Resource_Manager& rman)
 {
+  int counter = 0;


int counter = 0 in line 311 should probably be removed as well, otherwise line 321 won't return the correct value I guess (variable counter gets shadowed).

right! forgot to remove that one

mmd-osm · 2017-03-02T15:51:09Z

src/overpass_api/statements/area_query.cc

+        it(area_locations_db.flat_begin());
+        !(it == area_locations_db.flat_end()); ++it)
+    {
+      if (area->get_submitted_id() == it.object().id.val())


Can you check if you could use it.handle() ... instead to get the id (avoids some unneeded object instatiation) - not super urgent.

you mean just changing it to it.handle().object().id.val(), right? For me that seems to works as well.

It probably should also be changed on line 244…

I had it.handle().id() in mind - we have something similar elsewhere but I don't know if this also works in this context without further checking.

ah, yes, that makes more sense 🙈 -> 4d9992d

I think even the .val() at the end is not needed as id() already returns the right type. Can you maybe test this again?

Then I get the following compile-time error (gcc 6.2):

error: no match for ‘operator==’ (operand types are ‘long long int’ and ‘Area_Skeleton::Id_Type {aka Uint32_Index}’

Uint32_Index has the == operator implemented only for comparing with other Uint32_Index values while Area_Query_Statement (for some reason) stores the id it gets in a 64bit integer.

But I think that calling .val() here shouldn't be an issue, since the compiler can inline/optimize that away very easily.

Ok, fine, let's keep val(). I didn't have the source available when commenting and didn't exactly remember when it was necessary.

see drolbr#364 (comment)

mmd-osm · 2017-03-03T18:59:59Z

Cherry-picked the changes to my branch, tested both variants. Memory consumption looks good, there's a somewhat larger runtime in one case, which is due to some unrelated code in area_query.cc. Bottom line: result is still way better than before, though.

way["building"](area:3600285864);out;

Command being timed: "./osm3s_query"
User time (seconds): 5.54
System time (seconds): 0.10
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.65
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 123124
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 31456
Voluntary context switches: 13
Involuntary context switches: 487
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

area(3600285864);
way["building"](area._);
out;

Command being timed: "./osm3s_query"
User time (seconds): 3.58
System time (seconds): 0.09
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.67
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 122904
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 36025
Voluntary context switches: 13
Involuntary context switches: 191
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

mmd-osm · 2017-03-08T12:03:13Z

I had some issues to apply this PR cleanly, although Github claims that everything is fine. I think this a side effect of the most recent std:: additions across the whole source code. Can you merge the current master and update the pr? thanks!

tyrasd · 2017-03-08T13:00:22Z

although Github claims that everything is fine

I think that was simply because the PR was requested against the minor_issues branch. I've changed it to master now and am looking into the conflicts.

tyrasd · 2017-03-08T13:04:28Z

should be resolved now

mmd-osm · 2017-03-28T19:12:47Z

fyi: I deployed your patch on the following endpoint: http://dev.overpass-api.de/api_mmd/

mmd-osm · 2018-04-11T21:05:27Z

This one got superseded by dba448f -> this pull request can be closed, there's nothing left to do.

implement area size heuristic for area:id queries

200debd

fixes drolbr#359

mmd-osm reviewed Mar 2, 2017

View reviewed changes

don't initialize variable twice

1790d14

tyrasd added a commit to tyrasd/Overpass-API that referenced this pull request Mar 2, 2017

use iterator.handle() to fetch area ids

ad544de

see drolbr#364 (comment)

use iterator.handle() to fetch area ids

4d9992d

see drolbr#364 (comment)

tyrasd force-pushed the patch-359 branch from ad544de to 4d9992d Compare March 2, 2017 16:28

tyrasd changed the title ~~implement area size heuristic for area:id queries, fixes #359~~ implement area size heuristic for area:id queries Mar 2, 2017

tyrasd changed the base branch from minor_issues to master March 8, 2017 12:58

Merge branch 'master' into patch-359

7b049d2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement area size heuristic for area:id queries #364

implement area size heuristic for area:id queries #364

tyrasd commented Mar 2, 2017 •

edited

Loading

mmd-osm Mar 2, 2017

tyrasd Mar 2, 2017

mmd-osm Mar 2, 2017 •

edited

Loading

tyrasd Mar 2, 2017

mmd-osm Mar 2, 2017 •

edited

Loading

tyrasd Mar 2, 2017

mmd-osm Mar 3, 2017

tyrasd Mar 3, 2017

mmd-osm Mar 3, 2017

mmd-osm commented Mar 3, 2017 •

edited

Loading

mmd-osm commented Mar 8, 2017 •

edited

Loading

tyrasd commented Mar 8, 2017

tyrasd commented Mar 8, 2017

mmd-osm commented Mar 28, 2017

mmd-osm commented Apr 11, 2018 •

edited

Loading

implement area size heuristic for area:id queries #364

Are you sure you want to change the base?

implement area size heuristic for area:id queries #364

Conversation

tyrasd commented Mar 2, 2017 • edited Loading

mmd-osm Mar 2, 2017

Choose a reason for hiding this comment

tyrasd Mar 2, 2017

Choose a reason for hiding this comment

mmd-osm Mar 2, 2017 • edited Loading

Choose a reason for hiding this comment

tyrasd Mar 2, 2017

Choose a reason for hiding this comment

mmd-osm Mar 2, 2017 • edited Loading

Choose a reason for hiding this comment

tyrasd Mar 2, 2017

Choose a reason for hiding this comment

mmd-osm Mar 3, 2017

Choose a reason for hiding this comment

tyrasd Mar 3, 2017

Choose a reason for hiding this comment

mmd-osm Mar 3, 2017

Choose a reason for hiding this comment

mmd-osm commented Mar 3, 2017 • edited Loading

mmd-osm commented Mar 8, 2017 • edited Loading

tyrasd commented Mar 8, 2017

tyrasd commented Mar 8, 2017

mmd-osm commented Mar 28, 2017

mmd-osm commented Apr 11, 2018 • edited Loading

tyrasd commented Mar 2, 2017 •

edited

Loading

mmd-osm Mar 2, 2017 •

edited

Loading

mmd-osm Mar 2, 2017 •

edited

Loading

mmd-osm commented Mar 3, 2017 •

edited

Loading

mmd-osm commented Mar 8, 2017 •

edited

Loading

mmd-osm commented Apr 11, 2018 •

edited

Loading