Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

speedup getDisplacemenet #29

Open
webgisdeveloper opened this issue Mar 1, 2022 · 6 comments
Open

speedup getDisplacemenet #29

webgisdeveloper opened this issue Mar 1, 2022 · 6 comments
Assignees
@webgisdeveloper
Labels
enhancement

Comments

@webgisdeveloper
Copy link
Collaborator

need to speed up the whole process for large area
@webgisdeveloper webgisdeveloper self-assigned this Mar 1, 2022
@webgisdeveloper
Copy link
Collaborator Author

test paras:
{'scale': None, 'ref': None, 'ct': None, 'pt': None, 'eon': False, 'mon': False, 'dwin1': None, 'dwin2': None, 'vabs': False, 'analysisCenter': None, 'lat': 33, 'lon': -115, 'width': 4, 'height': 4, 'epoch1': '2010-01-01', 'epoch2': '2011-01-01', 'output': '1_profile'}

@webgisdeveloper
Copy link
Collaborator Author 

It took 70.53889632225037 seconds!
         3573883 function calls (3573522 primitive calls) in 70.539 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     7673   44.756    0.006   44.756    0.006 {method 'read' of '_ssl._SSLSocket' objects}
      105   12.634    0.120   12.634    0.120 {method 'do_handshake' of '_ssl._SSLSocket' objects}
      105    6.033    0.057    6.033    0.057 {method 'connect' of '_socket.socket' objects}
        1    2.431    2.431   70.539   70.539 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:132(getDisplacement)
   839419    0.958    0.000    1.756    0.000 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:69(__init__)
   846141    0.802    0.000    0.802    0.000 {method 'split' of 'str' objects}
      105    0.695    0.007    0.695    0.007 {method 'set_default_verify_paths' of '_ssl._SSLContext' objects}
      152    0.428    0.003    0.428    0.003 {method 'splitlines' of 'str' objects}
      104    0.284    0.003    0.284    0.003 {method 'write' of '_io.TextIOWrapper' objects}
      105    0.283    0.003    0.284    0.003 {built-in method _socket.getaddrinfo}
  1679174    0.213    0.000    0.213    0.000 {built-in method math.sqrt}
      105    0.165    0.002    0.185    0.002 /home/cicuser/miniconda3/envs/mom/lib/python3.9/socket.py:767(close)
      615    0.110    0.000    0.110    0.000 {method 'decode' of 'bytes' objects}
     7673    0.085    0.000   45.056    0.006 /home/cicuser/miniconda3/envs/mom/lib/python3.9/socket.py:690(readinto)
     7673    0.079    0.000   44.910    0.006 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1230(recv_into)
      108    0.070    0.001   36.330    0.336 {method 'read' of '_io.BufferedReader' objects}
     7673    0.056    0.000   44.816    0.006 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1090(read)
     7673    0.027    0.000    0.027    0.000 {method '_checkClosed' of '_io._IOBase' objects}
      154    0.024    0.000    0.027    0.000 {built-in method io.open}
      421    0.019    0.000    0.019    0.000 {built-in method __new__ of type object at 0x55caa10b2300}
     7673    0.018    0.000    0.033    0.000 {method '_checkReadable' of '_io._IOBase' objects}
     7778    0.015    0.000    0.015    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/socket.py:729(readable)
      105    0.013    0.000    0.019    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1298(_real_close)
17558/17493    0.012    0.000    0.012    0.000 {built-in method builtins.len}
       47    0.011    0.000    0.019    0.000 {method 'read' of '_io.TextIOWrapper' objects}
      105    0.010    0.000    0.025    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/email/feedparser.py:471(_parse_headers)
    15556    0.009    0.000    0.009    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1078(_checkClosed)
      105    0.009    0.000    0.009    0.000 {method 'write' of '_ssl._SSLSocket' objects}
      210    0.008    0.000    0.056    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/email/feedparser.py:218(_parsegen)
      735    0.008    0.000    0.014    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/email/message.py:462(get)
      105    0.008    0.000    0.009    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/http/client.py:415(close)
    10988    0.007    0.000    0.007    0.000 {built-in method builtins.print}
       47    0.007    0.000    0.007    0.000 {built-in method _codecs.utf_8_decode}
     5358    0.007    0.000    0.011    0.000 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:48(__init__)

@webgisdeveloper
Copy link
Collaborator Author

second run:

It took 3.0532331466674805 seconds!
         3433147 function calls (3432890 primitive calls) in 3.053 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.277    1.277    3.053    3.053 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:132(getDisplacement)
   839419    0.501    0.000    0.923    0.000 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:69(__init__)
   844789    0.424    0.000    0.424    0.000 {method 'split' of 'str' objects}
       28    0.296    0.011    0.296    0.011 {method 'read' of '_ssl._SSLSocket' objects}
      152    0.160    0.001    0.160    0.001 {method 'splitlines' of 'str' objects}
  1679174    0.112    0.000    0.112    0.000 {built-in method math.sqrt}
        1    0.109    0.109    0.109    0.109 {method 'do_handshake' of '_ssl._SSLSocket' objects}
        1    0.052    0.052    0.052    0.052 {method 'connect' of '_socket.socket' objects}
      151    0.040    0.000    0.066    0.000 {method 'read' of '_io.TextIOWrapper' objects}
      151    0.026    0.000    0.026    0.000 {built-in method _codecs.utf_8_decode}
    10989    0.006    0.000    0.006    0.000 {built-in method builtins.print}
      154    0.006    0.000    0.007    0.000 {built-in method io.open}
     5358    0.005    0.000    0.007    0.000 /home/cicuser/Projects/GPS_Services/webAPI/getDisplacement.py:48(__init__)
        1    0.005    0.005    0.005    0.005 {method 'set_default_verify_paths' of '_ssl._SSLContext' objects}
     4928    0.005    0.000    0.005    0.000 {method 'format' of 'str' objects}
        1    0.004    0.004    0.004    0.004 /home/cicuser/miniconda3/envs/mom/lib/python3.9/socket.py:767(close)
        1    0.003    0.003    0.005    0.005 {built-in method _socket.getaddrinfo}
      303    0.002    0.000    0.002    0.000 {method 'decode' of 'bytes' objects}
    17472    0.002    0.000    0.002    0.000 {built-in method math.cos}
      162    0.002    0.000    0.002    0.000 {built-in method posix.stat}
      151    0.001    0.000    0.234    0.002 /home/cicuser/Projects/GPS_Services/webAPI/setLocaldata.py:13(get_localdata)
    13888    0.001    0.000    0.001    0.000 {built-in method math.sin}
      154    0.001    0.000    0.001    0.000 {method '__exit__' of '_io._IOBase' objects}
        5    0.001    0.000    0.001    0.000 {built-in method __new__ of type object at 0x55bc05886300}
6273/6208    0.001    0.000    0.001    0.000 {built-in method builtins.len}
      151    0.001    0.000    0.026    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/codecs.py:319(decode)
      154    0.001    0.000    0.001    0.000 {built-in method _locale.nl_langinfo}
       38    0.001    0.000    0.001    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_compile.py:276(_optimize_charset)
        1    0.001    0.001    3.053    3.053 /home/cicuser/Projects/GPS_Services/webAPI/profile_displacemnet.py:36(main)
      286    0.000    0.000    0.000    0.000 /home/cicuser/Projects/GPS_Services/webAPI/GPS_service.py:21(__getattr__)
      154    0.000    0.000    0.001    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/_bootlocale.py:33(getpreferredencoding)
     55/6    0.000    0.000    0.001    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_parse.py:493(_parse)
     65/6    0.000    0.000    0.001    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_compile.py:71(_compile)
        3    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
       28    0.000    0.000    0.297    0.011 /home/cicuser/miniconda3/envs/mom/lib/python3.9/socket.py:690(readinto)
      151    0.000    0.000    0.002    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/genericpath.py:16(exists)
       28    0.000    0.000    0.296    0.011 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1230(recv_into)
        4    0.000    0.000    0.242    0.060 {method 'read' of '_io.BufferedReader' objects}
      151    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/codecs.py:309(__init__)
       10    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/urllib/request.py:436(add_handler)
        1    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1298(_real_close)
       28    0.000    0.000    0.296    0.011 /home/cicuser/miniconda3/envs/mom/lib/python3.9/ssl.py:1090(read)
        3    0.000    0.000    0.000    0.000 {method 'close' of '_io.TextIOWrapper' objects}
        7    0.000    0.000    0.000    0.000 {built-in method builtins.__build_class__}
    78/19    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_parse.py:174(getwidth)
      327    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_parse.py:233(__next)
     1040    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
      356    0.000    0.000    0.000    0.000 /home/cicuser/miniconda3/envs/mom/lib/python3.9/sre_parse.py:164(__getitem__)

@webgisdeveloper
Copy link
Collaborator Author

third run:

It took 3.3653359413146973 seconds!
         3433147 function calls (3432890 primitive calls) in 3.365 seconds

@webgisdeveloper
Copy link
Collaborator Author

fourth run"

It took 3.3811941146850586 seconds!
         3433147 function calls (3432890 primitive calls) in 3.381 seconds

@webgisdeveloper
Copy link
Collaborator Author

socket download data is slow

webgisdeveloper added a commit that referenced this issue Mar 2, 2022
@webgisdeveloper
#29 profile tool
62afaa6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@webgisdeveloper webgisdeveloper

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@webgisdeveloper