grass.jupyter: Add Query Button to InteractiveMap #3793

29riyasaxena · 2024-06-11T13:12:09Z

Hello Everyone,

This pull request introduces a new feature to grass.jupyter.interactivemap.py: a query button that allows users to retrieve raster/vector information associated with a specific point.

Here's how it operates:

Click the 'info' button ('i') to activate it.
Choose a point on the map to retrieve its relevant information.
Deactivate the button by toggling it off.

For a visual demonstration, please refer to the following recording:

grass_query_button.mp4

petrasovaa

For some reason, I can't see any results for a vector map. Also, it behaves strangely, when I click first time, it works (for raster) and then when I click second time on the map, the output disappears. Does it behave the same for you?

python/grass/jupyter/interactivemap.py

29riyasaxena · 2024-06-11T17:32:53Z

For some reason, I can't see any results for a vector map. Also, it behaves strangely, when I click first time, it works (for raster) and then when I click second time on the map, the output disappears. Does it behave the same for you?

It behaves like this:

Recording.2024-06-11.230149.mp4

petrasovaa · 2024-06-11T18:01:00Z

For some reason, I can't see any results for a vector map. Also, it behaves strangely, when I click first time, it works (for raster) and then when I click second time on the map, the output disappears. Does it behave the same for you?

It behaves like this:

Even there you can see strange flashing around 13th second.

I found out why I don't see any vector results, you don't set the threshold. vector_what has a distance parameter, but it needs to change based on the extent of the leaflet map, similarly like it is done in GUI:

https://github.com/OSGeo/grass/blob/main/gui/wxpython/mapdisp/frame.py#L1040

python/grass/jupyter/utils.py

python/grass/jupyter/interactivemap.py

python/grass/jupyter/utils.py

python/grass/jupyter/interactivemap.py

petrasovaa · 2024-06-26T06:41:20Z

The main problem here is still the graphical representation of the results, it doesn't look very well. The css would have to be tweaked to make the table look nicer. @29riyasaxena Do you agree? Is that something you want to work on as part of this PR? Or we could leave that as separate PR.

29riyasaxena · 2024-06-26T07:05:31Z

The main problem here is still the graphical representation of the results, it doesn't look very well. The css would have to be tweaked to make the table look nicer. @29riyasaxena Do you agree? Is that something you want to work on as part of this PR? Or we could leave that as separate PR.

I think I can do this since it is not a difficult task. Also, I couldn't find the tests of InteractiveMap.

petrasovaa · 2024-06-27T14:20:18Z

I think I can do this since it is not a difficult task. Also, I couldn't find the tests of InteractiveMap.

https://github.com/OSGeo/grass/blob/main/python/grass/jupyter/testsuite/interactivemap_test.py

29riyasaxena · 2024-06-27T18:50:41Z

I think I can do this since it is not a difficult task. Also, I couldn't find the tests of InteractiveMap.

https://github.com/OSGeo/grass/blob/main/python/grass/jupyter/testsuite/interactivemap_test.py

Hi Anna, Thank you for providing the link. Running tests give me the following error:
=========================================================== short test summary info ============================================================ FAILED interactivemap_test.py::test - SystemExit: True ERROR interactivemap_test.py::TestDisplay::test_basic - TypeError: can only concatenate str (not "NoneType") to str ERROR interactivemap_test.py::TestDisplay::test_save_as_html - TypeError: can only concatenate str (not "NoneType") to str

Could you please check what's going wrong here?

petrasovaa · 2024-06-27T19:59:18Z

Could you please check what's going wrong here?

It runs for me with this PR. How do you run the test? Is there any more output?

29riyasaxena · 2024-06-28T04:59:51Z

Could you please check what's going wrong here?

It runs for me with this PR. How do you run the test? Is there any more output?

Here's the complete output:


============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-8.1.0, pluggy-1.4.0
rootdir: /home/riya/grass
configfile: pyproject.toml
plugins: cov-4.1.0, hypothesis-6.100.1, anyio-4.3.0
collected 3 items

interactivemap_test.py FEE                                               [100%]

==================================== ERRORS ====================================
___________________ ERROR at setup of TestDisplay.test_basic ___________________

cls = <class 'interactivemap_test.TestDisplay'>, module = Module('g.region')
expecting_stdout = False, kwargs = {'raster': 'elevation'}
errors = 'ERROR: Raster map <elevation> not found\n\nSee available raster maps:\n'
re = <module 're' from '/usr/lib/python3.10/re.py'>

    @classmethod
    def runModule(cls, module, expecting_stdout=False, **kwargs):
        """Run PyGRASS module.
    
        Runs the module and raises an exception if the module ends with
        non-zero return code. Usually, this is the same as testing the
        return code and raising exception but by using this method,
        you give testing framework more control over the execution,
        error handling and storing of output.
    
        In terms of testing framework, this function causes a common error,
        not a test failure.
    
        :raises CalledModuleError: if the module failed
        """
        module = _module_from_parameters(module, **kwargs)
        _check_module_run_parameters(module)
        try:
>           module.run()

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py:1319: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py:838: in run
    self.wait()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Module('g.region')

    def wait(self):
        """Wait for the module to finish. Call this method if
        the run() call was performed with self.false_ = False.
    
        :return: A reference to this object
        """
        if self._finished is False:
            if self.stdin:
                self.stdin = encode(self.stdin)
            stdout, stderr = self._popen.communicate(input=self.stdin)
            self.outputs["stdout"].value = decode(stdout) if stdout else ""
            self.outputs["stderr"].value = decode(stderr) if stderr else ""
            self.time = time.time() - self.start_time
            self.returncode = self._popen.returncode
            self._finished = True
    
            if self._popen.poll():
>               raise CalledModuleError(
                    returncode=self._popen.returncode,
                    code=self.get_bash(),
                    module=self.name,
                    errors=stderr,
                )
E               grass.exceptions.CalledModuleError: Module run `g.region raster=elevation` ended with an error.
E               The subprocess ended with a non-zero return code: 1. See the following errors:
E               b'ERROR: Raster map <elevation> not found\n'

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py:859: CalledModuleError

During handling of the above exception, another exception occurred:

cls = <class 'interactivemap_test.TestDisplay'>

    @classmethod
    def setUpClass(cls):
        """Ensures expected computational region"""
        # to not override mapset's region (which might be used by other tests)
        cls.use_temp_region()
        # cls.runModule or self.runModule is used for general module calls
        # we'll use the elevation raster as a test display
>       cls.runModule("g.region", raster="elevation")

interactivemap_test.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'interactivemap_test.TestDisplay'>, module = Module('g.region')
expecting_stdout = False, kwargs = {'raster': 'elevation'}
errors = 'ERROR: Raster map <elevation> not found\n\nSee available raster maps:\n'
re = <module 're' from '/usr/lib/python3.10/re.py'>

    @classmethod
    def runModule(cls, module, expecting_stdout=False, **kwargs):
        """Run PyGRASS module.
    
        Runs the module and raises an exception if the module ends with
        non-zero return code. Usually, this is the same as testing the
        return code and raising exception but by using this method,
        you give testing framework more control over the execution,
        error handling and storing of output.
    
        In terms of testing framework, this function causes a common error,
        not a test failure.
    
        :raises CalledModuleError: if the module failed
        """
        module = _module_from_parameters(module, **kwargs)
        _check_module_run_parameters(module)
        try:
            module.run()
        except CalledModuleError:
            # here exception raised by run() with finish_=True would be
            # almost enough but we want some additional info to be included
            # in the test report
            errors = module.outputs.stderr
            # provide diagnostic at least in English locale
            # TODO: standardized error code would be handy here
            import re
    
            if re.search("Raster map.*not found", errors, flags=re.DOTALL):
                errors += "\nSee available raster maps:\n"
>               errors += call_module("g.list", type="raster")
E               TypeError: can only concatenate str (not "NoneType") to str

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py:1331: TypeError
_______________ ERROR at setup of TestDisplay.test_save_as_html ________________

cls = <class 'interactivemap_test.TestDisplay'>, module = Module('g.region')
expecting_stdout = False, kwargs = {'raster': 'elevation'}
errors = 'ERROR: Raster map <elevation> not found\n\nSee available raster maps:\n'
re = <module 're' from '/usr/lib/python3.10/re.py'>

    @classmethod
    def runModule(cls, module, expecting_stdout=False, **kwargs):
        """Run PyGRASS module.
    
        Runs the module and raises an exception if the module ends with
        non-zero return code. Usually, this is the same as testing the
        return code and raising exception but by using this method,
        you give testing framework more control over the execution,
        error handling and storing of output.
    
        In terms of testing framework, this function causes a common error,
        not a test failure.
    
        :raises CalledModuleError: if the module failed
        """
        module = _module_from_parameters(module, **kwargs)
        _check_module_run_parameters(module)
        try:
>           module.run()

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py:1319: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py:838: in run
    self.wait()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Module('g.region')

    def wait(self):
        """Wait for the module to finish. Call this method if
        the run() call was performed with self.false_ = False.
    
        :return: A reference to this object
        """
        if self._finished is False:
            if self.stdin:
                self.stdin = encode(self.stdin)
            stdout, stderr = self._popen.communicate(input=self.stdin)
            self.outputs["stdout"].value = decode(stdout) if stdout else ""
            self.outputs["stderr"].value = decode(stderr) if stderr else ""
            self.time = time.time() - self.start_time
            self.returncode = self._popen.returncode
            self._finished = True
    
            if self._popen.poll():
>               raise CalledModuleError(
                    returncode=self._popen.returncode,
                    code=self.get_bash(),
                    module=self.name,
                    errors=stderr,
                )
E               grass.exceptions.CalledModuleError: Module run `g.region raster=elevation` ended with an error.
E               The subprocess ended with a non-zero return code: 1. See the following errors:
E               b'ERROR: Raster map <elevation> not found\n'

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/pygrass/modules/interface/module.py:859: CalledModuleError

During handling of the above exception, another exception occurred:

cls = <class 'interactivemap_test.TestDisplay'>

    @classmethod
    def setUpClass(cls):
        """Ensures expected computational region"""
        # to not override mapset's region (which might be used by other tests)
        cls.use_temp_region()
        # cls.runModule or self.runModule is used for general module calls
        # we'll use the elevation raster as a test display
>       cls.runModule("g.region", raster="elevation")

interactivemap_test.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'interactivemap_test.TestDisplay'>, module = Module('g.region')
expecting_stdout = False, kwargs = {'raster': 'elevation'}
errors = 'ERROR: Raster map <elevation> not found\n\nSee available raster maps:\n'
re = <module 're' from '/usr/lib/python3.10/re.py'>

    @classmethod
    def runModule(cls, module, expecting_stdout=False, **kwargs):
        """Run PyGRASS module.
    
        Runs the module and raises an exception if the module ends with
        non-zero return code. Usually, this is the same as testing the
        return code and raising exception but by using this method,
        you give testing framework more control over the execution,
        error handling and storing of output.
    
        In terms of testing framework, this function causes a common error,
        not a test failure.
    
        :raises CalledModuleError: if the module failed
        """
        module = _module_from_parameters(module, **kwargs)
        _check_module_run_parameters(module)
        try:
            module.run()
        except CalledModuleError:
            # here exception raised by run() with finish_=True would be
            # almost enough but we want some additional info to be included
            # in the test report
            errors = module.outputs.stderr
            # provide diagnostic at least in English locale
            # TODO: standardized error code would be handy here
            import re
    
            if re.search("Raster map.*not found", errors, flags=re.DOTALL):
                errors += "\nSee available raster maps:\n"
>               errors += call_module("g.list", type="raster")
E               TypeError: can only concatenate str (not "NoneType") to str

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py:1331: TypeError
=================================== FAILURES ===================================
_____________________________________ test _____________________________________

    def test():
        """Run a test of a module."""
        # TODO: put the link to to the report only if available
        # TODO: how to disable Python code coverage for module and C tests?
        # TODO: we probably need to have different test  functions for C, Python modules,
        # and Python code
        # TODO: combine the results using python -m coverage --help | grep combine
        # TODO: function to anonymize/beautify file names (in content and actual filenames)
        # TODO: implement coverage but only when requested by invoker and only if
        # it makes sense for tests (need to know what is tested)
        # doing_coverage = False
        # try:
        #    import coverage
        #    doing_coverage = True
        #    cov = coverage.coverage(omit="*testsuite*")
        #    cov.start()
        # except ImportError:
        #    pass
        # TODO: add some message somewhere
    
        # TODO: enable passing omit to exclude also gunittest or nothing
        program = GrassTestProgram(
            module="__main__", exit_at_end=False, grass_location="all"
        )
        # TODO: check if we are in the directory where the test file is
        # this will ensure that data directory is available when it is requested
    
        # if doing_coverage:
        #    cov.stop()
        #    cov.html_report(directory='testcodecoverage')
    
        # TODO: is sys.exit the right thing here
>       sys.exit(not program.result.wasSuccessful())
E       SystemExit: True

/mnt/c/Users/29riy/Documents/projects/grass/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/main.py:113: SystemExit
----------------------------- Captured stderr call -----------------------------
E
======================================================================
ERROR: interactivemap_test (unittest.loader._FailedTest)
----------------------------------------------------------------------
AttributeError: module '__main__' has no attribute 'interactivemap_test'

----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
=============================== warnings summary ===============================
../../../../../.local/lib/python3.10/site-packages/_pytest/config/__init__.py:1437
  /home/riya/.local/lib/python3.10/site-packages/_pytest/config/__init__.py:1437: PytestConfigWarning: Unknown config option: timeout
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED interactivemap_test.py::test - SystemExit: True
ERROR interactivemap_test.py::TestDisplay::test_basic - TypeError: can only c...
ERROR interactivemap_test.py::TestDisplay::test_save_as_html - TypeError: can...
==================== 1 failed, 1 warning, 2 errors in 3.20s ====================

Is it because of wrong PYTHONPATH?
I am working in ~/grass.

echoix · 2024-06-28T05:24:08Z

pip install pytest-timeout

petrasovaa · 2024-06-28T06:28:50Z

It's saying Raster map elevation not found, you need to run it in with e.g. the nc_basic_spm_grass7 dataset.

29riyasaxena · 2024-06-30T07:16:18Z

Running tests like this: grass /home/riya/grass/data/grassdata/nc_basic_spm_grass7/user1 --exec python3 /home/riya/grass/python/grass/jupyter/testsuite/interactivemap_test.py worked for me.

petrasovaa · 2024-07-14T21:41:04Z

Could you please try to address the code quality issues from the CI?

echoix · 2024-07-14T21:46:00Z

Could you please try to address the code quality issues from the CI?

It should also be with a merge from main into this branch, it is 189 commits behind, and LOTS have changed since, especially for Python

python/grass/jupyter/utils.py

petrasovaa · 2024-07-18T21:02:47Z

To clarify this PR for others, could you please edit the description of this PR and include a brief summary and a most recent screenshot?

petrasovaa · 2024-07-31T17:37:16Z

Given #3838 is now merged, we need to make sure these modes (query, region) don't interact in a weird way. I think the toggle buttons should be exclusive, so when you toggle query button, the region button should untoggle (and hide region). There is ToggleButtons widget which does that, but probably more changes in the handling of both modes are needed.

python/grass/jupyter/interactivemap.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Co-authored-by: Anna Petrasova <[email protected]>

grass.jupyter: Add Query Button to InteractiveMap

2195ddf

github-actions bot added Python Related code is in Python libraries notebook labels Jun 11, 2024

petrasovaa reviewed Jun 11, 2024

View reviewed changes

petrasovaa added this to the 8.5.0 milestone Jun 11, 2024

Add toggle button and modify query function

b78f71d

petrasovaa reviewed Jun 14, 2024

View reviewed changes

python/grass/jupyter/utils.py Outdated Show resolved Hide resolved

petrasovaa reviewed Jun 14, 2024

View reviewed changes

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

Add support for multiple layers and display the ouput in popup

ab9ed38

cwhite911 reviewed Jun 15, 2024

View reviewed changes

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

petrasovaa reviewed Jun 15, 2024

View reviewed changes

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

petrasovaa reviewed Jun 15, 2024

View reviewed changes

python/grass/jupyter/utils.py Outdated Show resolved Hide resolved

petrasovaa reviewed Jun 15, 2024

View reviewed changes

python/grass/jupyter/utils.py Outdated Show resolved Hide resolved

Add info icon and compute distance dynamically

df9e06b

petrasovaa reviewed Jun 17, 2024

View reviewed changes

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

Resolve flickering issue in popup

8a21448

Add test for query mode

370acdd

github-actions bot added the tests Related to Test Suite label Jun 30, 2024

29riyasaxena and others added 2 commits July 15, 2024 16:30

Solve pylint error

faeaec8

Merge branch 'main' into allow_query_button

30ff143

petrasovaa reviewed Jul 16, 2024

View reviewed changes

python/grass/jupyter/utils.py Outdated Show resolved Hide resolved

29riyasaxena and others added 2 commits July 17, 2024 18:13

Modify query raster and query vector

287cb37

Merge branch 'main' into allow_query_button

c3da23f

petrasovaa reviewed Jul 18, 2024

View reviewed changes

python/grass/jupyter/utils.py Outdated Show resolved Hide resolved

petrasovaa marked this pull request as ready for review July 18, 2024 21:01

petrasovaa requested a review from cwhite911 July 18, 2024 21:03

29riyasaxena and others added 4 commits July 20, 2024 18:58

Modify utils

388796e

Merge branch 'main' into allow_query_button

9623978

Merge branch 'main' into allow_query_button

e8df263

Merge branch 'main' into allow_query_button

e3994ae

github-actions bot reviewed Jul 31, 2024

View reviewed changes

python/grass/jupyter/interactivemap.py Outdated Show resolved Hide resolved

petrasovaa and others added 8 commits July 31, 2024 15:43

Update python/grass/jupyter/interactivemap.py

4f4d3d1

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Resolve Conflict Changes

84279bc

Add toggle changes

927bf5c

Merge branch 'main' into allow_query_button

4b9f311

Fix issues

ff1b613

Merge branch 'main' into allow_query_button

2d4ee6b

simplify code, small tweaks

38273a3

fix clearing geometries, show region as fully transparent like in GUI

bc2ba5b

petrasovaa approved these changes Aug 9, 2024

View reviewed changes

petrasovaa enabled auto-merge (squash) August 9, 2024 20:06

petrasovaa merged commit 16c5bf9 into OSGeo:main Aug 9, 2024
26 checks passed

landam pushed a commit to landam/grass that referenced this pull request Aug 22, 2024

grass.jupyter: Add Query Button to InteractiveMap (OSGeo#3793)

9babe4a

Co-authored-by: Anna Petrasova <[email protected]>

Mahesh1998 pushed a commit to Mahesh1998/grass that referenced this pull request Sep 19, 2024

grass.jupyter: Add Query Button to InteractiveMap (OSGeo#3793)

32b07d7

Co-authored-by: Anna Petrasova <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grass.jupyter: Add Query Button to InteractiveMap #3793

grass.jupyter: Add Query Button to InteractiveMap #3793

29riyasaxena commented Jun 11, 2024 •

edited

Loading

petrasovaa left a comment

29riyasaxena commented Jun 11, 2024

petrasovaa commented Jun 11, 2024

petrasovaa commented Jun 26, 2024

29riyasaxena commented Jun 26, 2024

petrasovaa commented Jun 27, 2024

29riyasaxena commented Jun 27, 2024

petrasovaa commented Jun 27, 2024

29riyasaxena commented Jun 28, 2024

echoix commented Jun 28, 2024

petrasovaa commented Jun 28, 2024

29riyasaxena commented Jun 30, 2024

petrasovaa commented Jul 14, 2024

echoix commented Jul 14, 2024

petrasovaa commented Jul 18, 2024

petrasovaa commented Jul 31, 2024

grass.jupyter: Add Query Button to InteractiveMap #3793

grass.jupyter: Add Query Button to InteractiveMap #3793

Conversation

29riyasaxena commented Jun 11, 2024 • edited Loading

petrasovaa left a comment

Choose a reason for hiding this comment

29riyasaxena commented Jun 11, 2024

petrasovaa commented Jun 11, 2024

petrasovaa commented Jun 26, 2024

29riyasaxena commented Jun 26, 2024

petrasovaa commented Jun 27, 2024

29riyasaxena commented Jun 27, 2024

petrasovaa commented Jun 27, 2024

29riyasaxena commented Jun 28, 2024

echoix commented Jun 28, 2024

petrasovaa commented Jun 28, 2024

29riyasaxena commented Jun 30, 2024

petrasovaa commented Jul 14, 2024

echoix commented Jul 14, 2024

petrasovaa commented Jul 18, 2024

petrasovaa commented Jul 31, 2024

29riyasaxena commented Jun 11, 2024 •

edited

Loading