Added Streaming Tests to Testing Suite

.github/workflows/build_release_artifacts.yml

@@ -54,7 +54,7 @@ jobs:
         id: check_files
         uses: andstor/[email protected]
         with:
-          files: "${{runner.workspace}}/grpc-labview/tests/New_ATS/logs/*.xml"
+          files: "${{runner.workspace}}/grpc-labview/tests/New_ATS/logs/"
 
       - if: ${{always() && steps.check_files.outputs.files_exists == 'true' }}
         name: Download New_ATS Test Results if logs folder exists

tests/New_ATS/README.md

@@ -6,13 +6,13 @@
 
 2. The configuration of the testing suite is saved in the [testlist.json](pylib/testlist.json) file. We can modify it to run specific tests with specific labview version and bitness according to our needs.
 
-3. For each test, inside the test folder, we have a protofile, a python client and an `Impl` folder that stores pre-implemented `Start Sync.vi` and `Run Service.vi` for that particular protofile.
+3. For each test, inside the test folder, we have a protofile, a `Python_client` folder that contains the python clients and an `Impl` folder that stores pre-implemented `Start Sync.vi` and `Run Service.vi` for that particular protofile.
 
 4. When executing the testing suite, the following steps are performed for each test:
    - Delete the pre-existing `Generated Server` folder that contains the gRPC Server.
    - Regenerate the gRPC server using the protofile.
    - Copy the `Start Sync.vi` and `Run Service.vi` from the `Impl` folder into the new `Generated Server` folder.
-   - Run the pre-written python client which uses pytest to run all the testcases for each rpc. The testcases are defined in the form of json files in the `testcases` folder.
+   - Run the pre-written python clients in the `Python_client` which uses pytest to run all the testcases for each rpc. The testcases are defined in the form of json files in the `testcases` folder.
    - Prints the verbose output of each testcase onto the terminal.
 
 ### How to run?
@@ -88,13 +88,12 @@ Follow the below steps to add more tests in the testing suite.
 
 9. Inside the `testcases` folder, create a json file for each rpc method defined in the protofile. The name of the json file will be same as the rpc method's name. These will contain the testcases that the testing suite will run corresponding to each rpc method.
 
-10. Create a python client that will interact with the LabVIEW gRPC Server. The name of the client will be like `<test_name>_client.py`.
+10. Create a `Python_client` folder and add python clients for each rpc into it that will interact with the LabVIEW gRPC Server. The name of the client can be anything but should strictly end with `_client.py` like `<client_name>_client.py`.
 
 ### TODO:
 
 1. Add the following tests:
 
-   - Streaming tests (client streaming, server streaming, bi-directional streaming)
    - Reflection tests
    - Client tests (currently we are only testing gRPC Server)
    - Modification scenarios (do some modification after first generation and then generate and test again)

tests/New_ATS/RunPythonClient.bat

@@ -1,10 +1,16 @@
 @echo off
-set python_client_path=%1
-set test_name=%2
+setlocal
+
+set "python_client_path=%~1"
+set "test_name=%~2"
+set "python_client_name=%~3"
+
 pushd %~dp0
-set script_dir=%CD%
+set "script_dir=%CD%"
 popd
 
-echo Running Python Client
-if not exist "./logs" mkdir logs
-call %script_dir%\venv\Scripts\python.exe -m pytest %python_client_path% --junitxml=%script_dir%/logs/%test_name%_test_results.xml -vv
+if not exist "%script_dir%\logs" mkdir "%script_dir%\logs"
+if not exist "%script_dir%\logs\%test_name%" mkdir "%script_dir%\logs\%test_name%"
+
+call "%script_dir%\venv\Scripts\python.exe" -m pytest "%python_client_path%" --junitxml="%script_dir%\logs\%test_name%\%test_name%_%python_client_name%_test_results.xml" -vv
+endlocal

tests/New_ATS/Tests/all-datatypes-oneof/all-datatypes-oneof_client.py → tests/New_ATS/Tests/all-datatypes-oneof/Python_client/GetFeature_client.py

@@ -88,7 +88,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-GetFeature_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/GetFeature.json'
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
 
 @pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
 def test_SayHello(testcase):

tests/New_ATS/Tests/enum-oneof/enum-oneof_client.py → tests/New_ATS/Tests/enum-oneof/Python_client/GetFeature_client.py

@@ -26,7 +26,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-GetFeature_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/GetFeature.json'
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
 
 @pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
 def test_SayHello(testcase):

tests/New_ATS/Tests/helloworld/helloworld_client.py → tests/New_ATS/Tests/helloworld/Python_client/SayHello_client.py

@@ -10,7 +10,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-SayHello_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/SayHello.json'
+SayHello_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/SayHello.json'
 
 def get_SayHello_output(test_input):
     name = test_input['name']

tests/New_ATS/Tests/message-oneof/message-oneof_client.py → tests/New_ATS/Tests/message-oneof/Python_client/GetFeature_client.py

@@ -31,7 +31,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-GetFeature_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/GetFeature.json'
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
 
 @pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
 def test_SayHello(testcase):

tests/New_ATS/Tests/multiple-field-oneof/multiple-field-oneof_client.py → tests/New_ATS/Tests/multiple-field-oneof/Python_client/GetFeature_client.py

@@ -34,7 +34,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-GetFeature_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/GetFeature.json'
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
 
 @pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
 def test_SayHello(testcase):

tests/New_ATS/Tests/nested-oneof/nested-oneof_client.py → tests/New_ATS/Tests/nested-oneof/Python_client/GetFeature_client.py

@@ -50,7 +50,7 @@ def read_json(filepath):
         test_data = json.load(file)
     return test_data
 
-GetFeature_json_file_path = f'{os.path.dirname(os.path.abspath(__file__))}/testcases/GetFeature.json'
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
 
 @pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
 def test_SayHello(testcase):

tests/New_ATS/Tests/routeguide/Python_client/Getfeature_client.py

@@ -0,0 +1,40 @@
+import grpc
+import routeguide_pb2
+import routeguide_pb2_grpc
+import json
+import pytest
+import os
+
+def read_json(filepath):
+    with open(filepath, 'r') as file:
+        test_data = json.load(file)
+    return test_data
+
+GetFeature_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/GetFeature.json'
+
+def get_GetFeature_output(test_input):
+    latitude = test_input['latitude']
+    longitude = test_input['longitude']
+
+    with grpc.insecure_channel('localhost:50051') as channel:
+        stub = routeguide_pb2_grpc.RouteGuideStub(channel)
+        point = routeguide_pb2.Point(latitude=latitude, longitude=longitude)
+        response = stub.GetFeature(point)
+
+    return {
+        "name": response.name,
+        "location": {
+            "latitude": response.location.latitude,
+            "longitude": response.location.longitude
+        }
+    }
+
+@pytest.mark.parametrize('testcase', read_json(GetFeature_json_file_path))
+def test_GetFeature(testcase):
+    test_input = testcase['input']
+    expected = testcase['output']
+    assert get_GetFeature_output(test_input) == expected
+
+if __name__ == "__main__":
+    res = get_GetFeature_output({'latitude': 1, 'longitude': 2})
+    print(res)

tests/New_ATS/Tests/routeguide/Python_client/ListFeatures_client.py

@@ -0,0 +1,51 @@
+import grpc
+import routeguide_pb2
+import routeguide_pb2_grpc
+import json
+import pytest
+import os
+
+def read_json(filepath):
+    with open(filepath, 'r') as file:
+        test_data = json.load(file)
+    return test_data
+
+ListFeatures_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/ListFeatures.json'
+
+def get_ListFeatures_output(test_input):
+    lo = test_input['lo']
+    hi = test_input['hi']
+
+    output = []
+    with grpc.insecure_channel('localhost:50051') as channel:
+        stub = routeguide_pb2_grpc.RouteGuideStub(channel)
+        rectangle = routeguide_pb2.Rectangle(
+            lo=routeguide_pb2.Point(latitude=lo['latitude'], longitude=lo['longitude']),
+            hi=routeguide_pb2.Point(latitude=hi['latitude'], longitude=hi['longitude'])
+        )
+        response = stub.ListFeatures(rectangle)
+        stub.ListFeatures
+
+        for feature in response:
+            output.append({
+                "name": feature.name,
+                "location": {
+                    "latitude": feature.location.latitude,
+                    "longitude": feature.location.longitude
+                }
+            })
+    return output
+
+
+@pytest.mark.parametrize('testcase', read_json(ListFeatures_json_file_path))
+def test_ListFeatures(testcase):
+    test_input = testcase['input']
+    expected = testcase['output']
+    assert get_ListFeatures_output(test_input) == expected
+
+if __name__ == "__main__":
+    res = get_ListFeatures_output({
+        "lo": { "latitude": 1, "longitude": 2 },
+        "hi": { "latitude": 3, "longitude": 4 }
+    })
+    print(res)

tests/New_ATS/Tests/routeguide/Python_client/RecordRoute_client.py

@@ -0,0 +1,38 @@
+import grpc
+import routeguide_pb2
+import routeguide_pb2_grpc
+import json
+import pytest
+import os
+
+def read_json(filepath):
+    with open(filepath, 'r') as file:
+        test_data = json.load(file)
+    return test_data
+
+RecordRoute_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/RecordRoute.json'
+
+def get_RecordRoute_output(test_input):
+    points = [routeguide_pb2.Point(latitude=point['latitude'], longitude=point['longitude']) for point in test_input]
+
+    ans = None
+    with grpc.insecure_channel('localhost:50051') as channel:
+        stub = routeguide_pb2_grpc.RouteGuideStub(channel)
+        response = stub.RecordRoute(iter(points))
+        ans = {
+            "point_count": response.point_count,
+            "feature_count": response.feature_count,
+            "distance": response.distance,
+            "elapsed_time": response.elapsed_time
+        }
+    return ans;
+
+@pytest.mark.parametrize('testcase', read_json(RecordRoute_json_file_path))
+def test_RecordRoute(testcase):
+    test_input = testcase['input']
+    expected = testcase['output']
+    assert get_RecordRoute_output(test_input) == expected
+
+if __name__ == "__main__":
+    res = get_RecordRoute_output([{"latitude": 1, "longitude": 5}, {"latitude": 12, "longitude": 10}, {"latitude": 20, "longitude": 25}])
+    print(res)

tests/New_ATS/Tests/routeguide/Python_client/RouteChat_client.py

@@ -0,0 +1,57 @@
+import grpc
+import routeguide_pb2
+import routeguide_pb2_grpc
+import json
+import pytest
+import os
+
+def read_json(filepath):
+    with open(filepath, 'r') as file:
+        test_data = json.load(file)
+    return test_data
+
+RouteChat_json_file_path = f'{os.path.dirname(os.path.dirname(os.path.abspath(__file__)))}/testcases/RouteChat.json'
+
+def get_RouteChat_output(test_input):
+    output = []
+
+    with grpc.insecure_channel('localhost:50051') as channel:
+        stub = routeguide_pb2_grpc.RouteGuideStub(channel)
+
+        def generate_notes():
+            for point in test_input:
+                yield routeguide_pb2.RouteNote(
+                    location=routeguide_pb2.Point(
+                        latitude=point['location']['latitude'],
+                        longitude=point['location']['longitude']
+                    ),
+                    message=point['message']
+                )
+
+        response = stub.RouteChat(generate_notes())
+
+        for note in response:
+            output.append({
+                "location": {
+                    "latitude": note.location.latitude,
+                    "longitude": note.location.longitude
+                },
+                "message": note.message
+            })
+
+    return output
+
+@pytest.mark.parametrize('testcase', read_json(RouteChat_json_file_path))
+def test_RouteChat(testcase):
+    test_input = testcase['input']
+    expected = testcase['output']
+    assert get_RouteChat_output(test_input) == expected
+
+if __name__ == "__main__":
+    res = get_RouteChat_output([
+        { "location": { "latitude": 1, "longitude": 4 }, "message": "Point 1" },
+        { "location": { "latitude": 2, "longitude": 3 }, "message": "Point 2" },
+        { "location": { "latitude": 3, "longitude": 2 }, "message": "Point 3" },
+        { "location": { "latitude": 4, "longitude": 1 }, "message": "Point 4" }
+    ])
+    print(res)

tests/New_ATS/Tests/routeguide/routeguide.proto

@@ -0,0 +1,37 @@
+syntax = "proto3";
+
+package routeguide;
+
+service RouteGuide{
+  rpc GetFeature(Point) returns (Feature) {}
+  rpc ListFeatures(Rectangle) returns (stream Feature) {}
+  rpc RecordRoute(stream Point) returns (RouteSummary) {}
+  rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
+}
+
+message Point {
+  int32 latitude = 1;
+  int32 longitude = 2;
+}
+
+message Rectangle {
+  Point lo = 1;
+  Point hi = 2;
+}
+
+message Feature {
+  string name = 1;
+  Point location = 2;
+}
+
+message RouteNote {
+  Point location = 1;
+  string message = 2;
+}
+
+message RouteSummary {
+  int32 point_count = 1;
+  int32 feature_count = 2;
+  int32 distance = 3;
+  int32 elapsed_time = 4;
+}

tests/New_ATS/Tests/routeguide/testcases/GetFeature.json

@@ -0,0 +1,30 @@
+[
+  {
+    "input": { "latitude": 1, "longitude": 2 },
+    "output": {
+      "name": "1 x 2",
+      "location": { "latitude": 1, "longitude": 2 }
+    }
+  },
+  {
+    "input": { "latitude": 10, "longitude": 20 },
+    "output": {
+      "name": "10 x 20",
+      "location": { "latitude": 10, "longitude": 20 }
+    }
+  },
+  {
+    "input": { "latitude": 49, "longitude": 25 },
+    "output": {
+      "name": "49 x 25",
+      "location": { "latitude": 49, "longitude": 25 }
+    }
+  },
+  {
+    "input": { "latitude": 19, "longitude": 9 },
+    "output": {
+      "name": "19 x 9",
+      "location": { "latitude": 19, "longitude": 9 }
+    }
+  }
+]