Sanitize input #173

niranjank2022 · 2023-08-04T09:00:52Z

This PR addresses issue #118.

Currently, the handler.py files in every benchmark (000 to 500) are modified. Also, formatted strings using .format() are converted to f-strings.

niranjank2022 · 2023-08-04T09:06:37Z

Hello @mcopik !
I would like to hear from you whether I have done the first part correctly or if I have to revert anything.

mcopik

Looks like very good progress; thank you so much for your work!

I think the missing piece is modifying trigger and experiment code (should require only small changes), and running benchmarks to verify that they now work correctly.

mcopik · 2023-08-13T12:15:55Z

benchmarks/000.microbenchmarks/020.network-benchmark/python/function.py

 import socket
 from datetime import datetime
-from time import sleep
+from jsonschema import validate


Did you try running any of the benchmarks, e.g., on AWS? I think that jsonschema should be added to requirements.txt of each benchmark.

I haven't test-run the benchmarks on AWS. I really need assistance with this.

@niranjank2022 If the issue is that you cannot use the free tier of AWS, then please let me know - I'm happy to do the testing on this PR. Otherwise, if you found deploying and running benchmarks too complicated or not documented well enough, then I'm happy to help with that as well - just let me know.

mcopik · 2023-08-13T12:17:01Z

benchmarks/000.microbenchmarks/020.network-benchmark/python/function.py

+
+    client = storage.storage.get_instance()
+    key = client.upload(output_bucket, f'results-{request_id}.csv', '/tmp/data.csv') 
+    return { 'status': 'success', 'result': 'Returned with no error', 'measurement': key }


This is correct, but we also need to update the experiment in sebs/experiments/... to make sure it now checks for measurement key to download results.

mcopik · 2023-08-13T12:17:47Z

benchmarks/000.microbenchmarks/030.clock-synchronization/python/function.py

+    try:
+        validate(event, schema=schema)
+    except:
+        # !? To return 'measurement': {'bucket-key': None, 'timestamp': event['income-timestamp']}


Leftover comments?

I had a doubt to clarify. That's why I commented on it. When there is an exception, is it enough to return only {'result',' status' }? I noticed in some cases, that some values do exist and can be returned. For example, here, 'timestamp' can be returned in 'measurements'

@niranjank2022 Yes, in that scenario we should return the exception contents (as string) and a failure status.

mcopik · 2023-08-13T12:17:50Z

benchmarks/000.microbenchmarks/030.clock-synchronization/python/function.py

-                print("Can't setup the connection")
-                break
+                server_socket.close()
+                # !? To return 'measurement': {'bucket-key': None, 'timestamp': event['income-timestamp']}


Leftover comments?

mcopik · 2023-08-13T12:20:41Z

benchmarks/100.webapps/120.uploader/python/function.py

@@ -28,12 +46,12 @@ def handler(event):
    process_time = (process_end - process_begin) / datetime.timedelta(microseconds=1)
    upload_time = (upload_end - upload_begin) / datetime.timedelta(microseconds=1)
    return {
-            'result': {
+            'status': 'success',


We also need to update the results parsing in sebs/faas/function.py and sebs/{platform}/triggers.py - make sure that we check for status, and then retrieve results from measurement.

niranjank2022 · 2023-08-24T09:49:50Z

@mcopik
I don't know how to deploy the benchmarks in aws. I need assistance with that.

niranjank2022 · 2023-08-24T10:03:56Z

@mcopik I don't know how to deploy the benchmarks in aws. I need assistance with that.

Meanwhile, I tried launching the benchmarks locally in minio storage. I noticed that whenever we launch the Docker containers for the benchmarks using this command -
./sebs.py local start 110.dynamic-html test out_benchmark.json --config config/local_deployment.json --deployments 1
it modifies the out_benchmark.json. Input object is generated with a default value for each benchmark. I copy pasted that input as data to invoke the function. My doubt, is this the propriety way to test locally?

And while doing this, I observed that the benchmarks 020, 030 and 040 returned status: 'failure'. and 311 returned error. I wanted to bring this to your notice.

niranjank2022 · 2023-08-24T10:06:47Z

@mcopik I don't know how to deploy the benchmarks in aws. I need assistance with that.

Meanwhile, I tried launching the benchmarks locally in minio storage. I noticed that whenever we launch the Docker containers for the benchmarks using this command - ./sebs.py local start 110.dynamic-html test out_benchmark.json --config config/local_deployment.json --deployments 1 it modifies the out_benchmark.json. Input object is generated with a default value for each benchmark. I copy pasted that input as data to invoke the function. My doubt, is this the propriety way to test locally?

And while doing this, I observed that the benchmarks 020, 030 and 040 returned status: 'failure'. and 311 returned error. I wanted to bring this to your notice.

And I did some changes for incorporating jsonschema in requirements.txt. When I try to push my code, I am getting this warning:

remote: Permission to niranjank2022/serverless-benchmarks.git denied to niranjank2022.
fatal: unable to access 'https://github.com/niranjank2022/serverless-benchmarks.git/': The requested URL returned error: 403

Could you please help me resolve this?

mcopik · 2023-09-05T15:04:27Z

@mcopik I don't know how to deploy the benchmarks in aws. I need assistance with that.

Meanwhile, I tried launching the benchmarks locally in minio storage. I noticed that whenever we launch the Docker containers for the benchmarks using this command - ./sebs.py local start 110.dynamic-html test out_benchmark.json --config config/local_deployment.json --deployments 1 it modifies the out_benchmark.json. Input object is generated with a default value for each benchmark. I copy pasted that input as data to invoke the function. My doubt, is this the propriety way to test locally?

And while doing this, I observed that the benchmarks 020, 030 and 040 returned status: 'failure'. and 311 returned error. I wanted to bring this to your notice.

@niranjank2022 Correct - these are internal microbenchmarks that will not work outside of an experiment. I need to update the documentation for that.

Launching locally looks correct as long as you get a proper HTTP response- :)

EDIT: What error do you observe on benchmark 311? This one shouldn't fail.

mcopik · 2023-09-05T15:09:42Z

@mcopik I don't know how to deploy the benchmarks in aws. I need assistance with that.

@niranjank2022 There is documentation on configuring the AWS account. Once you do that, you can just run ./sebs.py benchmark invoke .... Everything benchmarks need for execution should be covered by AWS Free Tier.

Can you please elaborate on which of these steps are failing for you or if you don't know how to start with them?

https://github.com/spcl/serverless-benchmarks/blob/master/docs/platforms.md

mcopik · 2023-09-05T15:43:24Z

And I did some changes for incorporating jsonschema in requirements.txt. When I try to push my code, I am getting this warning:
remote: Permission to niranjank2022/serverless-benchmarks.git denied to niranjank2022.
fatal: unable to access 'https://github.com/niranjank2022/serverless-benchmarks.git/': The requested URL returned error: 403
Could you please help me resolve this?

This seems like a problem with your local git configuration: maybe replacing the address from https:// to [email protected] could help.

niranjank2022 · 2023-09-18T18:39:50Z

And I did some changes for incorporating jsonschema in requirements.txt. When I try to push my code, I am getting this warning:
remote: Permission to niranjank2022/serverless-benchmarks.git denied to niranjank2022.
fatal: unable to access 'https://github.com/niranjank2022/serverless-benchmarks.git/': The requested URL returned error: 403
Could you please help me resolve this?
This seems like a problem with your local git configuration: maybe replacing the address from https:// to [email protected] could help.

I resolved this problem of mine. I used Personal tokens with certain scopes.

mcopik · 2024-05-13T14:22:42Z

@niranjank2022 Hi! Do you plan to continue working on this PR? IIRC, it has not been tested on AWS?

niranjank2022 · 2024-05-13T14:37:24Z

@niranjank2022 Hi! Do you plan to continue working on this PR? IIRC, it has not been tested on AWS?

I am extremely sorry @mcopik. I got into some serious health problems and because of that, I have not got any time to work on Opensource. I will be mostly back by next month. Is it possible that I can continue from where I left off next month?

mcopik · 2024-05-13T14:41:45Z

@niranjank2022 Sure, no worries! Let me know if you have any questions.

niranjank2022 and others added 5 commits June 28, 2023 22:35

Add result and status in return dictionary

dcdceb0

Add result and status in return dictionary

07e30c6

Sanitize handler.py in 000, 100, 200

e243fb9

Sanitize 300, 400, 500

448e2cf

Merge branch 'spcl:master' into sanitize-input

e4cda49

mcopik requested changes Aug 13, 2023

View reviewed changes

niranjank2022 and others added 2 commits August 23, 2023 19:37

Merge branch 'spcl:master' into sanitize-input

c094568

incorporate jsonschema in requirements.txt

d47e7f7

Merge branch 'spcl:master' into sanitize-input

0ca8eb4

mcopik mentioned this pull request Mar 22, 2024

Sanitize input on Python benchmarks #118

Open

mcopik mentioned this pull request Jul 26, 2024

Standardize invocation output #68

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sanitize input #173

Sanitize input #173

niranjank2022 commented Aug 4, 2023

niranjank2022 commented Aug 4, 2023

mcopik left a comment

mcopik Aug 13, 2023

niranjank2022 Aug 13, 2023

mcopik Aug 23, 2023

mcopik Aug 13, 2023

mcopik Aug 13, 2023

niranjank2022 Aug 13, 2023

mcopik Aug 23, 2023

mcopik Aug 13, 2023

mcopik Aug 13, 2023

niranjank2022 commented Aug 24, 2023

niranjank2022 commented Aug 24, 2023

niranjank2022 commented Aug 24, 2023

mcopik commented Sep 5, 2023 •

edited

Loading

mcopik commented Sep 5, 2023

mcopik commented Sep 5, 2023

niranjank2022 commented Sep 18, 2023

mcopik commented May 13, 2024

niranjank2022 commented May 13, 2024

mcopik commented May 13, 2024

Sanitize input #173

Are you sure you want to change the base?

Sanitize input #173

Conversation

niranjank2022 commented Aug 4, 2023

niranjank2022 commented Aug 4, 2023

mcopik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niranjank2022 commented Aug 24, 2023

niranjank2022 commented Aug 24, 2023

niranjank2022 commented Aug 24, 2023

mcopik commented Sep 5, 2023 • edited Loading

mcopik commented Sep 5, 2023

mcopik commented Sep 5, 2023

niranjank2022 commented Sep 18, 2023

mcopik commented May 13, 2024

niranjank2022 commented May 13, 2024

mcopik commented May 13, 2024

mcopik commented Sep 5, 2023 •

edited

Loading