Merge pull request #56 from gizatechxyz/feature/add-verify-api

Add verification through API

docs/README.md

@@ -1,5 +1,5 @@
 ---
-description: Giza CLI 0.15.0
+description: Giza CLI 0.15.1
 ---
 
 # Giza CLI

docs/examples/full_transpilation.md

@@ -21,7 +21,7 @@ pip install -r requirements.txt
 Or:
 
 ```bash
-pip install giza-cli==0.15.0 onnx==1.14.1 torch==2.1.0 torchvision==0.16.0
+pip install giza-cli==0.15.1 onnx==1.14.1 torch==2.1.0 torchvision==0.16.0
 ```
 
 We will use the libraries for the following purposes:

docs/frameworks/cairo/deploy.md

@@ -115,6 +115,19 @@ To list the proofs for an endpoint, we can use the `list-proofs` command availab
 ]
 ```
 
+## Verify a proof
+
+After successfully creating a proof for your Orion Cairo model, the next step is to verify its validity. Giza offers a verification method using the `verify` command alongside the `endpoint-id` and `proof-id`.
+
+```console
+> giza endpoints verify --endpoint-id 1 --proof-id "b14bfbcf250b404192765d9be0811c9b"
+[giza][2024-02-20 15:40:48.560] Verifying proof...
+[giza][2024-02-20 15:40:49.288] Verification result: True
+[giza][2024-02-20 15:40:49.288] Verification time: 2.363822541
+```
+
+This way we can easily and quickly verify the proof generated for our model.
+
 ## Download the proof
 
 We can download the proof using the `download-proof` command available for the endpoints: 

docs/frameworks/cairo/verify.md

@@ -10,16 +10,20 @@ To verify a proof by providing the `proof-id`, use the following command:
 giza verify --model-id 1 --version-id 1 --proof-id 1 --size S
 ```
 
-Upon successful submission, you will see a confirmation message indicating that the verification job has been created, along with its name and ID. Once the verification process is complete, a success message will confirm the validity of the proof:
+Upon successful submission, you will see a confirmation message indicating that the verification procces has begun. Once the verification process is complete, a success message will confirm the validity of the proof:
 
 ```
-[giza][2023-12-04 19:43:37.686] Verification job created with name 'verify-cairo-20231204-32f44715' and id -> 1 ✅
-[giza][2023-12-04 19:45:18.683] Verification job is successful ✅
+[giza][2024-04-22 13:48:12.236] Verifying proof...
+[giza][2024-04-22 13:48:14.985] Verification result: True
+[giza][2024-04-22 13:48:14.985] Verification time: 2.363822541
+```
+
+A `--use-remote` flag can be added to the command to verify the proof using a job. This flag is optional and its strongly discouraged as verification time is 10x slower than the default method.
 ```
 
 ## Option 2: Verify by Providing the Proof Path
 
-Alternatively, you can verify a proof by specifying the path to the proof file directly:
+Alternatively, you can verify a proof by specifying the path to the proof file directly. This option uses by default the `--use-job` flag to verify the proof using a job. To verify a proof using the proof file path, use the following command:
 
 ```
 giza verify --proof /path/to/the/proof --size S

docs/reference/api/README.md

@@ -76,6 +76,7 @@
 - [`endpoints.list`](./commands.endpoints.md#function-list)
 - [`endpoints.list_jobs`](./commands.endpoints.md#function-list_jobs)
 - [`endpoints.list_proofs`](./commands.endpoints.md#function-list_proofs)
+- [`endpoints.verify`](./commands.endpoints.md#function-verify)
 - [`models.create`](./commands.models.md#function-create): Command to create a model. Asks for the new model's information and validates the input,
 - [`models.get`](./commands.models.md#function-get): Command to create a user. Asks for the new users information and validates the input,
 - [`models.list`](./commands.models.md#function-list): Command to list all models.

docs/reference/api/commands.actions.md

@@ -15,7 +15,7 @@
 ## <kbd>function</kbd> `new`
 
 ```python
-new(project_name: str = <typer.models.ArgumentInfo object at 0x1043e6550>)
+new(project_name: str = <typer.models.ArgumentInfo object at 0x105915950>)
 ```
 
 This command will create a new action by generating a Python project. 

docs/reference/api/frameworks.cairo.md

@@ -59,7 +59,7 @@ deploy(
     data: Optional[str] = None,
     size: ServiceSize = <ServiceSize.S: 'S'>,
     debug: Optional[bool] = DEBUG_OPTION
-) → str
+) → Endpoint
 ```
 
 Command to deploy a specific version of a model. This will create an endpoint for the specified version and check the status, once it finishes if COMPLETED the endpoint is ready to be used. 
@@ -139,7 +139,8 @@ verify(
     version_id: Optional[int],
     proof: Optional[str] = None,
     debug: Optional[bool] = False,
-    size: JobSize = <JobSize.S: 'S'>
+    size: JobSize = <JobSize.S: 'S'>,
+    use_job: Optional[bool] = False
 )
 ```
 

docs/reference/api/frameworks.ezkl.md

@@ -86,7 +86,7 @@ deploy(
     version_id: int,
     size: ServiceSize = <ServiceSize.S: 'S'>,
     debug: Optional[bool] = DEBUG_OPTION,
-) → str
+) → None
 ```
 
 Command to deploy a specific version of a model. This will create a endpoint for the specified version and check the status, once it finishes if COMPLETED the endpoint is ready to be used. 

docs/reference/api/utils.decorators.md

@@ -13,12 +13,12 @@
 
 ---
 
-<a href="https://github.com/gizatechxyz/giza-cli/blob/main/giza/utils/decorators.py#L8"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/gizatechxyz/giza-cli/blob/main/giza/utils/decorators.py#L11"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `auth`
 
 ```python
-auth(func: Callable)
+auth(func: Callable[~P, ~R]) → Callable[~P, ~R]
 ```
 
 Check that we have the token and it is not expired before executing 

examples/mnist/mnist_pytorch.ipynb

@@ -41,7 +41,7 @@
     "Or:\n",
     "\n",
     "```bash\n",
-    "pip install giza-cli==0.15.0 onnx==1.14.1 torch==2.1.0 torchvision==0.16.0\n",
+    "pip install giza-cli==0.15.1 onnx==1.14.1 torch==2.1.0 torchvision==0.16.0\n",
     "```\n",
     "\n",
     "We will use the libraries for the following purposes:\n",

examples/mnist/requirements.txt

@@ -1,4 +1,4 @@
-giza-cli==0.15.0
+giza-cli==0.15.1
 onnx==1.14.1
 tf2onnx==1.15.1
 torch==2.1.0

giza/__init__.py

@@ -1,5 +1,5 @@
 import os
 
-__version__ = "0.15.0"
+__version__ = "0.15.1"
 # Until DNS is fixed
 API_HOST = os.environ.get("GIZA_API_HOST", "https://api.gizatech.xyz")

giza/client.py

@@ -1,7 +1,7 @@
 import copy
 import json
 import os
-from io import BufferedReader
+from io import BufferedReader, TextIOWrapper
 from pathlib import Path
 from typing import Any, BinaryIO, Dict, List, Optional, Tuple, Union
 from urllib.parse import urlparse
@@ -20,6 +20,7 @@
 from giza.schemas.models import Model, ModelCreate, ModelList, ModelUpdate
 from giza.schemas.proofs import Proof, ProofList
 from giza.schemas.token import TokenResponse
+from giza.schemas.verify import VerifyResponse
 from giza.schemas.versions import Version, VersionCreate, VersionList, VersionUpdate
 from giza.schemas.workspaces import Workspace
 from giza.utils import echo
@@ -450,7 +451,7 @@ def create(
         model_id: int,
         version_id: int,
         endpoint_create: EndpointCreate,
-        f: BufferedReader,
+        f: Optional[BufferedReader] = None,
     ) -> Endpoint:
         """
         Create a new deployment.
@@ -483,7 +484,7 @@ def create(
         return Endpoint(**response.json())
 
     @auth
-    def list(self, params: Optional[Dict[str, str]] = None) -> EndpointsList:
+    def list(self, params: Optional[Dict[str, Any]] = None) -> EndpointsList:
         """
         List endpoints.
 
@@ -568,7 +569,7 @@ def list_proofs(self, endpoint_id: int) -> ProofList:
         return ProofList(root=[Proof(**proof) for proof in response.json()])
 
     @auth
-    def get_proof(self, endpoint_id: int, proof_id: int) -> Proof:
+    def get_proof(self, endpoint_id: int, proof_id: Union[int, str]) -> Proof:
         """
         Return information about a specific proof.
         `proof_if` is the identifier of the proof that can be a integer or the request id.
@@ -598,7 +599,7 @@ def get_proof(self, endpoint_id: int, proof_id: int) -> Proof:
         return Proof(**response.json())
 
     @auth
-    def download_proof(self, endpoint_id: int, proof_id: int) -> bytes:
+    def download_proof(self, endpoint_id: int, proof_id: Union[int, str]) -> bytes:
         """
         Download a proof.
 
@@ -691,6 +692,41 @@ def delete(self, endpoint_id: int) -> None:
         self._echo_debug(str(response))
         response.raise_for_status()
 
+    @auth
+    def verify_proof(
+        self, endpoint_id: int, proof_id: Union[str, int]
+    ) -> VerifyResponse:
+        """
+        Verify a proof.
+
+        Args:
+            endpoint_id: Endpoint identifier
+            proof_id: Proof identifier
+
+        Returns:
+            The verification response
+        """
+        headers = copy.deepcopy(self.default_headers)
+        headers.update(self._get_auth_header())
+
+        response = self.session.post(
+            "/".join(
+                [
+                    self.url,
+                    self.ENDPOINTS,
+                    str(endpoint_id),
+                    "proofs",
+                    f"{proof_id}:verify",
+                ]
+            ),
+            headers=headers,
+        )
+
+        self._echo_debug(str(response))
+        response.raise_for_status()
+
+        return VerifyResponse(**response.json())
+
 
 # For downstream dependencies until they are updated
 DeploymentsClient = EndpointsClient
@@ -926,8 +962,8 @@ def get(self, job_id: int, params: Optional[dict[str, str]] = None) -> Job:
     def create(
         self,
         job_create: JobCreate,
-        trace: BufferedReader,
-        memory: Optional[BufferedReader] = None,
+        trace: Optional[Union[BufferedReader, TextIOWrapper]] = None,
+        memory: Optional[Union[BufferedReader, TextIOWrapper]] = None,
     ) -> Job:
         """
         Create a new job.
@@ -945,8 +981,10 @@ def create(
         headers = copy.deepcopy(self.default_headers)
         headers.update(self._get_auth_header())
 
-        files = {"trace_or_proof": trace} if trace is not None else None
-        if trace is not None and memory is not None:
+        files: Optional[Dict[str, Any]] = (
+            {"trace_or_proof": trace} if trace is not None else None
+        )
+        if trace is not None and memory is not None and files is not None:
             files["memory"] = memory
         response = self.session.post(
             f"{self.url}/{self.JOBS_ENDPOINT}",
@@ -1027,7 +1065,7 @@ def get(self, model_id: int, version_id: int, job_id: int) -> Job:
 
     @auth
     def create(
-        self, model_id: int, version_id: int, job_create: JobCreate, f: BufferedReader
+        self, model_id: int, version_id: int, job_create: JobCreate, f: TextIOWrapper
     ) -> Job:
         """
         Create a new job.
@@ -1204,6 +1242,30 @@ def list(self) -> List[Proof]:
 
         return [Proof(**proof) for proof in response.json()]
 
+    @auth
+    def verify_proof(self, proof_id: int) -> VerifyResponse:
+        """
+        Verify a proof.
+
+        Args:
+            proof_id: Proof identifier
+
+        Returns:
+            The verification response
+        """
+        headers = copy.deepcopy(self.default_headers)
+        headers.update(self._get_auth_header())
+
+        response = self.session.post(
+            f"{self.url}/{self.PROOFS_ENDPOINT}/{proof_id}:verify",
+            headers=headers,
+        )
+
+        self._echo_debug(str(response))
+        response.raise_for_status()
+
+        return VerifyResponse(**response.json())
+
 
 class VersionsClient(ApiClient):
     """
@@ -1251,7 +1313,7 @@ def get(self, model_id: int, version_id: int) -> Version:
         return Version(**response.json())
 
     @auth
-    def upload_cairo(self, model_id: int, version_id: int, file_path: str) -> str:
+    def upload_cairo(self, model_id: int, version_id: int, file_path: str) -> Version:
         """
         Get the Cairo model URL.
 
@@ -1595,7 +1657,7 @@ def create(
         return Agent(**response.json())
 
     @auth
-    def list(self, params: Optional[Dict[str, str]] = None) -> AgentList:
+    def list(self, params: Optional[Dict[str, Any]] = None) -> AgentList:
         """
         List endpoints.
 

giza/commands/agents.py

@@ -27,13 +27,13 @@
     """,
 )
 def create(
-    model_id: int = typer.Option(
+    model_id: Optional[int] = typer.Option(
         None,
         "--model-id",
         "-m",
         help="The ID of the model used to create the agent",
     ),
-    version_id: int = typer.Option(
+    version_id: Optional[int] = typer.Option(
         None,
         "--version-id",
         "-v",

giza/commands/endpoints.py

@@ -369,3 +369,33 @@ def list_jobs(
         client = EndpointsClient(API_HOST)
         jobs = client.list_jobs(endpoint_id)
     print_json(jobs.json(exclude_unset=True))
+
+
+@app.command(
+    name="verify",
+    short_help="🔍 Verify a proof from an endpoint.",
+    help="""🔍 Verify a proof from an endpoint.
+    This command verifies a proof generated by a specific endpoint.
+    The verify information is printed in a json format for easy readability and further processing.
+    If the endpoint is not available, an error message is printed.
+    """,
+)
+def verify(
+    endpoint_id: int = typer.Option(
+        None,
+        "--deployment-id",
+        "-d",
+        "--endpoint-id",
+        "-e",
+        help="The ID of the endpoint",
+    ),
+    proof_id: str = typer.Option(
+        None, "--proof-id", "-p", help="The ID or request id of the proof"
+    ),
+    debug: Optional[bool] = DEBUG_OPTION,
+) -> None:
+    echo(f"Verifying proof from endpoint {endpoint_id} ✅ ")
+    with ExceptionHandler(debug=debug):
+        client = EndpointsClient(API_HOST)
+        verification = client.verify_proof(endpoint_id, proof_id)
+    print_json(verification.model_dump_json(exclude_unset=True))

giza/commands/verify.py

@@ -13,6 +13,7 @@ def verify(
     model_id: Optional[int] = typer.Option(None, "--model-id", "-m"),
     version_id: Optional[int] = typer.Option(None, "--version-id", "-v"),
     proof_id: Optional[int] = typer.Option(None, "--proof-id", "-p"),
+    use_job: Optional[bool] = typer.Option(False, "--use-job"),
     proof: Optional[str] = typer.Option(None, "--proof", "-P"),
     size: JobSize = typer.Option(JobSize.S, "--size", "-s"),
     framework: Framework = typer.Option(Framework.CAIRO, "--framework", "-f"),
@@ -26,6 +27,7 @@ def verify(
             size=size,
             debug=debug,
             proof=proof,
+            use_job=use_job,
         )
     elif framework == Framework.EZKL:
         ezkl.verify(