Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Values won't work #2

Open
AlexPaiva opened this issue Feb 6, 2024 · 2 comments
Open

Custom Values won't work #2

AlexPaiva opened this issue Feb 6, 2024 · 2 comments

Comments

@AlexPaiva
Copy link

import noble_tls


async def main():
    await noble_tls.update_if_necessary() # Update TLS client libs from bogdanfinn/tls-client
    
    session = noble_tls.Session(
        ja3_string="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0",
        h2_settings={
            "HEADER_TABLE_SIZE": 65536,
            "MAX_CONCURRENT_STREAMS": 1000,
            "INITIAL_WINDOW_SIZE": 6291456,
            "MAX_HEADER_LIST_SIZE": 262144
        },
        h2_settings_order=[
            "HEADER_TABLE_SIZE",
            "MAX_CONCURRENT_STREAMS",
            "INITIAL_WINDOW_SIZE",
            "MAX_HEADER_LIST_SIZE"
        ],
        supported_signature_algorithms=[
            "ECDSAWithP256AndSHA256",
            "PSSWithSHA256",
            "PKCS1WithSHA256",
            "ECDSAWithP384AndSHA384",
            "PSSWithSHA384",
            "PKCS1WithSHA384",
            "PSSWithSHA512",
            "PKCS1WithSHA512",
        ],
        supported_versions=["GREASE", "1.3", "1.2"],
        key_share_curves=["GREASE", "X25519"],
        cert_compression_algo="brotli",
        pseudo_header_order=[
            ":method",
            ":authority",
            ":scheme",
            ":path"
        ],
        connection_flow=15663105,
        header_order=[
            "accept",
            "user-agent",
            "accept-encoding",
            "accept-language"
        ]
    )

    res = await session.post(
        "https://www.example.com/",
        headers={
            "key1": "value1",
        },
        proxy="http://user:password@host:port"
    )
    print(res.text)

Won't work and give tls can't decode error.

@scrapoxy
Copy link

Hello,

I've got the same issue when I use the example:

import asyncio
import noble_tls

async def main():
    await noble_tls.update_if_necessary() # Update TLS client libs from bogdanfinn/tls-client

    session = noble_tls.Session(
        ja3_string="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0",
        h2_settings={
            "HEADER_TABLE_SIZE": 65536,
            "MAX_CONCURRENT_STREAMS": 1000,
            "INITIAL_WINDOW_SIZE": 6291456,
            "MAX_HEADER_LIST_SIZE": 262144
        },
        h2_settings_order=[
            "HEADER_TABLE_SIZE",
            "MAX_CONCURRENT_STREAMS",
            "INITIAL_WINDOW_SIZE",
            "MAX_HEADER_LIST_SIZE"
        ],
        supported_signature_algorithms=[
            "ECDSAWithP256AndSHA256",
            "PSSWithSHA256",
            "PKCS1WithSHA256",
            "ECDSAWithP384AndSHA384",
            "PSSWithSHA384",
            "PKCS1WithSHA384",
            "PSSWithSHA512",
            "PKCS1WithSHA512",
        ],
        supported_versions=["GREASE", "1.3", "1.2"],
        key_share_curves=["GREASE", "X25519"],
        cert_compression_algo="brotli",
        pseudo_header_order=[
            ":method",
            ":authority",
            ":scheme",
            ":path"
        ],
        connection_flow=15663105,
        header_order=[
            "accept",
            "user-agent",
            "accept-encoding",
            "accept-language"
        ]
    )

    r = await session.get(
        "https://tools.scrapfly.io/api/fp/ja3",
    )
    payload = r.json()
    ja3_digest = payload["ja3_digest"]
    ja3 = payload["ja3"]

    print("JA3 Hash: ", ja3_digest)
    print("JA3: ", ja3)

asyncio.run(main())

The error is:

/home/fabien/.virtualenvs/scrapy/bin/python /home/fabien/gitrepos/scrapynobletls/test_nobletls.py 
Traceback (most recent call last):
  File "/home/fabien/gitrepos/scrapynobletls/test_nobletls.py", line 60, in <module>
    asyncio.run(main())
  File "/home/fabien/.pyenv/versions/3.11.3/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/fabien/.pyenv/versions/3.11.3/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fabien/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/fabien/gitrepos/scrapynobletls/test_nobletls.py", line 50, in main
    r = await session.get(
        ^^^^^^^^^^^^^^^^^^
  File "/home/fabien/.virtualenvs/scrapy/lib/python3.11/site-packages/noble_tls/sessions.py", line 441, in get
    return await self.execute_request(method="GET", url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fabien/.virtualenvs/scrapy/lib/python3.11/site-packages/noble_tls/sessions.py", line 409, in execute_request
    raise TLSClientException(response_object["body"])
noble_tls.exceptions.exceptions.TLSClientException: failed to do request: Get "https://tools.scrapfly.io/api/fp/ja3": remote error: tls: unexpected message

Process finished with exit code 1

Any idea ?

@rawandahmad698
Copy link
Owner

Hello, apologies for the delay. As you are aware this library is simply a wrapper around the Go TLS executable, we simply pass arguments to it. I'm not sure what the issue is, but I will update you if I find something and fix it along the way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants