23.md

Results of GPT-4 on HumanEval/23

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 69

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """return len(string)

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate('') == 0
    assert candidate('x') == 1
    assert candidate('asdasnakj') == 9

Runs

✅ Run 1

Start time: 2023-07-18 18:47:21
Duration: 4.8s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 2

Start time: 2023-07-18 18:47:18
Duration: 3.35s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 3

Start time: 2023-07-18 18:47:12
Duration: 5.51s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 4

Start time: 2023-07-18 18:47:08
Duration: 4.45s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 5

Start time: 2023-07-18 18:47:04
Duration: 4.26s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 6

Start time: 2023-07-18 18:46:59
Duration: 4.58s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 7

Start time: 2023-07-18 18:46:53
Duration: 5.92s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 8

Start time: 2023-07-18 18:46:50
Duration: 3.31s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 9

Start time: 2023-07-18 18:46:45
Duration: 4.47s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 10

Start time: 2023-07-18 18:46:41
Duration: 3.68s
Completion tokens: 46
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)