Skip to content

Latest commit

 

History

History
233 lines (230 loc) · 6.94 KB

97.md

File metadata and controls

233 lines (230 loc) · 6.94 KB

Results of GPT-4 on HumanEval/97

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 124

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """return abs(a % 10) * abs(b % 10)

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate(148, 412) == 16, "First test error: " + str(candidate(148, 412))                    
    assert candidate(19, 28) == 72, "Second test error: " + str(candidate(19, 28))           
    assert candidate(2020, 1851) == 0, "Third test error: " + str(candidate(2020, 1851))
    assert candidate(14,-15) == 20, "Fourth test error: " + str(candidate(14,-15))      
    assert candidate(76, 67) == 42, "Fifth test error: " + str(candidate(76, 67))      
    assert candidate(17, 27) == 49, "Sixth test error: " + str(candidate(17, 27))      


    # Check some edge cases that are easy to work out by hand.
    assert candidate(0, 1) == 0, "1st edge test error: " + str(candidate(0, 1))
    assert candidate(0, 0) == 0, "2nd edge test error: " + str(candidate(0, 0))

Runs

✅ Run 1

Start time: 2023-07-18 18:16:42
Duration: 10.18s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 2

Start time: 2023-07-18 18:16:33
Duration: 9.25s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 3

Start time: 2023-07-18 18:16:19
Duration: 12.24s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 4

Start time: 2023-07-18 18:16:10
Duration: 9.17s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 5

Start time: 2023-07-18 18:15:59
Duration: 11.23s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 6

Start time: 2023-07-18 18:15:46
Duration: 12.44s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 7

Start time: 2023-07-18 18:15:31
Duration: 15.49s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 8

Start time: 2023-07-18 18:15:20
Duration: 10.37s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 9

Start time: 2023-07-18 18:15:08
Duration: 12.44s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)

✅ Run 10

Start time: 2023-07-18 18:14:56
Duration: 11.27s
Completion tokens: 115
Result: Pass

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """
    return (abs(a) % 10) * (abs(b) % 10)