Skip to content

Latest commit

 

History

History
143 lines (140 loc) · 3.79 KB

97.md

File metadata and controls

143 lines (140 loc) · 3.79 KB

Results of GPT-3.5 on HumanEval/97

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 124

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def multiply(a, b):
    """Complete the function that takes two integers and returns 
    the product of their unit digits.
    Assume the input is always valid.
    Examples:
    multiply(148, 412) should return 16.
    multiply(19, 28) should return 72.
    multiply(2020, 1851) should return 0.
    multiply(14,-15) should return 20.
    """return abs(a % 10) * abs(b % 10)

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate(148, 412) == 16, "First test error: " + str(candidate(148, 412))                    
    assert candidate(19, 28) == 72, "Second test error: " + str(candidate(19, 28))           
    assert candidate(2020, 1851) == 0, "Third test error: " + str(candidate(2020, 1851))
    assert candidate(14,-15) == 20, "Fourth test error: " + str(candidate(14,-15))      
    assert candidate(76, 67) == 42, "Fifth test error: " + str(candidate(76, 67))      
    assert candidate(17, 27) == 49, "Sixth test error: " + str(candidate(17, 27))      


    # Check some edge cases that are easy to work out by hand.
    assert candidate(0, 1) == 0, "1st edge test error: " + str(candidate(0, 1))
    assert candidate(0, 0) == 0, "2nd edge test error: " + str(candidate(0, 0))

Runs

✅ Run 1

Start time: 2023-07-18 17:12:58
Duration: 1.42s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 2

Start time: 2023-07-18 17:12:56
Duration: 1.44s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 3

Start time: 2023-07-18 17:12:55
Duration: 1.6s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 4

Start time: 2023-07-18 17:12:53
Duration: 1.76s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 5

Start time: 2023-07-18 17:12:51
Duration: 1.68s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 6

Start time: 2023-07-18 17:12:50
Duration: 1.29s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 7

Start time: 2023-07-18 17:12:48
Duration: 1.49s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 8

Start time: 2023-07-18 17:12:46
Duration: 1.89s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 9

Start time: 2023-07-18 17:12:45
Duration: 1.39s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])

✅ Run 10

Start time: 2023-07-18 17:12:43
Duration: 1.4s
Completion tokens: 21
Result: Pass

def multiply(a, b):
    return int(str(a)[-1]) * int(str(b)[-1])