Skip to content

Latest commit

 

History

History
266 lines (263 loc) · 6.65 KB

77.md

File metadata and controls

266 lines (263 loc) · 6.65 KB
Raw

Results of GPT-3.5 on HumanEval/77

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 5
Total runs: 10
Success rate: 50%

Prompt

Tokens: 136

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this ingeger is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this ingeger is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''a = abs(a)
    return int(round(a ** (1. / 3))) ** 3 == a

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate(1) == True, "First test error: " + str(candidate(1))
    assert candidate(2) == False, "Second test error: " + str(candidate(2))
    assert candidate(-1) == True, "Third test error: " + str(candidate(-1))
    assert candidate(64) == True, "Fourth test error: " + str(candidate(64))
    assert candidate(180) == False, "Fifth test error: " + str(candidate(180))
    assert candidate(1000) == True, "Sixth test error: " + str(candidate(1000))


    # Check some edge cases that are easy to work out by hand.
    assert candidate(0) == True, "1st edge test error: " + str(candidate(0))
    assert candidate(1729) == False, "2nd edge test error: " + str(candidate(1728))

Runs

❌ Run 1

Start time: 2023-07-18 17:16:17
Duration: 5.06s
Completion tokens: 135
Result: Fail

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    root = int(a ** (1/3))
    return root ** 3 == a

✅ Run 2

Start time: 2023-07-18 17:16:12
Duration: 5.13s
Completion tokens: 135
Result: Pass

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    root = round(a ** (1/3))
    return root ** 3 == a

❌ Run 3

Start time: 2023-07-18 17:16:10
Duration: 2.0s
Completion tokens: 30
Result: Fail

def iscube(a):
    return a >= 0 and round(a ** (1/3)) ** 3 == a

✅ Run 4

Start time: 2023-07-18 17:16:04
Duration: 5.37s
Completion tokens: 135
Result: Pass

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    root = round(a ** (1/3))
    return root ** 3 == a

✅ Run 5

Start time: 2023-07-18 17:15:59
Duration: 5.38s
Completion tokens: 135
Result: Pass

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = abs(a)
    root = round(a ** (1/3))
    return root ** 3 == a

✅ Run 6

Start time: 2023-07-18 17:15:54
Duration: 5.2s
Completion tokens: 131
Result: Pass

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    return round(a ** (1/3)) ** 3 == a

❌ Run 7

Start time: 2023-07-18 17:15:48
Duration: 5.13s
Completion tokens: 131
Result: Fail

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = abs(a)
    return int(a ** (1/3)) ** 3 == a

❌ Run 8

Start time: 2023-07-18 17:15:47
Duration: 1.69s
Completion tokens: 25
Result: Fail

def iscube(a):
    return a >= 0 and round(a ** (1/3)) ** 3 == a

✅ Run 9

Start time: 2023-07-18 17:15:41
Duration: 5.23s
Completion tokens: 133
Result: Pass

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    return int(round(a ** (1 / 3))) ** 3 == a

❌ Run 10

Start time: 2023-07-18 17:15:36
Duration: 4.94s
Completion tokens: 131
Result: Fail

def iscube(a):
    '''
    Write a function that takes an integer a and returns True 
    if this integer is a cube of some integer number.
    Note: you may assume the input is always valid.
    Examples:
    iscube(1) ==> True
    iscube(2) ==> False
    iscube(-1) ==> True
    iscube(64) ==> True
    iscube(0) ==> True
    iscube(180) ==> False
    '''
    if a < 0:
        a = -a
    return int(a ** (1/3)) ** 3 == a