Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

train_task_classify is searching for .rec in wrong folder #6

Closed
sipian opened this issue Apr 21, 2019 · 4 comments
Closed

train_task_classify is searching for .rec in wrong folder #6

sipian opened this issue Apr 21, 2019 · 4 comments

Comments

@sipian
Copy link

sipian commented Apr 21, 2019

When I am running train_task_classify.py

seq_files = [os.path.join(folder, f) for f in listing if f[-4:] == '.rec']
# training: Randomly pick programs
assert len(seq_files) >= samples_per_class, "Cannot sample " + str(samples_per_class) + " from " + str(
len(seq_files)) + " files found in " + folder
X_train += resample(seq_files, replace=False, n_samples=samples_per_class, random_state=seed)
y_train = np.concatenate([y_train, np.array([int(i)] * samples_per_class, dtype=np.int32)])

This assertion is failing (seq_files list is empty).
This is because the .rec files are generated in separate folders prefixed by seq.

ncc/task_utils.py

Lines 259 to 269 in 4e9eaeb

def llvm_ir_to_trainable(folder_ir):
####################################################################################################################
# Setup
assert len(folder_ir) > 0, "Please specify a folder containing the raw LLVM IR"
assert os.path.exists(folder_ir), "Folder not found: " + folder_ir
folder_seq = re.sub('ir', 'seq', folder_ir)
if len(folder_seq) > 0:
print('Preparing to write LLVM IR index sequences to', folder_seq)
if not os.path.exists(folder_seq):
os.makedirs(folder_seq)

But train_task_classify is searching for .rec files in the original IR folder ir_train.

@natalymr
Copy link

Hello, @sipian

I had the same problem and used a workaround described here, the second message: #4
i.e. just change a directory.

@tbennun
Copy link
Collaborator

tbennun commented Apr 27, 2019

Thanks for reporting @sipian and thank you for mentioning the workaround, @natalymr.

I'll try with a clean build and replace the lines as necessary.

Zacharias030 added a commit to Zacharias030/ncc that referenced this issue May 6, 2019
Made changes to let tensorflow only block as much GPU memory as necessary.
Fixed path for saving weights on ctrl+C and path scheme for models and predictions of classifyapp task.
tbennun added a commit that referenced this issue May 13, 2019
Fixes #4, #6 and minor improvements
@tbennun
Copy link
Collaborator

tbennun commented May 13, 2019

@sipian Did #9 fix the issue?

@tbennun
Copy link
Collaborator

tbennun commented Feb 26, 2020

Closing the issue due to old age. Please reopen if necessary!

@tbennun tbennun closed this as completed Feb 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants