Skip to content
This repository has been archived by the owner on Oct 31, 2023. It is now read-only.

KeyError: '</s>' #153

Open
pradeepdev-1995 opened this issue Jun 29, 2023 · 0 comments
Open

KeyError: '</s>' #153

pradeepdev-1995 opened this issue Jun 29, 2023 · 0 comments

Comments

@pradeepdev-1995
Copy link

Tried the below code

from models import InferSent
import torch
model_version = 1
MODEL_PATH = "/home/encoder/infersent1.pkl"
params_model = {'bsize': 64, 'word_emb_dim': 300, 'enc_lstm_dim': 2048,
                'pool_type': 'max', 'dpout_model': 0.0, 'version': model_version}
model = InferSent(params_model)
model.load_state_dict(torch.load(MODEL_PATH))

use_cuda = False
model = model.cuda() if use_cuda else model

W2V_PATH = '/home/GLOVE/glove.6B/glove.6B.300d.txt'
model.set_w2v_path(W2V_PATH)

model.build_vocab_k_words(K=100000)

# Load some sentences
sentences = ["I ate dinner.",
       "We had a three-course meal.",
       "Brad came to dinner with us.",
       "He loves fish tacos.",
       "In the end, we all felt like we ate too much.",
       "We all agreed; it was a magnificent evening."]
with open('samples.txt') as f:
    for line in f:
        sentences.append(line.strip())
print(len(sentences))

embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True)
print('nb sentences encoded : {0}'.format(len(embeddings)))

But it showed the error KeyError: '</s>'.
Where i got wrong?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant