Skip to content

Commit

Permalink
website: Use separate model and dataset files
Browse files Browse the repository at this point in the history
* This should allow the live website to use the files
  • Loading branch information
tedgravlin committed Dec 2, 2023
1 parent 9b02edf commit 91f30d1
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 9 deletions.
18 changes: 18 additions & 0 deletions docs/dataset/testdataset.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Label,Num Of Sections,TLD,TLD Length,Domain,Domain Length,URL
good,2,com,3,google,6,google.com
good,2,nl,2,google,6,google.nl
good,2,edu,3,emich,5,emich.edu
good,3,edu,3,emich,5,canvas.emich.edu
good,3,com,3,www.theverge,8,www.theverge.com
good,2,com,3,theverge,8,theverge.com
bad,2,com,3,824555,6,824555.com
bad,2,com,3,retajconsultancy,16,retajconsultancy.com
bad,2,info,4,9779,4,9779.info
bad,2,com,3,chinacxyy,9,chinacxyy.com
bad,3,vn,2,com,3,grasslandhotel.com.vn
bad,3,com,3,readmyweather,13,microencapsulation.readmyweather.com
bad,2,com,3,0068555,7,0068555.com
bad,2,com,3,haishundl,9,haishundl.com
bad,2,nl,2,zoetekroon,10,zoetekroon.nl
bad,2,com,3,socialsocietyedu,16,socialsocietyedu.com
bad,2,ru,2,3cf,3,3cf.ru
6 changes: 1 addition & 5 deletions docs/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
from pyscript import display
from sklearn.preprocessing import StandardScaler
from scipy.sparse import hstack
import os
from pyweb import pydom
from os.path import exists

# Get the input container element
input_container = pydom['#input-container']
Expand All @@ -21,13 +21,9 @@ def load_files():
tfidf = joblib.load('tfidf.pkl')
test_dataset = pd.read_csv("testdataset.csv")

progress_text.innerText = "Model load complete."

return model, tfidf, test_dataset

def test_model(model, tfidf, test_dataset):
progress_text.innerText = "Testing URL against model..."

# Turn the test dataset into a pandas data frame
dataframe = pd.DataFrame(test_dataset)
x = dataframe[['Num Of Sections', 'TLD', 'TLD Length', 'Domain', 'Domain Length', 'URL']]
Expand Down
3 changes: 3 additions & 0 deletions docs/models/model.pkl
Git LFS file not shown
3 changes: 3 additions & 0 deletions docs/models/tfidf.pkl
Git LFS file not shown
6 changes: 3 additions & 3 deletions docs/pyscript.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"scikit-learn"
],
"files": {
"../models/model.pkl": "model.pkl",
"../models/tfidf.pkl": "tfidf.pkl",
"../dataset/testdataset.csv": "testdataset.csv"
"./models/model.pkl": "model.pkl",
"./models/tfidf.pkl": "tfidf.pkl",
"./dataset/testdataset.csv": "testdataset.csv"
}
}
1 change: 0 additions & 1 deletion models/readme.md

This file was deleted.

0 comments on commit 91f30d1

Please sign in to comment.