Lasso Regression Tutorial

For this tutorial we will be using a dataset contained within a CSV file. You can obtain the CSV file I'm working with here. The CSV file contains data that pertain to the properties of a house when deciding to purchase one such as the size of the house, price, number of bathrooms, number of beds, etc.

⚠️ All of your data (features, observations, etc...) must be floating point values. You can utilize the MLDataManager class's convertMyDataToFloat method.

Downloading the data

Once you have downloaded the CSV file provided (above) we can now start modifying that data to be used with the framework.

⚠️ Note: We will be using the CSVReader framework to parse our CSV data. This framework is included with the installation of MLKit.

// Obtain data from csv file
let path = Bundle(for: [SELF OR OBJECT GOES HERE] ).path(forResource: "kc_house_data", ofType: "csv")
let csvUrl = NSURL(fileURLWithPath: path!)
let file = try! String(contentsOf: csvUrl as URL, encoding: String.Encoding.utf8)
let data = CSVReader(with: file)

Extracting Features

For this example I will be using two features: Square Feet Living ("sqft_living" column of CSV) and Number of Bedrooms ("bedrooms" column CSV). In order to obtain these two columns we will use the columns method like so:

// Setup the features we need and convert them to floats if necessary
let training_data_string = data.columns["sqft_living"]!
let training_data_2_string = data.columns["bedrooms"]!

Since MLKit primarily uses Floats, we will proceed to converting the training data into type Float:

// Features
let training_data = training_data_string.map { Float($0)! }
let training_data_2 = training_data_2_string.map { Float($0)! }

Define Your Observation/Output

// Output
let output_as_string = data.columns["price"]!
let output_data = output_as_string.map { Float($0)! }

Training Our Model

Now that we have extracted our features it's time that we train our model. In order to do so we must instantiate a LassoRegression Object.

let lassoModel = LassoRegression()

Next, we need to instantiate our weights. The weights chosen here are arbitrary. Here we are using the Matrix class which comes from the Upsurge framework.

let initial_weights = Matrix<Float>(rows: 3, columns: 1, elements: [0.0, 0.0, 0.0])

Training the Model

The last step is to train the model. For Lasso Regression, we must specify an l1_penalty and a tolerance. The l1_penalty and the tolerance in this example have been set arbitrarily.

let weights = try! lassoModel.train([feature1, feature2], output: output_data, initialWeights: initial_weights, l1Penalty: l1_penalty, tolerance: tolerance)

Your new weights are available in the weights variable above.

Evaluating the Cost of our Model

The method used to evaluate the cost of our model will be the residual sum of squares (RSS) equation. In order to view the RSS of your current model (after it has been trained) simply call the rss method.

let rss = try! lassoModel.RSS([training_data, training_data_2], observation: output_data)

Making Predictions

In order to create a prediction we simply call the predict method passing in our input values (features) and our weights (the weights we obtained after training our model).

let quickPrediction = lassoModel.predict([Float(1.0), Float(1180.0), Float(1.0)], yourWeights: weights.elements)

You can now access the estimated predicted price of a house (in our example) via the quickPrediction constant. Note that the last index of the input array is the intercept. This is always set to 1.0.