Quickstart
You should not need to build Breeze yourself---instead see the next section---but in case you do, here are the steps:
Breeze is hosted on github and is built with sbt.
Set up sbt following their instructions, and then run sbt. The following targets are useful:
-
compile-- Builds the library -
test-- Runs the unit tests -
doc-- Builds scaladoc for the public API -
publish-local-- Copies jars to local Ivy repository. -
assembly-- Builds a distributable jar -
console-- starts a scala repl
Note: you might need more than the default amount of memory to get Breeze to build. The following environment variable is known to work well:
export SBT_OPTS="-Xmx3g -XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"For SBT, Add these lines to your SBT project definition:
- For SBT versions 0.10.x or later
libraryDependencies ++= Seq(
// other dependencies here
"org.scalanlp" % "breeze_2.10" % "0.7-SNAPSHOT",
// native libraries are not included by default. add this if you want them (as of 0.7-SNAPSHOT)
// native libraries greatly improve performance, but increase jar sizes.
"org.scalanlp" % "breeze-natives_2.10" % "0.7-SNAPSHOT"
)
resolvers ++= Seq(
// other resolvers here
// if you want to use snapshot builds (currently 0.7-SNAPSHOT), use this.
"Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/",
"Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/releases/"
)
//snapshots:
libraryDependencies ++= Seq(
"org.scalanlp" %% "breeze" % "0.7-SNAPSHOT"
)Then run sbt update so SBT will download them from maven central.
Breeze is modeled on Scalala, and so if you're familiar with it, you'll be familiar with Breeze. First, import the linear algebra package:
scala> import breeze.linalg._Let's create a vector:
scala> val x = DenseVector.zeros[Double](5)
x: breeze.linalg.DenseVector[Double] = DenseVector(0.0, 0.0, 0.0, 0.0, 0.0)Here we make a column vector of zeros of type Double. And there are other ways we could create the vector - such as with a literal DenseVector(1,2,3) or with a call to fill or tabulate. Unlike Scalala, all Vectors are column vectors. Row vectors are represented as matrices.
The vector object supports accessing and updating data elements by their index in 0 to x.length-1. The vector is "dense" because it is backed by an Array[Double], but could as well have created a SparseVector.zeros[Double](5), which would not allocate memory for zeros.
scala> x(0)
res0: Double = 0.0
scala> x(1) = 2
scala> x
res2: breeze.linalg.DenseVector[Double] = DenseVector(0.0, 2.0, 0.0, 0.0, 0.0)Breeze also supports slicing. Note that slices using a Range are much, much faster than those with an arbitrary sequence.
scala> x(3 to 4) := .5
res3: breeze.linalg.DenseVector[Double] = DenseVector(0.5, 0.5)
scala> x
res4: breeze.linalg.DenseVector[Double] = DenseVector(0.0, 2.0, 0.0, 0.5, 0.5)The slice operator constructs a read-through and write-through view of the given elements in the underlying vector. You set its values using the vectorized-set operator :=. You could as well have set it to a compatibly sized Vector.
scala> x(0 to 1) := DenseVector(.1,.2)
scala> x
res6: breeze.linalg.DenseVector[Double] = DenseVector(0.1, 0.2, 0.0, 0.5, 0.5)Similarly, a DenseMatrix can be created with a constructor method call, and its elements can be accessed and updated.
scala> val m = DenseMatrix.zeros[Int](5,5)
m: breeze.linalg.DenseMatrix[Int] =
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0 The columns of m can be accessed as DenseVectors, and the rows as DenseMatrices.
scala> (m.rows, m.cols)
res10: (Int, Int) = (5,5)
scala> m(::,1)
res7: breeze.linalg.DenseVector[Int] = DenseVector(0, 0, 0, 0, 0)
scala> m(4,::) := DenseVector(1,2,3,4,5).t // transpose to match row shape
res8: breeze.linalg.DenseMatrix[Int] = 1 2 3 4 5
scala> m
res9: breeze.linalg.DenseMatrix[Int] =
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
1 2 3 4 5 Assignments with incompatible cardinality or a larger numeric type won't compile. Assignments with incompatible size will throw an exception:
scala> m := x
<console>:13: error: could not find implicit value for parameter op: breeze.linalg.operators.BinaryUpdateOp[breeze.linalg.DenseMatrix[Int],breeze.linalg.DenseVector[Double],breeze.linalg.operators.OpSet]
m := x
^
scala> m := DenseMatrix.zeros[Int](3,3)
java.lang.IllegalArgumentException: requirement failed: Matrixs must have same number of rowSub-matrices can be sliced and updated, and literal matrices can be specified using a simple tuple-based syntax. Unlike Scalala, only range slices are supported, and only the columns (or rows for a transposed matrix) can have a Range step size different form 1.
scala> m(0 to 1, 0 to 1) := DenseMatrix((3,1),(-1,-2))
res12: breeze.linalg.DenseMatrix[Int] =
3 1
-1 -2
scala> m
res13: breeze.linalg.DenseMatrix[Int] =
3 1 0 0 0
-1 -2 0 0 0
0 0 0 0 0
0 0 0 0 0
1 2 3 4 5 All Tensors support a set of operators, similar to those used in Matlab or Numpy. See Breeze Linear Algebra for a list of most of the operators and various operations. Some of the basic ones are reproduced here, to give you an idea.
| Operation | Breeze | Matlab | Numpy |
|---|---|---|---|
| Elementwise addition | a + b |
a + b |
a + b |
| Elementwise multiplication | a :* b |
a .* b |
a * b |
| Elementwise comparison | a :< b |
a < b (gives matrix of 1/0 instead of true/false) |
a < b |
| Inplace addition | a :+= 1.0 |
a += 1 |
a += 1 |
| Inplace elementwise multiplication | a :*= 2.0 |
a *= 2 |
a *= 2 |
| Vector dot product | a dot b |
dot(a,b) |
dot(a,b) |
| Elementwise sum | sum(a) |
sum(sum(a)) |
a.sum() |
| Elementwise max | a.max |
max(a) |
a.max() |
| Elementwise argmax | argmax(a) |
argmax(a) |
a.argmax() |
| Ceiling | ceil(a) |
ceil(a) |
ceil(a) |
| Floor | floor(a) |
floor(a) |
floor(a) |
Sometimes we want to apply an operation to every row or column of a matrix, as a unit. For instance, you might want to compute the mean of each row, or add a vector to every column. Adapting a matrix so that operations can be applied columnwise or rowwise is called broadcasting. Languages like R and numpy automatically and implicitly do broadcasting, meaning they won't stop you if you accidentally add a matrix and a vector. In Breeze, you have to signal your intent using the broadcasting operator *. The * is meant to evoke "foreach" visually. Here are some examples:
val dm = DenseMatrix((1.0,2.0,3.0),
(4.0,5.0,6.0))
val res = dm(::, *) + DenseVector(3.0, 4.0)
assert(res === DenseMatrix((4.0, 5.0, 6.0), (8.0, 9.0, 10.0)))
res(::, *) := DenseVector(3.0, 4.0)
assert(res === DenseMatrix((3.0, 3.0, 3.0), (4.0, 4.0, 4.0)))
val m = DenseMatrix((1.0, 3.0), (4.0, 4.0))
assert(mean(m(*, ::)) === DenseVector(2.0, 4.0))
assert(mean(m(::, *)) === DenseMatrix((2.5, 3.5)))Breeze also provides a fairly large number of probability distributions built in. These come with access to either probability density function (for discrete distributions) or pdf functions (for continuous distributions). Many distributions also have methods for giving the mean and the variance.
scala> val poi = new Poisson(3.0);
poi: breeze.stats.distributions.Poisson = <function1>
scala> poi.sample(10);
res21: List[Int] = List(3, 5, 5, 2, 2, 1, 1, 2, 4, 1)
scala> res21 map { poi.probabilityOf(_) }
res23: List[Double] = List(0.6721254229661636, 0.504094067224622, 0.504094067224622, 0.44808361531077556, 0.44808361531077556, 0.1493612051035918, 0.1493612051035918, 0.44808361531077556, 0.6721254229661628, 0.1493612051035918)
scala> val doublePoi = for(x <- poi) yield x.toDouble; // meanAndVariance requires doubles, but Poisson samples over Ints
doublePoi: java.lang.Object with breeze.stats.distributions.Rand[Double] = breeze.stats.distributions.Rand$$anon$2@2c98070c
scala> breeze.stats.DescriptiveStats.meanAndVariance(doublePoi.samples.take(1000));
res29: (Double, Double) = (3.018,2.9666426426426447)
scala> (poi.mean,poi.variance)
res30: (Double, Double) = (3.0,3.0)TODO: exponential families
Breeze's optimization package includes several convex optimization routines and a simple linear program solver. Convex optimization routines typically take a DiffFunction[T], which is a Function1 extended to have a gradientAt method, which returns the gradient at a particular point. Most routines will require a breeze.linalg-enabled type: something like a Vector or a Counter.
Here's a simple DiffFunction: a parabola along each vector's coordinate.
scala> import breeze.optimize._
import breeze.optimize._
scala> import breeze.linalg._
import breeze.linalg._
scala> val f = new DiffFunction[DenseVector[Double]] {
| def calculate(x: DenseVector[Double]) = {
| (norm((x - 3.) :^ 2.,1.),(x * 2.) - 6.);
| }
| }
f: java.lang.Object with breeze.optimize.DiffFunction[breeze.linalg.DenseVector[Double]] = $anon$1@617746b2Note that this function takes its minimum when all values are 3. (It's just a parabola along each coordinate.)
scala> f.valueAt(DenseVector(0,0,0))
res0: Double = 27.0
scala> f.valueAt(DenseVector(3,3,3))
res1: Double = 0.0
scala> f.gradientAt(DenseVector(3,0,1))
res2: breeze.linalg.DenseVector[Double] = DenseVector(0.0, -6.0, -4.0)
scala> f.calculate(DenseVector(0,0))
res3: (Double, breeze.linalg.DenseVector[Double]) = (18.0,DenseVector(-6.0, -6.0))
You can also use approximate derivatives, if your function is easy enough to compute:
scala> def g(x: DenseVector[Double]) = (x - 3.0):^ 2.0 sum
g: (x: breeze.linalg.DenseVector[Double])Double
scala> g(DenseVector(0.,0.,0.))
res5: Double = 27.0
scala> val diffg = new ApproximateGradientFunction(g)
diffg: breeze.optimize.ApproximateGradientFunction[Int,breeze.linalg.DenseVector[Double]] = <function1>
scala> diffg.gradientAt(DenseVector(3,0,1))
res5: breeze.linalg.DenseVector[Double] = DenseVector(1.000000082740371E-5, -5.999990000127297, -3.999990000025377)Ok, now let's optimize f. The easiest routine to use is just LBFGS, which is a quasi-Newton method that works well for most problems.
scala> val lbfgs = new LBFGS[DenseVector[Double]](maxIter=100, m=3) // m is the memory. anywhere between 3 and 7 is fine. The larger m, the more memory is needed.
lbfgs: breeze.optimize.LBFGS[breeze.linalg.DenseVector[Double]] = breeze.optimize.LBFGS@43e43267
scala> lbfgs.minimize(f,DenseVector(0,0,0))
scala> lbfgs.minimize(f,DenseVector(0,0,0))
res6: breeze.linalg.DenseVector[Double] = DenseVector(2.9999999999999973, 2.9999999999999973, 2.9999999999999973)
scala> f(res6)
res8: Double = 2.129924444096732E-29
That's pretty close to 0! You can also use a configurable optimizer, using FirstOrderMinimizer.OptParams. It takes several parameters:
case class OptParams(batchSize:Int = 512,
regularization: Double = 1.0,
alpha: Double = 0.5,
maxIterations:Int = -1,
useL1: Boolean = false,
tolerance:Double = 1E-4,
useStochastic: Boolean= false) {
// ...
}
batchSize applies to BatchDiffFunctions, which support using small minibatches of a dataset. regularization integrates L2 or L1 (depending on useL1) regularization with constant lambda. alpha controls the initial stepsize for algorithms that need it. maxIterations is the maximum number of gradient steps to be taken (or -1 for until convergence). tolerance controls the sensitivity of the convergence check. Finally, useStochastic determines whether or not batch functions should be optimized using a stochastic gradient algorithm (using small batches), or using LBFGS (using the entire dataset).
OptParams can be controlled using breeze.config.Configuration, which we described earlier.
We provide a DSL for solving linear programs, along with two methods for solving them. (I recommend InteriorPoint.) This package isn't industrial strength yet by any means, but it's good for simple problems. The DSL is pretty simple:
val lp = new LinearProgram()
import lp._
val x0 = Real()
val x1 = Real()
val x2 = Real()
val lpp = ( (x0 + x1 * 2 + x2 * 3 )
subjectTo ( x0 * -1 + x1 + x2 <= 20)
subjectTo ( x0 - x1 * 3 + x2 <= 30)
subjectTo ( x0 <= 40 )
)
val result = maximize( lpp)
assert( (result.result - DenseVector(40.,17.5,42.5)).norm(2) < 1E-4)
We also have specialized routines for bipartite matching (KuhnMunkres and CompetitiveLinking).
**This API is highly experimental. It may change greatly. **
Breeze continues most of the functionality of Scalala's plotting facilities, though the API is somewhat different (in particular, more object oriented.) These methods are documented in scaladoc (sbt doc) for the traits in the breeze.plot package object. First, let's plot some lines and save the image to file. All the actual plotting work is done by the excellent JFreeChart library.
import breeze.plot._
import breeze.linalg._
val f = Figure()
val p = f.subplot(0)
val x = linspace(0.0,1.0)
p += plot(x, x :^ 2.0)
p += plot(x, x :^ 3.0, '.')
p.xlabel = "x axis"
p.ylabel = "y axis"
f.saveas("lines.png") // save current figure as a .png, eps and pdf also supported
Then we'll add a new subplot and plot a histogram of 100,000 normally distributed random numbers into 100 buckets.
val p2 = f.subplot(2,1,1)
val g = breeze.stats.distributions.Gaussian(0,1)
p2 += hist(g.sample(100000),100)
p2.title = "A normal distribution"
f.saveas("subplots.png")
Breeze also supports the Matlab-like "image" command, here imaging a random matrix.
val f2 = Figure()
f2.subplot(0) += image(DenseMatrix.rand(200,200))
f2.saveas("image.png")
breeze.config.Configuration is a general mechanism for handling program configuration. It relies on reflection and bytecode munging to work. In particular, classes defined in the REPL won't work.
Configurations can be made in three ways:
- from a Map[String,String]:
scala Configuration.fromMap(m) - from Java-style properties files:
scala Configuration.fromPropertiesFiles(Seq(f1,f2,f3)) - from command line arguments:
scala CommandLineParser.parseArguments(args)
For basic usage, you can read in a property with scalaconfig.readIn[T]("property.name", [optional default]). If a property named "property.name" cannot be found, it will look for just "name". If that fails, it will use either the default or throw an exception, if none is provided.
Configuration also supports reflectively processing case classes. As an example, consider configuration options for training a classifier:
case class TrainerParams(
@Help(text="The kind of classifier to train. {Logistic,SVM,Pegasos}") `type`: String= "Logistic",
@Help(text="Input file in svm light format.") input: File= new java.io.File("train"),
@Help(text="Output file for the serialized classifier.") output: File = new File("classifier.ser"),
@Help(text="Optimization parameters.") opt: OptParams,
@Help(text="Prints this") help:Boolean = false)We can read in a TrainerParams by saying config.readIn[TrainerParams]("prefix"). Nested parameters have ".${name}" appended to this prefix.
The listing also illustrates a few other features. We have the Help annotation for displaying usage information with the GenerateHelp object. Configuration also supports Files natively. Recursive case classes are supported.
To specify the arguments, you can use command line arguments like --opt.batchSize 3 --useStochastic true --type Logistic. Anything not specified uses the default arguments provided by Scala. If no default is found, an error is thrown.
Sequences are supported by using --name.0 <value> --name.1 <value1> etc.
I'd like to outsource this package to someone else's library. This is a type-class oriented approach to writing and reading data. The idea is to provide a ReadWritable typeclass implicit for writing. For instance, DenseVector's ReadWritable instance looks like:
implicit object DenseVectorReadWritable extends ReadWritable[DenseVector[Double]] {
override def read(in : DataInput) = {
val offset = DataSerialization.read[Int](in)
val stride = DataSerialization.read[Int](in)
val length = DataSerialization.read[Int](in)
val data = DataSerialization.read[Array[Double]](in)
new DenseVector(data, offset,stride,length)
}
override def write(out : Output, v : DenseVector[Double]) {
DataSerialization.write(out,v.offset)
DataSerialization.write(out,v.stride)
DataSerialization.write(out,v.length)
DataSerialization.write(out,v.data)
}
}
Writing and reading are done though the obvious methods:
DataSerialization.write(output, t)
DataSerialization.read[T](input)