Chapter 22: Vectors and Matrices

In this chapter we look at built-in functions which support computation with vectors and matrices.

22.1 The Dot Product Conjunction

Recall the composition of verbs, from Chapter 08. A sum-of-products verb can be composed from sum and product with the @: conjunction.

P =: 2 3 4 Q =: 1 0 2 P * Q +/ P * Q P (+/ @: *) Q 
2 3 4
 
1 0 2
 
2 0 8
 
10
 
10

There is a conjunction . (dot, called "Dot Product"). It can be used instead of @: to compute the sum-of-products of two lists.

P Q P (+/ @: *) Q P (+/ . *) Q 
2 3 4
 
1 0 2
 
10
 
10

Evidently, the . conjunction is a form of composition, a variation of @: or @. We will see below that it is more convenient for working with vectors and matrices.

22.2 Scalar Product of Vectors

Recall that P is a list of 3 numbers. If we interpret these numbers as coordinates of a point in 3-dimensional space, then P can be regarded as defining a vector, a line-segment with length and direction, from the origin at 0 0 0 to the point P. We can refer to the vector P.

With P and Q interpreted as vectors, then the expression P (+/ . *) Q gives what is called the "scalar product" of P and Q. Other names for the same thing are "dot product", or "inner product", or "matrix product", depending on context. In this chapter let us stick to the neutral term "dot product", for which we define a function dot:

dot =: +/ . * P Q P dot Q 
+/ .*
 
2 3 4
 
1 0 2
 
10

A textbook definition of scalar product of vectors P and Q may appear in the form: 

      (magnitude P) * (magnitude Q) * (cos alpha)

where the magnitude (or length) of a vector is the square root of sum of squares of components, and alpha is the smallest non-negative angle between P and Q. To show the equivalence of this form with P dot Q, we can define utility-verbs ma for magnitude-of-a-vector and ca for cos-of-angle-between-vectors. 

   ma  =: %: @: (+/ @: *:)
   ca  =: 4 : '(-/ *: b,(ma x.-y.), c) % (2*(b=.ma x.)*(c=. ma y.))'

We expect the magnitude of vector 3 4 to be 5, and expect the angle between P and itself to be zero, and thus cosine to be 1.

ma 3 4 P ca P 
5
 
1

then we see that the dot verb is equivalent to the textbook form above

P Q P dot Q (ma P)*(ma Q)*(P ca Q) 
2 3 4
 
1 0 2
 
10
 
10

22.3 Matrix Product

The verb we called dot is "matrix product" for vectors and matrices.

M =: 3 4 ,: 2 3 V =: 3 5 V dot M M dot V M dot M 
3 4 
2 3
 
3 5
 
19 27
 
29 21
 
17 24 
12 17

To compute Z =: A dot B the last dimension of A must equal the first dimension of B. 

   A =: 2 5 $ 1
   B =: 5 4 $ 2

$ A $ B Z =: A dot B $ Z 
2 5
 
5 4
 
10 10 10 10 
10 10 10 10
 
2 4

The example shows that the last-and-first dimensions disappear from the result. If these two dimensions are not equal then an error is signalled.

$ B $ A B dot A 
5 4
 
2 5
 
error

22.4 Generalisations

22.4.1 Various Verbs

The "Dot Product" conjunction forms the dot-product verb with (+/ . *). Other verbs can be formed on the pattern (u.v).

For example, consider a relationship between people: person i is a child of person j, represented by a square boolean matrix true at row i column j. Using verbs +. (logical-or) and *. (logical-and). We can compute a grandchild relationship with the verb (+./ . *.). 

   g   =: +. / . *.

Taking the "child" relationship to be the matrix C: 

   C =: 4 4 $ 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0

Then the grandchild relationship is, so to speak, the child relationship squared.

C G =: C g C 
0 0 0 0 
1 0 0 0 
1 0 0 0 
0 1 0 0
 
0 0 0 0 
0 0 0 0 
0 0 0 0 
1 0 0 0

We can see from C that person 3 is a child of person 1, and person 1 is a child of person 0. Hence, as we see in G person 3 is a grandchild of person 0.

22.4.2 Symbolic Arithmetic

As arguments to the "Dot Product" conjunction we could supply verbs to perform symbolic arithmetic. Thus we might symbolically add the strings 'a' and 'b' to get the string 'a+b'. Here is a small collection of utility functions to do some limited symbolic arithmetic on strings. 

   pa     =: ('('&,) @: (,&')')   
   cp     =: [ ` pa @. (+./ @: ('+-*' & e.))
   symbol =: (1 : (':';'< (cp > x.), u., (cp > y.)')) " 0 0
   
   splus  =: '+' symbol 
   sminus =: '-' symbol 
   sprod  =: '*' symbol 
   
   a =: <'a'
   b =: <'b'
   c =: <'c'

a b c a splus b a sprod b splus c 
+-+ 
|a| 
+-+
 
+-+ 
|b| 
+-+
 
+-+ 
|c| 
+-+
 
+---+ 
|a+b| 
+---+
 
+-------+ 
|a*(b+c)| 
+-------+

As a variant of the symbolic product, we could elide the multiplication symbol to give an effect more like conventional notation: 

   sprodc =: '' symbol

a sprod b a sprodc b 
+---+ 
|a*b| 
+---+
 
+--+ 
|ab| 
+--+

Now for the dot verb, which we recall is (+/ . *), a symbolic version is: 

   sdot =: splus / . sprodc

To illustrate: 

   S =: 3 2 $ < "0 'abcdef'
   T =: 2 3 $ < "0 'pqrstu'

S T S sdot T 
+-+-+ 
|a|b| 
+-+-+ 
|c|d| 
+-+-+ 
|e|f| 
+-+-+
 
+-+-+-+ 
|p|q|r| 
+-+-+-+ 
|s|t|u| 
+-+-+-+
 
+-----+-----+-----+ 
|ap+bs|aq+bt|ar+bu| 
+-----+-----+-----+ 
|cp+ds|cq+dt|cr+du| 
+-----+-----+-----+ 
|ep+fs|eq+ft|er+fu| 
+-----+-----+-----+

22.4.3 Matrix Product in More than 2 Dimensions

An example in 3 dimensions will be sufficiently general. Symbolically: 

   A =: 1 2 3 $ <"0 'abcdef'
   B =: 3 2 2 $ <"0 'mnopqrstuvwx'

A B Z =: A sdot B $A $B $Z 
+-+-+-+ 
|a|b|c| 
+-+-+-+ 
|d|e|f| 
+-+-+-+
 
+-+-+ 
|m|n| 
+-+-+ 
|o|p| 
+-+-+ 
 
+-+-+ 
|q|r| 
+-+-+ 
|s|t| 
+-+-+ 
 
+-+-+ 
|u|v| 
+-+-+ 
|w|x| 
+-+-+
 
+----------+----------+ 
|am+(bq+cu)|an+(br+cv)| 
+----------+----------+ 
|ao+(bs+cw)|ap+(bt+cx)| 
+----------+----------+ 
 
+----------+----------+ 
|dm+(eq+fu)|dn+(er+fv)| 
+----------+----------+ 
|do+(es+fw)|dp+(et+fx)| 
+----------+----------+
 
1 2 3
 
3 2 2
 
1 2 2 2

The last dimension of A must equal the first dimension of B. The shape of the result Z is the leading dimensions of A followed by the trailing dimensions of B. The last-and-first dimension of A and B have disappeared, because each dimensionless scalar in Z combines a "row" of A with a "column" of B. We see in the result Z that each row of A is combined separately with the whole of B.

22.4.4 Dot Compared With @:

Recall from Chapter 07 that a dyadic verb v has a left and right rank. Here are some utility functions to extract the ranks from a given verb. 

   RANKS   =: 1 : 'x. b. 0'
   LRANK   =: 1 : '1 { (x. RANKS)'   NB. left rank only

* RANKS * LRANK 
0 0 0
 
0

The general scheme defining dyadic verbs of the form (u.v) is: 

         u.v   means  u @ (v " (1+L, _))   where L = (v LRANK)

or equivalently, 

          u.v   means (u @: v) " (1+L, _)
    and hence
         +/.*   means (+/ @: *)" 1 _

and so we see the difference between . and @:. For simple vector arguments they are the same, in which case the dimensions of the arguments must be the same, but this is not the condition we require for matrix multiplication in general, where (in the example above) each row of A is combined with the whole of B.

22.5 Determinant

The monadic verb (- / . *) computes the determinant of a matrix. 

   det =: - / . *

M det M (3*3)-(2*4) 
3 4 
2 3
 
1
 
1

Symbolically: 

   sdet =: sminus / . sprodc

S sdet S 
+-+-+ 
|a|b| 
+-+-+ 
|c|d| 
+-+-+ 
|e|f| 
+-+-+
 
+----------------------------+ 
|(a(d-f))-((c(b-f))-(e(b-d)))| 
+----------------------------+

The monadic rank of det is 2. 

   det RANKS
2 _ _

Hence det applied to a higher rank array will produce a separate result for each 2-cell (matrix).

A det A 
+-+-+-+ 
|a|b|c| 
+-+-+-+ 
|d|e|f| 
+-+-+-+
 
error

22.5.1 Singular Matrices

A matrix is said to be singular if the rows (or columns) are not linearly independent, that is, if one row (or column) can be obtained from another by multiplying by a constant. A singular matrix has a zero determinant. In the following example A is a (symbolic) singular matrix, with m the constant multiplier.

A =: 2 2 $ 'a';'b';'ma';'mb' sdet A 
+--+--+ 
|a |b | 
+--+--+ 
|ma|mb| 
+--+--+
 
+-------+ 
|amb-mab| 
+-------+

We see that the resulting term (amb-mab) must be zero for all a, b and m.

22.6 Matrix Divide

22.6.1 Simultaneous Equations

The built-in verb %. (percent dot) is called "Matrix Divide". It can be used to find solutions to systems of simultaneous linear equations. For example, consider the equations written conventionally as: 

          3x + 4y = 11
          2x + 3y =  8

Rewriting as a matrix equation, we have, informally, 

          M dot U = R

where M is the matrix of coefficients U is the vector of unknowns x,y and R is the vector of right-hand-side values:

M =: 3 4 ,: 2 3 R =: 11 8 
3 4 
2 3
 
11 8

The vector of unknowns U (that is, x,y) can be found by dividing R by matrix M.

M R U =: R %. M M dot U 
3 4 
2 3
 
11 8
 
1 2
 
11 8

We see that M dot U equals R, that is, U solves the equations.

22.6.2 Complex, Rational and Vector Variables

The equations to be solved may be in complex variables. For example:

M R =: 15j22 11j16 U =: R %. M M dot U 
3 4 
2 3
 
15j22 11j16
 
1j2 3j4
 
15j22 11j16

or in rationals. In this case both M and R must be rationals to give a rational result. 

   M =: 2 2 $ 3x 4x 2x 3x
   R =: 15r22 11r16

M R U =: R %. M M dot U 
3 4 
2 3
 
15r22 11r16
 
_31r44 123r176
 
15r22 11r16

In the previous examples the unknowns in U were scalars. Now suppose the unknowns are vectors and our equations for solving are: 

          3x + 4y = 15 22
          2x + 3y = 11 16

so we write: 

   M =: 2 2 $ 3 4 2 3
   R =: 2 2 $ 15 22 11 16

M R U =: R %. M M dot U 
3 4 
2 3
 
15 22 
11 16
 
1 2 
3 4
 
15 22 
11 16

The unknowns x and y are the rows of U, that is, vectors.

22.6.3 Curve Fitting

Suppose we aim to plot the best straight line fitting a set of data points. If the data points are x,y pairs given as: 

   x =: 0   1   2 
   y =: 3.1 4.9 7

we aim to find a and b for the equation: 

              y = a + bx

The 3 data points give us 3 equations in the 2 unknowns a and b. Conventionally: 

     1 . a  +  0 . b  =   3.1
     1 . a  +  1 . b  =   4.9
     1 . a  +  2 . b  =   7

so we take the matrix of coefficients M to be 

   M =: 3 2 $ 1 0 1 1 1 2

and divide y by matrix M to get the vector of unknowns U, (that is, a,b)

M y U =: y %. M M dot U 
1 0 
1 1 
1 2
 
3.1 4.9 7
 
3.05 1.95
 
3.05 5 6.95

Here we have more equations than unknowns, (more rows than columns in M) and so the solutions U are the best fit to all the equations together. We see that M dot U is close to, but not exactly equal to, y.

"Best fit" means that the sum of the squares of the errors is minimized, where the errors are given by y - M dot U. If the sum of squares is minimized, then we expect that by perturbing U slightly, the sum of squares is increased.

+/ , *: (y - M dot U) +/ , *: y - M dot U + 0.01 
0.015
 
0.0164

The method extends straightforwardly to fitting a polynomial to a set of data points. Suppose we aim to fit 

 
          y = a + bx + cx2
to the data points: 
   x =: 0   1  2  3 
   y =: 1   6 17 34.1

The four equations to be solved are: 

 
      1.a + bx0 + cx02 = y0 
      1 a + bx1 + cx12 = y1 
      1.a + bx2 + cx22 = y2 
      1.a + bx3 + cx32 = y3
and so the columns of matrix M are 1, x, x^2, conveniently given by x ^/ 0 1 2

M =: x ^/ 0 1 2 y U =: y %. M M dot U 
1 0 0 
1 1 1 
1 2 4 
1 3 9
 
1 6 17 34.1
 
1.005 1.955 3.025
 
1.005 5.985 17.015 34.095

There may be more equations than unknowns, as this example shows, but evidentlty there cannot be fewer. That is, in R %. M matrix M must have no more columns than rows.

22.6.4 Dividing by Higher-Rank Arrays

Here is an example of U =: R %. M, in which the divisor M is of rank 3. 

   M =: 3 2 2 $ 3 4 2 3 0 3 1 2 3 1 2 3
   R =: 21 42

M R U =: R %. M M dot U M dot"2 1 U 
3 4 
2 3 
 
0 3 
1 2 
 
3 1 
2 3
 
21 42
 
_105 84 
  28  7 
   3 12
 
error
 
21 42 
21 42 
21 42

Because the dyadic rank of %. is _ 2, 

   %. b. 0
2 _ 2

in this example the whole of R is combined separately with each of the 3 matrices in M. That is, we have 3 separate sets of equations, each with the same right-hand-side R Thus we have 3 separate solutions (the rows of U).

The condition R=M dot U evidently does not hold (because the last dimension of M is not equal to the first of U), but it does hold separately for each matrix in M with corresponding row of U.

22.7 Identity Matrix

A (non-singular) square matrix M divided by itself yields an "identity matrix", I say, such that (M dot I) = M. 

   M =: 3 3 $ 3 4 7 0 0 4 6 0 3

M I =: M %. M M dot I 
3 4 7 
0 0 4 
6 0 3
 
1 0 0 
0 1 0 
0 0 1
 
3 4 7 
0 0 4 
6 0 3

22.8 Matrix Inverse

The monadic verb %. computes the inverse of a matrix That is, %. M is equivalent to I %. M for a suitable identity matrix I:

M I =: M %. M I %. M %. M 
3 4 7 
0 0 4 
6 0 3
 
1 0 0 
0 1 0 
0 0 1
 
   0   _0.125 0.166667 
0.25 _0.34375   _0.125 
   0     0.25        0
 
   0   _0.125 0.166667 
0.25 _0.34375   _0.125 
   0     0.25        0

For a vector V, the inverse W has the reciprocal magnitude and the same direction. Thus the product of the magnitudes is 1 and the cosine of the angle between is 1.

V W =: %. V (ma V) * (ma W) V ca W 
3 5
 
0.0882353 0.147059
 
1
 
1

This is the end of Chapter 22.

NEXT
Table of Contents

Copyright © Roger Stokes 2002. This material may be freely reproduced, provided that this copyright notice is also reproduced.

last updated 3 March 2002