CMSC 435 Algorithm Analysis & Design

 

Matrix Chain Multiplication

 

            Let mij = the minimum number of multiplications to multiply the chain of

            matrices  Ai...Aj  

 

            Then we have the recurrence

 

                        mij = min (mik + mk+1j + pi-1pkpj)

                                 i£k<j

            where p = <p0, p1, p2,...,pn>  is a vector denoting the dimensions of the matrices

            in the chain.  A1 has dimension p0 x p1, A2 has dimension p1 x p2, etc.

 

            This recurrence leads directly to the recursive algorithm

 

                        int MatChain(int p[], int i, int j)  {

                              if (i == j)

                                    return 0;

                              int M =¥;

                              for (int k = i; k < j; j++) {

                                  int temp = MatChain(p, i, k)+MatChain(p, k+1,j) + p[i-1]p[k]p[j];

                                  if (temp < M)

                                        M = temp;

                              }

                              return M;

                        }

 

            This algorithm contains overlapping subproblems.  A top-down approach to

            eliminating this problem is to use a memoized version of the recurrence.

 

            Let M[n+1][n+1] be a matrix for holding structures containing two fields:

            a count field for holding the minimum number of multiplications, and

            a loc field for holding the location of k -- the place where parentheses will

            be added.

 

            First initialize M so that all cells hold a count value of ¥.

 

            int MatChain(int p[], int n)  {

                  for (int i = 1; i <= n; i++)

                        for (int j = 1; j <= n; j++) {

                             M[i][j].count = ¥;

                             M[i][j].loc = i;

                        }

                  return MatChainLookup(p, 1, n);

 

            Next modify the recursive algorithm to do a table lookup before issuing a

recursive call and a table write after returning from each recursive call.  If the

table lookup reveals a value that is less than ¥, then the subproblem has

previously been solved and the result is available from the table without having

to recursively solve the problem over again.

 

int MatChainLookup(int p[], int i, int j) {

     if (M[i][j].count < ¥)

            return M[i][j].count;

     for (int k = i; k < j; k++) {

            temp = MatChainLookup(p,i,k) + MatChainLookup(p,k+1,j) + p[i-1]*

                                    p[k]*p[j];

            if (temp < M[i][j].count)  {

                 M[i][j].count = temp;

                 M[i][j].loc = k;

            }

     }

     return M[i][j].count;

}

 

Dynamic Programming Solution

 

In the dynamic programming approach, we construct a table from the "bottom

up".   We begin examining chains of length 1 (consisting of a single matrix).  The

number of multiplications needed to form the product matrix is readily seen to

be zero.  There is no multiplication to be done.  All of the chains of length one

are the chains beginning and ending at i for i = 0 to i = n.  Thus all of the diagonal

elements of the matrix are set to zero.

Next we examine the chains of length l for l = 2 to l = the single chain of

length n.  For each value of l, we use the recursion relation

 

                        mij = min (mik + mk+1j + pi-1pkpj)

 

to fill the table entry i,j which can be found using entries i,k and k+1,j which

have been previously entered into the table.

       For each length l start the "caliper" at i = 1, and move it to the right until the back

       end includes An.  The caliper will be last positioned therefore at i = n- l +1.

 

            int MatChain (int p[], int i int j) {

                 for (int i = 1; i <= n; i++)  {

                        M[i][i].count = 0;

                        M[i][i].loc = i;

                 }

                 for (int l = 2; l <=n; l++)     //for each length

                        for (i = 1; i <= n - l -1; i++)  {  //for each initial placement

                             j = i + l -1;

                             M[i][j].count = ¥;

                             M[i][j].loc = i;

                             for (int k = i; k < j; k++) { //for each subinterval in the interval i..j

                                    int q = M[i][k].count + M[k+1][j].count + p[i-1]*p[k]*p[j];

                                    if (q < M[i][j].count)  {

                                          M[i][j].count = q;

                                          M[i][j].loc = k;

                                    }

                             }

                        }

                 return M[i][j].count.

            }

 

       Recursion Relation for the non-memoized algorithm

 

       Let T(n) = the number of statements executed in evaluating the recursive MatChain

       algorithm.  Then

                                         n-1

T(n) ³ 1 + å(T(k) + T(n-k) + 1)           where T(1) ³ 1

                                         k=1

       Then we find

 

                                      n-1

T(n) ³ 2 å(T(k) + n                              where T(1) ³ 1

                                      k=1

 

       Try to prove T(n) > 2n

 

                                       n-1

        =  2 å 2i + n = 2(2n - 1) + n = (2n+1 -2) + n ³ 2n

   k=1

       

       This recursion relation is a full history recursion relation much like the one

       that is obtained for quicksort, but here there is no averaging and T(n) is W(2n).

                   

Longest Common Subsequence

 

            Let X = <x1, x2, …, xm>  and Y = <y1, y2, …, yn>  be sequences and

            Z = <z1, z2,…, zk> be the longest common subsequence (LCS) of X and Y.

1.        If xm = yn, then zk = xm and Zk-1 is the LCS of Xm-1 and Yn-1.

2.        If xm ¹ yn, then zk ¹ xm implies that Z is an LCS of Xm-1 and Y.

3.        If xm ¹ yn, then zk ¹ yn implies that Z is an LCS of X and Yn-1.

 

A recursive solution to the Longest Common Subsequence problem

 

            Let c[i,j] denote the length of the LCS of two sequences of length i and j

            respectively.  Then

 

            if ((i == 0) || (j == 0))

                  c[i, j] = 0;

            else if (xi == yj)

                  c[i, j] = c[i-1, j-1] + 1

            else

                  c[i, j] = max(c[i, j-1], c[i-1, j])

 

            We can implement this solution as a recursive algorithm where C is a function

            That takes as its arguments the two sequences X and Y and the indices i and j,

and returns he length of the LCS of these two (sub) sequences.

 

int C (Comparable [] X, Comparable [] Y, int i, int j) {

    if ((i == 0) || (j == 0))

            return 0;

    if (X[i] == Y[j])

            return C(X, Y, i-1, j-1) + 1;

     else

            return max(C(X, Y, i, j-1), C(X, Y, i-1, j));

}

 

The concise code belies the hidden horror that this is a senile algorithm with

overlapping subproblems.

 

A dynamic programming solution computes the table entries for the length of the

LCS for all of the subsequences of X and Y. 

 

We construct a matrix C to hold each of these results.  This matrix will have

m+1 rows (for the number of subsequences of X including the empty one) and

n+1 columns. 

 

                       

0     y1     y2      y3      y4            yj                            yn

0

0

0

0

0

0

0

0

0

0

0

 

 

 

 

 

 

 

 

 

0

 

 

 

 

 

 

 

 

 

0

 

 

 

 

 

 

 

 

 

0

 

 

 

 

.

.

 

 

 

0

 

 

 

 

.

x

 

 

 

0

 

 

 

 

 

 

 

 

 

0

 

 

 

 

 

 

 

 

 

0

 

 

 

 

 

 

 

 

 

 

The first row and the first column correspond to i = 0 and j = 0 respectively, and

we know that C[0][j] and C[i][0] are 0 for all i and j.

To find the entry for the i, jth cell we need only look at three cells that have been

previously filled in.

 

Let    int C[m+1][n+1]  and  direction B[m+1][n+1] be two globally defined

matrices.   Then a dynamic programming solution to this problem can be

formulated as follows:

 

            void LCS(Comparable X[], Comparable Y[])  {

                  for (int i = 0; i <= m; i++)  {

                        C[i][0] = 0;

                        B[i][0] = "­";

                  }

                  for (int j = 0; j <= n; j++)  {

                        C[0][j] = 0;

                        B[0][j] = "¬";

                  }

                  for (i = 1; i <= m; i++)

                        for (j = 1; j <= n; j++) {

                              if (X[i] == Y[j])  {

                                    C[i][j] = C[i-1][j-1] +1;

                                    B[i][j] = " \ ";

                              }

                              else if (C[i-1][j] >= C[i][j-1])  {

                                    C[i][j] = C[i-1][j];

                                    B[i][j] = "­";

                              }

                              else  {

                                    C[i][j] = C[i][j-1];

                                    B[i][j] = "¬";

                              }

                        }

            }