New Approach for Accelerating Nonlinear Schwarz Iterations

abstract: The vector Epsilon algorithm is an effective extrapolation method used for accelerating the convergence of vector sequences. In this paper, this method is used to accelerate the convergence of Schwarz iterative methods for stationary linear and nonlinear partial differential equations (PDEs). The vector Epsilon algorithm is applied to the vector sequences produced by additive Schwarz (AS) and restricted additive Schwarz (RAS) methods after discretization. Some convergence analysis is presented, and several test-cases of analytical problems are performed in order to illustrate the interest of such algorithm. The obtained results show that the proposed algorithm yields much faster convergence than the classical Schwarz iterations.


Introduction
In scientific computing, the domain decomposition methods are now commonly used when solving large linear or nonlinear systems arising from discretization of partial differential equations (PDEs) [2,4,5,13].The first models of these methods have been established by H.A.Schwarz, the idea is to decompose a large problem into a series of smaller subproblems, and therefore more easily resolved.There are 2010 Mathematics Subject Classification: 65N55, 65B05.Submitted May 05, 2017.Published November 10, 2017 52 N. Nagid and H. Belhadj several variant of Schwarz method, for example additive Schwarz method (AS), and restricted additive schwarz method (RAS) [1,9,18,20].
So as to accelerate convergence of sequences produced by these methods, the Aitken process appears as an acceleration method suitable for many domain decomposition methods, in the case of linear problems, but this process does not warrant the convergence when the problems are nonlinear.The generalization of Aitken process for nonlinear sequences leads us to focus on the shanks transformation and it's derivatives algorithms.In practice, calculating determinants being very costly, shanks transformation is calculated just for the low values of k, in particular for k = 1, where the ∆ 2 Aitken algorithm is obtained.The most common method for calculating the shanks transformation is the Epsilon algorithm (ε-algorithm) proposed by Peter Wynn [3,6,7,10].There exist different variants of the Epsilon algorithm that can be used with vector sequences: the vector Epsilon algorithm, or the scalar Epsilon algorithm applied to each component of the vector sequences [7,10].There have many works that have treated the acceleration of domain decomposition methods, for example in [19], the authors accelerate the nonlinear Schwarz iterations by reduced rank extrapolation method.Another idea was described in [16], to accelerate Schwarz iterations for ordinary differential equations ODEs.There exist many other works that have treated the acceleration of domain decomposition methods, see for examples [8,14,15,17].The purpose of this paper is to accelerate the nonlinear iterative Schwarz, using the vector Epsilon algorithm for PDEs, this algorithm is applied to the sequences of vectors produced by AS and RAS methods, we show experimentally that the proposed algorithm can provide faster convergence measured both in number of iterations and in CPU Times.

Linear Schwarz iterations
We consider the following problem where L is a linear operator, B is a boundary operator and Ω is a bounded domain of R d (d = 1, 2, ..).H.A.Schwarz proposed an iterative method for the solution of classical boundary value problems.There are several variants of Schwarz algorithms, additive, multiplicative, and several hybrid types, a number of them are discussed in detail in [2,4,5,12,13], in the present work, we have considered the additive Schwarz method .
The discretization of problem (2.1) leads to a linear system of equations of the form where A is the discretization matrix by a numerical methods (Finite element, Finite Difference , or Finite volume).We use the same notation f after discretization.A stationary iterative method for (2.2) is given by with a given initial approximation u 0 to the solution of (2.2).Algebraic domain decomposition methods group the unknowns into subsets, u j = R j u, j = 1, ..., p, where R j are rectangular restriction matrices.Coefficient matrices for subdomain problems are defined by A j = R j AR T j .The additive Schwarz (AS) preconditioner, and the restricted additive Schwarz (RAS) preconditioner (see [1,9,18,20]) are defined by: where the ∼ R j correspond to a non-overlapping decomposition, and it consists of zeroes and ones, in such a way that p j=1 ∼ R T j R j = I.The additive Schwarz method constructs the sequence of approximations {u n } n∈N by setting: (without the Richardson acceleration).The restricted additive Schwarz (RAS) algorithm is given by: N. Nagid and H. Belhadj

Nonlinear Schwarz iterations
We consider now the problem (2.1) with a nonlinear operator L. After discretization, we obtain an algebric nonlinear system we transform this problem to a fixed point form where F and G are two mappings from R n → R n , using the same notation as before, we define G on each subdomain Ω j , j = 1, 2, ..., p, as follows: The corresponding nonlinear additive Schwarz method is defined by and the nonlinear restricted additive Schwarz method is defined by we also consider for the solution of (3.2) the Schwarz-Newton methods, where in each subdomain, the nonlinear problem is solved by a Newton, see [21].

Vector Epsilon algorithm
The vector Epsilon algorithm is a nonlinear extrapolation method for accelerating the convergence of sequences, one can say also that this is a generalization of Aitken method.There exist several versions of the Epsilon algorithm (topological, scalar, and vector Epsilon algorithm).In this work, we are only interested in the vector form.We consider thereafter the fundamental algebraic results in the theory of the vector Epsilon algorithm [3,6,7,10].
First, we recall some results concerning the Aitken's process.Let U = (u n ) n∈N is a sequence that converges to u, the convergence acceleration methods consists in transforming U = (u n ) n∈N into another sequence (ε 2 ) which converges faster to the same limit u.
Among these transformation methods, the best-known are the Richardson methods and ∆ 2 Aitken.We define the operator ∆ such as Definition 4.2.Let U = (u n ) n∈N be a sequence of real numbers, the ∆ 2 Aitken process consists in transforming the sequence (u n ) into a new sequence (ε 2 ) defined by : then the sequence ε n 2 converges to u faster than u n+1 .

Proof:
Using the definition 4.1 we have: if the condition of the theorem is satisfied, then (ε n 2 ) converges to u faster than (u n+1 ).✷ Now, we seek the conditions on (u n ) in order that ε We have seen that: based on determinants: , we want to have for this determinant to be zero, it is necessary and sufficient that there exist a 0 and a 1 such that: if a 0 + a 1 = 0 we remark that u n = u n+1 ∀n and then the ∆ 2 Aitken process cannot be applied to u n , and if a 0 + a 1 = 0 then, we have ε , therefore, we have the following theorems.
Theorem 4.4.A necessary and sufficient condition to have ε This theorem can be generalized to high order using a nonlinear acceleration method, the Shanks transformation [3,6,7].This transformation called e n k (U ) is built such that e n k (U ) = u ∀n > N, and it consists in computing the quantities e n k (U ) as follows from these equations, it is easy to obtain a determinantal formula for e n k (U ) New Approach for Accelerating Nonlinear Schwarz Iterations 57 the transformed expression given above is to a ratio of two determinants having a particular structure, and it's a part of the hankel determinants [6,7].
The previous results leads to the following theorem.
Theorem 4.5.If for a fixed k, the sequence U is such that there exists u ∈ R and The proof of theorems 4.4 and 4.5 are given for example in [6,7,8].
Remark 4.6.A recursive rule for computing the quantities e n k (U ) of shanks transformation has been given by [6,7], these quantities can be computed by the following Epsilon algorithm: where the inverse of a vector y is defined by: Using theorem 4.5, it has been proved that the vector Epsilon algorithm provides a direct method for solving the linear systems of equations [6,7].a i t i the minimal polynomial of A for the vector u 0 − u, the definition of the minimal polynomial of a matrix for a vector is: a i = 0 on the other hand, we a i (u n+i − u) = 0 ∀n, so using theorem 4.5 we prove that

✷
Now, let to be solve the following nonlinear problem where F : R p −→ R p is differentiable in the sense of Frechet in a neighborhood of x, knowing x 0 we set u 0 = x n and we solve for k = 1, ..., 2m − r the following iterative problem To calculate ε (r) 2(m−r) we applied the Epsilon algorithm to the vectors u 0 , ..., u 2m−r , then we take , where m is the degree of minimal polynomial of F ′ (x) for the vector x n − x and r is the multiplicity of the root (λ = 0) for this minimal polynomial.The nonlinear fixed point problems can be accelerated by the Epsilon algorithm if the conditions of theorem 4.8 is verified.
Theorem 4.8.Let F : R p −→ R p such as there exist x ∈ R p wich satisfies x = F (x), such that F is differentiable in the sense of Frechet in a neighborhood of x, and such that I−F ′ (x) is invertible.Then there exists a neighborhood V of x such that for any x 0 ∈ V the previous algorithm converges to x at least quadratically, ie: Proof: If F is differentiable in the sense of Frechet in a neighborhood of x, we have: where o( z k 2 ) refers to a vector Let p(t) = m i=0 a i t i the minimal polynomial of F ′ (x) for the vector we have: New Approach for Accelerating Nonlinear Schwarz Iterations 59 and and using theorems 4.4 and 4.5 we get

Vector Epsilon algorithm applied to AS/RAS for linear systems
We consider the following problem: in Ω, Bu = g on ∂Ω. (5.1) In the case where the operator L is linear, a discretization of the equation (5.1) leads to a linear system of equations of the form Au = f. (5. 2) The additive Schwarz methods allows to compute the sequence of approximations {u n } n∈N by setting: the equivalent system becomes y n+1 = M −1 AS P y n = By n , using theorem 4.7, we have ε n 2m = u, where m is the degree of the minimal polynomial of B, and we have New Approach for Accelerating Nonlinear Schwarz Iterations 61 where γ n are the coefficients of the polynomial P d such that P d (1) = 1.✷

Vector Epsilon algorithm applied to AS/RAS for nonlinear systems
We consider now the nonlinear reaction diffusion problem defined by: we can write Lu = G(u) in the sense of D ′ (Ω).( 6. 2) The corresponding discretized problem can be written as follows: where A is the matrix of the discretized operator L, obtained by the finite elements on a regular grid, G : R p −→ R p is the nonlinear function, and U is the vector containing the approximation of the solution of the continuous problem to grid points.We remark, that if we put F = A −1 G(.), then the problem (6.3) is equivalent to U = F (U ).Let solve the following problem: where F : R p −→ R p is differentiable in the sense of Frechet, in a neighborhood of x, m is the degree of minimal polynomial of F ′ (x) for the vector x n − x and r is the multiplicity of the root (λ = 0) for this minimal polynomial.Knowing x 0 we set u 0 = x n and we solve for k = 1, ..., 2m − r, the following iterative problem u k = F (u k−1 ).
To calculate ε −r) we apply the Epsilon algorithm to the vectors u 0 , ..., u 2m−r .then we take The application of the Epsilon algorithm to the nonlinear RAS provides a method of resolution with quadratic convergence, see [11].
two spatial dimensions and all computational experiments presented were carried out using Freefem++.
We compare results for different number of nonoverlapping subdomains and different discretizations.In all plots, the labels AS and RAS refer to the additive and restricted additive Schwarz methods, respectively.The labels Epsilon-AS and Epsilon-RAS refer to the vector Epsilon algorithm applied to sequences constructed by the AS and RAS methods, respectively.Application to the Helmholtz Problem We consider the Helmholtz problem Let k be a constant, we take k = 10, g = y(y − 1) and we use a finite element discretization on an equidistant grid on the domain Ω = [0, 1] × [0, 1] with homogeneous Neumann boundary conditions.Figure 1 illustrates the computational result on all domain Ω using FreeFem ++.
Figure 1: Helmholtz problem: Solution on Ω Now, using the Epsilon-RAS algorithm, we solve the problem on multiple subdomains.Let p the number of subdomains, figure 2 shows the solution for p=4, and table 1 shows the behaviour of the error norm (L ∞ ) when we apply the Epsilon-RAS algorithm to the problem (7.1).
Figure 3 shows the behaviour of the error norm using a logarithmic scale versus number of iterations for all algorithms, when p=16.The domain is the unit square Ω = [0, 1] × [0, 1] decomposed uniformly into p nonoverlapping subdomains.
b is chosen so that the solution is known to be the vector of all ones, using a finite element discretization, we obtain the following nonlinear system of equations For this problem, we use the nonlinear additive and restricted additive Schwarz iterations, respectively; and their acceleration with vector Epsilon algorithm.In each subdomain, we use the nonlinear SSOR method to solve the smaller nonlinear problem.
We chose the values of λ = 6.998 and Ω = 0.5.Using FreeFem++, Figure 6 shows the solution of problem (7.2) using nonlinear Epsilon-RAS algorithm on p=8 nonoverlapping subdomains.The following results reported in table 3, show the L ∞ Error norm when we apply the nonlinear Epsilon-RAS algorithm to the problem (7.2).
To show experimentally that the vector Epsilon algorithm can indeed provide a good acceleration, we show the behaviour of the error norm using a logarithmic scale versus number of iterations for all methods, when p=16, see figure 7.
As in the linear case, it can be observed that both nonlinear additive and restricted additive Schwarz iterations take too long to converge, whereas nonlinear Epsilon-AS and Epsilon-RAS require far fewer iterations for convergence.As in the previous example, When we compare the CPU Time, one can observe New Approach for Accelerating Nonlinear Schwarz Iterations 67 Table 3: Bratu problem: Error norm for nonlinear Epsilon-RAS algorithm with p=8

Conclusion
We have proposed an accelerated form of Schwarz iterations for nonlinear problems (AS-RAS) using the vector Epsilon algorithm.Comparing CPU-Time and the number of iterations, we show that this accelerated method is fast and it has a better accuracy than the direct classical Schwarz method.As perspective of the present work, we can generalize the acceleration method for non stationary PDEs, and apply it to a real modelling case.

Theorem 4 . 7 .
If we apply the vector Epsilon algorithm to the sequence {u n } produced by u n+1 = Au n + b, with a given u 0 and A is a real square matrix such that I − A is invertible, then we have ε (n) 2m = u for n = 0, 1, ... where u = (I − A) −1 b and m is the degree of minimal polynomial of A for the vector u 0 − u.Proof: Let p(t) = m i=0 .4) where B = I − M −1 AS A. Theorem 5.1.Suppose that A and M −1 AS a real square matrices that have the same size such that C = M −1 AS A is non singular.If we apply the vector Epsilon algorithm to the aditive Schwarz sequence (5.4) then ε n 2m = u, where u is the solution of the linear system Cu = F, and m the degree of the minimal polynomial of B .Proof: The solution u is a fixed point of the operator u → u + M −1 AS (f − Au), let P = M AS − A is the difference between A and M AS , when (5.3) converge, it converges to the solution of the preconditioned system M −1 AS Au = M −1 AS f, by setting y n = u n − u we obtain

Figure 4 :
Figure 4: Helmholtz problem.CPU Time versus number of subdomains

Figure 7 :
Figure 7: Bratu problem: Convergence for different number of subdomains p

Figure 8 :
Figure 8: Bratu problem: CPU Time versus number of subdomains

Table 1 :
Helmholtz problem: Error norm for Epsilon-RAS algorithm with p=4

Table 2 :
Helmholtz problem: CPU Time versus Number of subdomains

Table 4 :
Bratu problem: CPU Time versus Number of subdomains