The Gamma function is implemented as Gamma(x). At integer values n of the argument, Gamma(n) is computed exactly. Because of overflow, it only makes sense to compute exact integer factorials for small numbers n. Currently a warning message is printed if a factorial of n>65535 is requested.
For half-integer arguments Gamma(x) is also computed exactly, using the following identities (here n is a nonnegative integer and we use the factorial notation):
If the factorial of a large integer or half-integer n needs to be computed not exactly but only with a certain floating-point precision, it is faster (for large enough Abs(n)) not to evaluate an exact integer product, but to use the floating-point numerical approximation. This method is currently not implemented in Yacas.
There is also the famous Stirling's asymptotic formula for large factorials,
Repeated partial integration gives the expansion
The method gives the Gamma-function only for arguments with positive real part; at negative values of the real part of the argument, the Gamma-function is computed via the identity
The Lanczos-Spouge approximation formula depends on a parameter a,
The coefficients c[k] and the parameter a can be chosen to achieve a greater precision of the approximation formula. However, the recipe for the coefficients c[k] given in the paper by Lanczos is too complicated for practical calculations in arbitrary precision: the time it would take to compute the array of N coefficients c[k] grows as N^3. Therefore it is better to use less precise but much simpler formulae derived by Spouge.
At version 1.2.2, Yacas is limited in its internal arbitrary precision facility that does not support true floating-point computation but rather uses fixed-point logic; this hinders precise calculations with floating-point numbers. In the current version of the Internal'GammaNum() function, two workarounds are implemented. First, a Horner scheme is used to compute the sum; this is somewhat faster and leads to smaller round-off errors. Second, intermediate calculations are performed at 40% higher precision than requested. This is much slower but allows to obtain results at desired precision.
If strict floating-point logic is used, the working precision necessary to compensate for the cancellations must be 1.1515*P digits for P digits of the result. This can be shown as follows.
The sum converges to a certain value S which is related to the correct value of the Gamma function at z. After some algebra we find that S is of order Sqrt(a) if z>a and of order a^(1/2-z) if a>z. Since a is never a very large number, we can consider the value of S to be roughly of order 1, compared with exponentially large values of some of the terms c[k] of this sum. The magnitude of a coefficient c[k] is estimated by Stirling's formula,
For a given (large) value of Abs(x), the terms of this series decrease at first, but then start to grow. (Here x can be a complex number.) There exist estimates for the error term of the asymptotic series (see [Abramowitz et al. 1964], 6.1.42). Roughly, the error is of the order of the first discarded term.
We can estimate the magnitude of the terms using the asymptotic formula for the Bernoulli numbers (see below). After some algebra, we find that the value of n at which the series starts to grow and diverge is n[0]<=>Pi*Abs(x)+2. Therefore at given x we can only use the asymptotic series up to the n[0]-th term.
For example, if we take x=10, then we find that the 32-nd term of the asymptotic series has the smallest magnitude (about 10^(-28)) but the following terms start to grow.
To be on the safe side, we should drop a few more terms from the series. Define the number of terms by n[0]:=Pi*Abs(x). Then the order of magnitude of the n[0]-th term is Exp(-2*Pi*Abs(x))/(2*Pi^2*Abs(x)). This should be compared with the magnitude of the sum of the series which is of order Abs(x*Ln(x)). We find that the relative precision of P decimal digits or better is achieved if
For very large P, the inequality is satisfied when roughly x>P*Ln(10)/Ln(P). Assuming that the Bernoulli numbers are precomputed, the complexity of this method is that of computing a Taylor series with n[0] terms, which is roughly O(Sqrt(P))*M(P).
What if x is not large enough? Using the identity x*Gamma(x)=Gamma(x+1), we can reduce the computation of Gamma(x) to Gamma(x+M) for some integer M. Then we can choose M to be large enough so that the asymptotic series gives the required precision when evaluated at x+M. We shall have to divide the result M times by some long numbers to obtain Gamma(x). Therefore, the complexity of this method for given (x, P) is increased by M(P)*(P*Ln(10)/Ln(P)-x). For small x this will be the dominant contribution to the complexity.
On the other hand, if the Bernoulli numbers are not available precomputed, then their calculation dominates the complexity of the algorithm.
This method works well when 1<=x<2 (other values of x need to be reduced first). The idea is to represent the Gamma function as a sum of two integrals,
The first integral in this equation can be found as a sum of the Taylor series (expanding Exp(-u) near u=0),
Now we can estimate the number of terms in the above series. We know that the value of the Gamma function is of order 1. The condition that n-th term of the series is smaller than 10^(-P) gives n*Ln(n/e*M)>P*Ln(10). With the above value for M, we obtain n=P*Ln(10)/W(1/e) where W is Lambert's function; W(1/e)<=>0.2785.
The terms of the series are however not monotonic: first the terms grow and then they start to decrease, like in the Taylor series for the exponential function evaluated at a large argument. The ratio of the ( k+1)-th term to the k-th term is approximately M/(k+1). Therefore the terms with k<=>M will be the largest and will have the magnitude of order M^M/M! <=>Exp(M)<=>10^P. In other words, we will be adding and subtracting large numbers with P digits before the decimal point, but we need to obtain a result with P digits after the decimal point. Therefore to avoid the round-off error we need to increase the working precision to 2*P floating-point decimal digits.
It is quicker to compute this series if x is a small rational number, because then the long multiplications can be avoided, or at high enough precision the binary splitting can be used. Calculations are also somewhat faster if M is chosen as an integer value.
If the second integral is approximated by an asymptotic series instead of a constant Exp(-M), then it turns out that the smallest error of the series is Exp(-2*M). Therefore we can choose a smaller value of M and the round-off error gets somewhat smaller. According to [Brent 1978], we then need only 3/2*P digits of working precision, rather than 2*P, for computing the first series (and only P/2 digits for computing the second series). However, this computational savings may not be significant enough to justify computing a second series.
The basic formulae for the "fast" method (Brent's method "B1") are:
First, the sequence H[n] is defined as the partial sum of the harmonic series:
According to [Brent et al. 1980], the error of this approximation of gamma, assuming that S(n) and V(n) are computed exactly, is
The required number of terms k[max] in the summation over k to get S(n) and V(n) with this precision can be approximated as usual via Stirling's formula. It turns out that k[max] is also proportional to the number of digits, k[max]<=>2.07*P.
Therefore, this method of computing gamma has "linear convergence", i.e. the number of iterations is linear in the number of correct digits we need in the result. Of course, all calculations need to be performed with the working precision. The working precision must be a few digits more than P because we accumulate about Ln(k[max])/Ln(10) digits of round-off error by performing k[max] arithmetic operations.
Brent mentions a small improvement on this method (his method "B3"). It consists of estimating the error of the approximation of gamma by an asymptotic series. Denote W(n) the function
This trick can be formulated for any sequence A[k] of the form A[k]=B[k]*C[k], where the sequences B[k] and C[k] are given by the recurrences B[k]=p(k)*B[k-1] and C[k]=q(k)+C[k-1]. Here we assume that p(k) and q(k) are known functions of k that can be computed to P digits using O(P) operations, e.g. rational functions with short constant coefficients. Instead of evaluating B[k] and C[k] separately and multiplying them using a long multiplication, we note that p(k)*A[k-1]=B[k]*C[k-1]. This allows to compute A[k] by using the following two recurrences:
Also, it turns out that we can use a variant of the fast "rectangular method" to evaluate the series for U(n) and V(n) simultaneously. (We can consider these series as Taylor series in n^2.) This however does not speed up the evaluation of gamma. This happens because the rectangular method requires long multiplications and leads in this case to increased round-off errors. The rectangular method for computing a power series in x is less efficient than a straightforward computation when x is a "short" rational or integer number.
The "rectangular method" for computing Sum(k,0,N,x^k*A[k]) needs to be able to convert a coefficient of the Taylor series into the next coefficient A[k+1] by "short" operations, more precisely, by some multiplications and divisions by integers of order k. The j-th column of the rectangle ( j=0, 1, ...) consists of numbers x^(r*j)*A[r*j], x^(r*j)*A[r*j+1], ..., x^r*A[r*j+r-1]. The numbers of this column are computed sequentially by short operations, starting from the x^(r*j)*A[j*r] which is known from the end of the previous column. The recurrence relation for A[k] is not just some multiplication by rational numbers, but also contains an addition of B[k]. However, if we also use the rectangular method for V(n), the number x^(r*j)*B[r*j] will be known and so we will be able to use the recurrence relation to get x^(r*j)*A[r*j+1] and all following numbers of the column.
To obtain P decimal digits of relative precision, we need to take at most P*Ln(10)/Ln(4) terms of the series. The sum can be efficiently evaluated using Horner's scheme, for example
A drawback of this scheme is that it requires a separate high-precision computation of Pi, Sqrt(3) and of the logarithm.
This method combined with Brent's summation trick (see the section on the Euler constant) was used in [Fee 1990]. Brent's trick allows to avoid a separate computation of the harmonic sum and all long multiplications. Catalan's constant is obtained as a limit of G[k] where G[0]=B[0]=1/2 and
A third formula is more complicated but the convergence is much faster and there is no need to evaluate any other transcendental functions. This formula is called "Broadhurst's series".
We need to take only P*Ln(10)/Ln(16) terms of the first series and P*Ln(10)/Ln(4096) terms of the second series. However, each term is about six times more complicated than one term of the first method's series. So there are no computational savings (unless Ln(x) is excessively slow).
The classic book [Bateman et al. 1953], vol. 1, describes many results concerning the properties of Zeta(s).
For the numerical evaluation of Riemann's Zeta function with arbitrary precision to become feasible, one needs special algorithms. Recently P. Borwein [Borwein 1995] gave a simple and quick approximation algorithm for Re(s)>0. See also [Borwein et al. 1999] for a review of methods.
It is the "third" algorithm (the simplest one) from P. Borwein's paper which is implemented in Yacas. The approximation formula valid for Re(s)> -(n-1) is
This method requires to compute n times the exponential and the logarithm to find the power (j+1)^(-s). This power can be computed in asymptotic time O(M(P)*Ln(P)), unless s is an integer, in which case this computation is O(M(P)), the cost of one division by an integer (j+1)^s. Therefore the complexity of this method is at most O(P*M(P)*Ln(P)).
The function Zeta(s) calls Internal'ZetaNum(s) to compute this approximation formula for Re(s)>1/2 and uses the identity above to get the value for other s.
For very large values of s, it is faster to use more direct methods implemented in the routines Internal'ZetaNum1(s,N) and Internal'ZetaNum2(s,N). If the required precision is P digits and s>1+Ln(10)/Ln(P)*P, then it is more efficient to compute the defining series for Zeta(s),
Alternatively, one can use Internal'ZetaNum2(n,N) which computes the infinite product over prime numbers p[i]
The value Zeta(3), also known as the Apery's constant, can be computed using the following geometrically convergent series:
For other odd integers n there is no general analogous formula. The corresponding expressions for Zeta(5) and Zeta(7) are
In these series the term Bin(2*k,k) grows approximately as 4^k and therefore one can take no more than P*Ln(10)/Ln(4) terms in the series to get P decimal digits of relative precision.
For odd integer n there are the following special relations: for n:=Mod(3,4),
These relations contain geometrically convergent series, and it suffices to take P*Ln(10)/(2*Pi) terms to obtain P decimal digits of relative precision.
Finally, [Kanemitsu et al. 2001] gave a curious formula for the values of Riemann's Zeta function at rational values between 0 and 2 (their "Corollary 1"). This formula is very complicated but contains a geometrically convergent series.
We shall have to define several auxiliary functions to make the formula more understandable. We shall be interested in the values of Zeta(p/N) where p, N are integers. For integer h, N such that 0<=h<=N, and for arbitrary real x,
Practical calculations using this formula are of the same asymptotic complexity as Borwein's method above. (It is not clear whether this method has a significant computational advantage.) The value of x can be chosen at will, so we should find such x as to minimize the cost of computation. There are two series to be computed: the terms in the first one decay as Exp(-n^N*x) while the terms in the second one (containing f) decay only as
For a target precision of P decimal digits, the required numbers of terms n[1], n[2] for the first and the second series can be estimated as n[1]<=>((P*Ln(10))/x)^(1/N), n[2]<=>x/(2*Pi)*((P*Ln(10))/(2*Pi))^N. (Here we assume that N, the denominator of the fraction p/N, is at least 10. This scheme is impractical for very large N because it requires to add O(N) slightly different variants of the second series.) The total cost is proportional to the time it takes to compute Exp(x) or Cos(x) and to roughly n[1]+N*n[2]. The value of x that minimizes this cost function is approximately
Asymptotics of Lambert's W function are
Here are some inequalities to help estimate W(x) at large x (more exactly, for x>e):
One can also find uniform rational approximations, e.g.:
There exists a uniform approximation of the form
The numerical procedure uses Halley's method. Halley's iteration for the equation W*Exp(W)=x can be written as
The initial value is computed using one of the uniform approximation formulae. The good precision of the uniform approximation guarantees rapid convergence of the iteration scheme to the correct root of the equation, even for complex arguments x.
For large values of Abs(x), there is the following asymptotic series:
The error of a truncated asymptotic series is not larger than the first discarded term if the number of terms is larger than n-1/2. (See the book [Olver 1974] for derivations. It seems that each asymptotic series requires special treatment and yet in all cases the error is about the same as the first discarded term.)
Currently Yacas can compute BesselJ(n,x) for all x where n is an integer and for Abs(x)<=2*Gamma(n) when n is a real number. Yacas currently uses the Taylor series when Abs(x)<=2*Gamma(n) to compute the numerical value:
If Abs(x)>2*Gamma(n) and n is an integer, then Yacas uses the forward recurrence relation:
We see from the definition that when Abs(x)<=2*Gamma(n), the absolute value of each term is always decreasing (which is called absolutely monotonely decreasing). From this we know that if we stop after i iterations, the error will be bounded by the absolute value of the next term. So given a set precision, turn this into a value epsilon, so that we can check if the current term will contribute to the sum at the prescribed precision. Before doing this, Yacas currently increases the precision by 20% to do interim calculations. This is a heuristic that works, it is not backed by theory. The value epsilon is given by epsilon:=5*10^(-prec), where prec was the previous precision. This is directly from the definition of floating point number which is correct to prec digits: A number correct to prec digits has a rounding error no greater than 5*10^(-prec). Beware that some books incorrectly have .5 instead of 5.
Bug: Something is not right with complex numbers, but pure imaginary are OK.
Bernoulli numbers and polynomials are used in various Taylor series expansions, in the Euler-Maclauren series resummation formula, in Riemann's Zeta function and so on. For example, the sum of (integer) p-th powers of consecutive integers is given by
The Bernoulli polynomials B(x)[n] can be found by first computing an array of Bernoulli numbers up to B[n] and then applying the above formula for the coefficients.
We consider two distinct computational tasks: evaluate a Bernoulli number exactly as a rational, or find it approximately to a specified floating-point precision. There are also two possible problem settings: either we need to evaluate all Bernoulli numbers B[n] up to some n (this situation occurs most often in practice), or we only need one isolated value B[n] for some large n. Depending on how large n is, different algorithms can be chosen in these cases.
Here is an estimate of the cost of BernoullliArray. Suppose M(P) is the time needed to multiply P-digit integers. The required number of digits P to store the numerator of B[n] is asymptotically P<>n*Ln(n). At each of the n iterations we need to multiply O(n) large rational numbers by large coefficients and take a GCD to simplify the resulting fractions. The time for GCD is logarithmic in P. So the complexity of this algorithm is O(n^2*M(P)*Ln(P)) with P<>n*Ln(n).
For large (even) values of the index n, a single Bernoulli number B[n] can be computed by a more efficient procedure: the integer part and the fractional part of B[n] are found separately (this method is also well explained in [Gourdon et al. 2001]).
First, by the theorem of Clausen -- von Staudt, the fractional part of (-B[n]) is the same as the fractional part of the sum of all inverse prime numbers p such that n is divisible by p-1. To illustrate the theorem, take n=10 with B[10]=5/66. The number n=10 is divisible only by 1, 2, 5, and 10; this corresponds to p=2, 3, 6 and 11. Of these, 6 is not a prime. Therefore, we exclude 6 and take the sum 1/2+1/3+1/11=61/66. The theorem now says that 61/66 has the same fractional part as -B[10]; in other words, -B[10]=i+f where i is some unknown integer and the fractional part f is a nonnegative rational number, 0<=f<1, which is now known to be 61/66. Indeed -B[10]= -1+61/66. So one can find the fractional part of the Bernoulli number relatively quickly by just checking the numbers that might divide n.
Now one needs to obtain the integer part of B[n]. The number B[n] is positive if Mod(n,4)=2 and negative if Mod(n,4)=0. One can use Riemann's Zeta function identity for even integer values of the argument and compute the value zeta(n) precisely enough so that the integer part of the Bernoulli number is determined. The required precision is found by estimating the Bernoulli number from the same identity in which one approximates zeta(n)=1, i.e.
At such large values of the argument n, it is feasible to use the routines Internal'ZetaNum1(n, N) or Internal'ZetaNum2(n,N) to compute the zeta function. These routines approximate zeta(n) by the defining series
For example, let us compute B[20] using this method.
In> 1/2 + 1/3 + 1/5 + 1/11; Out> 371/330; |
In> N(1+1/2^20) Out> 1.0000009536; |
In> N( 2*20! /(2*Pi)^20*1.0000009536 ) Out> 529.1242423667; |
In> -(529+41/330); Out> -174611/330; |
All these steps are implemented in the routine Bernoulli1. The function Bernoulli1Threshold() returns the smallest n for which B[n] is to be computed via this routine instead of the recursion relation. Its current default value is 20. This value can be set with SetBernoulli1Threshold(threshold).
The complexity of Bernoulli1 is estimated as the complexity of finding all primes up to n plus the complexity of computing the factorial, the power and the Zeta function. Finding the prime numbers up to n by checking all potential divisors up to Sqrt(n) requires O(n^(3/2)*M(Ln(n))) operations with precision O(Ln(n)) digits. For the second step we need to evaluate n!, Pi^n and zeta(n) with precision of P=O(n*Ln(n)) digits. The factorial is found in n short multiplications with P-digit numbers (giving O(n*P)), the power of pi in Ln(n) long multiplications (giving O(M(P)*Ln(n))), and Internal'ZetaNum2(n) (the asymptotically faster algorithm) requires O(n*M(P)) operations. The Zeta function calculation dominates the total cost because M(P) is slower than O(P). So the total complexity of Bernoulli1 is O(n*M(P)) with P<>n*Ln(n).
Note that this is the cost of finding just one Bernoulli number, as opposed to the O(n^2*M(P)*Ln(P)) cost of finding all Bernoulli numbers up to B[n] using the first algorithm Internal'BernoulliArray. If we need a complete table of Bernoulli numbers, then Internal'BernoulliArray is only marginally (logarithmically) slower. So for finding complete Bernoulli tables, Bernoulli1 is better only for very large n.
However, the recurrence relation used in Internal'BernoulliArray turns out to be numerically unstable and needs to be replaced by another [Brent 1978]. Brent's algorithm computes the Bernoulli numbers divided by factorials, C[n]:=B[2*n]/(2*n)! using a (numerically stable) recurrence relation
The numerical instability of the usual recurrence relation
The eigenvalue of the sequence e[k] can be found approximately for large k if we notice that the recurrence relation for e[k] is similar to the truncated Taylor series for Sin(x). Substituting e[k]=lambda^k into it and disregarding a very small number (2*Pi)^(-2*k) on the right hand side, we find
By a very similar calculation one finds that the inverse powers of 4 in Brent's recurrence make the largest eigenvalue of the error sequence e[k] almost equal to 1 and therefore the recurrence is stable. Brent gives the relative error in the computed C[k] as O(k^2) times the round-off error in the last digit of precision.
The complexity of Brent's method is given as O(n^2*P+n*M(P)) for finding all Bernoulli numbers up to B[n] with precision P digits. This computation time can be achieved if we compute the inverse factorials and powers of 4 approximately by floating-point routines that know how much precision is needed for each term in the recurrence relation. The final long multiplication by (2*k)! computed to precision P adds M(P) to each Bernoulli number.
The non-iterative method using the Zeta function does not perform much better if a Bernoulli number B[n] has to be computed with significantly fewer digits P than the full O(n*Ln(n)) digits needed to represent the integer part of B[n]. (The fractional part of B[n] can always be computed relatively quickly.) The Zeta function needs 10^(P/n) terms, so its complexity is O(10^(P/n)*M(P)) (here by assumption P is not very large so 10^(P/n)<n/(2*Pi*e); if n>P we can disregard the power of 10 in the complexity formula). We should also add O(Ln(n)*M(P)) needed to compute the power of 2*Pi. The total complexity of Bernoulli1 is therefore O(Ln(n)*M(P)+10^(P/n)*M(P)).
If only one Bernoulli number is required, then Bernoulli1 is always faster. If all Bernoulli numbers up to a given n are required, then Brent's recurrence is faster for certain (small enough) n.
Currently Brent's recurrence is implemented as Internal'BernoulliArray1() but it is not used by Bernoulli because the internal arithmetic is not yet able to correctly compute with floating-point precision.
The complementary error function Erfc(x) is defined for real x as
The imaginary error function Erfi(x) is defined for real x as
Numerical computation of the error function Erf(z) needs to be performed by different methods depending on the value of z and its position in the complex plane, and on the required precision. We follow the book [Tsimring 1988] and the paper [Thacher 1963]. (These texts, however, do not describe arbitrary-precision computations.)
The function Erf(z) has the following approximations that are useful for its numerical computation:
Here we shall analyze the convergence and precision of these methods. We need to choose a good method to compute Erf(z) with (relative) precision P decimal digits for a given (complex) number z, and to obtain estimates for the necessary number of terms to take.
Both Taylor series converge absolutely for all z, but they do not converge uniformly fast; in fact these series are not very useful for large z because a very large number of slowly decreasing terms gives a significant contribution to the result, and the round-off error (especially for the first series with the alternating signs) becomes too high. Both series converge well for Abs(z)<1.
Consider method 1 (the first Taylor series). We shall use the method 1 only for Abs(z)<=1. If the absolute error of the truncated Taylor series is estimated as the first discarded term, the precision after taking all terms up to and including z^(2*n) is approximately z^(2*n+2)/(n+2)!. The factorial can be approximated by Stirling's formula, n! <=>n^n*e^(-n). The value of Erf(z) at small z is of order 1, so we can take the absolute error to be equal to the relative error of the series that starts with 1. Therefore, to obtain P decimal digits of precision, we need the number of terms n that satisfies the inequality
Consider method 3 (the asymptotic series). Due to limitations of the asymptotic series, we shall use the method 3 only for large enough values of z and small enough precision.
There are two important cases when calculating Erf(z) for large (complex) z: the case of z^2>0 and the case of z^2<0. In the first case (e.g. a real z), the function Erf(z) is approximately 1 for large Abs(z) (if Re(z)>0, and approximately -1 if Re(z)<0). In the second case (e.g. pure imaginary z=I*t) the function Erf(z) rapidly grows as Exp(-z^2)/z at large Abs(z).