Using Faster Exponential Approximation

In some scenarios/applicatons, where the precision may not be so critically important but the speed (performance) is, you may be willing to sacrifice some extent of accuracy for the speed.

In neutral networks, where the math function where n is usually small (less than 2, for instance), you can avoid the expensive exp() provided by math.h (for other programming languages, similar inbuilt system functions are provided)

The (exponential function) can be considered as the following:

In practice, n  cannot approach to the infinity but we can achieve a relatively good accuracy by using a large n.

For example, if we put , then we can multiply by itself 8 times due to the fact

With this in mind, we can come up with the following approximation:

1
2
3
4
5
6
7

inline
double exp1(double x) {
  x = 1.0 + x / 256.0;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x; x *= x; x *= x;
  return x;
}

inline
double exp1(double x) {
  x = 1.0 + x / 256.0;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x; x *= x; x *= x;
  return x;
}

We can also multiply a few more times, to increase the accuracy.

1
2
3
4
5
6
7
8

inline
double exp2(double x) {
  x = 1.0 + x / 1024;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x;
  return x;
}

inline
double exp2(double x) {
  x = 1.0 + x / 1024;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x; x *= x; x *= x;
  x *= x; x *= x;
  return x;
}

Now, you have the pattern, but for now, we need to test how accurate these approximations are:

The above plots 3 curves, which are the exp provided by math.h, the exp 256 and the exp 1024. They show very good agreement for input smaller than 5.

We plot the difference to make it easier to see.

Wow, it really can be a faster alternative if the required input range smaller than 5. For negative inputs, the difference won’t be so noticeable because the value itself is so tiny that can’t be observed visually in graph.

The exp 256 is 360 times faster than the traditional exp and the exp 1024 is 330 times faster than the traditional exp.

–EOF (Coding For Speed) —