Casio FX-55 User Guide - Page 15

Calculation Ranges, Number of Digits, and Precision

Page 15 highlights

l= F=5, 5 6 l= 5 6 When further simplification is not possible, "Simplified" appears momentarily on the display and then the calculation result is displayed. When a divisor of 3 is specified: 18 = 6 21 7 18 ' 21 =l 3 = F=3, 6 7 or 18 ' 21 el 3 = F=3, 6 7 Calculation Ranges, Number of Digits, and Precision The calculation range, number of digits used for internal calculation, and calculation precision depend on the type of calculation you are performing. Calculation Range and Precision Calculation Range Number of Digits for Internal Calculation Precision ±1 × 10-99 to ±9.999999999 × 1099 or 0 15 digits In general, ±1 at the 10th digit for a single calculation. Precision for exponential display is ±1 at the least significant digit. Errors are cumulative in the case of consecutive calculations. Function Calculation Input Ranges and Precision Functions Input Range 10x -9.999999999 × 1099 Ϲ x Ϲ 99.99999999 'x 0 Ϲ x Ͻ 1 × 10100 x2 |x| Ͻ 1 × 1050 x -1 |x| Ͻ 1 × 10100 ; x G 0 |a|, b, c Ͻ 1 × 10100; 0 Ϲ b, c °' " The display seconds value is subject to an error of ±1 at the second decimal place. x Ͼ 0: -1 × 10100 Ͻ ylogx Ͻ 100 x = 0: y Ͼ 0 xy x Ͻ 0: y = n, m (m, n are integers) 2n+1 However: -1 × 10100 Ͻ ylog |x| Ͻ 100 a b/c Total of integer, numerator, and denominator must be 10 digits or less (including division marks). E-14

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20

E-14
l
=
F=5,
5
6
l
=
5
6
When further simplification is not possible, “Simplified” appears momentarily
on the display and then the calculation result is displayed.
When a divisor of 3 is specified:
18
21
=
6
7
18
'
21
=
l
3
=
F=3,
6
7
or
18
'
21
e
l
3
=
F=3,
6
7
Calculation Ranges, Number of Digits,
and Precision
The calculation range, number of digits used for internal calculation, and
calculation precision depend on the type of calculation you are performing.
Calculation Range and Precision
Calculation Range
±
1
×
10
–99
to
±
9.999999999
×
10
99
or 0
Number of Digits for Internal
Calculation
15 digits
Precision
In general,
±
1 at the 10th digit for a single
calculation. Precision for exponential
display is
±
1 at the least significant digit.
Errors are cumulative in the case of
consecutive calculations.
Function Calculation Input Ranges and Precision
Functions
Input Range
10
x
–9.999999999
×
10
99
³
x
³
99.99999999
'
x
0
³
x
´
1
×
10
100
x
2
|
x
|
´
1
×
10
50
x
–1
|
x
|
´
1
×
10
100
;
x
G
0
°’ ”
|
a
|
,
b
,
c
´
1
×
10
100
; 0
³
b
,
c
The display seconds value is subject to an error of ±1 at
the second decimal place.
x
y
x
±
0: –1
×
10
100
´
y
log
x
´
100
x
= 0:
y
±
0
x
´
0:
y
=
n
,
m
2
n
+1
(
m
,
n
are integers)
However: –1
×
10
100
´
y
log
|
x
|
´
100
a
b
/
c
Total of integer, numerator, and denominator must be
10 digits or less (including division marks).