Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(square (square (square 1000))) incorrect -- unable to handle large integers #47

Open
ptdecker opened this issue Aug 17, 2020 · 0 comments

Comments

@ptdecker
Copy link
Owner

(square (square (square 1000))) calculates incorrectly. Value returned is '5726162197579952128' while the correct answer is '1008028056070056028008001'. Probably due to the implementation of integers using c's 'long' type and LONG_MAX is '9223372036854775807' (32-bit). Switching to type 'long long' (64-bit) would correct the problem for this example. Switching to GMP or some other library to handle arbitrary precision arithmetic would be perhaps better. For space efficiency, might be great to sense overflows and automatically switch base types

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant