The best use of the following list would be to first assess yourself, and then work on the parts that you are weakest at first. All the resources are chosen for your convenience, among many alternatives, and decided to be one of the best in their relative fields.
At the end, we also provide resources for learning cryptography itself as well.
Beginner’s resource: “Discrete mathematics : mathematical reasoning and proof with puzzles, patterns, and games” by Douglas E. Ensley and J. Winston Crawley.
More comprehensive resouce (includes topics that are directly related to Computer Science, and partly cryptography): “Discrete Mathematics with Applications” by Susanna Epp.
“An Introduction to Probability Theory and Its Applications”, by William Feller
Introductory resource: “Linear Algebra” by Jim Hefferon.
Advanced resource: “Linear Algebra” by Georgi E. Shilov.
Many exercises: “Linear Algebra: A Modern Introduction” by David Poole.
“A friendly introduction to number theory” by Joseph H. Silverman.
Mostly, the eventual knowledge we are trying to learn here is the complexity classes using Turing machines. Things like complexity class P and NP, concepts such as NP-completeness and reductions. Once the introductory knowledge has been gained, cryptography researchers are suggested to continue with the advanced resource.
Advanced resource: “Computational Complexity: A Modern Approach” by Sanjeev Arora and Boaz Barak.
Introductory resources (mostly on information theory):
“An Introduction to Information Theory: Symbols, Signals and Noise” by John R. Pierce.
“Introduction to Coding and Information Theory” by Steven Roman.
More resources (mostly on coding theory):
“A First Course in Coding Theory” by Raymond Hill.
“Coding Theory: A First Course” by San Ling and Chaoping Xing.
“Unix Network Programming, Volume 1: The Sockets Networking API” by W. Richard Stevens.
“The Design of the UNIX Operating System” by Maurice J. Bach.
“Operating Systems Design and Implementation” by Andrew S. Tanenbaum.