Most books or research papers are written for people with college level mathematical training. At this moment, you have to bite the bullet and look at books in indvidual areas and keep googling it. One reason is that most theories are developed decades ago without the context of ML.

Being said, I am in the process of writing a series of articles for the Maths in ML. I am releasing it one topic a week. Afterwards, I may open source it that may eventually become a book format.