All references listed here have been mentioned somewhere in the course pages or exercises. Feel free to suggest supplementary references (links to books, websites, blogs, wikipedia pages, etc.)
Browse them, look at images, find the paper(s), implement.
Ward, M. O., G. Grinstein, et al. (2010). Interactive Data Visualization: Foundations, Techniques, and Applications, A. K. Peters.
This is the main book we used to get inspiration when writing this course. The book covers all issues concerning Visual Analytics. What I particularly like about the book is that each chapter is followed by suggestions for exercises and projects – if you look closely, you'll recognize exercises and projects I have borrowed.
Lebart, L., M. Piron, et al. (2006). Statistiques exploratoire multidimensionnelle - Visualisations et inférences en fouille de données, Dunod.
There must be thousands of similar introductory books on PCA and data analysis. But I specially like the style of these authors. And they moreover have a taste for visualization (!). Have a look at the first chapter – relates to the PCA part concerning high-dimensional data.
Ingwer Borg, Patrick J. F. Groenen (2005). Modern multidimensional scaling: theory and applications. Springer Series in Statistics.
This book provides a clear exposition about MDS, together with historical background and considerations on (ultra)metric spaces and applications in social sciences. The web pages by van Deun and Delbeke also offer quite a good introduction to the topic.
Graph Drawing is now well covered in these general audience books:
Handbook of Graph Drawing and Visualization. Roberto Tamassia, Editor, CRC Press (to appear).
Kaufmann, M. and D. Wagner, Eds. (2001). Drawing Graphs, Methods and Models. Lecture Notes in Computer Science, Springer.
di Battista, G., P. Eades, et al. (1998). Graph Drawing: Algorithms for the Visualisation of Graphs, Prentice Hall.
The books by Edward Tufte remain foundational texts, and even gems of literature. Try to have a look at it if they're available at your local library.
Feel free to browse the web, add a link and a comment.