User Tools

Site Tools


Visual Analytics Course

Text and documents

Tag clouds

Vector space model

This is where statistics and linguistics meet. Being able to decide of a relevant set of keywords to index documents, or even a set of keywords that best describe a document or set of document is difficult. There are tricks like using the tf-idf score of words to try to determine how relevant they are with respect to a given set of documents.

Consider a set of keywords as given . Any documents may then be described with respect to how many times each of these keywords occur in the document. This information can be summarized into a co-occurrence matrix where each of the lines correspond to documents and columns correspond to keywords. The entry thus computes the number of occurrences of in document .

The line vector thus describes document and can be used to compute several different statistics on the dataset.

/net/html/perso/melancon/Visual_Analytics_Course/data/pages/text_and_documents.txt · Last modified: 2012/10/11 14:36 by melancon