# Visual Analytics Course

### Site Tools

text_and_documents

# Differences

This shows you the differences between two versions of the page.

 text_and_documents [2011/12/23 00:04]melancon [Vector space model] text_and_documents [2012/10/11 14:36] (current)melancon Both sides previous revision Previous revision 2012/10/11 14:36 melancon 2011/12/23 00:04 melancon [Vector space model] 2011/12/22 23:16 melancon 2011/09/15 05:19 melancon created 2012/10/11 14:36 melancon 2011/12/23 00:04 melancon [Vector space model] 2011/12/22 23:16 melancon 2011/09/15 05:19 melancon created Line 13: Line 13: This is where statistics and linguistics meet. Being able to decide of a relevant set of keywords to index documents, or even a set of keywords that best describe a document or set of document is difficult. There are tricks like using the tf-idf score of words to try to determine how relevant they are with respect to a given set of documents. This is where statistics and linguistics meet. Being able to decide of a relevant set of keywords to index documents, or even a set of keywords that best describe a document or set of document is difficult. There are tricks like using the tf-idf score of words to try to determine how relevant they are with respect to a given set of documents. - Consider a set of keywords as given w_1, ..., w_m. Any documents may then be described with respect to how many times each of these keywords occur in the document. This information can be summarized into a n x m co-occurrence matrix where each of the n lines correspond to documents and columns correspond to keywords. The c_ij entry thus computes the number of occurrences of w_j in document d_i. + Consider a set of keywords as given $w_1, \ldots, w_m$. Any documents may then be described with respect to how many times each of these keywords occur in the document. This information can be summarized into a $n \times ​m$ co-occurrence matrix where each of the $n$ lines correspond to documents and columns correspond to keywords. The $c_{ij}$ ​entry thus computes the number of occurrences of $w_j$ in document ​$d_i$. - The line vector w_i = (di1, ..., d_im) thus describes document d_i and can be used to compute several different statistics on the dataset. + The line vector ​$w_i = (d_{i_1}, \ldots, d_{i_m})$ thus describes document ​$d_i$ and can be used to compute several different statistics on the dataset.