Data · dataset · 2007
Replication data for: A Correlated Topic Model of Science
Listed in Harvard Dataverse
Topic models, such as latent Dirichlet allocation (LDA), can be useful tools for the statistical analysis of document collections and other discrete data.
Description
The LDA model assumes that the words of each document arise from a mixture of topics, each of which is a distribution over the vocabulary. A limitation of LDA is the inability to model topic correlation even though, for example, a document about genetics is more likely to also be about disease than X-ray astronomy.
This limitation stems from the use of the Dirichlet distribution to model the variability among the topic proportions. In this paper we develop the correlated topic model (CTM), where the topic proportions exhibit correlation via the logistic normal distribution [J. Roy. Statist.
Read the rest (2 more)
Soc. Ser. B 44 (1982) 139–177].
We derive a fast variational inference algorithm for approximate posterior inference in this model, which is complicated by the fact that the logistic normal is not conjugate to the multinomial. We apply the CTM to the articles from Science published from 1990–1999, a data set that comprises 57M words. The CTM gives a better fit of the data than LDA, and we demonstrate its use as an exploratory tool of large document collections.
Links
Where it is published
- Dataverse dataset page dataverse.harvard.edu/dataset.xhtml?persistentId=doi%3A10.7910%2FDVN%2F12VGO7 ↗
landing page · from Harvard Dataverse
- DOI doi.org/10.7910/dvn/12vgo7 ↗
DOI / persistent id · from Harvard Dataverse
Catalogue records · 1
- Dataverse API dataverse.harvard.edu/api/datasets/:persistentId/?persistentId=doi%3A10.7910%2FDVN%2… ↗
metadata API · from Harvard Dataverse
Topics
- Inferred from text
- Disease 75%
Provenance · 1 source records, 7 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Harvard Dataverse | doi:10.7910/DVN/12VGO7 | 12 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| concepts[disease].local:disease:disease | enrichment · Harvard Dataverse | keyword-concept-rules@1.0.0 | title+description (75%) |
| created_date | source · Harvard Dataverse | connector:dataverse@1.0.0 | |
| description | source · Harvard Dataverse | connector:dataverse@1.0.0 | /description |
| publication_date | source · Harvard Dataverse | connector:dataverse@1.0.0 | |
| title | source · Harvard Dataverse | connector:dataverse@1.0.0 | /name |
| updated_date | source · Harvard Dataverse | connector:dataverse@1.0.0 | |
| version_label | source · Harvard Dataverse | connector:dataverse@1.0.0 |