Beyond the Zipf-Mandelbrot law in quantitative linguistics

Authors
Citation
Ma. Montemurro, Beyond the Zipf-Mandelbrot law in quantitative linguistics, PHYSICA A, 300(3-4), 2001, pp. 567-578
Citations number
14
Language
INGLESE
art.tipo
Article
Categorie Soggetti
Physics
Journal title
PHYSICA A
ISSN journal
0378-4371 → ACNP
Volume
300
Issue
3-4
Year of publication
2001
Pages
567 - 578
Database
ISI
SICI code
0378-4371(20011115)300:3-4<567:BTZLIQ>2.0.ZU;2-Y
Abstract
In this paper the Zipf-Mandelbrot law is revisited in the context of lingui stics. Despite its widespread popularity the Zipf-Mandelbrot law can only d escribe the statistical behaviour of a rather restricted fraction of the to tal number of words contained in some given corpus. In particular, we focus our attention on the important deviations that become statistically releva nt as larger corpora are considered and that ultimately could be understood as salient features of the underlying complex process of language generati on. Finally, it is shown that all the different observed regimes can be acc urately encompassed within a single mathematical framework recently introdu ced by C. Tsallis. (C) 2001 Elsevier Science B.V. All rights reserved.