holger Posted October 3, 2007 at 12:57 PM Report Posted October 3, 2007 at 12:57 PM Hello, I'm looking for a language corpus of written or spoken chinese language for computational linguistic purposes. It should consist of a balanced selection of genres and stratas. There should be a distinction between modern and archaic language, either of them would be nice but i'm looking for both. And the corpus should be available under appropriate copyright for publishing any statistic data retrieved from it. Btw. what is the adaequate translation for corpus in linguistic context? 语料库 ? Thanks in advance! Quote
c_redman Posted October 3, 2007 at 03:11 PM Report Posted October 3, 2007 at 03:11 PM Lancaster Corpus of Mandarin Chinese. To get the actual data, click on the Oxford Text Archive link, and then search for the name. It has texts in many different genres, which it separates into files for each category. As a bonus, words are segmented and tagged for part of speech. Unfortunately, you will need some familiarity with XML to get any use out of it. I just put up my rough perl scripts to a website here, if you needed an idea to get started. It parses the xml into an SQLite database, after which I can run any queries I need, including extracting all the text into readable files. According to their header file, the license is: Freely available for non-commercial useprovided [sic] that this header is included in its entirety with any copy distributed Quote
in_lab Posted October 4, 2007 at 08:32 AM Report Posted October 4, 2007 at 08:32 AM After extracting the text I got 45731 lines. That's about 2 mb compressed. Not too big, but it's free, so I'm happy. I didn't know there were any freely available corpora. Thanks. Quote
Recommended Posts
Join the conversation
You can post now and select your username and password later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.