Jump to content
Chinese-Forums
  • Sign Up

looking for a chinese text corpus


Recommended Posts

Posted

Hello,

I'm looking for a language corpus of written or spoken chinese language for computational linguistic purposes. It should consist of a balanced selection of genres and stratas. There should be a distinction between modern and archaic language, either of them would be nice but i'm looking for both. And the corpus should be available under appropriate copyright for publishing any statistic data retrieved from it.

Btw. what is the adaequate translation for corpus in linguistic context? 语料库 ?

Thanks in advance!

Posted

Lancaster Corpus of Mandarin Chinese. To get the actual data, click on the Oxford Text Archive link, and then search for the name. It has texts in many different genres, which it separates into files for each category. As a bonus, words are segmented and tagged for part of speech. Unfortunately, you will need some familiarity with XML to get any use out of it. I just put up my rough perl scripts to a website here, if you needed an idea to get started. It parses the xml into an SQLite database, after which I can run any queries I need, including extracting all the text into readable files.

According to their header file, the license is:

Freely available for non-commercial useprovided [sic] that this header is included in its entirety with any copy distributed
Posted

After extracting the text I got 45731 lines. That's about 2 mb compressed.

Not too big, but it's free, so I'm happy. I didn't know there were any freely available corpora. Thanks.

Join the conversation

You can post now and select your username and password later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Click here to reply. Select text to quote.

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...