Links: LIN353C introduction to computational linguistics


Natural Language Processing tools

  • The Stanford NLP tools, including tools for automatic part-of-speech tagging, parsing, coreference resolution, and named entity recognition for English
  • FreeLing: Natural Language Processing tools for a variety of languages
  • TreeTagger: a fast part-of-speech tagger and lemmatizer for a variety of languages


Python

Installing Python:

Learning Python:

General Python pages:

The Natural Language Toolkit:

NLTK Installation Tips

General installation tips

If you typed "sudo pip install -U nltk" but got an error message that pip could not be found, try this instead:

sudo python3 -m pip install -U nltk

On Mac, use the Terminal

Do not try to type the NLTK installation commands into a Python or Idle shell. It's a language problem: Idle speaks only Python. The installation commands are shellscript commands. So Idle does not understand them.

Under "Applications", select the subfolder "Utilities", and in there the application "Terminal". This gives you a small window where you can type in the NLTK installation commands.

Error message on a Mac: compiler not found

After getting this error message, on student used the following steps to get NLTK working on her Mac:

  1. Download the appropriate version of Xcode (takes about 3 hours). You need to go here: https://developer.apple.com/xcode/,  click on "downloads" and register as an Apple developer.

    If you get an error message that iTunes needs to be closed before Xcode can install, and that error message won't go away even after you have closed iTunes, try this: http://osxdaily.com/2011/08/02/cant-install-xcode-because-itunes-fix/

  2. During install, make sure the following boxes are checked when prompted:
    • UNIX Development (includes command line tools)
    • System Tools
    • X11 SDK (may be another version of SDK)

  3. Once you've installed, type the following in terminal:
        sudo easy_install pip
        sudo pip install numpy
        sudo pip install nltk

Error on Windows: could not install nltk because of 64-bit Python
NLTK can only work with 32-bit Python

One student who encountered this problem said:

    I managed to fix my problem installing NLTK by reinstalling Python with the 32-bit installer
    (admittedly, the nltk page says to avoid 64-bit, but it doesn't emphasize it).
    I also had to set a PYTHONPATH environment variable (to C:\Python27 in my case),
    but I think that might just be because the old installation was causing conflicts.

Other bug fixes

In some people's systems, the nltk.app.chunkparser() application within NLTK crashes and takes the whole Python with it. This is due to a problem with the Python matplotlib library and its TK interface. Here is a solution for the Mac OS:

* Open a Terminal.
* In the terminal, type:
mkdir -p ~/.matplotlib
echo "backend: TkAgg" > ./.matplotlib/matplotlibrc
This makes a file with setup information for matplotlib and should fix the issue.


Comments