PyStemmer
=========

What is PyStemmer?
------------------

PyStemmer is a Python interface to the stemming algorithms from the Snowball
project (https://snowballstem.org/).

Snowball can generate pure-Python stemmer code, but if you want to stem a
lot of words this can be rather slow.

PyStemmer instead wraps the "libstemmer_c" library which is built from C
code generated by Snowball.  An unmodified copy of this library is contained
within the PyStemmer distribution.  If you wish to upgrade PyStemmer
to a more recent version of libstemmer_c (or the snowball algorithms), it
should suffice to download a new copy of libstemmer_c from the snowball
project, and replace the contents of the libstemmer_c subdirectory with the
contents of the download.

An alternative to using PyStemmer directly is to use the snowballstemmer
module from Snowball, which will automatically use PyStemmer if available,
falling back to the pure Python implementations if not.  This allows your
users to choose between the convenience of only dealing with pure Python
code and the better performance of PyStemmer.

What is Stemming?
-----------------

Stemming maps different forms of the same word to a common "stem" - for
example, the English stemmer maps *connection*, *connections*, *connective*,
*connected*, and *connecting* to *connect*.  So a searching for *connected*
would also find documents which only have the other forms.

This stem form is often a word itself, but this is not always the case as this
is not a requirement for text search systems, which are the intended field of
use.  We also aim to conflate words with the same meaning, rather than all
words with a common linguistic root (so *awe* and *awful* don't have the same
stem), and over-stemming is more problematic than under-stemming so we tend not
to stem in cases that are hard to resolve.  If you want to always reduce words
to a root form and/or get a root form which is itself a word then Snowball's
stemming algorithms likely aren't the right answer.

Requirements
------------

Python header files should be installed.

This version of PyStemmer has been CI tested using Python series 2.6, 2.7, 3.3,
3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, pypy and pypy3.

Installation
------------

PyStemmer uses distutils, so all that is necessary to build and install
PyStemmer is the usual distutils invocation::

    python setup.py install

You can also install using ``pip``:

    * from PyPI: ``pip install pystemmer``
    * from a local copy of the code: ``pip install .``
    * from git: ``pip install git+git://github.com/snowballstem/pystemmer``

API
---

PyStemmer's API is documented by documentation comments.

A brief overview can be found in docs/quickstart.txt

License
-------

PyStemmer is copyright (c) 2006, Richard Boulton, and is licensed under the MIT
license: see the file "LICENSE" for the full text of this.  It is was inspired
by an earlier implementation (which was copyright (c) 2001, Andreas Jung, and
also licensed under the MIT license, but no portions of which remain in this
package, and had a different API).

The snowball algorithms, and the snowball library, are copyright (c) 2001-2006,
Dr Martin Porter and Richard Boulton, and are licensed under the BSD license.
