Go to file
2019-09-11 13:39:51 +02:00
wrapper Use latest image tag for master. 2019-09-11 13:39:51 +02:00
.gitlab-ci.yml Update CI script and remove unused code. 2019-07-31 11:39:54 +02:00
Dockerfile Change the documentation style. 2019-09-11 13:34:01 +02:00
nlp Update 2019-05-20 12:08:13 +02:00
README.md Update for unprivileged usage 2 2019-06-03 13:32:37 +02:00
spacy_nlp Set a more specific python version. 2019-05-23 10:09:01 +02:00

Natural language processing

This repository provides all code that is needed to build a container image for natural language processing utilizing spaCy.

Build image

  1. Clone this repository and navigate into it:
git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nlp.git && cd nlp
  1. Build image:
docker build -t sfb1288inf/nlp:latest .

Alternatively build from the GitLab repository without cloning:

  1. Build image:
docker build -t sfb1288inf/nlp:latest https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nlp.git

Download prebuilt image

The GitLab registry provides a prebuilt image. It is automatically created, utilizing the conquaire build servers.

  1. Download image:
docker pull gitlab.ub.uni-bielefeld.de:4567/sfb1288inf/nlp:latest

Run

  1. Create input and output directories for the NLP software:
mkdir -p /<mydatalocation>/files_for_nlp /<mydatalocation>/files_from_nlp
  1. Place your text files inside the /<mydatalocation>/files_for_nlp directory. Files should all contain text of the same language.

  2. Start the NLP process.

docker run \
    --rm \
    -it \
    -u $(id -u $USER):$(id -g $USER) \
    -v /<mydatalocation>/files_for_nlp:/input \
    -v /<mydatalocation>/files_from_nlp:/output \
    sfb1288inf/nlp:latest \
        -i /input \
        -l <languagecode> \
        -o /output

The arguments below sfb1288inf/nlp:latest are described in the NLP arguments part.

If you want to use the prebuilt image, replace sfb1288inf/nlp:latest with gitlab.ub.uni-bielefeld.de:4567/sfb1288inf/nlp:latest.

  1. Check your results in the /<mydatalocation>/files_from_nlp directory.

NLP arguments

-i path

  • Sets the input directory using the specified path.
  • required = True

-o path

  • Sets the output directory using the specified path.
  • required = True

-l languagecode

  • Tells spaCy which language will be used.
  • options = de (German), el (Greek), en (English), es (Spanish), fr (French), it (Italian), nl (Dutch), pt (Portuguese)
  • required = True