mirror of
https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
synced 2024-12-25 19:04:18 +00:00
70 lines
3.6 KiB
Markdown
70 lines
3.6 KiB
Markdown
# nopaque
|
|
|
|
nopaque bundles various tools and services that provide humanities scholars with DH methods and thus can support their various individual research processes. Using nopaque, researchers can subject digitized sources to Optical Character Recognition (OCR). The resulting text files can then be used as a data basis for Natural Language Processing (NLP). The texts are automatically subjected to various linguistic annotations. The data processed via NLP can then be summarized in the web application as corpora and analyzed by means of an information retrieval system through complex search queries. The range of functions of the web application will be successively extended according to the needs of the researchers.
|
|
|
|
## Prerequisites and requirements
|
|
|
|
1. Install docker for your system. Following the official instructions.
|
|
2. Install docker-compose. Following the official instructions.
|
|
|
|
|
|
## Configuration and startup
|
|
|
|
### **Create Docker swarm**
|
|
|
|
The generated computational workload is handled by a [Docker](https://docs.docker.com/) swarm. A swarm is a group of machines that are running Docker and joined into a cluster. It consists out of two different kinds of members, manager and worker nodes. The swarm setup process is described best in the [Docker documentation](https://docs.docker.com/engine/swarm/swarm-tutorial/).
|
|
|
|
### **Create network storage**
|
|
|
|
A shared network space is necessary so that all swarm members have access to all the data. To achieve this a [samba](https://www.samba.org/) share is used.
|
|
``` bash
|
|
# Example: Create a Samba share via Docker
|
|
# More details can be found under https://hub.docker.com/r/dperson/samba/
|
|
username@hostname:~$ sudo mkdir -p /srv/samba/nopaque
|
|
username@hostname:~$ docker run \
|
|
--name opaque_storage \
|
|
-v /srv/samba/nopaque:/srv/samba/nopaque \
|
|
-p 139:139 \
|
|
-p 445:445 \
|
|
dperson/samba \
|
|
-p -r -s "nopaque;/srv/samba/nopaque;no;no;no;nopaque" -u "nopaque;nopaque"
|
|
|
|
# Mount the Samba share on all swarm nodes (managers and workers)
|
|
username@hostname:~$ sudo mkdir /mnt/nopaque
|
|
username@hostname:~$ sudo mount --types cifs --options gid=${USER},password=nopaque,uid=${USER},user=nopaque,vers=3.0 //<SAMBA-SERVER-IP>/nopaque /mnt/nopaque
|
|
```
|
|
|
|
### **Download, configure and build nopaque**
|
|
|
|
``` bash
|
|
# Clone the nopaque repository
|
|
username@hostname:~$ git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
|
|
# Create data directories for the database and message queue
|
|
username@hostname:~$ mkdir data/{db,mq}
|
|
username@hostname:~$ cp db.env.tpl db.env
|
|
username@hostname:~$ cp .env.tpl .env
|
|
# Fill out the variables within these files.
|
|
username@hostname:~$ <YOUR EDITOR> db.env
|
|
username@hostname:~$ <YOUR EDITOR> .env
|
|
# Create docker-compose.override.yml file
|
|
username@hostname:~$ touch docker-compose.override.yml
|
|
# Tweak the docker-compose.override.yml to satisfy your needs. (You can find examples in docker-compose.<example>.yml)
|
|
username@hostname:~$ <YOUR EDITOR> docker-compose.override.yml
|
|
# Build docker images
|
|
username@hostname:~$ docker-compose build
|
|
```
|
|
|
|
### Start your instance
|
|
``` bash
|
|
# Create log files
|
|
touch nopaque.log nopaqued.log
|
|
# For background execution add the -d flag
|
|
username@hostname:~$ docker-compose up
|
|
# To scale your app use the following command after starting it normally
|
|
username@hostname:~$ docker-compose -f docker-compose.yml \
|
|
-f docker-compose.override.yml
|
|
-f docker-compose.scale.yml
|
|
up
|
|
-d --no-recreate --scale nopaque=<NUM_INSTANCES>
|
|
```
|