sporada/nopaque
1
0
Fork 0
mirror of https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git synced 2024-11-14 16:55:42 +00:00
Code Issues Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
Patrick Jentsch 2020-06-09 13:27:02 +02:00
parent d59268dd29
commit d92efba805
1 changed files with 41 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
README.md
View File

@ -1,6 +1,6 @@
# nopaque
_nopaque_ bundles various tools and services that provide humanities scholars with DH methods and thus can support their various individual research processes. Using _nopaque_, researchers can subject digitized sources to Optical Character Recognition (OCR). The resulting text files can then be used as a data basis for Natural Language Processing (NLP). The texts are automatically subjected to various linguistic annotations. The data processed via NLP can then be summarized in the web application as corpora and analyzed by means of an information retrieval system through complex search queries. The range of functions of the web application will be successively extended according to the needs of the researchers.
nopaque bundles various tools and services that provide humanities scholars with DH methods and thus can support their various individual research processes. Using nopaque, researchers can subject digitized sources to Optical Character Recognition (OCR). The resulting text files can then be used as a data basis for Natural Language Processing (NLP). The texts are automatically subjected to various linguistic annotations. The data processed via NLP can then be summarized in the web application as corpora and analyzed by means of an information retrieval system through complex search queries. The range of functions of the web application will be successively extended according to the needs of the researchers.
## Prerequisites and requirements
@ -8,20 +8,20 @@ _nopaque_ bundles various tools and services that provide humanities scholars wi
2. Install docker-compose. Following the official instructions. (LINK)
## Configuration and startup: Run a docker swarm and setup a Samba share
## Configuration and startup
1. **Create Docker swarm:**
### **Create Docker swarm**
The generated computational workload is handled by a [Docker](https://docs.docker.com/) swarm. A swarm is a group of machines that are running Docker and joined into a cluster. It consists out of two different kinds of members, manager and worker nodes. The swarm setup process is described best in the [Docker documentation](https://docs.docker.com/engine/swarm/swarm-tutorial/).
2. **Create a network storage:**
### **Create network storage**
A shared network space is necessary so that all swarm members have access to all the data. To achieve this a [samba](https://www.samba.org/) share is used.
``` bash
# Example: Create a Samba share via Docker
# More details can be found under https://hub.docker.com/r/dperson/samba/
sudo mkdir -p /srv/nopaque/storage
docker run \
username@hostname:~$ sudo mkdir -p /srv/nopaque/storage
username@hostname:~$ docker run \
--name opaque_storage \
-v /srv/nopaque/storage:/srv/nopaque/storage \
-p 445:445 \
@ -31,56 +31,38 @@ docker run \
-u nopaque;nopaque
# Mount the Samba share on all swarm nodes (managers and workers)
sudo mkdir /mnt/nopaque
sudo mount --types cifs --options gid=${USER},password=nopaque,uid=${USER},user=nopaque,vers=3.0 //<SAMBA-SERVER-IP>/storage.nopaque /mnt/nopaque
username@hostname:~$ sudo mkdir /mnt/nopaque
username@hostname:~$ sudo mount --types cifs --options gid=${USER},password=nopaque,uid=${USER},user=nopaque,vers=3.0 //<SAMBA-SERVER-IP>/storage.nopaque /mnt/nopaque
```
## Download, configure and build _nopaque_**
3. **Download, configure and build _nopaque_**
### **Download, configure and build nopaque**
``` bash
git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
mkdir logs
cp nopaque.env.tpl nopaque.env
<YOUR EDITOR> nopaque.env # Fill out the variables within this file. For the GitLab variables either use your credentials (not recommended) or create an access token with the read_registry scope. If this repository is public no credentials are needed.
username@hostname:~$ git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
username@hostname:~$ mkdir logs
username@hostname:~$ cp .env.tpl .env
# Fill out the variables within this file.
username@hostname:~$ <YOUR EDITOR> .env
username@hostname:~$ cp docker-compose.development-override.yml.tpl docker-compose.override.yml
# Tweak the docker-compose.override.yml to satisfy your needs.
username@hostname:~$ <YOUR EDITOR> docker-compose.override.yml
```
### Variables and their values are explained here:
The first three DOCKER variables should only be used if you want to use the Docker HTTP API. Check the daemon readme to see how to create certificates and activate the API.
FLASK_CONFIG=development|testing|production \
SECRET_KEY=92b461ba136e4ca48e430003acd56977 \
Uses this for example to create a secret key: `python -c "import uuid; print(uuid.uuid4().hex)"`
#### Configuration variables in detail
The **GitLab Registry** Variables are not needed if this repository is public. If needed use your GitLab username and a token as a password
The variables prefixed with **DOCKER** should only be filled out if you want to use the Docker HTTP API. Check the [Docker Documentation](https://docs.docker.com/engine/security/https/) to see how to create certificates, configure and activate the Docker HTTP API.
**Flask Mail** variables are needed for sending password reset mails etc. Use your own mail server configs here.\
MAIL_SERVER=smtp.example.com \
MAIL_PORT=587 \
MAIL_USE_TLS=true \
MAIL_USERNAME=user@example.com \
MAIL_PASSWORD=password \
NOPAQUE_MAIL_SENDER=Nopaque Admin <user@example.com> _Name shown as sender._
The variables prefixed with **GITLAB** hold your login information of this GitLab instance. Either use your credentials (not recommended) or create an access token with the read_registry scope.
**Nopaque** variables are needed for the web app.\
NOPAQUE_ADMIN=yourmail@example.com _If a user is registered using this mail the user will automatically be granted admin rights._ \
NOPAQUE_CONTACT=contactmailadress@example.com _Contact mail address shown in the footer of the web application._\
NOPAQUE_DOMAIN=yourdomain.com _The domain your nopaque installation is hosted on. use https://nopaque.localhost for a local running instance._ \
NOPAQUE_LOG_LEVEL=WARNING|INFO|ERROR|DEBUG \
NOPAQUE_STORAGE=path/to/your/samba/share
The value of the **NOPAQUE_MAIL_SENDER** variable is shown as a sender of emails generated by nopaque.
**PostgreSQL Database** credentials: \
POSTGRES_DB_NAME=dbanme \
POSTGRES_USER=username \
POSTGRES_PASSWORD=password
On registration the email address stored in the **NOPAQUE_ADMIN** variable will automatically be granted the administrator role.
The email address stored in **NOPAQUE_CONTACT** will be used within the contact button of the footer within the websites.
### Start your instance
``` bash
cp .env.tpl .env
<YOUR EDITOR> .env # Fill out the variables within this file.
docker-compose build
```
**Start your instance**
```bash
# for background execution add the -d flag and to scale the app, add --scale web=<NUM-INSTANCES>
docker-compose up
# For background execution add the -d flag and to scale the app, add --scale web=<NUM-INSTANCES>
username@hostname:~$ docker-compose up
```