mirror of
https://gitlab.ub.uni-bielefeld.de/sfb1288inf/nopaque.git
synced 2024-11-14 16:55:42 +00:00
Update readme
This commit is contained in:
parent
a7e94b8c9b
commit
08a277d967
100
README.md
100
README.md
@ -6,83 +6,61 @@ Opaque is designed as a web application which can be easily used by researchers
|
||||
|
||||
In particular researchers can use Opaque to start OCR jobs for digitized sources. The text output of these OCR jobs can then be used as an input for tagging processes (POS, NER etc.).
|
||||
|
||||
As a last step texts can be loaded into an information retrieval system to query for specific words, phrases in connection with POS tags.
|
||||
As a last step texts can be loaded into an information retrieval system to query for specific words, phrases in connection with linguistic features.
|
||||
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Docker: https://www.docker.com/
|
||||
- Python 3.5+
|
||||
- cifs-utils
|
||||
- Docker
|
||||
- Docker Compose
|
||||
|
||||
|
||||
## Setup
|
||||
0. **Create log files**
|
||||
``` bash
|
||||
mkdir /logs
|
||||
```
|
||||
|
||||
## Configuration and startup
|
||||
|
||||
1. **Create Docker swarm:**
|
||||
The generated computational workload is handled by a [Docker](https://docs.docker.com/) swarm. A swarm is a group of machines that are running Docker and joined into a cluster. It consists out of two different kinds of members, managers and workers. Currently it is not possible to specify a dedicated Docker host, instead Opaque expects the executing system to be a swarm manager of a swarm with at least one dedicated worker machine. The [swarm setup](https://docs.docker.com/engine/swarm/swarm-tutorial/) process is described best in the Docker documentation.
|
||||
The generated computational workload is handled by a [Docker](https://docs.docker.com/) swarm. A swarm is a group of machines that are running Docker and joined into a cluster. It consists out of two different kinds of members, managers and workers. Currently it is not possible to specify a dedicated Docker host, instead Opaque expects the executing system to be a swarm manager of a cluster with at least one dedicated worker machine. The swarm setup process is described best in the [Docker documentation](https://docs.docker.com/engine/swarm/swarm-tutorial/).
|
||||
|
||||
2. Create a dedicated user `opaque` on all swarm members with `sudo useradd opaque`.
|
||||
|
||||
3. Create shared network storage
|
||||
A shared network space is necessary so that all swarm members have access to all the data. To achieve this a [Samba](https://www.samba.org/) share is used.
|
||||
2. **Create a network storage**
|
||||
A shared network space is necessary so that all swarm members have access to all the data. To achieve this a [Samba](https://www.samba.org/) can be used.
|
||||
``` bash
|
||||
# Start a samba service on a swarm manager node
|
||||
SAMBA_DIRECTORY=</ABSOLUT/PATH>
|
||||
SAMBA_HOSTNAME=<HOSTNAME>
|
||||
SAMBA_PASSWORD=<SET_PASSWORD>
|
||||
|
||||
docker service create \
|
||||
--constraint node.hostname==$SAMBA_HOSTNAME \
|
||||
--mount type=bind,src=$SAMBA_DIRECTORY,dst=/storage.opaque \
|
||||
--name samba_opaque \
|
||||
--publish published=139,target=139,mode=host \
|
||||
--publish published=445,target=445,mode=host \
|
||||
dperson/samba \
|
||||
-p \
|
||||
-s "storage.opaque;/storage.opaque;no;no;no;opaque" \
|
||||
-u "opaque;$SAMBA_PASSWORD"
|
||||
# Example: Create a Samba share via Docker
|
||||
# More details can be found under https://hub.docker.com/r/dperson/samba/
|
||||
$ sudo mkdir -p /srv/nopaque/storage
|
||||
$ docker run \
|
||||
--name opaque_storage \
|
||||
-v /srv/nopaque/storage:/srv/nopaque/storage \
|
||||
-p 445:445 \
|
||||
dperson/samba \
|
||||
-p \
|
||||
-s storage.nopaque;/srv/nopaque/storage;no;no;no;nopaque \
|
||||
-u nopaque;nopaque
|
||||
|
||||
|
||||
# The following steps need to be executed on all swarm members
|
||||
# Login as opaque user
|
||||
sudo su opaque
|
||||
# Create mount point for opaque storage
|
||||
mkdir -p $HOME/mnt/opaque
|
||||
# Mount the samba share
|
||||
sudo mount -t cifs -o gid=opaque,password=$SAMBA_PASSWORD,uid=opaque,user=opaque,vers=3.0 //$SAMBA_HOSTNAME/storage.opaque $HOME/mnt/opaque
|
||||
# Mount the Samba share on all swarm member nodes with the following code
|
||||
$ sudo mkdir /mnt/nopaque
|
||||
$ sudo mount --types cifs --options gid=${USER},password=nopaque,uid=${USER},user=nopaque,vers=3.0 //<YOUR IP>/storage.nopaque /mnt/nopaque
|
||||
```
|
||||
|
||||
4. Clone the Opaque repository to the swarm manager, that should execute the Opaque server software
|
||||
```
|
||||
git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/opaque.git
|
||||
cd opaque
|
||||
```
|
||||
|
||||
4.1 Create a configuration file
|
||||
3. **Download Opaque**
|
||||
``` bash
|
||||
touch .env
|
||||
# Account information of a mail account for sending emails to opaque users.
|
||||
echo "MAIL_USERNAME=opaque@example.com" >> .env
|
||||
echo "MAIL_PASSWORD=password" >> .env
|
||||
echo "MAIL_SERVER=smtp.example.com" >> .env
|
||||
echo "MAIL_PORT=587" >> .env
|
||||
echo "MAIL_USE_TLS=true" >> .env
|
||||
# A user registering with this email address will automatically promoted as an admin.
|
||||
echo "NOPAQUE_ADMIN=admin.opaque@example.com" >> .env
|
||||
# Absolut path to an existing directory to save all opaque files.
|
||||
echo "OPAQUE_STORAGE=/home/opaque/mnt/opaque" >> .env
|
||||
$ git clone https://gitlab.ub.uni-bielefeld.de/sfb1288inf/opaque.git
|
||||
$ cd opaque
|
||||
$ docker-compose pull
|
||||
```
|
||||
|
||||
4.2 Create Python virtual environment, activate it and install the required python packages.
|
||||
```
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
4. **Configure your instance**
|
||||
``` bash
|
||||
$ cp nopaque.env.tpl nopaque.env
|
||||
$ <YOUR EDITOR> nopaque.env # Fill out the empty variables within this file.
|
||||
```
|
||||
|
||||
5. Start the server: `python opaque.py`
|
||||
6. test
|
||||
5. **Start your instance**
|
||||
``` bash
|
||||
# Execute the following 3 steps only on first startup
|
||||
$ docker-compose run web flask upgrade
|
||||
$ docker-compose run web flask insert-initial-database-entries
|
||||
$ docker-compose down
|
||||
|
||||
$ docker-compose up
|
||||
```
|
||||
|
Loading…
Reference in New Issue
Block a user