nopaque/docs/platform_basics_brainstorming_2019-04-17.md

2.5 KiB

Web Server

  • Solutions:
    • Apache (with mod WSGI)
    • or nginx (with gunicorn I guess)
  • Goal/Function:
    • Serves content
    • handels HTTP requests etc.
    • handels encryption
      • with SSL/TLS and Let's encrypt
    • serves forms for user request and inputs
    • has copy of Joblist to display those for the user according to requests etc.
    • talks to the Manager service
    • Users CANNOT talk directly with the manager
    • has list of all currently running user sessions (maybe used for authentication)

Application Server

  • Solution: Flask

Authentication and session management

  • Solutions:
    • Flask-Login (minimal)
    • Flask-Session (maybe a bit more functions)
  • Goal/Function
    • handels both internal and external users
  • Relational Database
    • Solutions_
      • PostgreSQL
      • MariaDB
  • Object Relational Mapper
    • Solutions:
      • Flask-SQLAlchemy

Manager Service:

  • Part of the Application Server
  • manages also files on file server
  • Joblist
  • REST API
    • Solution
      • Flask internal
      • and also part of celery
    • Goal/Function
      • Passes requests to the joblist/celery
      • Functions:
        • create_job
        • delete_job
        • get_job (JSON Object or metadata or both?)
        • alter_job
  • Mail notifications
    • Solution:
      • Flask-Mail
    • Goal/Function
      • Sends Mails to users if a OCR job has finished

OCR containers with tesseract

  • Goal/Function
    • celery checks joblists continiously
    • job start commands will be passed to the containers
    • jobs will be started accordingly

Compute pool: Docker Cluster

  • Solutions:
    • Kubernetes
    • swarm
  • Goal/Function
    • How to handle job and resource management for user Jobs and processes.
    • gets requests and tasks from the manager

File Server (Scans, pdfs etc.)

  • Goal/Function
    • stores user input and
    • output files
    • Upload
    • Download
  • Solutions:
    • WebDAV/Samba/Docker Volume

Additional Functions

Information retrival system

  • Solutions:
    • CEQUL with CWB Server
    • Lucene
  • Functions/Goals
    • KWIC
    • KWIC with complex querys (POS, NER, Lemma querys)
    • Frequency lists
    • n-grams
    • complex n-grams
    • etc.