From cdd16f8401a1630ee17571d2baad253ea38c5ad0 Mon Sep 17 00:00:00 2001 From: Stephan Porada Date: Mon, 1 Jul 2019 14:18:35 +0200 Subject: [PATCH] Update docs/platform_basics_brainstorming_2019-04-17.md --- ...latform_basics_brainstorming_2019-04-17.md | 66 +++++++++++++------ 1 file changed, 46 insertions(+), 20 deletions(-) diff --git a/docs/platform_basics_brainstorming_2019-04-17.md b/docs/platform_basics_brainstorming_2019-04-17.md index 6486a050..aab08987 100644 --- a/docs/platform_basics_brainstorming_2019-04-17.md +++ b/docs/platform_basics_brainstorming_2019-04-17.md @@ -1,39 +1,65 @@ -## Manager Service: -- Joblist - - Thread safe - - REST API (https://github.com/zalando/connexion) - - CreateNewJob - - DelteJobs - - GetJobInfo - - AlterJob - - Scheduling - - Ressource management -- manages also files on file server - ## Web Server -- serves content. +- **Solutions**: + - Apache (with mod WSGI) + - or nginx (with gunicorn I guess) +- Serves content +- handels HTTP requests etc. - serves forms for user request and inputs - has copy of Joblist to display those for the user according to requests etc. - talks to the Manager service - Users CANNOT talk directly with the manager - has list of all currently running user sessions (maybe used for authentication) -## Authentication via LDAP (not sure if we need that) -- university internal authentication with LDAP -- LDAP and connexion can use (OAuth 2) - - https://ldapwiki.com/wiki/Best%20Practices%20for%20LDAP%20Security - - https://connexion.readthedocs.io/en/latest/security.html?highlight=authentication -- university external authentication with something else +## Application Server +- **Solution**: Flask + +### Authentication and session management + - **Solutions**: + - Flask-Login (minimal) + - Flask-Session (maybe a bit more functions) +- handels both internal and external users +- Relational Database + - **Solutions**_ + - PostgreSQL + - MariaDB +- Object Relational Mapper + - **Solutions**: + - Flask-SQLAlchemy + +### Manager Service: +- Part of the Application Server +- Joblist + - **Solution**: + - http://www.celeryproject.org/ + - Thread safe + - Scheduling + - Ressource management +- REST API + - **Solution** + - Flask internal + - Passes requests to the joblist/celery + - Functions: + - create_job + - delete_job + - get_job (JSON Object or metadata or both?) + - alter_job +- manages also files on file server + ## Compute pool: Docker Cluster +- Solutions: + - Kubernetes + - swarm - How to handle job and resource management for user Jobs and processes. - gets requests and tasks from the manager -## File Server +## File Server (Scans, pdfs etc.) - stores user input and - output files - Maybe WebDAV ## Database - document based data base like mongoDB preferred +- could be possible that a python framework for jobs and tasks exist +mongo Db would not be necessary then